Package 'randomPlantedForest'

Description

Method for bundle::bundle() wrapping rpf_marshal()/rpf_unmarshal(), so rpf models work with the standard tidymodels serialization workflow. Training data is not included; see rpf_marshal() for the implications. Pruned forests from prune_components() or rpf() are bundled the same way.

Usage

## S3 method for class 'rpf'
bundle(x, ...)

## S3 method for class 'rpf_pruned'
bundle(x, ...)

Arguments

Value

An object of class bundled_rpf (bundled_rpf_pruned) for bundle(), or the restored rpf (rpf_pruned) object for unbundle().

Examples

fit <- rpf(mpg ~ wt + cyl, data = mtcars, n_families = 10, prune = FALSE)
b <- bundle::bundle(fit)
tmp <- tempfile(fileext = ".rds")
saveRDS(b, tmp)
restored <- bundle::unbundle(readRDS(tmp))
predict(restored, mtcars)
unlink(tmp)

Description

The points of greediness that rpf(tune = ) searches: pairs of n_try_split_points (split points tried per split candidate) and share_candidates (share of the possible splits drawn as candidates in each round), greediest first. Each step down the ladder halves share_candidates and n_try_split_points (rounded up, at least 1), from 40 / 1 to 1 / 0.003125, except the first, which takes share_candidates from 1 to 0.8. Row 3 is the other split structures' default, 10 / 0.4; row 5 is rpf()'s default for split_structure = "leaf", 3 / 0.1.

Usage

greediness_ladder()

Value

A data.frame with columns n_try_split_points and share_candidates, one row per point, greediest first.

See Also

tune_control(), rpf()

Examples

greediness_ladder()

Description

The numbers of splits at which rpf() with oob_path = TRUE evaluates the out-of-bag (OOB) error: a geometric grid with four points per doubling, round(25 * 2^(j / 4)) from 6 up to n_splits (it contains 12, 25, 50, ..., 1600, 3200), and n_splits itself; every number up to 12 if n_splits <= 12. The error changes slowly in log(n_splits), so a geometric grid resolves it evenly, and the path's memory, 2 * nrow(x) * length(checkpoints) doubles, stays small; every split would cost 2 * nrow(x) * n_splits.

Usage

oob_path_checkpoints(n_splits, ...)

Arguments

Value

An increasing integer vector.

See Also

rpf() (oob_path)

Examples

oob_path_checkpoints(400)

Description

Illustrates how a Random Planted Forest predicts by drawing the trees of one tree family. Each tree belongs to a set of variables, and its leaves are boxes in those variables with a value each. The prediction for a point is the root value plus the values of all leaves containing it, averaged over tree families.

Usage

## S3 method for class 'rpf'
plot(
  x,
  family = 1L,
  ...,
  type = c("tree", "boxes"),
  max_interaction = 2L,
  trees = NULL,
  class = NULL,
  factor_levels = TRUE
)

Arguments

Details

• type = "tree" draws a split diagram in the style of Figure 1b of Hiabu et al. (2020), with the trees in rows by number of variables, each leaf filled by its value. The forest stores only its final leaves, not the order of splits or which leaf a tree grew from, so the splits within a tree are reconstructed: the diagram shows a split structure consistent with the leaves, and dashed lines link trees to the trees they could have grown from. Leaves that overlap other leaves of the same tree hang from dotted lines. Readable for small fits, e.g. n_families = 1 and n_splits = 10.

• type = "boxes" draws one panel per tree in variable space: leaves of trees on one variable as bars from zero to their value, leaves of trees on two variables as rectangles filled by their value. Overlapping leaves add up, which the translucent drawing shows.

Trees on more than max_interaction variables, and trees not in trees, are not drawn but summarized as a remainder, similar to the remainder term of predict_components().

After purify(), the trees are the components of the functional decomposition and only type = "boxes" applies. To plot components averaged over all tree families, use predict_components(), e.g. with the plotting functions in glex.

Value

A ggplot for type = "tree", a patchwork for type = "boxes".

References

Hiabu, M., Mammen, E., Meyer, J. T. (2020). Random Planted Forest: a directly interpretable tree ensemble. arXiv:2012.14563.

Examples

rpfit <- rpf(mpg ~ wt + hp, data = mtcars, n_families = 1, n_splits = 10, prune = FALSE)
plot(rpfit)
purify(rpfit)
plot(rpfit, type = "boxes")

Description

Evaluates the components of a model at the rows of new_data: its intercept and one column per main effect and interaction. They are a functional decomposition of the prediction: for every row, the intercept plus the components equals the prediction. For a pruned model, the default of rpf(), these are its kept components.

Usage

predict_components(object, new_data, ...)

## S3 method for class 'rpf'
predict_components(
  object,
  new_data,
  max_interaction = NULL,
  predictors = NULL,
  ...
)

## S3 method for class 'rpf_pruned'
predict_components(
  object,
  new_data,
  max_interaction = NULL,
  predictors = NULL,
  ...
)

Arguments

Details

Extracts all possible components up to max_interaction degrees, up to the value set when calling rpf(). The intercept is always included. Optionally predictors can be specified to only include components made of the given predictors; the sum of the other components is returned as remainder. If max_interaction is greater than length(predictors), the max_interaction will be lowered accordingly.

All components are evaluated in one pass over the rows of new_data. Components that the purified forest does not hold, such as sets of predictors that never appear together in a tree, are zero columns.

The components are only unique for a purified forest; purify() says how they are identified. An unpurified forest is purified in place on first use with the defaults of purify(). For other settings, such as more threads, call purify() before predict_components().

Value

A list with elements:

• m (data.table): Components for each main effect and interaction term, representing the functional decomposition of the prediction. All components together with the intercept sum up to the prediction. For multiclass classification, the number of output columns is multiplied by the number of target_levels, the levels of the outcome: one column per component and level, named ⁠<component>__class:<level>⁠. With loss = "logit" or "exponential" a component's columns sum to 0 across the levels, as the scores of predict() with type = "link" do.

• intercept (numeric): Expected value of the prediction. For multiclass classification, one value per target level, named by target_levels.

• x (data.table): The predictors selected by predictors, as the forest sees them in new_data (with a formula, its terms, such as log(x1)).

• target_levels (character): For multiclass classification only: Vector of target levels which can be used to disassemble m, as names include both term and target level.

• remainder (numeric): Only if m leaves out components of the forest: if predictors leaves out some of the forest's predictors, if max_interaction (lowered to the number of predictors) is below that of the rpf object, or if the forest was purified with a maxp_interaction below it (see purify()). The prediction minus intercept and m's row sums, so that the three sum to the prediction. For multiclass classification, one column per target level.

Pruned forests

For an rpf_pruned object from prune_components() or rpf(), m holds only the kept components, each scaled by its coefficient beta_S, and intercept is the pruned model's intercept alpha, so that they sum to predict() of the pruned model. Components that are not kept are absent, not zero columns. If predictors or max_interaction leave out kept components, their sum is returned as remainder.

Note

Depending on the number of predictors and max_interaction, the number of components will increase drastically to sum(choose(ncol(new_data), seq_len(max_interaction))).

See Also

purify() for how the components are identified, prune_components() for the pruned model's components. The glex package plots the result with plot().

Examples

# The default model: its kept components add up to its prediction
set.seed(1)
model <- rpf(mpg ~ wt + hp + cyl, data = mtcars)
components <- predict_components(model, mtcars[1:3, ])
components$m
components$intercept + rowSums(components$m)
predict(model, mtcars[1:3, ])

# A forest: all its components up to its max_interaction
train <-  mtcars[1:20, 1:4]
test <-  mtcars[21:32, 1:4]

set.seed(23)
rpfit <- rpf(mpg ~ ., data = train, max_interaction = 3, n_families = 30, prune = FALSE)

# Extract all components, including main effects and interaction terms up to `max_interaction`
(components <- predict_components(rpfit, test))

# sums to prediction
cbind(
  m_sum = rowSums(components$m) + components$intercept,
  prediction = predict(rpfit, test)
)

# Only get components with interactions of a lower degree, ignoring 3-way interactions
predict_components(rpfit, test, max_interaction = 2)

# Only retrieve main effects
(main_effects <- predict_components(rpfit, test, max_interaction = 1))

# The difference is the combined contribution of interaction effects
cbind(
  m_sum = rowSums(main_effects$m) + main_effects$intercept,
  prediction = predict(rpfit, test)
)

Description

Predictions of a forest (class rpf, from rpf() with prune = FALSE) for new data: the mean of its tree families' predictions. For classification, type chooses probabilities, classes or the raw scores. A pruned model, the default of rpf(), has its own method, predict.rpf_pruned(), with the same arguments.

Usage

## S3 method for class 'rpf'
predict(
  object,
  new_data,
  type = ifelse(object$mode == "regression", "numeric", "prob"),
  nthreads = NULL,
  ...
)

Arguments

Details

new_data is prepared as the training data were: by the formula or recipe of the fit, and with factor levels coded as in the fit (see "Factor predictors" in rpf()). Missing values are an error.

Value

For regression: A tbl with column .pred with the same number of rows as new_data.

For classification: A tbl with one column ⁠.pred_<level>⁠ for each level in y containing class probabilities if type = "prob". For type = "class", one column .pred_class with class predictions is returned. For type = "numeric" or "link", raw predictions are returned: one column .pred for binary outcomes (the log-odds of the second level with loss = "logit" or "exponential"), and one column ⁠.pred_<level>⁠ per level for multiclass outcomes. With loss = "logit" or "exponential", these are one score per class that sum to 0 across the classes, and the probabilities are their softmax. (The multiclass logit loss fits K - 1 scores against the first level; they are reported with the first level's score 0 joined and centred across the K classes, which leaves the probabilities as they are.)

See Also

predict.rpf_pruned() for the pruned model, predict_components() for the prediction as a sum of components.

Examples

# Regression: a tibble with the column .pred
rpfit <- rpf(mpg ~ cyl + wt, data = mtcars, n_families = 20, prune = FALSE)
predict(rpfit, mtcars[1:3, ])

# Classification: probabilities by default, or classes, or the raw scores
rpfit <- rpf(Species ~ ., data = iris, n_families = 20, prune = FALSE)
predict(rpfit, iris[c(1, 51, 101), ])
predict(rpfit, iris[c(1, 51, 101), ], type = "class")
predict(rpfit, iris[c(1, 51, 101), ], type = "link")

Description

The prediction of a pruned forest is alpha + sum(beta_S * f_S(x)) over the kept components f_S of the forest, with the intercept alpha and the coefficients beta_S (see prune_components()). For classification these are the scores, one per score column, which type turns into probabilities or classes as predict.rpf() does.

Usage

## S3 method for class 'rpf_pruned'
predict(
  object,
  new_data,
  type = ifelse(object$fit$mode == "regression", "numeric", "prob"),
  nthreads = NULL,
  ...
)

Arguments

Value

A tbl with the same number of rows as new_data: for regression column .pred; for classification the columns of predict.rpf() for type.

See Also

prune_components(), predict_components().

Examples

set.seed(1)
x <- matrix(runif(300 * 3, -1, 1), ncol = 3, dimnames = list(NULL, paste0("x", 1:3)))
y <- x[, 1] + 2 * x[, 2] * x[, 3] + rnorm(300, sd = 0.3)
fit <- rpf(x, y, max_interaction = 2, n_families = 10, n_splits = 30, prune = FALSE)
pruned <- prune_components(fit, x, y, rule = "R", tau = 0.01)
predict(pruned, x[1:5, ])

Description

Shows a forest (class rpf, from rpf() with prune = FALSE): the kind of model and its formula, the number of tree families and of splits (with how the splits were chosen), the predictors and the interaction order, whether the forest is purified, the settings of the split search and the number of threads. A pruned model, the default of rpf(), is printed by print.rpf_pruned().

Usage

## S3 method for class 'rpf'
print(x, ...)

Arguments

Value

Invisibly: x.

See Also

rpf(), print.rpf_pruned().

Examples

rpf(mpg ~ cyl + wt + drat, data = mtcars, max_interaction = 2, n_families = 10, prune = FALSE)

Description

The flattened forest that rpf() stores with export_forest = TRUE (as ⁠$forest⁠) is a long nested list of trees. These methods print a one-line summary instead of the whole list.

Usage

## S3 method for class 'rpf_forest'
print(x, ...)

## S3 method for class 'rpf_forest'
str(object, ...)

Arguments

Value

Invisibly: x (object for str()).

See Also

rpf

Examples

rpfit <- rpf(mpg ~ cyl + wt, data = mtcars, n_families = 10, export_forest = TRUE, prune = FALSE)
print(rpfit$forest)
str(rpfit$forest)

Description

Shows the model: its intercept and its components, each with its importance, the component's share of the variance of the model's prediction on the training rows (components$model_share). How the components were chosen (the forest's families and splits, given or chosen out of bag, the rule and threshold, what was left out, the heat-map forms) follows briefly at the end.

Usage

## S3 method for class 'rpf_pruned'
print(x, n_show = 20, ...)

Arguments

Value

Invisibly: x.

See Also

prune_components().

Examples

set.seed(1)
x <- matrix(runif(300 * 3, -1, 1), ncol = 3, dimnames = list(NULL, paste0("x", 1:3)))
y <- x[, 1] + 2 * x[, 2] * x[, 3] + rnorm(300, sd = 0.3)
fit <- rpf(x, y, max_interaction = 2, n_families = 10, n_splits = 30, prune = FALSE)
prune_components(fit, x, y, rule = "R", tau = 0.01)

Description

Drops the small components of a forest's functional decomposition and, optionally, refits one coefficient per kept component. The rule and the threshold are chosen out of bag by default, from the forest itself without refits, or by K-fold cross-validation on the training rows. By default every kept interaction of three or more predictors is then coarsened so that it can be shown in at most max_panels heat maps, and the model is pruned again.

Usage

prune_components(
  object,
  data,
  y = NULL,
  ...,
  max_panels = 10,
  rule = c("D", "R"),
  tau = c(0, 1e-04, 3e-04, 0.001, 0.003, 0.01, 0.03, 0.1),
  select = c("1se", "min"),
  select_by = c("oob", "cv"),
  folds = 5L,
  fold_id = NULL,
  seed = NULL,
  nthreads = NULL,
  verbose = FALSE,
  keep_forest = FALSE
)

Arguments

Details

The forest is decomposed into its purified components f_S, one per set S of predictors (see purify() and predict_components()), at all orders up to max_interaction, and its intercept f_0. The size of a component is its share of the variance of the prediction f on the training rows, share_S = Var(f_S) / Var(f). The shares are descriptive sizes, not a partition: the components are not orthogonal on the data, so they need not add up to 1. A component is kept if its share is at least tau (and it is not identically zero on the training rows). Two rules are available:

• "D" (drop): the prediction is f_0 plus the kept components. rule = "D" with tau = 0 is the unpruned forest.

• "R" (drop and refit): the prediction is alpha + sum(beta_S * f_S) over the kept components, with the intercept alpha and the coefficients beta_S by least squares of y on the kept components (the minimum-norm solution if the components are collinear). The components themselves are not changed, only their scale. With select_by = "oob" (the default, see below), the least squares use each training row's out-of-bag components instead of the forest's own, for a chosen and for a given rule and tau. The forest's own components have fitted the training rows' noise, so least squares on them scale the components up. With select_by = "cv", and where the default falls back to it (see below), the least squares use the forest's own components on the training rows.

The choice. If more than one pair of rule and tau is given (the default: both rules and the grid 0, 0.01%, 0.03%, 0.1%, 0.3%, 1%, 3%, 10%), the pair is chosen by its out-of-bag (select_by = "oob", the default) or cross-validated ("cv") RMSE. With select = "1se" (the default), the largest tau whose MSE is within one standard error of the minimum is chosen (the SE is the standard deviation of the per-fold MSEs of the minimum, divided by sqrt(K)), and at that tau the rule with the lower RMSE. With select = "min", the pair with the lowest RMSE is chosen (ties, up to a relative 1e-12: the larger tau, then "D"). "1se" gives sparse models: with "R" fitted out of bag the out-of-bag error changes little over a range of tau, and "min" keeps small components that add nothing on new data.

Out of bag (select_by = "oob"). Every tree family of object is grown on a bootstrap sample of the training rows, and the purification works family by family. So every family's components (see Purification below) are also evaluated at the training rows outside its bootstrap sample. A row's out-of-bag (OOB) component is the mean over its OOB families (about 37% of them), and its OOB intercept likewise. The purification uses the predictors of all training rows, not the outcome, so no outcome information of a row enters its OOB components.

• The kept components of every tau are those of the final model (the shares of object on the training rows).

• "D" is scored by the OOB intercept plus the kept OOB components.

• For "R", alpha and beta_S are fitted as for the final model, by least squares on the OOB components, but only on the rows outside each of K folds (folds, fold_id, seed). They are scored with the OOB components of the fold's rows, so that no row scores coefficients fitted on it. The final model's are fitted on all rows with at least one OOB family.

• An OOB mean averages m_i families (about 37% of B = n_families), the final model all B, so the OOB prediction scatters more around its expectation. The OOB MSE is therefore corrected: per row it subtracts (1 / m_i - 1 / B) * v_i, with v_i the across-family variance of the candidate's prediction, estimated from the OOB means over the even and over the odd families (for a row whose OOB families are all even or all odd: the mean estimate of the other rows). Without the correction, the noise of the OOB components makes the candidates look worse than they are. The uncorrected values are reported as oob_rmse_raw.

• If the correction exceeds a candidate's uncorrected MSE, its corrected MSE is reported as 0. That is not a perfect candidate but an overshooting, unreliable estimate (it happens with very few rows, e.g. "R" with about as many coefficients as rows, or with almost no noise). Such candidates are not chosen, unless all candidates are at 0 (then they tie).

• The OOB RMSE is taken over the rows with at least one OOB family, the SE for "1se" from the per-fold MSEs.

• With few families the OOB means average only one to a few families, and many rows have none (16% at 4 families). From 10 families on, the OOB choice is usually as good as the CV's, or better where the CV's fold fits choose poorly. Below 10 families (n_families), the default select_by scores by CV instead (with a message); an explicit select_by = "oob" warns.

• A deterministic forest has no out-of-bag rows, and a forest restored by rpf_unmarshal() from a blob made by an earlier version has no in-bag counts (with a message): with the default select_by, they are scored by CV instead.

• A given rule "R" and tau (no choice) is fitted on the OOB components too, with the same fallbacks: below 10 families or without in-bag counts its coefficients are fitted in sample (with a message), and so they are for a deterministic forest. A given "D" needs no OOB components.

• Factor predictors are coded by the outcome of all training rows: their levels are ordered by the mean outcome (regression), by the share of the second class (binary classification) or by the first principal component of the class proportions (three or more classes; Coppersmith, Hong and Hosking 1999). So a row's own outcome enters the coding its OOB families split on: with factors the OOB error is somewhat optimistic.

Cross-validation (select_by = "cv"). For each fold, the forest is refitted with the parameters of object on the other folds (after set.seed(seed), the same for every fold), its components, shares and refit are computed on those rows, and every candidate is scored on the held-out fold. The CV RMSE is sqrt(sum(SSE_k) / n), summing the squared errors over the K folds.

The chosen (or given) pair is then applied to object itself (not refitted): its components and shares on data, and for "R" the least squares on the OOB components of data's rows (out of bag) or on its components on data (by CV). data and y must therefore be the rows object was trained on, in the order of the fit (checked).

The display (max_panels, default 10; Inf switches it off). A component of two predictors is one heat map. A kept component f_S of three or more predictors is shown as heat maps over two of its predictors, one per cell of the others. If its own cut points need more than max_panels such heat maps, it is coarsened:

• Two predictors of S stay at full resolution. Each other predictor k is cut into q_k intervals of equal numbers of training rows (ties merged), with prod(q) <= max_panels; q_k = 1 averages k out. Only the maximal q vectors (no q_k can grow) are considered; they are listed directly, not searched among all ⁠max_panels^(|S| - 2)⁠ vectors.

• Within a cell, f_S is averaged over the coarse predictors with the product of their training marginals. The coarsened component is not purified again: purify() identifies the components under the joint distribution of the training rows, not under the product of the marginals, and the averaging changes the component's means along its two full-resolution predictors and its mean over the training rows. It is used as it is (no part of it is moved into the intercept or into components of lower order).

• Of all choices of the two full predictors and the q vector, the one keeping the largest share of the variance of f_S on the training rows is taken (ties: the first in lexicographic order).

Then the same rule prunes again, on the components the first pruning kept, with the coarsened ones in place of the originals. Its shares, which tau thresholds, are of the variance of that model (the intercept plus these components) on the training rows, not of the forest's prediction. A pair chosen out of bag is chosen again out of bag, with the same candidates and folds, scored with each family's coarsened components at its out-of-bag rows (R's coefficients out of bag); a given rule and tau, or a pair chosen by CV, is applied again as it is (R's coefficients out of bag if the first pruning fitted them so, else in sample; a CV choice is not repeated, as it would need the display in every fold's refit). The prediction then uses the coarsened components, so the pruning judges the model that is shown. This second pruning is usually a little more accurate than pruning and then coarsening, or than not coarsening. If no kept component needs coarsening, the model is the first pruning's. The display costs about as much again as the out-of-bag pruning: a purification restricted to the components being coarsened (two at a time) for their candidates, and one for their out-of-bag values; the stored coarsened components take a few MB each on large data at order 3, more at higher orders.

Purification. The pruning needs every component's values at the training rows (and out of bag, its OOB means), not the purified forest: purify() keeps every component of every tree family as pieces, which at high orders takes far more memory than the forest (see Cost). So each family is purified in turn, as purify() does (mode 2, all orders), its components are evaluated at the training rows and added up, and the family's pieces are freed. The pruned model then stores the pieces of its kept components only, merged over the families exactly as purify() stores them, and predict() and predict_components() evaluate them from there; a coarsened component (the display) is stored in its coarsened form. The components equal those of the purified forest up to rounding (about 1e-15 relative), and the stored ones are the purified forest's bit for bit. object itself is not purified: an earlier purify(), at any maxp_interaction or mode, is left as it is and does not change the result.

Random numbers. The folds are drawn after set.seed(seed), and the CV's fold fits after it too. Afterwards the random number stream is restored to its state on entry (after drawing seed if it is NULL), so the call does not reset the caller's stream.

Cost. The out-of-bag choice purifies the families of object once (see Purification); it needs no fit. The CV fits and purifies the forest K more times (each fold's forest as purify() does). Only the components the forest has are evaluated (at most sum(choose(p, seq_len(max_interaction))), usually far fewer: the subsets of the trees' predictor sets). The memory is the forest's plus about n doubles per component, three times that for the out-of-bag choice, plus one family's components at a time per thread; storing the kept components passes over the families once more. So the whole call of rpf(n_splits = NULL, prune = TRUE) can need much less memory than purify() of the forest alone, most so at high orders, where most of it goes to the coarsened components the display keeps. nthreads speeds up the fits and the purification.

Classification. A classification forest is pruned the same way, with its scores: one for a binary outcome, one per class otherwise. Every component then has a column per class, and:

• A component's share sums its variance over the classes' scores, centred across the classes (as predict() reports them with type = "link": the part all classes share does not count), over the same sum for the prediction. One kept list serves all classes. (Internally loss = "logit" fits K - 1 scores against the first level; centred, the shares, the criterion and the model do not depend on which level is first.)

• "R" fits one coefficient per kept component, shared by the classes, and one intercept per class (up to a constant shared by the classes, which the probabilities ignore; they are reported summing to 0), out of bag as for regression.

• The criterion is the forest's loss's own: for loss = "L1" or "L2" the squared error of the raw scores summed over the columns (the Brier score; least squares for "R"), for "logit" and "exponential" the log-loss of the probabilities that predict(type = "prob") makes (maximum likelihood for "R", by Newton's method from rule "D"'s model). The OOB correction is its second-order form (see rpf(), "Classification"). The tables then have oob_loss, se_loss, oob_loss_raw and se_loss_raw (the mean criterion) instead of the RMSE columns, and cv_loss and se_loss for CV; select = "1se" uses the SE of the mean criterion. A binary forest with "L1" or "L2" has one 0/1 column and is pruned as a regression forest of that response, with the RMSE columns.

• The display coarsens a component with one form (C, q, cuts) for all score columns, chosen by the variance kept of the centred columns; its stored form is one per score column.

• predict() takes type as predict.rpf() does, and predict_components() has one column per component and class, named as for the forest (⁠<component>__class:<level>⁠); with "logit" or "exponential" each component's columns, and the intercepts, sum to 0 across the classes.

Value

An object of class rpf_pruned, a list with elements:

• fit: the description of object (its blueprint, mode, params, and with n_splits = NULL its oob_path and splits_oob), without the forest (fit$fit is NULL), so that a use of the forest from it is an error that says so. With keep_forest = TRUE, object as it was (not purified by the pruning).

• rule, tau: the chosen rule and threshold.

• intercept: alpha (the forest's intercept f_0 for "D"); for classification one per score column of the forest (K - 1 for multiclass loss = "logit", against the first level; with "exponential" and rule "R" the first is 0). predict_components() and print() report them as one per class, centred.

• beta: named vector of the coefficients of the kept components (all 1 for "D"). For "R" with select_by "oob", intercept and beta are fitted on the out-of-bag components (see Details).

• sets: the predictor sets of the kept components, as column indices.

• components: a data.frame of every nonzero component of object on the training rows (after a second pruning of the display, only of those the first pruning kept: see below): component (its name, as in predict_components()), order, share, kept, beta (NA if dropped), and model_share: the share of the final model's component, ⁠Var(beta_S f_S) / Var(prediction of the model)⁠ on the training rows (with the display, of f_S in its heat-map form; for classification, as share, over the centred score columns; NA if dropped). share is what tau thresholds; model_share is the component's importance in the model, which print() shows. As the components need not be uncorrelated, the shares need not sum to 1.

• oob: NULL unless the pair was chosen out of bag; otherwise a data.frame with one row per candidate: rule, tau, oob_rmse (corrected; used for the choice; 0 where the correction exceeds the uncorrected MSE, see Details), se_mse (from its per-fold MSEs), oob_rmse_raw and se_mse_raw (uncorrected), n_kept, min and one_se (the two choices). Its attribute n_rows is the number of rows with an OOB family.

• cv: NULL unless the pair was chosen by CV; otherwise a data.frame with one row per candidate: rule, tau, cv_rmse, se_mse (SE of the CV MSE), n_kept (mean over the folds), min and one_se.

• select, select_by, fold_id, seed: as used (fold_id and seed are NULL if no choice was made). For a given rule and tau, select_by is "oob" if R's coefficients were fitted out of bag, otherwise NULL.

• criterion (classification only): "brier" or "logloss", what the candidates were scored by.

• With the display (finite max_panels) and a component coarsened, a second pruning runs, and these describe it: its candidates are the components the first pruning kept, with the coarsened ones in place, and oob is its table. components then lists only these components, and their share (which the second pruning's tau thresholds) is of the variance of their model (the intercept plus these components) on the training rows, not of the forest's prediction; every nonzero component of object with its share of the forest's prediction is in display$first$components. Without a coarsened component, they are the first pruning's. With the display, components also has form ("full" for one or two predictors, "asis" if its own cut points need at most max_panels heat maps, "coarsened"; NA for a component of three or more predictors the first pruning dropped, so only without a second pruning) and heat_maps. The element display is a list: max_panels; first, the first pruning's model (without fit); coarse, per coarsened component (named as the components): S (all the component's predictors), C (the coarse predictors, ⁠|S| - 2⁠ of S: the panel predictors), q (the numbers of intervals aimed at; merged ties can leave fewer, lengths(cuts) + 1), cuts (per coarse predictor), cells (the number of heat maps, prod(lengths(cuts) + 1)), var_kept (the share of the component's variance kept), forest_panels, kept (in the final model), candidates (every candidate's coarse set, q, cells and variance kept), stored (what prediction needs; no training data) and seconds (the time to choose and build its form); asis, per component kept as it is (form "asis"), its panels with the same names and meaning: S (all the component's predictors), C (the panel predictors: the ⁠|S| - 2⁠ of S with the fewest own cut points; of ties, the first in S), cuts (per predictor of C, its own cut points: the leaf bounds of the trees containing S inside the range of the training values), forest_panels (prod(lengths(cuts) + 1)) and kept; forms, per component of three or more predictors the first pruning kept, its form, heat maps and variance kept; second (whether a second pruning ran); seconds (the time of each step); cache, an environment holding the coarsened components' search indices, built at their first predict() (a cache, not part of the model: rebuilt when missing or invalid, as after saving and loading). In coarse and asis alike, a panel is a cell of cuts (a value x of predictor k of C lies in cell findInterval(x, cuts[[k]])), shown as a heat map over the two predictors setdiff(S, C) at full resolution.

• store: the pieces of the kept components that are not coarsened, from which the model predicts (see Purification), and cache, an environment holding them built for prediction.

Use predict() and predict_components() on it. Without its forest, save it with saveRDS(); with it, with rpf_marshal() (which works either way).

See Also

predict.rpf_pruned(), predict_components(), purify(); rpf() with prune = TRUE fits and prunes in one call.

Examples

# A main effect and a pure interaction
set.seed(1)
x <- matrix(runif(600 * 4, -1, 1), ncol = 4, dimnames = list(NULL, paste0("x", 1:4)))
y <- x[, 1] + 2 * x[, 2] * x[, 3] + rnorm(600, sd = 0.3)
fit <- rpf(x, y, max_interaction = 2, n_families = 20, n_splits = 50, prune = FALSE)

# Rule and threshold chosen out of bag (no refits): the largest tau within one SE
# of the lowest error
pruned <- prune_components(fit, x, y, seed = 1)
pruned
pruned$oob

# ... or the lowest error
prune_components(fit, x, y, select = "min", seed = 1)

# ... or by 5-fold CV
pruned_cv <- prune_components(fit, x, y, select_by = "cv", seed = 1)
pruned_cv$cv

# A given rule and threshold: no choice; R's coefficients are still fitted out of bag
prune_components(fit, x, y, rule = "R", tau = 0.01)

predict(pruned, x[1:5, ])

# A three-way interaction, shown in at most 4 heat maps
y3 <- x[, 1] + 4 * x[, 2] * x[, 3] * x[, 4] + rnorm(600, sd = 0.3)
fit3 <- rpf(x, y3, max_interaction = 3, n_families = 20, n_splits = 100, prune = FALSE)
shown <- prune_components(fit3, x, y3, max_panels = 4, seed = 1)
shown
shown$display$forms

Description

Purification makes the components of the forest's functional decomposition unique, which predict_components() relies on. Unless rpf() was called with purify = TRUE, predict_components() purifies the forest on first use.

Usage

purify(x, ..., maxp_interaction = NULL, mode = 2L, nthreads = NULL)

is_purified(x)

Arguments

Details

The components are identified by the training data (the marginal identification of Hiabu, Meyer and Wright, 2023). With f the forest's prediction and ⁠X_1, ..., X_n⁠ the training rows, the component of a set S of predictors is

⁠m_S(x_S) = sum over the subsets V of S of (-1)^(|S| - |V|) * mean_i f(x_V, X_(i,-V))⁠,

where X_(i,-V) are training row i's values of the predictors outside V. The intercept, the component of the empty set, is the mean prediction on the training rows. The averages are over the joint distribution of the training rows, so with dependent predictors a component need not have mean zero along each of its predictors.

The forest is modified in place: x and every copy of it are purified, whether or not the result is assigned. If the forest is already purified with all the orders asked for, purify() returns it unmodified, whatever mode and nthreads are. A forest purified with a maxp_interaction below the order asked for (all orders with the default NULL) is purified again, which needs its training data: a forest restored by rpf_unmarshal() without it can only be purified up to its earlier cap.

Value

purify() returns x invisibly. is_purified() returns TRUE or FALSE.

References

Hiabu, M., Meyer, J. T., & Wright, M. N. (2023). Unifying local and global model explanations by functional decomposition of low dimensional structures. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 206, 7040–7060. https://proceedings.mlr.press/v206/hiabu23a.html

Hiabu, M., Mammen, E., & Meyer, J. T. (2020). Random Planted Forest: a directly interpretable tree ensemble. doi:10.48550/arXiv.2012.14563

Examples

rpfit <- rpf(mpg ~ ., data = mtcars, max_interaction = 2, n_families = 10, prune = FALSE)
is_purified(rpfit)
purify(rpfit)
is_purified(rpfit)

Description

Fits a random planted forest (Hiabu, Mammen and Meyer, 2020) for regression or classification. The model is a sum of trees that each split on at most max_interaction predictors, so it decomposes into main effects and interaction components; see predict_components().

Usage

rpf(x, ...)

## S3 method for class 'formula'
rpf(
  formula,
  data,
  max_interaction = 3,
  n_families = 200,
  n_splits = NULL,
  split_structure = "leaf",
  n_try_split_points = NULL,
  share_candidates = NULL,
  max_candidates = 50,
  split_decay_rate = 0.1,
  delete_leaves = TRUE,
  min_leaf_rows = 1,
  interaction_constraints = NULL,
  loss = NULL,
  exact_step = TRUE,
  prune = NULL,
  max_panels = 10,
  keep_forest = FALSE,
  tune = FALSE,
  purify = FALSE,
  nthreads = 1,
  export_forest = FALSE,
  deterministic = FALSE,
  keep_level_order = FALSE,
  oob_path = FALSE,
  ...
)

## S3 method for class 'data.frame'
rpf(
  x,
  y,
  max_interaction = 3,
  n_families = 200,
  n_splits = NULL,
  split_structure = "leaf",
  n_try_split_points = NULL,
  share_candidates = NULL,
  max_candidates = 50,
  split_decay_rate = 0.1,
  delete_leaves = TRUE,
  min_leaf_rows = 1,
  interaction_constraints = NULL,
  loss = NULL,
  exact_step = TRUE,
  prune = NULL,
  max_panels = 10,
  keep_forest = FALSE,
  tune = FALSE,
  purify = FALSE,
  nthreads = 1,
  export_forest = FALSE,
  deterministic = FALSE,
  keep_level_order = FALSE,
  oob_path = FALSE,
  ...
)

## S3 method for class 'matrix'
rpf(
  x,
  y,
  max_interaction = 3,
  n_families = 200,
  n_splits = NULL,
  split_structure = "leaf",
  n_try_split_points = NULL,
  share_candidates = NULL,
  max_candidates = 50,
  split_decay_rate = 0.1,
  delete_leaves = TRUE,
  min_leaf_rows = 1,
  interaction_constraints = NULL,
  loss = NULL,
  exact_step = TRUE,
  prune = NULL,
  max_panels = 10,
  keep_forest = FALSE,
  tune = FALSE,
  purify = FALSE,
  nthreads = 1,
  export_forest = FALSE,
  deterministic = FALSE,
  keep_level_order = FALSE,
  oob_path = FALSE,
  ...
)

## S3 method for class 'recipe'
rpf(
  x,
  data,
  max_interaction = 3,
  n_families = 200,
  n_splits = NULL,
  split_structure = "leaf",
  n_try_split_points = NULL,
  share_candidates = NULL,
  max_candidates = 50,
  split_decay_rate = 0.1,
  delete_leaves = TRUE,
  min_leaf_rows = 1,
  interaction_constraints = NULL,
  loss = NULL,
  exact_step = TRUE,
  prune = NULL,
  max_panels = 10,
  keep_forest = FALSE,
  tune = FALSE,
  purify = FALSE,
  nthreads = 1,
  export_forest = FALSE,
  deterministic = FALSE,
  keep_level_order = FALSE,
  oob_path = FALSE,
  ...
)

Arguments

Details

By default, rpf() returns the pruned model that can be shown, in one call:

1. it grows 200 tree families of up to three-way interactions, with the number of splits chosen out of bag while they grow (n_splits = NULL);

2. it prunes the forest's components by prune_components() with its defaults (prune = NULL): the sparse rule, chosen out of bag;

3. it coarsens every kept component of three or more predictors to at most max_panels = 10 heat maps and prunes again.

The result, of class rpf_pruned, has methods for predict(), predict_components() and print(). It holds the kept components, not the forest (keep_forest = TRUE keeps the forest too). prune = FALSE returns the forest (class rpf) instead, and ⁠n_splits = <number>⁠ fits a fixed number of splits.

Choosing parameters

The defaults were chosen for regression and carried over to classification.

max_interaction sets which effects the model can represent: 1 for an additive model, 2 to add pairwise interactions, 3 (the default) three-way interactions. Order 3 is often more accurate than order 2. Higher orders gain only where the truth has such interactions, cost much more time (no limit, 0, is often much slower), and the pruned model rarely keeps a component of more than three predictors, so it is shown in at most max_panels heat maps anyway. With a higher max_interaction, splits are spread over more possible components.

n_splits sets how closely the model follows the data. By default it is chosen out of bag while the forest grows (see "Choosing n_splits out of bag"), so it need not be tuned.

n_families averages over bootstrap samples to reduce variance, and fitting and prediction time grow linearly with it. Each tree family is itself a complete model, a sum of trees covering all components, rather than a single tree, but averaging more of them still pays off: more families usually lower the test error a little, and the pruned models are usually sparser. Hence the default of 200, at the price of time; use fewer for a quick look.

The split search parameters (split_structure, n_try_split_points, share_candidates, max_candidates, split_decay_rate) trade speed for a more thorough, more greedy search. For "leaf", a less greedy search is usually better, so its default is 3 split points per candidate and 10% of the possible splits per round, against 10 and 40% for the other structures. tune searches this greediness out of bag.

For classification, "logit" (the default) and "exponential" fit on the link scale and are transformed to probabilities; see "Classification losses". "L1" and "L2" fit class indicators directly and their probabilities are truncated to ⁠[0, 1]⁠.

Classification losses

"exponential" and "logit" fit scores on the link scale: with two classes one score, the log-odds of the second class; with more, one score per class, summing to 0 across the classes. The probabilities are the softmax of the scores. "exponential" with more than two classes is the multiclass exponential loss SAMME of Zhu, Zou, Rosset and Hastie (2009), whose row weight is exp(-g), g the score of its class. "logit" is the multinomial logit; it fits K - 1 scores against the first level (whose score is 0), which predict() (type = "link"), predict_components(), rpf_boxes() and the pruned model report with the first level's 0 joined and centred across the K classes, so both losses report the same layout. In exact arithmetic the fit and its pruned model do not depend on which level is first; in floating point, candidate splits that tie exactly (as duplicated bootstrap rows make) can be broken differently.

Every split moves the scores of the rows of each new leaf by one step, which minimises the leaf's penalised loss: the loss of its rows, plus that of one pseudo-row (split evenly over the classes) at the leaf's current mean score. The pseudo-row pulls the step towards no change, so a leaf whose rows all have one class moves towards a finite level (about ⁠log(2 n + 1)⁠ for n rows and two classes) and is not moved further once it is there, and no clamp of the class shares is needed. A split is scored by its gain over the parent leaf's own step, and each new leaf pays for its pseudo-row, so splitting a leaf of one class gains nothing, nor does repeating a split whose leaves are at their optimum, and such splits are not made. A round in which no candidate gains makes no split. The candidates' gains come from sums over their two sides (the second-order Newton gain for "logit", the exact gain for "exponential"), built as regression builds its sums (along the leaf's sorted rows, so that a split point costs the same whatever the leaf's size; for "tree_then_leaf" one pass over the leaf per split point). "exponential"'s step is exact; with exact_step = TRUE (the default) "logit"'s steps are made exact too, otherwise they are one Newton step.

"logit" is the default, with two classes and with more: its test log-loss is usually as low as that of "exponential" or lower, more so with three or more classes. "exponential" can need many more splits with many classes.

The level of the classification losses

"logit" and "exponential" average their tree families on the score scale. Where the families disagree, the probability of their mean score is not the mean of their probabilities: for a rare class (probability below 1/2, where the probability is convex in the score) it is lower, the more so the more splits the families make. So the forest would predict a rare class too rarely. After the fit the forest's score therefore gets one constant per score column, its level, which is part of its intercept: the maximum-likelihood intercept of the log-loss given the forest's out-of-bag scores, so that the mean out-of-bag probability of every class is its frequency in the training rows (corrected, as the out-of-bag criterion is, for an out-of-bag mean averaging fewer families than the forest). With deterministic = TRUE, which has no out-of-bag rows, the forest's own predictions of the training rows are balanced instead. The out-of-bag choice of n_splits scores every number of splits at its own level, and the pruning starts from the forest with its level. ⁠$level⁠ holds it (one value per score column) and print() shows it. options(randomPlantedForest.level = FALSE) fits without it. The setting is stored in the fit's ⁠$params$level⁠, and refits (the pruning's cross-validation) follow it; a fit without the setting is refitted without a level.

Factor predictors

Factor predictors, and character predictors, which are converted to factors, are coded as integers ⁠1, 2, ...⁠ in an order of their levels, and splits are thresholds on these codes. A split therefore separates the levels below a cut from those above it in that order.

By default, the levels are ordered by the outcome, so that the levels most alike in the outcome are neighbours (Wright and König, 2019):

• regression: by the mean outcome of each level;

• binary classification: by the share of the second outcome level;

• more than two classes: by the first principal component of the class proportions of each level (Coppersmith et al., 1999).

This applies to ordered factors too. With keep_level_order, the chosen factors are instead coded in the order of levels(), as given, ordered or not, so that every split separates neighbouring levels in that order.

The coding is stored in the fit as ⁠$factor_levels⁠, the levels of each factor in the order of their codes. predict(), predict_components() and purify() code new data by these levels, by name, so the order of the levels in new_data does not matter.

split_structure

In each round, a share_candidates share of all possible splits (capped at max_candidates) is drawn as candidates with weights exp(-split_decay_rate * age). split_structure defines what a candidate is and how its split points are tried. The names say the candidate, then how the leaf to split is found.

• "leaf" (was "leaves"): Split candidates are (leaf, split-dimension) pairs. For each sampled candidate, n_try_split_points split points are drawn uniformly without replacement from the distinct values of that predictor among the leaf's rows, leaving out the smallest and, with min_leaf_rows, those that leave fewer rows on either side (all of them if there are fewer), and evaluated to choose the best split.

• "leaf_binned" (was "hist"): As "leaf", but split points are drawn from boundaries of quantile bins computed once per predictor, which makes evaluating them faster on large data. With min_leaf_rows, only from the boundaries that leave that many rows on each side, and candidates without one are not drawn.

• "tree_then_leaf" (was "cur_trees_1"): Split candidates are (current-tree, split-dimension) pairs. For each sampled candidate, perform n_try_split_points evaluations. Each evaluation samples a leaf from the set of valid current trees (with probability proportional to its number of available split points) and then uniformly samples a single split point within that leaf. With min_leaf_rows, a leaf's available split points are those that leave that many rows on each side, so a leaf without one is not drawn.

• "tree_then_all_leaves" (was "cur_trees_2"): Split candidates are (current-tree, split-dimension) pairs. For each sampled candidate, iterate through every valid leaf. Within each leaf, sample n_try_split_points split points uniformly and evaluate them. With min_leaf_rows, only split points that leave that many rows on each side, and leaves without one are skipped.

• "target_tree_then_leaf" (was "res_trees"): Split candidates are resulting trees. For each sampled candidate, run n_try_split_points evaluations by sampling a (split-dimension, leaf) pair from all valid pairs (with probability proportional to its number of available split points), then uniformly sampling one split point within that pair. With min_leaf_rows, the available split points are those that leave that many rows on each side, as for "tree_then_leaf".

Renamed arguments

Four arguments and the split structures were renamed after version 0.5.0: ntrees is now n_families, splits is n_splits, split_try is n_try_split_points and t_try is share_candidates; the split structures are listed above. The prefix gives the unit: n_ a count, share_ a share in ⁠(0, 1]⁠, max_ a cap. The old names still work and give the same fit, with a deprecation warning once per session for each old name (as lifecycle's soft deprecation; options(lifecycle_verbosity = "warning") warns every time, "quiet" never); giving an argument under both names is an error. The fitted model's ⁠$params⁠ keeps the old names (ntrees, splits, split_try, t_try, and the old split_structure values), so code that reads them, and models saved by 0.5.0, keep working.

The draw of candidates for "leaf"

With split_structure = "leaf" (regression and classification), the global option randomPlantedForest.exact_draw chooses how the candidates are drawn. With FALSE (the default), the candidates are drawn from a Fenwick tree over the weights, kept up to date between rounds, which costs a few random numbers per candidate. With TRUE, the exact draw of earlier versions, every possible split gets a random number in every round. Both draw from the same distribution, but with other random numbers, so the same seed gives another (equally good) forest. Ties between equally good candidates may go to different candidates under the two draws, so with many splits, near saturation, their fits differ in distribution, though not in accuracy. With min_leaf_rows, both leave out the same candidates: those whose leaf has no threshold with that many rows on each side. The fast draw is faster on large data with many splits. The option is read when the forest is fitted and stored in params$exact_draw; it does not affect the other split structures. To reproduce a regression fit of version 0.5.0, set it and give 0.5.0's defaults, as the example "A regression forest of version 0.5.0" does:

options(randomPlantedForest.exact_draw = TRUE)

Classification fits of 0.5.0 cannot be reproduced: they changed for every split structure and loss (see NEWS: classification uses regression's split structures, the largest value of a predictor can be split off, and three or more classes with the exponential loss use SAMME).

Reproducibility

With the same seed, rpf() gives the same forest on the same platform, also with more threads. Another platform can give another forest for the same seed, because the integer sampling of the C++ standard library (std::uniform_int_distribution) differs between implementations, such as libc++ and libstdc++.

The out-of-bag path

With the same seed, the forest grown with k splits is exactly the first k splits of every tree family of a forest grown with more: nothing in the fit depends on the total number of splits. Each family is grown on a bootstrap sample, so every training row is out of bag for the families that did not draw it (about 37% of them). With oob_path, each family's prediction at its OOB rows is updated split by split while it grows, and copied at each checkpoint. So one fit gives the OOB error of every forest with fewer splits, for example to choose n_splits without cross-validation. This costs 2 * nrow(x) * length(checkpoints) doubles and a little time; the fit itself is unchanged.

The OOB prediction of a row is the mean over its OOB families, and the plain error (oob_rmse_raw) is the root mean squared error of these predictions over the rows with an OOB family. That mean averages fewer families than the forest, so the plain error overstates the forest's error by the families' extra variance, more so with more splits. The corrected error (oob_rmse) subtracts this excess, (1 / m_i - 1 / n_families) * v_i for row i with m_i OOB families, where v_i is the variance across families estimated from the OOB means over the families of even and of odd index, v_i = (g_even - g_odd)^2 / (1 / m_even + 1 / m_odd); this is also the correction of prune_components()' OOB choice. A row whose OOB families are all even or all odd is not corrected here (prune_components() gives it the mean estimate of the other rows): about 2% of the rows at 20 families, 18% at 10 and over half at 4 (none at 50), so with few families the corrected error is biased upwards, and above the corrected error prune_components() gives the same forest (rule "D" at tau = 0). Factor levels are ordered by the outcome of all training rows: by the mean outcome (regression), by the share of the second class (binary classification) or by the first principal component of the class proportions (three or more classes). So a row's own outcome enters the ordering its OOB families split on, and the OOB error is somewhat optimistic with factor predictors, most so with many levels.

Choosing n_splits out of bag

With n_splits = NULL (not deterministic), the number of splits is chosen while the forest grows, by the corrected OOB error of the out-of-bag path. The families grow to checkpoints, four per doubling of the splits (oob_path_checkpoints() up to 25,600), and the forest's corrected error is followed checkpoint by checkpoint. Growing stops once the lowest error is one doubling of the splits back (for example, no new minimum from 100 to 200 splits), or once it is on a plateau: the lowest error improved over the last doubling by at most getOption("randomPlantedForest.adaptive_tolerance", 1) standard errors of that improvement (paired over the training rows: the standard deviation of the rows' differences in corrected error between the two checkpoints, divided by the square root of their number). A tolerance of 0 stops only on the first rule. Growing also stops at 25,600 splits, with a warning if the error was still falling there. The result is the forest at the lowest error (ties go to fewer splits; a corrected error at or below 0, a correction that overshot, is not chosen unless all are): the forest that rpf() with that many n_splits and the same seed returns. The first rule alone usually chooses the same n_splits as the lowest error along a long fixed budget of splits, while growing fewer. The plateau stop usually chooses the same n_splits as the first rule alone, while growing fewer splits; it matters most on long, flat out-of-bag curves.

The families grow one at a time, nthreads together, each to the stop estimated from the families grown before it. A grown family keeps only its OOB predictions at the checkpoints and a record of its splits (40 bytes a split), from which the chosen forest is rebuilt, without a refit. A family that reaches the estimate waits there with its growing state while the waiting families fit in getOption("randomPlantedForest.adaptive_pause_gb", 0.25) GB, so that it can grow on cheaply if the stop comes later; otherwise it is freed, and grown again from its seed if needed. The option trades memory for time; the fit depends neither on it nor on nthreads. A smaller budget lowers the peak memory, while a budget of 0 can cost much more time (families grown again). Growing all families together would hold every family's growing state until the stop, which on large data needs several times the memory of growing them one at a time.

Pruning

By default (prune = NULL, or TRUE), rpf() fits the forest, then prunes its components with prune_components() at that function's defaults, on the training rows: the rule and the threshold are chosen out of bag (by cross-validation below 10 tree families, with a message), the sparse choice select = "1se", and every kept component of three or more predictors is shown in at most max_panels heat maps before the model is pruned again. The result is the pruned model, of class rpf_pruned, with methods for predict(), predict_components() and print(). It is the same, bit for bit, as prune_components(rpf(..., prune = FALSE), data) from the same random number state, for every interface. For other settings of the pruning (rule, tau, select, select_by, folds or seed), fit with prune = FALSE and call prune_components(). Together with n_splits = NULL, one call chooses the number of splits out of bag, prunes and returns the model to show. Pruning needs out-of-bag rows, so with deterministic = TRUE prune = NULL means FALSE and prune = TRUE is an error.

The pruned model does not keep the forest (keep_forest = FALSE): it predicts, prints and shows its components from what it stores, the same, bit for bit, as with the forest, and is a fraction of the forest's size. What needs the forest is then an error that says so: pruning again, purify(), the forest's own predict(), predict_components() and plot(). With keep_forest = TRUE the forest is in ⁠$fit⁠, so that prune_components(model$fit, data, ...) prunes it again.

Classification

A classification forest goes through the same steps (the OOB path, n_splits = NULL, prune, max_panels and tune) with its score columns: one for a binary outcome, one per class otherwise. Every out-of-bag choice is scored by the loss's own criterion, corrected for the OOB family count as above:

• loss = "L1" or "L2": the squared error of the raw scores summed over the columns (the Brier score, before the scores are truncated to probabilities). With two classes this is the regression criterion of the 0/1 outcome, and the choices are the regression ones, bit for bit.

• loss = "logit" or "exponential": the log-loss of the probabilities that predict(type = "prob") returns. The correction is its second-order form, ⁠(1 / m_i - 1 / n_families) * d' H d / (2 * (1 / m_even + 1 / m_odd))⁠ with d = g_even - g_odd and ⁠H = diag(p) - p p'⁠ over the score columns (for the squared error ⁠H = 2 I⁠, the correction above).

The pruning keeps or drops a component for all classes, sizes it by its variance summed over the score columns centred across the classes, and fits one coefficient per kept component shared by the classes (see prune_components()). The defaults were chosen for regression and carried over.

Tuning out of bag

With tune, rpf() searches the greediness of the split search (n_try_split_points and share_candidates, on the ladder of greediness_ladder()) and the minimum leaf size min_leaf_rows, without cross-validation. Every forest of the search is grown with n_splits chosen out of bag and pruned by prune_components() with its defaults (without the display), and forests are compared by the corrected out-of-bag RMSE of their pruned model (for classification by the root of its mean corrected criterion); the lowest wins. A pruned model at 0 (every candidate's correction overshot, see prune_components()) is not chosen unless all are. The search is coordinate-wise (tune_control()): with tune = TRUE the ladder from 10 / 0.4 to 1 / 0.0125 at min_leaf_rows = 1, then min_leaf_rows 1, 5, 20 and 100 at the chosen point, then the point's two ladder neighbours, about 10 forests in all. tune = "quick" grows two forests, 3 / 0.1 and 1 / 0.025 at min_leaf_rows = 1. The chosen forest's pruned model is returned, with the display at max_panels, and the search in its element tune. Every forest is grown from the same random number state, so the result is the model that rpf(n_splits = NULL, prune = TRUE) returns at the chosen settings from that state.

The full search often lowers the test error clearly against the greediest point, 10 / 0.4, and does as well as 5-fold cross-validation over the same settings, or better, at a small fraction of its cost; the quick preset gets most of the gain. A search costs about as many fits as it grows forests, and their prunings.

Value

With the defaults, an object of class rpf_pruned (see below). With prune = FALSE, an object of class "rpf" with model object contained in ⁠$fit⁠. With oob_path, ⁠$oob_path⁠ holds the OOB path: a data frame with one row per checkpoint, n_splits, the corrected and the plain OOB root mean squared error (oob_rmse, oob_rmse_raw), and the standard errors of the two mean squared errors (se_mse, se_mse_raw). Its attribute n_rows is the number of training rows with an OOB tree family. For classification with loss = "logit" or "exponential", or more than two classes, the columns are the mean corrected and plain criterion (oob_loss, oob_loss_raw) and their standard errors (se_loss, se_loss_raw), and the attribute criterion names it (see "Classification" in Details). With n_splits = NULL, ⁠$params$splits⁠ is the chosen number of splits (⁠$params⁠ keeps the names of 0.5.0, see "Renamed arguments" in Details), ⁠$oob_path⁠ the path of the checkpoints grown, and ⁠$splits_oob⁠ a list with the chosen n_splits, stopped_at (the number of splits where growing stopped) and capped (whether it stopped at the cap while the error was still falling, neither stop rule reached). When pruning (the default), an object of class rpf_pruned (see prune_components()), whose element fit describes the forest with the elements above: the pruned model's OOB path is ⁠$fit$oob_path⁠, and with n_splits = NULL the chosen number of splits ⁠$fit$params$splits⁠ and the stop ⁠$fit$splits_oob⁠. Without keep_forest, fit holds no forest (⁠$fit$fit⁠ is NULL); with keep_forest = TRUE it is the forest (not purified). With tune, the rpf_pruned model of the chosen forest, with an element tune: search, the search table (one row per forest grown, in the order grown: step, n_try_split_points, share_candidates, min_leaf_rows, the chosen n_splits, stopped_at, capped, the forest's corrected OOB RMSE oob_rmse_forest (as the pruning scores it: its rule "D" at tau = 0, which keeps every nonzero component; the OOB path's oob_rmse at the chosen n_splits leaves the rows without OOB families of both halves uncorrected and is higher), its pruned model's rule, tau, n_kept and corrected OOB RMSE oob_rmse (the score; for classification both are the square root of the mean corrected criterion), seconds, whether each step chose it (chosen_step1, chosen_step2, chosen_step3; NA if not a candidate of the step) and final); steps, each step's candidates and choice; settings, the chosen n_try_split_points, share_candidates, min_leaf_rows and n_splits; control, the tune_control() used; and seconds.

References

Hiabu, M., Mammen, E., & Meyer, J. T. (2020). Random Planted Forest: a directly interpretable tree ensemble. doi:10.48550/arXiv.2012.14563

Wright, M. N. & König, I. R. (2019). Splitting on categorical predictors in random forests. PeerJ 7:e6339. doi:10.7717/peerj.6339

Coppersmith, D., Hong, S. J. & Hosking, J. R. (1999). Partitioning nominal attributes in decision trees. Data Mining and Knowledge Discovery 3:197–217. doi:10.1023/A:1009869804967

See Also

predict() and print() of the pruned model, predict_components() for its components, prune_components() for other pruning rules, tune_control() for tune, and rpf_marshal() to save a model.

Examples

# The default: splits chosen out of bag, pruned, every component shown
set.seed(1)
model <- rpf(mpg ~ ., data = mtcars)
model
predict(model, mtcars[1:3, ])

# Classification, the same steps
set.seed(1)
model <- rpf(Species ~ ., data = iris, n_families = 50)
predict(model, iris[c(1, 51, 101), ], type = "prob")

# The forest instead of the pruned model, with x and y, pairwise interactions
# and a fixed number of splits
set.seed(1)
rpfit <- rpf(
  x = mtcars[, c("cyl", "wt", "hp")],
  y = mtcars$mpg,
  max_interaction = 2,
  n_splits = 30,
  prune = FALSE
)
rpfit

# A factor that keeps its level order: splits separate neighbouring levels
cars <- mtcars
cars$gear <- factor(cars$gear, levels = c(3, 4, 5))
rpfit <- rpf(mpg ~ gear + wt, data = cars, keep_level_order = "gear")
rpfit$factor_levels

# The out-of-bag error along the splits, from one fit
set.seed(1)
rpfit <- rpf(
  mpg ~ .,
  data = mtcars,
  n_families = 20,
  n_splits = 100,
  oob_path = TRUE,
  prune = FALSE
)
rpfit$oob_path[which.min(rpfit$oob_path$oob_rmse), ]


# A regression forest of version 0.5.0: its defaults and the exact draw
old <- options(randomPlantedForest.exact_draw = TRUE)
set.seed(1)
rpfit <- rpf(
  mpg ~ cyl + wt,
  data = mtcars,
  max_interaction = 2,
  n_families = 50,
  n_splits = 30,
  n_try_split_points = 10,
  share_candidates = 0.4,
  prune = FALSE
)
options(old)



# Tuning out of bag: the quick preset, two forests
set.seed(1)
x <- matrix(runif(400 * 4, -1, 1), ncol = 4, dimnames = list(NULL, paste0("x", 1:4)))
y <- x[, 1] + 2 * x[, 2] * x[, 3] + rnorm(400, sd = 0.3)
tuned <- rpf(x, y, n_families = 10, tune = "quick")
tuned$tune$search

Description

Every tree in a Random Planted Forest is a set of boxes (leaves) with a value each, and the prediction is the sum of the values of all boxes containing the point, averaged over tree families. rpf_boxes() returns the boxes of one tree family as a data frame.

Usage

rpf_boxes(x, family = 1L, ...)

Arguments

Details

Before purification, these are the leaves as grown: boxes in one tree may overlap, and the trees are not an identified decomposition. After purify(), they are the cells of the purified grid, one non-overlapping set per component of the functional decomposition. As purify() modifies the forest in place, extract the unpurified boxes before purifying. After purify(mode = 2), the default, the cells are computed from the training data, so a forest restored by rpf_unmarshal() needs to have been marshaled with include_data = TRUE.

Factor predictors are encoded as integer codes in the order of x$factor_levels, with a box covering the codes c with ⁠lower <= c < upper⁠.

Value

A base::data.frame with one row per box and columns

• tree: The tree, named by its sorted variables as in predict_components(), e.g. "x1:x2", or "(Intercept)".

• order: The number of variables of the tree, 0 for the intercept.

• box: The index of the box within its tree.

• value: The value of the box. For multiclass classification, one column ⁠value_<class>⁠ per outcome level instead (for loss = "logit" and "exponential", they sum to 0 across the levels, as the scores of predict() with type = "link" do).

• ⁠<variable>_lower⁠, ⁠<variable>_upper⁠ for every predictor: The bounds of the box, NA for variables the tree does not use.

See Also

plot.rpf() to plot the boxes.

Examples

rpfit <- rpf(mpg ~ wt + hp, data = mtcars, n_families = 5, n_splits = 10, prune = FALSE)
head(rpf_boxes(rpfit))

Description

The C++ forest does not survive saveRDS(); an rpf object, or an rpf_pruned one fitted with keep_forest = TRUE, restored via readRDS() without rpf_marshal()/rpf_unmarshal() is unusable, and predict() on it is an error that says so. A pruned model without its forest (keep_forest = FALSE, the default) has no C++ forest and is always valid; the fit of such a model is not (it has no forest to use).

Usage

rpf_is_valid(x)

Arguments

Value

TRUE if the underlying model can be used, FALSE otherwise (also for the fit of a pruned model without its forest).

See Also

rpf_marshal() to save a model so that it can be restored.

Examples

fit <- rpf(mpg ~ wt + cyl, data = mtcars, n_families = 10, prune = FALSE)
tmp <- tempfile(fileext = ".rds")

# saveRDS() alone loses the C++ forest
saveRDS(fit, tmp)
rpf_is_valid(readRDS(tmp))

# rpf_marshal() keeps it
saveRDS(rpf_marshal(fit), tmp)
rpf_is_valid(rpf_unmarshal(readRDS(tmp)))
unlink(tmp)

Description

rpf_marshal() converts an rpf object into a plain R list safe for saveRDS(); rpf_unmarshal() reverses it. The C++ forest behind an rpf object does not survive R serialization, so this explicit round-trip is required to store or transfer fitted models.

Usage

rpf_marshal(x, include_data = FALSE)

rpf_unmarshal(blob)

Arguments

Details

Training data is only included with include_data = TRUE; without it, a restored forest can predict (including purified prediction if the forest was purified before marshaling) but can never be purified afterwards. With it, the in-bag counts of the bootstrap samples are stored too, so that prune_components() can choose out of bag after restoring.

A pruned forest from prune_components() or rpf() (class rpf_pruned) is marshaled the same way: its forest as above, and the pruning (rule, coefficients, components, the table of the choice, the stored kept components, the display's coarsened components) as it is. The restored model predicts as before, from its stored components, so it needs no training data for that. A pruned model without its forest (the default of rpf() and prune_components(), keep_forest = FALSE) is plain R data: saveRDS() stores it as it is, and its blob is the model without the caches of its built indices (include_data has nothing to include).

Value

rpf_marshal() returns a list of class rpf_marshaled (rpf_pruned_marshaled for a pruned forest); rpf_unmarshal() returns an object of class rpf (rpf_pruned).

See Also

rpf_is_valid(); bundle() for the same through the bundle package.

Examples

fit <- rpf(mpg ~ wt + cyl, data = mtcars, n_families = 10, prune = FALSE)
blob <- rpf_marshal(fit)
tmp <- tempfile(fileext = ".rds")
saveRDS(blob, tmp)
restored <- rpf_unmarshal(readRDS(tmp))
all.equal(predict(fit, mtcars), predict(restored, mtcars))
unlink(tmp)

Description

What rpf(tune = ) searches. The search is coordinate-wise, so that it needs few forests (9 to 11 with the defaults):

1. the points search of ladder at the first min_leaf_rows;

2. every min_leaf_rows at the point chosen in step 1 (skipped if only one is given);

3. the two ladder neighbours of that point at the min_leaf_rows chosen in step 2 (if neighbours).

Usage

tune_control(
  ladder = greediness_ladder(),
  search = 3:8,
  min_leaf_rows = c(1, 5, 20, 100),
  neighbours = TRUE
)

Arguments

Details

Every forest has its number of splits chosen out of bag while it grows (as rpf(n_splits = NULL)) and is pruned by prune_components() with its defaults, without the display. A step chooses, among all forests grown at the coordinate it varies, the one whose pruned model has the lowest corrected out-of-bag RMSE (ties: the greedier point, the larger min_leaf_rows). Step 3 chooses among every forest grown at the chosen min_leaf_rows, so with min_leaf_rows 1 it also compares step 1's.

rpf(tune = TRUE) is tune_control(), and rpf(tune = "quick") is tune_control(search = c(5, 7), min_leaf_rows = 1, neighbours = FALSE): two forests, 3 / 0.1 and 1 / 0.025.

Value

An object of class rpf_tune_control, a list with the four settings, for rpf(tune = ).

See Also

greediness_ladder(), rpf()

Examples

# The full search and the quick preset
tune_control()
tune_control(search = c(5, 7), min_leaf_rows = 1, neighbours = FALSE)


# A small search: two points of greediness, then two leaf sizes
set.seed(1)
x <- matrix(runif(400 * 4, -1, 1), ncol = 4, dimnames = list(NULL, paste0("x", 1:4)))
y <- x[, 1] + 2 * x[, 2] * x[, 3] + rnorm(400, sd = 0.3)
tuned <- rpf(
  x,
  y,
  max_interaction = 2,
  n_families = 10,
  tune = tune_control(search = c(3, 5), min_leaf_rows = c(1, 20), neighbours = FALSE)
)
tuned$tune$search