|
Nalenz, Malte; Augustin, Thomas
(11. November 2021):
Cultivated Random Forests: Robust Decision Tree
Learning through Tree Structured Ensembles.
Department of Statistics: Technical Reports, No.240
|
![[img]](https://epub.ub.uni-muenchen.de/77861/1.hassmallThumbnailVersion/CRF_TR.pdf)  Preview |
|
463kB |
Abstract
We propose a robust decision tree induction method that mitigates the problems
of instability and poor generalization on unseen data. In the spirit of model
imprecision and robust statistics, we generalize decision trees by replacing internal
nodes with two types of ensemble modules that pool together a set of decisions into
a soft decision: (1) option modules consisting of all reasonable variable choices at
each step of the induction process, (2) robust split modules including all elements
of a neighbourhood of an optimal split-point as reasonable alternative split-points.
We call the resulting set of trees cultivated random forest as it corresponds to an
ensemble of trees which is centered around a single tree structure, alleviating the loss
of interpretability of traditional ensemble methods. The explicit modelling of nonprobabilistic uncertainty about the tree structure also provides an estimate of the
reliability of predictions, allowing to abstain from predictions when the uncertainty
is too high. On a variety of benchmark datasets, we show that our method is often
competitive with random forests, while being structurally substantially simpler and
easier to interpret.
