Statistical issues in machine learning [Elektronische Ressource] : towards reliable split selection and variable importance measures / vorgelegt von: Carolin Strobl

Statistical Issues in Machine Learning –Towards Reliable Split Selection andVariable Importance MeasuresDissertationamInstitut fur¨ StatistikderFakult¨at fu¨r Mathematik, Informatik und StatistikderLudwig-Maximilians-Universit¨at Munc¨ henVorgelegt von: Carolin StroblMunc¨ hen, den 26. Mai 2008Erstgutachter: Prof. Dr. Thomas AugustinZweitgutachter: Prof. Dr. Gerhard TutzExterner Gutachter: Prof. Dr. Kurt HornikRigorosum: 2. Juli 2008AbstractRecursivepartitioningmethodsfrommachinelearningarebeingwidelyappliedinmanyscientificfieldssuchas, e.g., geneticsandbioinformatics. Thepresentworkisconcernedwiththetwomainproblems that arise in recursive partitioning, instability and biased variable selection, from astatistical point of view. With respect to the first issue, instability, the entire scope of methodsfrom standard classification trees over robustified classification trees and ensemble methods suchas TWIX, bagging and random forests is covered in this work. While ensemble methods prove tobe much more stable than single trees, they also loose most of their interpretability. Therefore anadaptive cutpoint selection scheme is suggested with which a TWIX ensemble reduces to a singletree if the partition is sufficiently stable. With respect to the second issue, variable selectionbias, the statistical sources of this artifact in single trees and a new form of bias inherent inensemble methods based on bootstrap samples are investigated.