### TMVA

TMVA version 4.0.1 is included in this root release:

#### Main changes and new features introduced with TMVA 4

- Reorganisation of internal data handling and constructors of methods to allow to build arbitrary composite MVA methods, and to deal with multi-class classification and multi-target regression
- Extended TMVA to multivariate multi-target regression
- Any TMVA method can now be boosted (linearly or non-linearly)
- Transformation of input variables can be chained as wished
- Weight files are now in XML format
- New MVA methods "PDE-Foam" and "LD", both featuring classification and regression

##### Comments

On XML format:
The old text format is obsolete though still readable in the application. Backward compatibility is NOT guaranteed. Please contact the authors if you require the reading of old text weight files in TMVA 4.

Standard macros:
The structure of the standard macros has changed: macros are still in the "$ROOTSYS/tmva/test" directory, but distinguished for classification and regression examples:

    TMVAClassification.C, TMVAClassificationApplication.C TMVARegression.C, TMVARegressionApplication.C

Classification and regression analysis (training) is analysed as usual via standard macros that can be called from dedicated GUIs.

Regression:

- Not yet available for all MVA methods. It exists for: PDE-RS, PDE-Foam, K-NN, LD, FDA, MLP, BDT for single targets (1D), and MLP for multiple targets (nD).
- Not all transformation of input variables are available (only "Norm" so far).
- Regression requires specific evaluation tools:
  - During the training we provide a ranking of input variables, using various criteria: correlations, transposed correlation, correlation ratio, and "mutual information" between input variables and regression target. (Correlation ratio and mutual information implmentations provided by Moritz Backes, Geneva U)
  - After the training, the trained MVA methods are ranked wrt. the deviations between regression target and estimate.
  - Macros plot various deviation and correlation quantities. A new GUI (macros/TMVARegGui.C) collects these macros.

#### Improvements of / new features for MVA methods

- Linear Discriminant: Re-implementation of "Fisher" method as general linear discriminant ("LD"), which is also regression capable (so far: single-target only)
- PDEFoam: PDE-Foam is a variation of the PDE-RS method using a self-adapting binning method to divide the multi-dimensional variable space into a finite number of hyper-rectangles (cells). The binning algorithm adjusts the size and position of a predefined number of cells such that the variance of the signal and background densities inside the cells reaches a minimum.
- BDT: Introduced gradient boosting and stochastic gradient boosting for classification with BDT (as desribed by Friedman 1999). See "BDTG" example in TMVAClassification.C/cxx. A new option allows to restrict the maximum tree depth. This may be used to avoid overtraining and often gives better performance than pruning. (The pruning mechanism needs to be revisited)
- MLP: Introduced recognition of convergence via general ConvergenceTest-class for interrupting computations when convergence is reached. This feature has is used now in MethodMLP. Improved treatment of event-weights in BFGS training. Implemented random and importance sampling of events in DataSet. Implemented the usage of this feature for MLP.
- TMlpANN (interface to TMultiLayerPerceptron) now also uses event weights and writes standalone C++ class.
- k-NN: A new global knn search function has been added to NodekNN that searches for k-nearest neighbor using event weights instead of raw event counts. ModulekNN has been modified to allow searches using "weight" or "count" option, where "count" is default. Added UseWeight option to MethodKNN to allow using of "weight" or "count". (Work by Rustem Ospanov, CERN).
- Likelihood (and general PDF treatment): Adaptive smoothing the PDF class, allowing it to smooth between MinSmoothNum (for regions with more signal) and MaxSmoothNum (for regions with less signal). Configuration of the PDF parameters from the option string moved to PDF class, allowing the user to define all the PDF functionalities in every classifier the PDF is used (i.e., also for the MVA PDFs). The reading of these variables was removed from MethodBase and MethodLikelihood. This also allows improved (full) PDF configuration of MVA output via the "CreateMvaPdf" option. (Work by Or Cohen, CERN & Weizmann)
- New generalisation methods:
  - MethodCompositeBase: combines more than one classifier within one.
  - MethodBoost: boosts/bags any classifier type. A special booking procedure for it was added to Factory class.
  - MethodDT: a classifier composed of a single decision tree, boosted using MethodBoost. Results are compatible with BDT, but BDT remains the default for boosted decision trees, because it has pruning (among other additional features).

##### Other improvements

- Improved handling of small Likelihood values such that Likelihood performance increases in analyses with many variables (~>10). Thanks to Ralph Schaefer (Bonn U.) for reporting this.
- Nicer plotting: custom variable titles and units can be assigned in "AddVariable" call.
- Introduced the inverse transformation InverseTransform for the variable transformations into the framework. While this is not necessary for classification, it is necessary for regression. The inverse transformation of the normalization transformation has been implemented.
- Started to extend the variable transformations to the regression targets as well.
- MethodCuts now produces the 'optimal-cut' histograms needed by macro mvaeffs.C. (macro 5a of TMVAGui.C)
- MsgLogger can be silenced in order to prevent excess output during boosting.
- Third dataset type added centrally (Training, Validation and Testing). The validation data is split off the original training data set.
- Update of GUI and other Macros according to the new features of PDF and the addition of MethodBoost.

#### Updates in TMVA 4.0.1

- "Spectator" variables can be defined now which are computed just as the input variables and which are written out into the TestTree, but which don't participate in any MVA calculation (useful for correlation studies).
- New booking option "IgnoreNegWeightsInTraining" to test the effect of events with negative weights on the training. This is especially useful for methods, which do not properly deal with such events. Note that this new option is not available for all methods (a training interrupt is issued if not available).

##### Bug fixes:

- Fixed regression bug in VariableNormalizeTransform (Use number of targets from Event instead of DataSet)
- Fixed Multitarget-Regression in PDEFoam, foam dimensions were miscalculated
- Added writing of targets to the weight files in regression mode to fix problems in RegressionApplication
- Added missing standard C++ header files missing to some classes, which lead to compilation failures on some architectures (thanks to Lucian Ancu, Nijmegen, for reporting these).
- Added checks for unused options to Factory and DataSetFactory configuration options interpretation. Will now complain if wrong option labels are used.
- Fixed standard creation of correlation matrix plots
- Fixed internal mapping problem giving a fatal error message when destroying and recreating the Factory.