|
|
: Parsing option string: : ... "V:!Silent:Color:DrawProgressBar:AnalysisType=Classification" : The following options are set: : - By User: : V: "True" [Verbose flag] : Color: "True" [Flag for coloured screen output (default: True, if in batch mode: False)] : Silent: "False" [Batch mode: boolean silent flag inhibiting any output from TMVA after the creation of the factory class object (default: False)] : DrawProgressBar: "True" [Draw progress bar to display training, testing and evaluation schedule (default: True)] : AnalysisType: "Classification" [Set the analysis type (Classification, Regression, Multiclass, Auto) (default: Auto)] : - Default: : VerboseLevel: "Info" [VerboseLevel (Debug/Verbose/Info)] : Transformations: "I" [List of transformations to test; formatting example: "Transformations=I;D;P;U;G,D", for identity, decorrelation, PCA, Uniform and Gaussianisation followed by decorrelation transformations] : Correlations: "False" [boolean to show correlation in output] : ROC: "True" [boolean to show ROC in output] : ModelPersistence: "True" [Option to save the trained model in xml file or using serialization]DataSetInfo : [MatchNNDataSet] : Added class "Signal" : Add Tree Signal of type Signal with 7718 eventsDataSetInfo : [MatchNNDataSet] : Added class "Background" : Add Tree Bkg of type Background with 11895204 events : Dataset[MatchNNDataSet] : Class index : 0 name : Signal : Dataset[MatchNNDataSet] : Class index : 1 name : BackgroundFactory : Booking method: [1mmatching_mlp[0m : : Parsing option string: : ... "!H:V:TrainingMethod=BP:NeuronType=ReLU:EstimatorType=CE:VarTransform=Norm:NCycles=700:HiddenLayers=N+2,N:TestRate=50:Sampling=1.0:SamplingImportance=1.0:LearningRate=0.02:DecayRate=0.01:!UseRegulator" : The following options are set: : - By User: : <none> : - Default: : Boost_num: "0" [Number of times the classifier will be boosted] : Parsing option string: : ... "!H:V:TrainingMethod=BP:NeuronType=ReLU:EstimatorType=CE:VarTransform=Norm:NCycles=700:HiddenLayers=N+2,N:TestRate=50:Sampling=1.0:SamplingImportance=1.0:LearningRate=0.02:DecayRate=0.01:!UseRegulator" : The following options are set: : - By User: : NCycles: "700" [Number of training cycles] : HiddenLayers: "N+2,N" [Specification of hidden layer architecture] : NeuronType: "ReLU" [Neuron activation function type] : EstimatorType: "CE" [MSE (Mean Square Estimator) for Gaussian Likelihood or CE(Cross-Entropy) for Bernoulli Likelihood] : V: "True" [Verbose output (short form of "VerbosityLevel" below - overrides the latter one)] : VarTransform: "Norm" [List of variable transformations performed before training, e.g., "D_Background,P_Signal,G,N_AllClasses" for: "Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)"] : H: "False" [Print method-specific help message] : TrainingMethod: "BP" [Train with Back-Propagation (BP), BFGS Algorithm (BFGS), or Genetic Algorithm (GA - slower and worse)] : LearningRate: "2.000000e-02" [ANN learning rate parameter] : DecayRate: "1.000000e-02" [Decay rate for learning parameter] : TestRate: "50" [Test for overtraining performed at each #th epochs] : Sampling: "1.000000e+00" [Only 'Sampling' (randomly selected) events are trained each epoch] : SamplingImportance: "1.000000e+00" [ The sampling weights of events in epochs which successful (worse estimator than before) are multiplied with SamplingImportance, else they are divided.] : UseRegulator: "False" [Use regulator to avoid over-training] : - Default: : RandomSeed: "1" [Random seed for initial synapse weights (0 means unique seed for each run; default value '1')] : NeuronInputType: "sum" [Neuron input function type] : VerbosityLevel: "Default" [Verbosity level] : CreateMVAPdfs: "False" [Create PDFs for classifier outputs (signal and background)] : IgnoreNegWeightsInTraining: "False" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)] : EpochMonitoring: "False" [Provide epoch-wise monitoring plots according to TestRate (caution: causes big ROOT output file!)] : SamplingEpoch: "1.000000e+00" [Sampling is used for the first 'SamplingEpoch' epochs, afterwards, all events are taken for training] : SamplingTraining: "True" [The training sample is sampled] : SamplingTesting: "False" [The testing sample is sampled] : ResetStep: "50" [How often BFGS should reset history] : Tau: "3.000000e+00" [LineSearch "size step"] : BPMode: "sequential" [Back-propagation learning mode: sequential or batch] : BatchSize: "-1" [Batch size: number of events/batch, only set if in Batch Mode, -1 for BatchSize=number_of_events] : ConvergenceImprove: "1.000000e-30" [Minimum improvement which counts as improvement (<0 means automatic convergence check is turned off)] : ConvergenceTests: "-1" [Number of steps (without improvement) required for convergence (<0 means automatic convergence check is turned off)] : UpdateLimit: "10000" [Maximum times of regulator update] : CalculateErrors: "False" [Calculates inverse Hessian matrix at the end of the training to be able to calculate the uncertainties of an MVA value] : WeightRange: "1.000000e+00" [Take the events for the estimator calculations from small deviations from the desired value to large deviations only over the weight range]matching_mlp : [MatchNNDataSet] : Create Transformation "Norm" with events from all classes. : : Transformation, Variable selection : : Input : variable 'chi2' <---> Output : variable 'chi2' : Input : variable 'teta2' <---> Output : variable 'teta2' : Input : variable 'distX' <---> Output : variable 'distX' : Input : variable 'distY' <---> Output : variable 'distY' : Input : variable 'dSlope' <---> Output : variable 'dSlope' : Input : variable 'dSlopeY' <---> Output : variable 'dSlopeY'matching_mlp : Building Network. : Initializing weightsFactory : [1mTrain all methods[0m : Rebuilding Dataset MatchNNDataSet : Parsing option string: : ... "SplitMode=random:V:nTrain_Signal=0:nTrain_Background=20000.0:nTest_Signal=1000.0:nTest_Background=5000.0" : The following options are set: : - By User: : SplitMode: "Random" [Method of picking training and testing events (default: random)] : nTrain_Signal: "0" [Number of training events of class Signal (default: 0 = all)] : nTest_Signal: "1000" [Number of test events of class Signal (default: 0 = all)] : nTrain_Background: "20000" [Number of training events of class Background (default: 0 = all)] : nTest_Background: "5000" [Number of test events of class Background (default: 0 = all)] : V: "True" [Verbosity (default: true)] : - Default: : MixMode: "SameAsSplitMode" [Method of mixing events of different classes into one dataset (default: SameAsSplitMode)] : SplitSeed: "100" [Seed for random event shuffling] : NormMode: "EqualNumEvents" [Overall renormalisation of event-by-event weights used in the training (NumEvents: average weight of 1 per event, independently for signal and background; EqualNumEvents: average weight of 1 per event for signal, and sum of weights for background equal to sum of weights for signal)] : ScaleWithPreselEff: "False" [Scale the number of requested events by the eff. of the preselection cuts (or not)] : TrainTestSplit_Signal: "0.000000e+00" [Number of test events of class Signal (default: 0 = all)] : TrainTestSplit_Background: "0.000000e+00" [Number of test events of class Background (default: 0 = all)] : VerboseLevel: "Info" [VerboseLevel (Debug/Verbose/Info)] : Correlations: "True" [Boolean to show correlation output (Default: true)] : CalcCorrelations: "True" [Compute correlations and also some variable statistics, e.g. min/max (Default: true )] : Building event vectors for type 2 Signal : Dataset[MatchNNDataSet] : create input formulas for tree Signal : Building event vectors for type 2 Background : Dataset[MatchNNDataSet] : create input formulas for tree BkgDataSetFactory : [MatchNNDataSet] : Number of events in input trees : : : Dataset[MatchNNDataSet] : Weight renormalisation mode: "EqualNumEvents": renormalises all event classes ... : Dataset[MatchNNDataSet] : such that the effective (weighted) number of events in each class is the same : Dataset[MatchNNDataSet] : (and equals the number of events (entries) given for class=0 ) : Dataset[MatchNNDataSet] : ... i.e. such that Sum[i=1..N_j]{w_i} = N_classA, j=classA, classB, ... : Dataset[MatchNNDataSet] : ... (note that N_j is the sum of TRAINING events : Dataset[MatchNNDataSet] : ..... Testing events are not renormalised nor included in the renormalisation factor!) : Number of training and testing events : --------------------------------------------------------------------------- : Signal -- training events : 6718 : Signal -- testing events : 1000 : Signal -- training and testing events: 7718 : Background -- training events : 20000 : Background -- testing events : 5000 : Background -- training and testing events: 25000 : DataSetInfo : Correlation matrix (Signal): : -------------------------------------------------------- : chi2 teta2 distX distY dSlope dSlopeY : chi2: +1.000 -0.083 +0.248 +0.242 +0.206 +0.042 : teta2: -0.083 +1.000 +0.038 +0.508 +0.191 +0.637 : distX: +0.248 +0.038 +1.000 -0.175 +0.681 +0.107 : distY: +0.242 +0.508 -0.175 +1.000 +0.349 +0.484 : dSlope: +0.206 +0.191 +0.681 +0.349 +1.000 +0.349 : dSlopeY: +0.042 +0.637 +0.107 +0.484 +0.349 +1.000 : --------------------------------------------------------DataSetInfo : Correlation matrix (Background): : -------------------------------------------------------- : chi2 teta2 distX distY dSlope dSlopeY : chi2: +1.000 -0.024 +0.242 +0.209 +0.046 +0.055 : teta2: -0.024 +1.000 +0.245 +0.652 +0.371 +0.483 : distX: +0.242 +0.245 +1.000 +0.017 +0.776 +0.198 : distY: +0.209 +0.652 +0.017 +1.000 +0.312 +0.554 : dSlope: +0.046 +0.371 +0.776 +0.312 +1.000 +0.392 : dSlopeY: +0.055 +0.483 +0.198 +0.554 +0.392 +1.000 : --------------------------------------------------------DataSetFactory : [MatchNNDataSet] : : Factory : [MatchNNDataSet] : Create Transformation "I" with events from all classes. : : Transformation, Variable selection : : Input : variable 'chi2' <---> Output : variable 'chi2' : Input : variable 'teta2' <---> Output : variable 'teta2' : Input : variable 'distX' <---> Output : variable 'distX' : Input : variable 'distY' <---> Output : variable 'distY' : Input : variable 'dSlope' <---> Output : variable 'dSlope' : Input : variable 'dSlopeY' <---> Output : variable 'dSlopeY'TFHandler_Factory : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : chi2: 14.879 7.6783 [ 0.35410 29.998 ] : teta2: 0.0053594 0.015677 [ 5.5206e-06 0.34331 ] : distX: 74.975 63.347 [ 0.00024414 487.68 ] : distY: 35.490 43.750 [ 8.3923e-05 497.42 ] : dSlope: 0.35788 0.24459 [ 6.4602e-05 1.2881 ] : dSlopeY: 0.0073112 0.012369 [ 3.9814e-08 0.14883 ] : ----------------------------------------------------------- : Ranking input variables (method unspecific)...IdTransformation : Ranking result (top variable is best ranked) : -------------------------------- : Rank : Variable : Separation : -------------------------------- : 1 : chi2 : 9.921e-02 : 2 : distY : 8.773e-02 : 3 : dSlopeY : 2.784e-02 : 4 : teta2 : 2.748e-02 : 5 : dSlope : 2.662e-02 : 6 : distX : 1.420e-02 : --------------------------------Factory : Train method: matching_mlp for Classification : TFHandler_matching_mlp : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : chi2: -0.020078 0.51803 [ -1.0000 1.0000 ] : teta2: -0.96881 0.091329 [ -1.0000 1.0000 ] : distX: -0.69253 0.25979 [ -1.0000 1.0000 ] : distY: -0.85730 0.17591 [ -1.0000 1.0000 ] : dSlope: -0.44439 0.37979 [ -1.0000 1.0000 ] : dSlopeY: -0.90175 0.16622 [ -1.0000 1.0000 ] : ----------------------------------------------------------- : Training Network : : Elapsed time for training with 26718 events: [1;31m57.7 sec[0m matching_mlp : [MatchNNDataSet] : Evaluation of matching_mlp on training sample (26718 events) : Elapsed time for evaluation of 26718 events: [1;31m0.0346 sec[0m : Creating xml weight file: [0;36mMatchNNDataSet/weights/TMVAClassification_matching_mlp.weights.xml[0m : Creating standalone class: [0;36mMatchNNDataSet/weights/TMVAClassification_matching_mlp.class.C[0m : Write special histos to file: matching_ghost_mlp_training.root:/MatchNNDataSet/Method_MLP/matching_mlpFactory : Training finished : : Ranking input variables (method specific)...matching_mlp : Ranking result (top variable is best ranked) : -------------------------------- : Rank : Variable : Importance : -------------------------------- : 1 : distY : 1.467e+02 : 2 : teta2 : 6.884e+01 : 3 : distX : 6.627e+01 : 4 : dSlopeY : 3.066e+01 : 5 : dSlope : 1.175e+01 : 6 : chi2 : 2.632e+00 : --------------------------------Factory : === Destroy and recreate all methods via weight files for testing === : : Reading weight file: [0;36mMatchNNDataSet/weights/TMVAClassification_matching_mlp.weights.xml[0mmatching_mlp : Building Network. : Initializing weightsFactory : [1mTest all methods[0mFactory : Test method: matching_mlp for Classification performance : matching_mlp : [MatchNNDataSet] : Evaluation of matching_mlp on testing sample (6000 events) : Elapsed time for evaluation of 6000 events: [1;31m0.0118 sec[0m Factory : [1mEvaluate all methods[0mFactory : Evaluate classifier: matching_mlp : TFHandler_matching_mlp : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : chi2: 0.10881 0.51711 [ -1.0020 0.99902 ] : teta2: -0.96093 0.10865 [ -0.99988 0.46950 ] : distX: -0.67673 0.27337 [ -0.99968 0.75285 ] : distY: -0.82663 0.20236 [ -0.99997 0.83868 ] : dSlope: -0.46394 0.38477 [ -0.99839 0.97924 ] : dSlopeY: -0.89235 0.16561 [ -1.0000 0.93883 ] : -----------------------------------------------------------matching_mlp : [MatchNNDataSet] : Loop over test events and fill histograms with classifier response... : TFHandler_matching_mlp : Variable Mean RMS [ Min Max ] : ----------------------------------------------------------- : chi2: 0.10881 0.51711 [ -1.0020 0.99902 ] : teta2: -0.96093 0.10865 [ -0.99988 0.46950 ] : distX: -0.67673 0.27337 [ -0.99968 0.75285 ] : distY: -0.82663 0.20236 [ -0.99997 0.83868 ] : dSlope: -0.46394 0.38477 [ -0.99839 0.97924 ] : dSlopeY: -0.89235 0.16561 [ -1.0000 0.93883 ] : ----------------------------------------------------------- : : Evaluation results ranked by best signal efficiency and purity (area) : ------------------------------------------------------------------------------------------------------------------- : DataSet MVA : Name: Method: ROC-integ : MatchNNDataSet matching_mlp : 0.842 : ------------------------------------------------------------------------------------------------------------------- : : Testing efficiency compared to training efficiency (overtraining check) : ------------------------------------------------------------------------------------------------------------------- : DataSet MVA Signal efficiency: from test sample (from training sample) : Name: Method: @B=0.01 @B=0.10 @B=0.30 : ------------------------------------------------------------------------------------------------------------------- : MatchNNDataSet matching_mlp : 0.075 (0.082) 0.476 (0.467) 0.841 (0.828) : ------------------------------------------------------------------------------------------------------------------- : Dataset:MatchNNDataSet : Created tree 'TestTree' with 6000 events : Dataset:MatchNNDataSet : Created tree 'TrainTree' with 26718 events : Factory : [1mThank you for using TMVA
