You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
280 lines
24 KiB
280 lines
24 KiB
: Parsing option string:
|
|
: ... "V:!Silent:Color:DrawProgressBar:AnalysisType=Classification"
|
|
: The following options are set:
|
|
: - By User:
|
|
: V: "True" [Verbose flag]
|
|
: Color: "True" [Flag for coloured screen output (default: True, if in batch mode: False)]
|
|
: Silent: "False" [Batch mode: boolean silent flag inhibiting any output from TMVA after the creation of the factory class object (default: False)]
|
|
: DrawProgressBar: "True" [Draw progress bar to display training, testing and evaluation schedule (default: True)]
|
|
: AnalysisType: "Classification" [Set the analysis type (Classification, Regression, Multiclass, Auto) (default: Auto)]
|
|
: - Default:
|
|
: VerboseLevel: "Info" [VerboseLevel (Debug/Verbose/Info)]
|
|
: Transformations: "I" [List of transformations to test; formatting example: "Transformations=I;D;P;U;G,D", for identity, decorrelation, PCA, Uniform and Gaussianisation followed by decorrelation transformations]
|
|
: Correlations: "False" [boolean to show correlation in output]
|
|
: ROC: "True" [boolean to show ROC in output]
|
|
: ModelPersistence: "True" [Option to save the trained model in xml file or using serialization]
|
|
DataSetInfo : [MatchNNDataSet] : Added class "Signal"
|
|
: Add Tree Signal of type Signal with 2175608 events
|
|
DataSetInfo : [MatchNNDataSet] : Added class "Background"
|
|
: Add Tree Bkg of type Background with 14040318 events
|
|
: Dataset[MatchNNDataSet] : Class index : 0 name : Signal
|
|
: Dataset[MatchNNDataSet] : Class index : 1 name : Background
|
|
Factory : Booking method: [1mmatching_mlp[0m
|
|
:
|
|
: Parsing option string:
|
|
: ... "!H:V:TrainingMethod=BP:NeuronType=ReLU:EstimatorType=CE:VarTransform=Norm:NCycles=700:HiddenLayers=N+2,N:TestRate=50:Sampling=1.0:SamplingImportance=1.0:LearningRate=0.02:DecayRate=0.01:!UseRegulator"
|
|
: The following options are set:
|
|
: - By User:
|
|
: <none>
|
|
: - Default:
|
|
: Boost_num: "0" [Number of times the classifier will be boosted]
|
|
: Parsing option string:
|
|
: ... "!H:V:TrainingMethod=BP:NeuronType=ReLU:EstimatorType=CE:VarTransform=Norm:NCycles=700:HiddenLayers=N+2,N:TestRate=50:Sampling=1.0:SamplingImportance=1.0:LearningRate=0.02:DecayRate=0.01:!UseRegulator"
|
|
: The following options are set:
|
|
: - By User:
|
|
: NCycles: "700" [Number of training cycles]
|
|
: HiddenLayers: "N+2,N" [Specification of hidden layer architecture]
|
|
: NeuronType: "ReLU" [Neuron activation function type]
|
|
: EstimatorType: "CE" [MSE (Mean Square Estimator) for Gaussian Likelihood or CE(Cross-Entropy) for Bernoulli Likelihood]
|
|
: V: "True" [Verbose output (short form of "VerbosityLevel" below - overrides the latter one)]
|
|
: VarTransform: "Norm" [List of variable transformations performed before training, e.g., "D_Background,P_Signal,G,N_AllClasses" for: "Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)"]
|
|
: H: "False" [Print method-specific help message]
|
|
: TrainingMethod: "BP" [Train with Back-Propagation (BP), BFGS Algorithm (BFGS), or Genetic Algorithm (GA - slower and worse)]
|
|
: LearningRate: "2.000000e-02" [ANN learning rate parameter]
|
|
: DecayRate: "1.000000e-02" [Decay rate for learning parameter]
|
|
: TestRate: "50" [Test for overtraining performed at each #th epochs]
|
|
: Sampling: "1.000000e+00" [Only 'Sampling' (randomly selected) events are trained each epoch]
|
|
: SamplingImportance: "1.000000e+00" [ The sampling weights of events in epochs which successful (worse estimator than before) are multiplied with SamplingImportance, else they are divided.]
|
|
: UseRegulator: "False" [Use regulator to avoid over-training]
|
|
: - Default:
|
|
: RandomSeed: "1" [Random seed for initial synapse weights (0 means unique seed for each run; default value '1')]
|
|
: NeuronInputType: "sum" [Neuron input function type]
|
|
: VerbosityLevel: "Default" [Verbosity level]
|
|
: CreateMVAPdfs: "False" [Create PDFs for classifier outputs (signal and background)]
|
|
: IgnoreNegWeightsInTraining: "False" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)]
|
|
: EpochMonitoring: "False" [Provide epoch-wise monitoring plots according to TestRate (caution: causes big ROOT output file!)]
|
|
: SamplingEpoch: "1.000000e+00" [Sampling is used for the first 'SamplingEpoch' epochs, afterwards, all events are taken for training]
|
|
: SamplingTraining: "True" [The training sample is sampled]
|
|
: SamplingTesting: "False" [The testing sample is sampled]
|
|
: ResetStep: "50" [How often BFGS should reset history]
|
|
: Tau: "3.000000e+00" [LineSearch "size step"]
|
|
: BPMode: "sequential" [Back-propagation learning mode: sequential or batch]
|
|
: BatchSize: "-1" [Batch size: number of events/batch, only set if in Batch Mode, -1 for BatchSize=number_of_events]
|
|
: ConvergenceImprove: "1.000000e-30" [Minimum improvement which counts as improvement (<0 means automatic convergence check is turned off)]
|
|
: ConvergenceTests: "-1" [Number of steps (without improvement) required for convergence (<0 means automatic convergence check is turned off)]
|
|
: UpdateLimit: "10000" [Maximum times of regulator update]
|
|
: CalculateErrors: "False" [Calculates inverse Hessian matrix at the end of the training to be able to calculate the uncertainties of an MVA value]
|
|
: WeightRange: "1.000000e+00" [Take the events for the estimator calculations from small deviations from the desired value to large deviations only over the weight range]
|
|
matching_mlp : [MatchNNDataSet] : Create Transformation "Norm" with events from all classes.
|
|
:
|
|
: Transformation, Variable selection :
|
|
: Input : variable 'chi2' <---> Output : variable 'chi2'
|
|
: Input : variable 'teta2' <---> Output : variable 'teta2'
|
|
: Input : variable 'distX' <---> Output : variable 'distX'
|
|
: Input : variable 'distY' <---> Output : variable 'distY'
|
|
: Input : variable 'dSlope' <---> Output : variable 'dSlope'
|
|
: Input : variable 'dSlopeY' <---> Output : variable 'dSlopeY'
|
|
matching_mlp : Building Network.
|
|
: Initializing weights
|
|
Factory : [1mTrain all methods[0m
|
|
: Rebuilding Dataset MatchNNDataSet
|
|
: Parsing option string:
|
|
: ... "SplitMode=random:V:nTrain_Signal=50000.0:nTrain_Background=500000.0:nTest_Signal=20000.0:nTest_Background=100000.0"
|
|
: The following options are set:
|
|
: - By User:
|
|
: SplitMode: "Random" [Method of picking training and testing events (default: random)]
|
|
: nTrain_Signal: "50000" [Number of training events of class Signal (default: 0 = all)]
|
|
: nTest_Signal: "20000" [Number of test events of class Signal (default: 0 = all)]
|
|
: nTrain_Background: "500000" [Number of training events of class Background (default: 0 = all)]
|
|
: nTest_Background: "100000" [Number of test events of class Background (default: 0 = all)]
|
|
: V: "True" [Verbosity (default: true)]
|
|
: - Default:
|
|
: MixMode: "SameAsSplitMode" [Method of mixing events of different classes into one dataset (default: SameAsSplitMode)]
|
|
: SplitSeed: "100" [Seed for random event shuffling]
|
|
: NormMode: "EqualNumEvents" [Overall renormalisation of event-by-event weights used in the training (NumEvents: average weight of 1 per event, independently for signal and background; EqualNumEvents: average weight of 1 per event for signal, and sum of weights for background equal to sum of weights for signal)]
|
|
: ScaleWithPreselEff: "False" [Scale the number of requested events by the eff. of the preselection cuts (or not)]
|
|
: TrainTestSplit_Signal: "0.000000e+00" [Number of test events of class Signal (default: 0 = all)]
|
|
: TrainTestSplit_Background: "0.000000e+00" [Number of test events of class Background (default: 0 = all)]
|
|
: VerboseLevel: "Info" [VerboseLevel (Debug/Verbose/Info)]
|
|
: Correlations: "True" [Boolean to show correlation output (Default: true)]
|
|
: CalcCorrelations: "True" [Compute correlations and also some variable statistics, e.g. min/max (Default: true )]
|
|
: Building event vectors for type 2 Signal
|
|
: Dataset[MatchNNDataSet] : create input formulas for tree Signal
|
|
: Building event vectors for type 2 Background
|
|
: Dataset[MatchNNDataSet] : create input formulas for tree Bkg
|
|
DataSetFactory : [MatchNNDataSet] : Number of events in input trees
|
|
: Dataset[MatchNNDataSet] : Signal requirement: "chi2<15 && distX<250 && distY<400 && dSlope<1.5 && dSlopeY<0.15"
|
|
: Dataset[MatchNNDataSet] : Signal -- number of events passed: 2151182 / sum of weights: 2.15118e+06
|
|
: Dataset[MatchNNDataSet] : Signal -- efficiency : 0.988773
|
|
: Dataset[MatchNNDataSet] : Background requirement: "chi2<15 && distX<250 && distY<400 && dSlope<1.5 && dSlopeY<0.15"
|
|
: Dataset[MatchNNDataSet] : Background -- number of events passed: 7175761 / sum of weights: 7.17576e+06
|
|
: Dataset[MatchNNDataSet] : Background -- efficiency : 0.511083
|
|
: Dataset[MatchNNDataSet] : you have opted for interpreting the requested number of training/testing events
|
|
: to be the number of events AFTER your preselection cuts
|
|
:
|
|
: Dataset[MatchNNDataSet] : you have opted for interpreting the requested number of training/testing events
|
|
: to be the number of events AFTER your preselection cuts
|
|
:
|
|
: Dataset[MatchNNDataSet] : Weight renormalisation mode: "EqualNumEvents": renormalises all event classes ...
|
|
: Dataset[MatchNNDataSet] : such that the effective (weighted) number of events in each class is the same
|
|
: Dataset[MatchNNDataSet] : (and equals the number of events (entries) given for class=0 )
|
|
: Dataset[MatchNNDataSet] : ... i.e. such that Sum[i=1..N_j]{w_i} = N_classA, j=classA, classB, ...
|
|
: Dataset[MatchNNDataSet] : ... (note that N_j is the sum of TRAINING events
|
|
: Dataset[MatchNNDataSet] : ..... Testing events are not renormalised nor included in the renormalisation factor!)
|
|
: Number of training and testing events
|
|
: ---------------------------------------------------------------------------
|
|
: Signal -- training events : 50000
|
|
: Signal -- testing events : 20000
|
|
: Signal -- training and testing events: 70000
|
|
: Dataset[MatchNNDataSet] : Signal -- due to the preselection a scaling factor has been applied to the numbers of requested events: 0.988773
|
|
: Background -- training events : 500000
|
|
: Background -- testing events : 100000
|
|
: Background -- training and testing events: 600000
|
|
: Dataset[MatchNNDataSet] : Background -- due to the preselection a scaling factor has been applied to the numbers of requested events: 0.511083
|
|
:
|
|
DataSetInfo : Correlation matrix (Signal):
|
|
: --------------------------------------------------------
|
|
: chi2 teta2 distX distY dSlope dSlopeY
|
|
: chi2: +1.000 +0.197 +0.512 +0.603 +0.392 +0.418
|
|
: teta2: +0.197 +1.000 +0.456 +0.649 +0.399 +0.581
|
|
: distX: +0.512 +0.456 +1.000 +0.445 +0.555 +0.606
|
|
: distY: +0.603 +0.649 +0.445 +1.000 +0.529 +0.568
|
|
: dSlope: +0.392 +0.399 +0.555 +0.529 +1.000 +0.647
|
|
: dSlopeY: +0.418 +0.581 +0.606 +0.568 +0.647 +1.000
|
|
: --------------------------------------------------------
|
|
DataSetInfo : Correlation matrix (Background):
|
|
: --------------------------------------------------------
|
|
: chi2 teta2 distX distY dSlope dSlopeY
|
|
: chi2: +1.000 +0.001 +0.370 +0.305 +0.002 +0.084
|
|
: teta2: +0.001 +1.000 +0.173 +0.650 +0.280 +0.455
|
|
: distX: +0.370 +0.173 +1.000 +0.043 +0.627 +0.195
|
|
: distY: +0.305 +0.650 +0.043 +1.000 +0.240 +0.458
|
|
: dSlope: +0.002 +0.280 +0.627 +0.240 +1.000 +0.362
|
|
: dSlopeY: +0.084 +0.455 +0.195 +0.458 +0.362 +1.000
|
|
: --------------------------------------------------------
|
|
DataSetFactory : [MatchNNDataSet] :
|
|
:
|
|
Factory : [MatchNNDataSet] : Create Transformation "I" with events from all classes.
|
|
:
|
|
: Transformation, Variable selection :
|
|
: Input : variable 'chi2' <---> Output : variable 'chi2'
|
|
: Input : variable 'teta2' <---> Output : variable 'teta2'
|
|
: Input : variable 'distX' <---> Output : variable 'distX'
|
|
: Input : variable 'distY' <---> Output : variable 'distY'
|
|
: Input : variable 'dSlope' <---> Output : variable 'dSlope'
|
|
: Input : variable 'dSlopeY' <---> Output : variable 'dSlopeY'
|
|
TFHandler_Factory : Variable Mean RMS [ Min Max ]
|
|
: -----------------------------------------------------------
|
|
: chi2: 4.1539 4.6460 [ 1.3264e-05 15.000 ]
|
|
: teta2: 0.0079252 0.017224 [ 1.2100e-06 0.43619 ]
|
|
: distX: 27.109 38.586 [ 3.8147e-06 250.00 ]
|
|
: distY: 20.564 28.494 [ 1.5259e-05 399.49 ]
|
|
: dSlope: 0.28782 0.22814 [ 2.2016e-06 1.3026 ]
|
|
: dSlopeY: 0.0054782 0.0099926 [ 1.8626e-09 0.14834 ]
|
|
: -----------------------------------------------------------
|
|
: Ranking input variables (method unspecific)...
|
|
IdTransformation : Ranking result (top variable is best ranked)
|
|
: --------------------------------
|
|
: Rank : Variable : Separation
|
|
: --------------------------------
|
|
: 1 : chi2 : 6.095e-01
|
|
: 2 : distX : 4.727e-01
|
|
: 3 : distY : 1.428e-01
|
|
: 4 : dSlope : 7.613e-02
|
|
: 5 : dSlopeY : 5.967e-02
|
|
: 6 : teta2 : 5.937e-02
|
|
: --------------------------------
|
|
Factory : Train method: matching_mlp for Classification
|
|
:
|
|
TFHandler_matching_mlp : Variable Mean RMS [ Min Max ]
|
|
: -----------------------------------------------------------
|
|
: chi2: -0.44615 0.61947 [ -1.0000 1.0000 ]
|
|
: teta2: -0.96367 0.078975 [ -1.0000 1.0000 ]
|
|
: distX: -0.78313 0.30869 [ -1.0000 1.0000 ]
|
|
: distY: -0.89705 0.14265 [ -1.0000 1.0000 ]
|
|
: dSlope: -0.55809 0.35029 [ -1.0000 1.0000 ]
|
|
: dSlopeY: -0.92614 0.13472 [ -1.0000 1.0000 ]
|
|
: -----------------------------------------------------------
|
|
: Training Network
|
|
:
|
|
: Elapsed time for training with 550000 events: [1;31m1.28e+03 sec[0m
|
|
matching_mlp : [MatchNNDataSet] : Evaluation of matching_mlp on training sample (550000 events)
|
|
: Elapsed time for evaluation of 550000 events: [1;31m0.785 sec[0m
|
|
: Creating xml weight file: [0;36mMatchNNDataSet/weights/TMVAClassification_matching_mlp.weights.xml[0m
|
|
: Creating standalone class: [0;36mMatchNNDataSet/weights/TMVAClassification_matching_mlp.class.C[0m
|
|
: Write special histos to file: matching_ghost_mlp_training.root:/MatchNNDataSet/Method_MLP/matching_mlp
|
|
Factory : Training finished
|
|
:
|
|
: Ranking input variables (method specific)...
|
|
matching_mlp : Ranking result (top variable is best ranked)
|
|
: --------------------------------
|
|
: Rank : Variable : Importance
|
|
: --------------------------------
|
|
: 1 : teta2 : 7.346e+02
|
|
: 2 : distX : 1.891e+02
|
|
: 3 : dSlopeY : 5.900e+01
|
|
: 4 : distY : 4.639e+01
|
|
: 5 : dSlope : 1.074e+01
|
|
: 6 : chi2 : 2.093e+00
|
|
: --------------------------------
|
|
Factory : === Destroy and recreate all methods via weight files for testing ===
|
|
:
|
|
: Reading weight file: [0;36mMatchNNDataSet/weights/TMVAClassification_matching_mlp.weights.xml[0m
|
|
matching_mlp : Building Network.
|
|
: Initializing weights
|
|
Factory : [1mTest all methods[0m
|
|
Factory : Test method: matching_mlp for Classification performance
|
|
:
|
|
matching_mlp : [MatchNNDataSet] : Evaluation of matching_mlp on testing sample (120000 events)
|
|
: Elapsed time for evaluation of 120000 events: [1;31m0.169 sec[0m
|
|
Factory : [1mEvaluate all methods[0m
|
|
Factory : Evaluate classifier: matching_mlp
|
|
:
|
|
TFHandler_matching_mlp : Variable Mean RMS [ Min Max ]
|
|
: -----------------------------------------------------------
|
|
: chi2: -0.15813 0.62626 [ -1.0000 0.99995 ]
|
|
: teta2: -0.97140 0.071651 [ -1.0000 0.85686 ]
|
|
: distX: -0.67477 0.35263 [ -1.0000 0.99866 ]
|
|
: distY: -0.87476 0.15583 [ -1.0000 0.99415 ]
|
|
: dSlope: -0.49635 0.36886 [ -0.99993 0.97372 ]
|
|
: dSlopeY: -0.91980 0.13029 [ -1.0000 1.0219 ]
|
|
: -----------------------------------------------------------
|
|
matching_mlp : [MatchNNDataSet] : Loop over test events and fill histograms with classifier response...
|
|
:
|
|
TFHandler_matching_mlp : Variable Mean RMS [ Min Max ]
|
|
: -----------------------------------------------------------
|
|
: chi2: -0.15813 0.62626 [ -1.0000 0.99995 ]
|
|
: teta2: -0.97140 0.071651 [ -1.0000 0.85686 ]
|
|
: distX: -0.67477 0.35263 [ -1.0000 0.99866 ]
|
|
: distY: -0.87476 0.15583 [ -1.0000 0.99415 ]
|
|
: dSlope: -0.49635 0.36886 [ -0.99993 0.97372 ]
|
|
: dSlopeY: -0.91980 0.13029 [ -1.0000 1.0219 ]
|
|
: -----------------------------------------------------------
|
|
:
|
|
: Evaluation results ranked by best signal efficiency and purity (area)
|
|
: -------------------------------------------------------------------------------------------------------------------
|
|
: DataSet MVA
|
|
: Name: Method: ROC-integ
|
|
: MatchNNDataSet matching_mlp : 0.970
|
|
: -------------------------------------------------------------------------------------------------------------------
|
|
:
|
|
: Testing efficiency compared to training efficiency (overtraining check)
|
|
: -------------------------------------------------------------------------------------------------------------------
|
|
: DataSet MVA Signal efficiency: from test sample (from training sample)
|
|
: Name: Method: @B=0.01 @B=0.10 @B=0.30
|
|
: -------------------------------------------------------------------------------------------------------------------
|
|
: MatchNNDataSet matching_mlp : 0.543 (0.551) 0.936 (0.936) 0.985 (0.985)
|
|
: -------------------------------------------------------------------------------------------------------------------
|
|
:
|
|
Dataset:MatchNNDataSet : Created tree 'TestTree' with 120000 events
|
|
:
|
|
Dataset:MatchNNDataSet : Created tree 'TrainTree' with 550000 events
|
|
:
|
|
Factory : [1mThank you for using TMVA![0m
|
|
: [1mFor citation information, please visit: http://tmva.sf.net/citeTMVA.html[0m
|
|
Transforming neural_net_training/result/MatchNNDataSet/weights/TMVAClassification_matching_mlp.class.C ...
|
|
Found minimum and maximum values for 6 variables.
|
|
Found 3 matrices:
|
|
1. fWeightMatrix0to1 with 7 columns and 8 rows
|
|
2. fWeightMatrix1to2 with 9 columns and 6 rows
|
|
3. fWeightMatrix2to3 with 7 columns and 1 rows
|