Models trained with RMVA are saved in .Rdata files.
For all of the models in
name of the file is hardcoded e.g:
TString path = GetWeightFileDir() + "/RXGBModel.RData";
so when one books multiple variations of the same model with
factory->BookMethod(dataloader, TMVA::Types::kRXGB, "kRXGB", "!V:NRounds=80:MaxDepth=2:Eta=1");
TMVA keeps overwriting the same file and at the end reports the same results for all variants.
An easy fix which I tested to be working is to replace the lines like
TString path = GetWeightFileDir() + "/RXGBModel.RData
TString path = GetWeightFileDir() + "/" + GetName() + ".RData";
everywhere in *.cxx files above. This will use a file name based on the method title the same way it is done for the default TMVA methods as long as a user uses unique titles when booking using the line above