Uploaded image for project: 'ROOT'
  1. ROOT
  2. ROOT-9783 RDataFrame-related tickets
  3. ROOT-9438

[DF] Separate computation graph from dataset it should run on



    • Sub-task
    • Status: Open (View Workflow)
    • High
    • Resolution: Unresolved
    • None
    • None
    • RDataFrame
    • None


      Advanced users (e.g. TMVA internals) might want to define a computation graph once (e.g. by jitting) and then apply it to several different data-sets or several subsets of a dataset in sequence, without having to re-generate the computation graph each time.

      The introduction of a new type of head node, a "computation manager", which can attach to different "loop managers", would be one relatively simple way to allow this.

      Common usage of RDataFrame would remain the same, with the loop manager using a computation manager to "speak" to the computation graph under the hood. Explicit usage of the computation manager node would allow to attach to different loop managers.

      At least the schema of the dataset (or the subset that one expects to read) should probably still be known a priori, to allow RDataFrame to error out early in case of misspelled column name, inexistent branch name and so on.

      To allow the reusage of the same computation graph, actions should not be deleted at the end of the event loop as it is currently done but kept around for the next run.


        Issue Links



              eguiraud Enrico Guiraud
              eguiraud Enrico Guiraud
              1 Vote for this issue
              6 Start watching this issue