Uploaded image for project: 'ROOT'
  1. ROOT
  2. ROOT-9783 RDataFrame-related tickets
  3. ROOT-9438

[DF] Separate computation graph from dataset it should run on

    XMLWordPrintable

Details

    • Sub-task
    • Status: Open (View Workflow)
    • High
    • Resolution: Unresolved
    • None
    • None
    • RDataFrame
    • None

    Description

      Advanced users (e.g. TMVA internals) might want to define a computation graph once (e.g. by jitting) and then apply it to several different data-sets or several subsets of a dataset in sequence, without having to re-generate the computation graph each time.

      The introduction of a new type of head node, a "computation manager", which can attach to different "loop managers", would be one relatively simple way to allow this.

      Common usage of RDataFrame would remain the same, with the loop manager using a computation manager to "speak" to the computation graph under the hood. Explicit usage of the computation manager node would allow to attach to different loop managers.

      At least the schema of the dataset (or the subset that one expects to read) should probably still be known a priori, to allow RDataFrame to error out early in case of misspelled column name, inexistent branch name and so on.

      To allow the reusage of the same computation graph, actions should not be deleted at the end of the event loop as it is currently done but kept around for the next run.

      Attachments

        Issue Links

          Activity

            People

              eguiraud Enrico Guiraud
              eguiraud Enrico Guiraud
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: