Uploaded image for project: 'ROOT'
  1. ROOT
  2. ROOT-10454

Consider adding randomized tests to RHist

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open (View Workflow)
    • Priority: Low
    • Resolution: Unresolved
    • Affects Version/s: master
    • Fix Version/s: None
    • Component/s: Core Libraries
    • Labels:
      None
    • Development:

      Description

      Randomized testing, as provided by the various variants of Haskell's quickcheck test framework and fuzz testing tools, has proven surprisingly potent at probing areas of a program's parameter space which a developer would not think about manually testing. I personally found a number of interesting bugs in RHist by using it, and therefore think that RHist could benefit from using this technique in its test suite too.

      I am not proposing to completely replace RHist's test suite with such tests, because that would require using a pseudorandom data generator that is specifically tuned to adversarially favor known-tricky data, which is hard to implement and does not really have a good pre-built implementation in C++. Instead, I am proposing to complement RHist's existing hand-written tests with a simple randomized test suite which just pokes across the parameter space with a uniform law, much like the test suite of my work-in-progress RHist -> THx conversion tool is doing.

      Test reproducibility concerns may be alleviated by either...

      • Using a fixed PRNG seed. We don't really need true randomness to benefit from randomized testing, as we only care about the good N-dimensional parameter space coverage properties of pseudo-random numbers.
      • Picking a random PRNG seed by default, but printing it on test startup and allowing the dev to replay a test with a specific PRNG seed, as e.g. the verrou floating-point error checker does

        Attachments

          Activity

            People

            Assignee:
            axel Axel Naumann
            Reporter:
            hgraslan Hadrien Benjamin Grasland
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: