Uploaded image for project: 'ROOT'
  1. ROOT
  2. ROOT-10454

Consider adding randomized tests to RHist



    • Improvement
    • Status: Open (View Workflow)
    • Low
    • Resolution: Unresolved
    • master
    • None
    • Core Libraries
    • None


      Randomized testing, as provided by the various variants of Haskell's quickcheck test framework and fuzz testing tools, has proven surprisingly potent at probing areas of a program's parameter space which a developer would not think about manually testing. I personally found a number of interesting bugs in RHist by using it, and therefore think that RHist could benefit from using this technique in its test suite too.

      I am not proposing to completely replace RHist's test suite with such tests, because that would require using a pseudorandom data generator that is specifically tuned to adversarially favor known-tricky data, which is hard to implement and does not really have a good pre-built implementation in C++. Instead, I am proposing to complement RHist's existing hand-written tests with a simple randomized test suite which just pokes across the parameter space with a uniform law, much like the test suite of my work-in-progress RHist -> THx conversion tool is doing.

      Test reproducibility concerns may be alleviated by either...

      • Using a fixed PRNG seed. We don't really need true randomness to benefit from randomized testing, as we only care about the good N-dimensional parameter space coverage properties of pseudo-random numbers.
      • Picking a random PRNG seed by default, but printing it on test startup and allowing the dev to replay a test with a specific PRNG seed, as e.g. the verrou floating-point error checker does




            axel Axel Naumann
            hgraslan Hadrien Benjamin Grasland
            0 Vote for this issue
            1 Start watching this issue