Currently, the -f both means:
-
that we include data from failing runs, which can be caused either by benchmarks aborting or because the validation of results fails or output indicates other issues
-
that the rebench exit code indicates no failure
We may want to separate these concerns out.
It might be useful to have a --dont-fail option, just to keep the exit code 0.
Where --faulty is only about including data from runs that indicate some issue.
Requested by @OctaveLarose
Currently, the
-fboth means:that we include data from failing runs, which can be caused either by benchmarks aborting or because the validation of results fails or output indicates other issues
that the
rebenchexit code indicates no failureWe may want to separate these concerns out.
It might be useful to have a
--dont-failoption, just to keep the exit code 0.Where
--faultyis only about including data from runs that indicate some issue.Requested by @OctaveLarose