papersSEP 10 04:00 UTC
Benchmarks and Evaluation Protocols Skew Results in Provenance-Based Intrusion Detection
A study examines how the choice of benchmark and evaluation protocol affects reported performance of provenance-based intrusion detection systems, showing that conclusions about detector effectiveness can shift substantially depending on these decisions. The authors caution that favorable results may reflect testing setup rather than true detection capability, and outline practices for more rigorous evaluation.