Show HN: An open source benchmark for prompt-injection detectors
Category: security
Tags: prompt-injection, benchmark, llm-security
Score: 7.3/10 (Innovation: 7, Technical: 7, Documentation: 8, Utility: 7)
This project provides a reproducible, threshold-agnostic benchmark for prompt-injection detectors, measuring both detection rate and false positives on real traffic. It offers a standardized methodology and leaderboard to compare detectors fairly, addressing a critical gap in LLM security evaluation. The open, model-agnostic design with committed raw scores ensures transparency and reproducibility.
Target audience: security engineers, AI researchers, backend devs
Repository: https://github.com/bastion-soft/pi-detector-bench · Python · NOASSERTION
View on Hacker News