Show HN: Promptloop – create, run, and improve prompt evals from the terminal

Category: cli-tool

Tags: cli, prompt-eval, llm

Score: 6.0/10 (Innovation: 5, Technical: 6, Documentation: 7, Utility: 6)

Promptloop is a CLI agent that automates the prompt evaluation loop, allowing users to create test cases, run evals, generate reports, and propose prompt diffs interactively from the terminal. It leverages LangChain's deepagents to turn prompt iteration into a repeatable, structured workflow, making it interesting for teams refining LLM prompts in production.

Target audience: backend devs, data engineers, ai-ml engineers

Repository: https://github.com/Bella3202019/promptloop · Python · 14 stars

View on Hacker News