Show HN: YieldOS-Lite – A simulator for LLM inference control-plane governance

Category: infrastructure

Tags: llm-inference, resource-governance, simulator

Score: 6.8/10 (Innovation: 7, Technical: 7, Documentation: 8, Utility: 5)

YieldOS-Lite is a research simulator for testing resource-governance policies in heterogeneous LLM inference, focusing on predictive SLO governance and value-aware KV cache management. It provides a clean, trace-driven environment to evaluate control-plane ideas before production integration, with strong experimental rigor and documentation. The project is a novel combination of scheduling, governance, and heterogeneity analysis in the LLM serving space.

Target audience: researchers and engineers working on LLM serving systems

Repository: https://github.com/nikitph/yieldos · Python

View on Hacker News