Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM

Category: library

Tags: llm-inference, java, llama-cpp, spring-boot, project-panama, ffm

Score: 6.8/10 (Innovation: 6, Technical: 7, Documentation: 8, Utility: 6)

mochallama embeds llama.cpp directly inside the JVM using Project Panama's Foreign Function & Memory API, eliminating the need for JNI, separate daemons, or native compilation. It provides OpenAI-compatible APIs, tool calling, and Spring Boot integration, making local LLM inference accessible to Java developers without leaving their ecosystem.

Target audience: backend devs

Repository: https://deemwar-products.github.io/mochallama/ · Java · MIT

View on Hacker News