Show HN: ShadowPEFT – Centralized and Detachable Parameter-Efficient Fine-Tuning

Category: library

Tags: parameter-efficient-fine-tuning, llm, huggingface

Score: 7.3/10 (Innovation: 7, Technical: 8, Documentation: 8, Utility: 6)

ShadowPEFT is a parameter-efficient fine-tuning framework that introduces a separate, pretrainable 'shadow' network parallel to a frozen base model, enabling detachable and modular adaptation without modifying backbone weights. It is interesting because it combines architectural decoupling with the ability to reuse smaller pretrained models as adaptation modules, offering novel flexibility for edge deployment and model versioning.

Target audience: machine learning engineers, nlp researchers, ai developers

Repository: https://github.com/ShadowLLM/shadow-peft · Python · MIT · 25 stars

View on Hacker News