Monthly Update: personal-ai project
Made solid progress—most notably, standing up a lightweight services layer that exposes my own OpenAI-style API for local HuggingFace models.
Following the previous post, I’ve decided to share monthly updates highlighting the most notable developments in my personal-ai project.
🛠️ Key Improvements
The biggest improvements were in the services layer. Other updates focused on MCP integration and backend layer quality enhancement.
Here’s what I’ve got so far:
OpenAI-Compatible APIs
A basic model registry system
An initial version of the LM Studio proxy
The APIs serving OSS models from HuggingFace, tested on an old GTX 1070.
To run it yourself, see instructions here.
To get the models I’ve tested with, just run:
pytest -vv .\tests\test_registry.py
Why Build My Own OpenAI Interface?
Initially, I planned to lean on LM Studio. But when I tried facebook/vjepa2-vitg-fpc64-384, its quantized version in LM Studio performed poorly. So I went for the full model—and one thing led to another. I now have a (simplistic) way to host models directly.
Future Plans
The LM Studio proxy will evolve into a load balancer across multiple instances of the same model. That should unlock more scalability for local deployments.
I plan to provide vector stores as well, and see how to expose MCP servers as FastAPI APIs.
Final Notes
That’s it for this update. I push code pretty regularly, so if you’re curious or want to collaborate, let me know—I’ve got plenty of ideas to explore.
Everything I write are my opinions and perspectives and do not represent my past, current of future employers.