GitHub - janhq/nitro: A fast, lightweight, embeddable inference engine to supercharge your apps with local AI. OpenAI-compatible API
{{ message }}
A fast, lightweight, embeddable inference engine to supercharge your apps with local AI. OpenAI-compatible API
License
About
A fast, lightweight, embeddable inference engine to supercharge your apps with local AI. OpenAI-compatible API
Topics
ai cuda llama accelerated inference-engine openai-api llm stable-diffusion llms llamacpp llama2 gguf tensorrt-llm
Resources
License
Stars
Watchers
Forks
Packages
No packages published
Languages
- C++ 98.8%
- Other 1.2%