Call 30+ LLMs with a single API.
- Send multiple prompts to multiple LLMs and get the results back in a single response.
- Zero dependencies (under 10kB minified)
- Bring your own API keys
- Works anywhere (Node, Deno, browser)
Install
npm
yarn
Example
import { LLM, MODEL } from 'llm.ts'; (async function () { await new LLM({ apiKeys: { openAI: process.env.OPENAI_API_KEY ?? '', cohere: process.env.COHERE_API_KEY ?? '', huggingface: process.env.HF_API_TOKEN ?? '', } }).completion({ prompt: [ 'Repeat the following sentence: "I am a robot."', 'Repeat the following sentence: "I am a human."', ], model: [ // use the model name 'text-ada-001', // or specify a specific provider 'cohere/command-nightly', // or use enums to avoid typos MODEL.HF_GPT2, ], }).then(resp => { console.log(resp); }) })()
{
"created": 1683079463217,
"choices": [
{
"text": "\n\nI am a robot.",
"index": 0,
"model": "text-ada-001",
"promptIndex": 0,
"created": 1683079462
},
{
"text": "\n\nI am a human.",
"index": 1,
"model": "text-ada-001",
"promptIndex": 1,
"created": 1683079462
},
{
"text": "\nI am a robot.",
"index": 2,
"model": "command-nightly",
"promptIndex": 0,
"created": 1683079463217
},
{
"text": "\nI am a human.",
"index": 3,
"model": "command-nightly",
"promptIndex": 1,
"created": 1683079463216
},
{
"text": " \"Is that your question? I was expecting the answer.\" \"Then why do you think you are being asked!\" 1. \"What are you?\" \"What are you?\" \"Why are you",
"index": 4,
"model": "gpt2",
"promptIndex": 0,
"created": 1683079463088
},
{
"text": " — this quote is most often cited in reference to the Qur'an. (e.g. Ibn `Allaahu `udayyyih, Al-Rai`an, Al",
"index": 5,
"model": "gpt2",
"promptIndex": 1,
"created": 1683079463091
}
]
}Models supported (want to add one, open a PR!):
- text-ada-001
- text-babbage-001
- text-curie-001
- text-davinci-002
- text-davinci-003
- cohere-command
- cohere-command-nightly
- cohere-command-light
- cohere-command-light-nightly
- gpt2
- bloom-1b
- bloom-3b
- bloom-7b1
- llama-7b
- llama-13b
- llama-30b
- llama-65b
- gptj-6b
- gptj-2.7b
- gpt-neo-125m
- gpt-neo-1.3b
- gpt-neo-20b
- cerebras-gpt-111m
- cerebras-gpt-1.3b
- cerebras-gpt-2.7b
- santacoder
- codegen-350m
- codegen-2b
- stablelm-tuned-3b
- stablelm-tuned-7b
- pythia-70m
- pythia-160m
- pythia-12b
- distilgpt2
Hosting providers supported (want to add one, open a PR!):
- OpenAI
- Cohere
- HuggingFace
License
MIT