Loading...
Loading...
The RAG Starter Kit uses a provider factory pattern that supports multiple LLM backends. The default is OpenRouter, which gives access to hundreds of models including free ones.
OpenRouter provides a unified API for 200+ models from OpenAI, Anthropic, Google, Meta, and more. It includes free models (with rate limits) so you can evaluate the kit without spending anything.
OPENROUTER_API_KEY=sk-or-...Get a key at openrouter.ai/keys. The model is selected in the chat UI via the model picker.
OPENAI_API_KEY=sk-...Supports GPT-4o, GPT-4o-mini, and o1 models. The API key is also used for OpenAI embeddings when configured.
Access Claude models directly through the Anthropic API.
ANTHROPIC_API_KEY=sk-ant-...Available when selected in the chat model picker via OpenRouter, or configure directly for Claude-only usage.
Use Gemini models for both chat and embeddings.
GOOGLE_GENERATIVE_AI_API_KEY=AIza...Free tier includes generous usage limits. Works for both embedding and chat when configured.
Run models locally for complete data privacy and zero API costs.
OLLAMA_BASE_URL=http://localhost:11434ollama pull llama3.2
ollama pull mistralRequires Ollama running locally. Supports any model available in the Ollama library. Ideal for air-gapped environments or when data must not leave your network.
The provider factory (src/lib/ai/llm/factory.ts) resolves the LLM provider from the model ID selected in the chat UI. The model picker shows available models based on which API keys are configured. If only OpenRouter is configured, you see all OpenRouter models. If additional keys are set, those providers become available too.