homelab/karakeep
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: wire AI inference to local ollama #1

Merged
litelotus merged 1 commit from feat/ollama-ai into main 2026-09-14 13:29:02 +00:00
Conversation 0 Commits 1 Files changed 1 +24 -1
litelotus commented 2026-09-14 13:21:15 +00:00
Owner
Copy link

Enables karakeep's AI layer (auto-tagging, OCR, semantic search) against the homelab's local ollama (CUDA 12.9 build on Maxwell-class GPUs).

Config-only change — the current image runs web+workers via s6-overlay in one container, so no new workloads:

  • OLLAMA_BASE_URL -> the k3s master node's ollama service (native ollama API; OPENAI_API_KEY deliberately absent — it would take precedence and break ollama inference)
  • INFERENCE_TEXT_MODEL=qwen3:8b — tagging/summarization/chat
  • INFERENCE_IMAGE_MODEL=deepseek-ocr — OCR of image assets
  • EMBEDDING_TEXT_MODEL=nomic-embed-text (768-dim) + auto-indexing → semantic/hybrid search via Meilisearch
  • INFERENCE_CONTEXT_LENGTH=4096, INFERENCE_JOB_TIMEOUT_SEC=120 (local inference is slower than cloud), OLLAMA_KEEP_ALIVE=5m

Models are pulled by the flake's services.ollama.loadModels (separate .nix change).

Deploy: no crow CI on this repo — merge, then pulumi up -s dev from the workstation. Pod restarts, s6 runs the DB migration, AI jobs start processing new saves.

If tagging comes back empty: flip INFERENCE_OUTPUT_SCHEMA=plain (qwen3 structured output via ollama may need it).

Enables karakeep's AI layer (auto-tagging, OCR, semantic search) against the homelab's local ollama (CUDA 12.9 build on Maxwell-class GPUs). **Config-only change** — the current image runs web+workers via s6-overlay in one container, so no new workloads: - `OLLAMA_BASE_URL` -> the k3s master node's ollama service (native ollama API; `OPENAI_API_KEY` deliberately absent — it would take precedence and break ollama inference) - `INFERENCE_TEXT_MODEL=qwen3:8b` — tagging/summarization/chat - `INFERENCE_IMAGE_MODEL=deepseek-ocr` — OCR of image assets - `EMBEDDING_TEXT_MODEL=nomic-embed-text` (768-dim) + auto-indexing → semantic/hybrid search via Meilisearch - `INFERENCE_CONTEXT_LENGTH=4096`, `INFERENCE_JOB_TIMEOUT_SEC=120` (local inference is slower than cloud), `OLLAMA_KEEP_ALIVE=5m` Models are pulled by the flake's `services.ollama.loadModels` (separate .nix change). **Deploy**: no crow CI on this repo — merge, then `pulumi up -s dev` from the workstation. Pod restarts, s6 runs the DB migration, AI jobs start processing new saves. **If tagging comes back empty**: flip `INFERENCE_OUTPUT_SCHEMA=plain` (qwen3 structured output via ollama may need it).
Enable karakeep's AI layer against the homelab ollama (CUDA 12.9/M10):

- OLLAMA_BASE_URL -> the k3s master node's ollama service (native API)
- INFERENCE_TEXT_MODEL=qwen3:8b (tagging/summarization/chat)
- INFERENCE_IMAGE_MODEL=deepseek-ocr (image OCR)
- EMBEDDING_TEXT_MODEL=nomic-embed-text, 768-dim, auto-indexing on
  (semantic/hybrid search)
- INFERENCE_CONTEXT_LENGTH=4096, INFERENCE_JOB_TIMEOUT_SEC=120 (local
  inference is slower than OpenAI), OLLAMA_KEEP_ALIVE=5m

OPENAI_API_KEY is intentionally absent — karakeep gives it precedence over
OLLAMA_BASE_URL and would try gpt models on ollama.

No new workloads: the current image runs web+workers via s6-overlay, so AI
processing happens in the existing deployment. Models are pulled via the
flake's services.ollama.loadModels (separate change).
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
homelab/karakeep!1
No description provided.