This repository contains the Ollama project configuration for local development. The project uses the Ollama framework to run large language models (LLMs) locally, along with OpenWebUI (web interface) and SearXNG (search engine).

Environment Details

OS: Linux (6.8.0-117-generic)
Working Directory: /opt/ollama
Model: qwen2.5-coder:7b-instruct-q4_K_M-64k
Context Limit: 40960 tokens (64k context window)

Project Features

Local model serving with Ollama
Custom OpenAI-compatible provider
7B parameter model with 64k context window
Web interface with OpenWebUI
Search capabilities with SearXNG

How to Run

1. Start the model

curl http://localhost:11434/api/generate -d '{
  "model": "qwen2.5-coder:7b-instruct-q4_K_M-64k",
  "prompt": "Hello, world!"
}'"

2. Check model list

curl http://localhost:11434/api/models

3. Start OpenWebUI (web interface)

docker run -p 3000:3000 -v /opt/ollama/openwebui:/app/data -d --name openwebui openwebui/openwebui:latest

4. Start SearXNG (search engine)

docker run -p 8888:8888 -v /opt/ollama/searxng:/app/data -d --name searxng searxng/searxng:latest

Notes

The model has a 64k context window limit
No Docker required for Ollama - uses native API
OpenWebUI accessible at http://localhost:3000
SearXNG accessible at http://localhost:8888
All operations are performed in the /opt/ollama directory
For GPU acceleration, ensure CUDA is properly configured