ollama/README.md

# Ollama Project Setup

## Project Overview

This repository contains the Ollama project configuration for local development. The project uses the [Ollama framework](https://ollama.ai) to run large language models (LLMs) locally, along with [OpenWebUI](https://openwebui.ai) (web interface) and [SearXNG](https://searxng.net) (search engine).

## Environment Details

- **OS**: Linux (6.8.0-117-generic)
- **Working Directory**: /opt/ollama
- **Model**: qwen2.5-coder:7b-instruct-q4_K_M-64k
- **Context Limit**: 40960 tokens (64k context window)

## Project Features

- Local model serving with Ollama
- Custom OpenAI-compatible provider
- 7B parameter model with 64k context window
- Web interface with OpenWebUI
- Search capabilities with SearXNG

## How to Run

### 1. Start the model
   ```bash
   curl http://localhost:11434/api/generate -d '{
     "model": "qwen2.5-coder:7b-instruct-q4_K_M-64k",
     "prompt": "Hello, world!"
   }'" 
   ```

### 2. Check model list
   ```bash
   curl http://localhost:11434/api/models
   ```

### 3. Start OpenWebUI (web interface)
   ```bash
   docker run -p 3000:3000 -v /opt/ollama/openwebui:/app/data -d --name openwebui openwebui/openwebui:latest
   ```

### 4. Start SearXNG (search engine)
   ```bash
   docker run -p 8888:8888 -v /opt/ollama/searxng:/app/data -d --name searxng searxng/searxng:latest
   ```

## Notes

- The model has a 64k context window limit
- No Docker required for Ollama - uses native API
- OpenWebUI accessible at http://localhost:3000
- SearXNG accessible at http://localhost:8888
- All operations are performed in the /opt/ollama directory
- For GPU acceleration, ensure CUDA is properly configured