diff --git a/README.md b/README.md new file mode 100644 index 0000000..60f5da8 --- /dev/null +++ b/README.md @@ -0,0 +1,54 @@ +# Ollama Project Setup + +## Project Overview + +This repository contains the Ollama project configuration for local development. The project uses the [Ollama framework](https://ollama.ai) to run large language models (LLMs) locally, along with [OpenWebUI](https://openwebui.ai) (web interface) and [SearXNG](https://searxng.net) (search engine). + +## Environment Details + +- **OS**: Linux (6.8.0-117-generic) +- **Working Directory**: /opt/ollama +- **Model**: qwen2.5-coder:7b-instruct-q4_K_M-64k +- **Context Limit**: 40960 tokens (64k context window) + +## Project Features + +- Local model serving with Ollama +- Custom OpenAI-compatible provider +- 7B parameter model with 64k context window +- Web interface with OpenWebUI +- Search capabilities with SearXNG + +## How to Run + +### 1. Start the model + ```bash + curl http://localhost:11434/api/generate -d '{ + "model": "qwen2.5-coder:7b-instruct-q4_K_M-64k", + "prompt": "Hello, world!" + }'" + ``` + +### 2. Check model list + ```bash + curl http://localhost:11434/api/models + ``` + +### 3. Start OpenWebUI (web interface) + ```bash + docker run -p 3000:3000 -v /opt/ollama/openwebui:/app/data -d --name openwebui openwebui/openwebui:latest + ``` + +### 4. Start SearXNG (search engine) + ```bash + docker run -p 8888:8888 -v /opt/ollama/searxng:/app/data -d --name searxng searxng/searxng:latest + ``` + +## Notes + +- The model has a 64k context window limit +- No Docker required for Ollama - uses native API +- OpenWebUI accessible at http://localhost:3000 +- SearXNG accessible at http://localhost:8888 +- All operations are performed in the /opt/ollama directory +- For GPU acceleration, ensure CUDA is properly configured \ No newline at end of file