DS Log

In my blog, I delve into the world of programming web technologies, Linux, Unix-like, and graphic design using free tools on Linux.

Tuesday, October 7, 2025

How to install AI and use it locally (Ubuntu)

Ollama is a desktop/server application that lets you run large language models locally and expose them via a simple CLI and HTTP API. It runs models on your machine or server so no cloud is required by default. It provides a command-line interface with commands like pull, list, chat, run, and serve. When you run ollama serve it hosts an HTTP API at http://127.0.0.1:11434. Ollama supports multiple models (community and commercially licensed); you pull models with ollama pull model. It runs models using local CPU/GPU acceleration if available, so performance depends on your hardware. Configuration and models are stored under ~/.ollama by default. Ollama can integrate with Docker/OCI-style model packages in some cases. Common uses include local development and testing with LLMs, private or offline model inference, and serving a local API for apps or bots.

Go to Olamma website:

https://ollama.com/download/linux

Run in terminal:

curl -fsSL https://ollama.com/install.sh | sh

CodeLlama is specifically fine-tuned for coding tasks. It's based on Llama 2 and excels at generating, completing, and explaining code in languages like Python, Bash, C++, JavaScript, and more. It's great for writing scripts, debugging errors, or even converting pseudocode to real code.

You will need to download the model with command:

ollama pull codellama

And to run it, use:

ollama run codellama

deepseek-coder is one of the top models for code generation and understanding, often outperforming CodeLlama in benchmarks for multi-language support and complex logic (e.g., algorithms, APIs). It's trained on vast code datasets, so it's excellent for real-world dev tasks like integrating libraries or refactoring.

You will need to download the model with command:

ollama pull deepseek-coder

And to run it, use:

ollama run deepseek-coder

WebAPI

When you run ollama serve it hosts an HTTP API at http://127.0.0.1:11434

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "codellama",
  "prompt": "Write a Bash script to check disk usage and alert if over 80%.",
  "stream": false
}'

To get the list of models:

http://127.0.0.1:11434/v1/models

Ollama has a variety of open-source models optimized for these tasks, and the "best" one depends on your hardware (e.g., CPU/GPU, RAM) and needs, but I'll prioritize those with strong performance in code-related tasks while also handling general system knowledge well.

To stop service use:

sudo systemctl stop ollama

posted by damirs @ October 07, 2025