LLM ServingSep 4, 2026(Updated Oct 10, 2026)

How to Run DeepSeek R1 Locally on Linux with Ollama (Step-by-Step VRAM Guide)

By B. Cho · Systems & AI Infrastructure Engineer

VRAM Sizing & Ollama Launch Commands for DeepSeek R1

To install Ollama and run a quantized DeepSeek-R1 distilled model immediately on Linux with NVIDIA GPU acceleration, run:

# 1. Install Ollama binary & systemd service
curl -fsSL https://ollama.com/install.sh | sh

# 2. Run the 8B distilled model (Requires ~6GB VRAM, fits RTX 3060/4060)
ollama run deepseek-r1:8b

Once loaded, the interactive prompt handles streaming responses including DeepSeek R1’s <think>...</think> reasoning trace.

1. DeepSeek R1 Model Sizes & Hardware VRAM Matrix

DeepSeek R1 was trained at a full 671B parameter scale (Mixture-of-Experts), then distilled into dense architectures using Qwen-2.5 and Llama-3. From the official Ollama deepseek-r1 model registry, select the distilled parameter size that fits your GPU unshared VRAM:

Model Tag	Base Architecture	Context Window	Min VRAM (Q4_K_M)	Recommended GPU
`deepseek-r1:1.5b`	Qwen-2.5-Math-1.5B	32k	1.8 GB	Any modern GPU / CPU only
`deepseek-r1:7b`	Qwen-2.5-7B	32k	5.2 GB	RTX 3060 (6GB mobile), RTX 2060
`deepseek-r1:8b`	Llama-3.1-8B	32k	5.8 GB	RTX 3060 / 4060 (8GB)
`deepseek-r1:14b`	Qwen-2.5-14B	32k	9.6 GB	RTX 3060 (12GB), RTX 4070 (12GB)
`deepseek-r1:32b`	Qwen-2.5-32B	32k	20.5 GB	RTX 3090 / RTX 4090 (24GB)
`deepseek-r1:70b`	Llama-3.3-70B	32k	43.0 GB	2x RTX 3090 / 2x RTX 4090 (48GB total)
`deepseek-r1:671b`	MoE (37B active)	128k	420+ GB	Multi-node cluster (8x H100 80GB)

The 14B model (deepseek-r1:14b) delivers the highest reasoning performance per watt for standard 12GB to 16GB consumer GPUs. The 32B model matches proprietary reasoning models for complex programming tasks.

If your workload demands high multi-user concurrent throughput rather than desktop exploration, review our Ollama vs vLLM local serving comparison for PagedAttention architectural details.

2. Linux System Preparation & GPU Driver Verification

Verify that your NVIDIA drivers and CUDA runtime are exposed to system services before downloading multi-gigabyte model weights:

nvidia-smi

Ensure your Driver Version is at least 535 and CUDA Version is at least 12.0. If running on Ubuntu or Debian without proprietary drivers:

sudo apt update && sudo apt install -y ubuntu-drivers-common
sudo ubuntu-drivers install
sudo reboot

If you encounter device enumeration issues or invalid index errors, refer to our guide on resolving CUDA invalid device ordinal errors in Linux. For systems experiencing dynamic library loading failures, consult fixing bitsandbytes CUDA setup failures.

If you deploy Ollama within containerized infrastructure, configure the NVIDIA Container Toolkit to pass GPU devices correctly, as detailed in our guide on fixing Docker NVIDIA GPU passthrough on Linux.

3. Configuring the Ollama Service for Performance

By default, Ollama binds only to 127.0.0.1 and uses conservative memory settings. To expose the inference API to your local network, enable Flash Attention, and allow concurrent streams, configure a systemd override.

Create /etc/systemd/system/ollama.service.d/override.conf:

sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo tee /etc/systemd/system/ollama.service.d/override.conf > /dev/null <<EOF
[Service]
# Allow LAN access for web UIs and IDE extensions
Environment="OLLAMA_HOST=0.0.0.0:11434"

# Enable Flash Attention (reduces VRAM consumption for long reasoning chains)
Environment="OLLAMA_FLASH_ATTENTION=1"

# Enable 8-bit quantized KV-cache to save up to 40% context memory
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"

# Keep model loaded in VRAM for 30 minutes after last request
Environment="OLLAMA_KEEP_ALIVE=30m"

# Number of parallel request slots
Environment="OLLAMA_NUM_PARALLEL=2"
EOF

Reload and restart the daemon:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Verify the service is active and running:

systemctl status ollama --no-pager

If containerized applications report connection failures when communicating with Ollama, verify loopback bindings using our walkthrough on fixing Ollama connection refused in Docker on Linux.

4. Expanding the Context Window (Crucial for Reasoning)

By default, Ollama limits inference context (num_ctx) to 2,048 tokens unless specified. Because DeepSeek R1 outputs hundreds or thousands of tokens inside <think>...</think> tags before returning the final solution, a 2k context window will truncate code generation mid-thought.

Create a custom Modelfile to set num_ctx to 32,768:

tee Modelfile > /dev/null <<EOF
FROM deepseek-r1:14b

# Set context window to 32k tokens
PARAMETER num_ctx 32768

# Recommended sampling parameters from DeepSeek team
PARAMETER temperature 0.6
PARAMETER top_p 0.95
EOF

Build the custom model:

ollama create deepseek-r1-32k -f Modelfile

Launch the expanded context model:

ollama run deepseek-r1-32k

For deeper mechanics on token buffer sizing, read our dedicated troubleshooting guide on fixing Ollama context window truncation and num_ctx settings. If you scale up to vLLM, also check vLLM max sequence length allocation on Linux.

5. Integrating with Coding Tools & APIs

Ollama provides an OpenAI-compatible endpoint at http://localhost:11434/v1.

Verification via cURL

Test the API and inspect the reasoning trace:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-r1:8b",
    "messages": [
      {"role": "user", "content": "Write a Python function to find all anagrams in an array."}
    ],
    "stream": false
  }'

Cursor & Continue Extension Setup

Provider: OpenAI Compatible
Base URL: http://localhost:11434/v1
API Key: ollama (any placeholder string)
Model Name: deepseek-r1:8b or deepseek-r1-32k

For automated developer workflows, see our comparative evaluation of top AI coding agents in 2026. If your agents execute structured tool calls, prevent schema errors using our guide on fixing malformed tool calls in local LLMs.

6. Troubleshooting Common Linux Errors

1. Slow Token Generation (< 3 tokens/sec) & 100% CPU Usage

Root Cause: The model failed to offload layers to the GPU and fell back to CPU system RAM.
Diagnosis: Check GPU layer distribution with ollama ps:
```
ollama ps
```
Look at the PROCESSOR column. If it displays 100% CPU or partial offload:
Fix: Ensure your user or the ollama system user has access to GPU compute groups:
```
sudo usermod -a -G video,render ollama
sudo systemctl restart ollama
```

2. CUDA Out of Memory (OOM) on Launch

Root Cause: The requested model layer weights plus context KV-cache exceed physical GPU VRAM.
Fix: Either downscale from 14b to 8b, or reduce num_ctx in your Modelfile from 32768 to 16384. Each 8k context window expansion adds approximately 1.2GB to 2.0GB to VRAM usage depending on model dimensions. For advanced PyTorch memory management, review fixing CUDA out of memory on Linux.

3. Model Skips `<think>` and Gives Shallow Answers

Root Cause: Setting temperature=0 or injecting aggressive system prompts that demand “Answer directly without thinking”.
Fix: Keep temperature between 0.5 and 0.7. DeepSeek R1 models rely on native reasoning tokens to structure complex answers. Do not force zero-shot format constraints into the system message.

4. Custom Weights and LoRA Fine-Tuning

If you plan to adapt DeepSeek R1 to proprietary codebases or private APIs, explore our guide on fine-tuning DeepSeek R1 distill models with Unsloth on Linux GPUs for memory-efficient training workflows.

Summary Reference

Lightweight / Laptop (8GB VRAM): ollama run deepseek-r1:8b
Mid-tier Workstation (12GB VRAM): ollama run deepseek-r1:14b
High-end Workstation (24GB VRAM): ollama run deepseek-r1:32b
System Config Location: /etc/systemd/system/ollama.service.d/override.conf
API Endpoint: http://localhost:11434/v1

Written by B. Cho

Systems & AI Infrastructure Engineer

Specializing in GPU inference engines (vLLM, SGLang, Ollama), CUDA memory allocation, and Model Context Protocol (MCP) infrastructure on Linux workstations.