Run Llama 3.1 / 3.3 Locally: Complete Guide for Windows, Mac & Linux

<div className="tldr-box"> <h3>Key Takeaways</h3> <ul> <li>Overview Meta's Llama 3.1 and Llama 3.3 models represent the state of the art in openweight language models.</li> <li>Llama 3.1 comes in 8B, 70B, and 405B sizes.</li> <li>Llama 3.3 improved the 70B model to match the 405B's quality at a fraction of the compute cost.</li> <li>This guide covers running these models on your own hardware — whether you have a gaming PC, a MacBook, or a Linux server.</li> </ul> </div>

Overview

Meta's Llama 3.1 and Llama 3.3 models represent the state of the art in open-weight language models. Llama 3.1 comes in 8B, 70B, and 405B sizes. Llama 3.3 improved the 70B model to match the 405B's quality at a fraction of the compute cost.

This guide covers running these models on your own hardware — whether you have a gaming PC, a MacBook, or a Linux server.

Model Comparison

| Model | Parameters | Quality | Hardware Needed | Best For | |---|---|---|---|---| | Llama 3.1 8B | 8B | Good | 8 GB RAM | Basic tasks, chatbots | | Llama 3.1 70B | 70B | Excellent | 40 GB RAM (Q4) | Complex reasoning, writing | | Llama 3.1 405B | 405B | State-of-art | 220 GB RAM (Q4) | Research, enterprise | | Llama 3.3 70B | 70B | Matches 405B | 40 GB RAM (Q4) | Best quality-to-size ratio |

Hardware Requirements

Llama 3.1 8B

| Quantization | RAM Needed | VRAM Needed | Quality | |---|---|---|---| | Q4_K_M | 6 GB | 6 GB | Excellent (recommended) | | Q5_K_M | 7 GB | 7 GB | Near-lossless | | Q8_0 | 9 GB | 9 GB | Lossless | | FP16 | 16 GB | 16 GB | Full precision |

Llama 3.3 70B

| Quantization | RAM Needed | VRAM Needed | Quality | |---|---|---|---| | Q2_K | 28 GB | 28 GB | Usable (reduced quality) | | Q3_K_M | 33 GB | 33 GB | Good balance | | Q4_K_M | 40 GB | 40 GB | Excellent (recommended) | | Q5_K_M | 48 GB | 48 GB | Near-lossless | | Q8_0 | 70 GB | 70 GB | Lossless |

Llama 3.1 405B (Q4_K_M)

• Minimum: 220 GB unified memory (Mac Studio / AMD Ryzen AI Max)
• Multi-GPU: 10x RTX 3090 or 5x A6000
• CPU only: 256 GB RAM, very slow (0.5–1 tok/s)

Method 1: Ollama (Easiest)

Ollama is the simplest way to run Llama models on any platform.

Installation

# Windows: Download from ollama.com
# macOS: brew install ollama
# Linux: curl -fsSL https://ollama.com/install.sh | sh

Pull Llama Models

# Llama 3.1 8B (4.7 GB)
ollama pull llama3.1:8b

# Llama 3.3 70B (40 GB)
ollama pull llama3.3:70b

# Llama 3.1 405B (230 GB Q4)
ollama pull llama3.1:405b

Run Inference

# Interactive chat
ollama run llama3.3:70b

# One-shot generation
ollama run llama3.3:70b "Explain neural networks to a 10-year-old"

API Mode

import requests

response = requests.post("http://localhost:11434/api/generate", json={
    "model": "llama3.3:70b",
    "prompt": "Write a Python function to sort a list",
    "stream": False
})
print(response.json()["response"])

Method 2: LM Studio (GUI-Friendly)

LM Studio provides a graphical interface for downloading, configuring, and running Llama models.

Installation

• Download from lmstudio.ai
• Install and launch

Steps

1. Search for "Llama 3.3 70B" in the model browser
2. Download a GGUF version (recommended: Q4_K_M)
3. Load the model — click the model and select a quantization
4. Configure GPU offloading — drag the GPU slider to max
5. Chat in the built-in interface
6. Enable API — start the local inference server for API access

Configuration Tips

| Setting | Recommendation | |---|---| | GPU offload | 100% (all layers on GPU if VRAM allows) | | Context length | Start at 4096, increase if you have VRAM | | Batch size | 512 for GPU, 32 for CPU | | Thread count | 4–8 for CPU offloading |

Method 3: llama.cpp (Advanced)

For maximum control and performance, use llama.cpp directly.

Build

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

# Windows (CMake)
mkdir build && cd build
cmake .. -DLLAMA_CUBLAS=ON
cmake --build . --config Release

# macOS (Metal)
make LLAMA_METAL=1

# Linux (CUDA)
make LLAMA_CUDA=1

Download GGUF Files

# Download Llama 3.3 70B Q4_K_M from Hugging Face
wget https://huggingface.co/bartowski/Llama-3.3-70B-Instruct-GGUF/resolve/main/Llama-3.3-70B-Instruct-Q4_K_M.gguf

Run

./main -m Llama-3.3-70B-Instruct-Q4_K_M.gguf \
  -n 512 \
  -p "What is machine learning?" \
  -ngl 99 \
  -t 8

Flags:

• -n: Number of tokens to generate
• -p: Prompt
• -ngl: GPU layers (99 = all)
• -t: Thread count

Performance Optimization

Batch Prompt Processing

For processing many prompts at once, increase batch size:

ollama run llama3.3:70b --num-batch 1024

Flash Attention

Enable Flash Attention for faster inference with long contexts:

# In LM Studio: Settings → Advanced → Enable Flash Attention
# In llama.cpp: add --flash-attn flag

Context Length Tradeoffs

| Context Length | VRAM (Llama 3.3 70B Q4) | Use Case | |---|---|---| | 2,048 | 38 GB | Simple Q&A, chat | | 8,192 | 40 GB | Document analysis | | 32,768 | 48 GB | Code understanding | | 131,072 | 80 GB | Full book analysis |

FAQ

Can I run Llama 3.3 70B on a single RTX 4090?

No — 24 GB VRAM is insufficient. You need at least 40 GB (Q4). Use CPU offloading (slower) or a dual-GPU setup.

What's the best value setup for Llama 3.3 70B?

A Mac Studio with 64 GB ($3,999) or an AMD Ryzen AI Max 128 GB ($2,999) mini PC. Both run the 70B at Q4_K_M with decent speed.

How fast will it run?

On a Mac Studio M2 Ultra (192 GB): ~15 tok/s. On dual RTX 4090: ~20 tok/s. On CPU only (64 GB RAM): ~2–4 tok/s.