Best Local AI Models for a 16GB GPU

A 16GB graphics card can run capable local models for chat, writing, coding, image understanding and generation, private document work, and short videos. This guide targets the GeForce RTX 5070 Ti, with an emphasis on models that run well rather than the largest ones that can be forced to load.

Recommended Models

Quick favors speed and headroom, Primary is the recommended everyday balance, and Ceiling favors greater capability while accepting slower CPU/RAM offloading.

General Assistant

Tier	Recommendation
Quick	Qwen3.5 9B Q4
Primary	Qwen3.5 9B Q8
Ceiling	Qwen3.6 27B Q4

Coding / Repositories / Tools

Tier	Recommendation
Quick	Qwen3-Coder 30B-A3B UD-Q2
Primary	Qwen3-Coder 30B-A3B UD-Q3
Ceiling	Qwen3-Coder 30B-A3B UD-Q4

Specialized Models

Role	Recommendation
Conversation & Persona	Hermes 4 14B Q5
Creative Writing	Gemma 4 12B StyleTune Q6
Photo Generation	HiDream-O1 Image MXFP8
Video Generation	Wan 2.2 TI2V 5B FP16

The specialized table gives one everyday recommendation per role. Exact identifiers, download sizes, and runtime guidance appear in the detailed sections below. Language models run through Ollama; photo and video use ComfyUI.

Ollama residency guidance assumes a 16K context, one loaded model, one request at a time, and no other GPU-heavy workload. ComfyUI guidance assumes a current release, batch size 1, typical 1024px images, or short 720p clips.

Hardware and Selection Rules

The target PC uses an RTX 5070 Ti with 16GB of VRAM, a Ryzen 5 7600, and 32GB of RAM on Ubuntu. VRAM is the main constraint. GPU-resident means the model weights remain in VRAM; CPU/RAM offload means some use slower system memory. Offload is intentional for the two language-model Ceilings, while ComfyUI automatically moves media-workflow parts as needed. With 64GB of system RAM, larger image and video models or heavier workflows would become more practical, so the photo and video recommendations could be different.

Each language model runs locally through Ollama and was selected for its role rather than simply for being the largest model that would load. The General Assistant selections accept images. Qwen3-Coder is text-only, which suits code, files, terminal output, repositories, and tool results; use a General Assistant model when a coding task starts with a screenshot.

Running the Ollama Models

Install Ollama for Linux (opens in a new tab), keep Ollama and the NVIDIA driver current, and confirm that nvidia-smi lists the RTX 5070 Ti. The commands below assume Ollama’s standard Ubuntu systemd service.

These settings establish the 16K baseline used by the tables. Flash Attention and the Q8 cache (opens in a new tab) reduce its memory cost, while the concurrency limits keep model loading predictable. Run sudo systemctl edit ollama.service and add:

[Service]
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="OLLAMA_CONTEXT_LENGTH=16384"
Environment="OLLAMA_NUM_PARALLEL=1"
Environment="OLLAMA_MAX_LOADED_MODELS=1"

Apply the change:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Run any language-model identifier from its description with ollama run MODEL_ID; Ollama downloads it if needed. Vision-capable models also accept an image path in the prompt. After loading, use ollama ps to confirm the context and GPU/CPU split; connected apps may override the 16K default.

Running the ComfyUI Models

On Ubuntu, install or update ComfyUI (opens in a new tab), then open Workflow → Browse Templates. The linked model guides below identify the official workflow, checkpoint, and supporting files.

Before generating, close other GPU-heavy applications and free any model held by Ollama with ollama stop MODEL_ID. Start with batch size 1, a 1024px image, or a short 720p clip, then increase resolution or length only after confirming a stable run.

General Assistant

Qwen combines broad reasoning, tools, languages, and vision, making it the strongest general-purpose fit for these tiers.

Quick: Qwen 3.5 9B

qwen3.5:9b-q4_K_M (opens in a new tab) is the 6.6GB fast choice for chat, summaries, lightweight reasoning, tools, and visual questions. It retains Qwen 3.5 9B’s strong general and multimodal capabilities while leaving substantial GPU headroom.

Primary: Qwen 3.5 9B Q8

qwen3.5:9b-q8_0 (opens in a new tab) is the 11GB everyday choice for planning, documents, structured output, tools, languages, and images. Q8 retains more fidelity than Q4 and is expected to remain fully GPU-resident, although it has less spare capacity than the other 9B selections. The official model card (opens in a new tab) shows strong results across knowledge, reasoning, instruction following, tools, languages, and vision.

Ceiling: Qwen 3.6 27B

qwen3.6:27b-q4_K_M (opens in a new tab) is the 17GB choice for difficult reasoning, planning, and visual work. Its package plus runtime memory produces the small CPU/RAM offload shown in the table. In the official direct evaluation (opens in a new tab), this dense 27B model leads the larger 35B A3B variant on most relevant measures while requiring much less offload.

Coding, Repositories, and Tools

Qwen3-Coder 30B-A3B is used for all three tiers. It has 30.5B total parameters but activates only 3.3B at a time, combining broad repository knowledge with efficient generation. The official model card (opens in a new tab) highlights agentic coding, tool use, browser use, and repository-scale context.

Quick: Qwen3-Coder 30B-A3B UD-Q2

hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:UD-Q2_K_XL (opens in a new tab) is the 11.8GB fast choice for code questions, focused edits, debugging, shell commands, and lightweight tool use. It should remain fully GPU-resident at the article’s 16K baseline, trading some fidelity for speed and headroom.

Primary: Qwen3-Coder 30B-A3B UD-Q3

hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:UD-Q3_K_XL (opens in a new tab) is the recommended 13.8GB balance for repository work, multi-file changes, debugging, terminal tasks, and repeated tool calls. It keeps more model detail than UD-Q2 while still targeting full GPU residency with tighter headroom.

Ceiling: Qwen3-Coder 30B-A3B UD-Q4

hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:UD-Q4_K_XL (opens in a new tab) is the 17.7GB ceiling for difficult repository analysis, larger patches, and complex tool-driven tasks. It preserves more fidelity but requires some CPU/RAM offload on a 16GB card, making it slower than the resident UD-Q2 and UD-Q3 builds.

Specialized Models

Conversation and Persona: Hermes 4 14B

hf.co/bartowski/NousResearch_Hermes-4-14B-GGUF:Q5_K_M (opens in a new tab) is a 10.51GB quant tuned for natural conversation, persona adherence, personal-assistant behavior, and roleplay. This Ollama build is text-only, so use a General Assistant selection when images matter.

Creative Writing: Gemma 4 12B StyleTune

hf.co/mradermacher/Gemma-4-12B-StyleTune-GGUF:Q6_K (opens in a new tab) is a 10.6GB text model for fiction, characters, dialogue, scenes, prose, roleplay, and longer storytelling. The creator’s 200-prompt evaluation (opens in a new tab) reports 56% fewer clichés than the base instruct model. Q6 provides very good quality while leaving useful runtime headroom.

Photo Generation: ComfyUI

HiDream-O1-Image Full MXFP8 (opens in a new tab), using hidream_o1_image_mxfp8.safetensors, is the single photo recommendation. The full 8B model handles text-to-image generation, image editing, subject references, storyboards, and text-heavy layouts at up to 2048×2048. Its developer-published evaluation (opens in a new tab) leads FLUX.2 Dev and Z-Image Turbo on human preference, composition, and dense prompt alignment. The 8.92GB checkpoint is a healthy fit at 1024px; a 16GB test (opens in a new tab) also ran it at 2048×2048.

Video Generation: ComfyUI

Wan2.2 TI2V 5B FP16 (opens in a new tab), using wan2.2_ti2v_5B_fp16.safetensors, is the single video recommendation. One checkpoint handles both text-to-video and image-to-video at 720p and 24 fps, with strong motion, prompt adherence, and cinematic control. ComfyUI says the 5B workflow fits well on an 8GB card because it automatically moves parts that are not currently in use to system RAM. That makes 16GB of VRAM and 32GB of RAM a healthy fit.