Run Local AI on Windows with Ollama: Setup and Checks

Updated September 8, 2026 · Published by FarsiVid

You can run a small AI model on a Windows PC with Ollama, then check whether it handles your own tasks well enough to keep. Start with a modest model and a repeatable test. Download size, available memory and answer quality are different things; a model that launches successfully can still give you a confidently wrong answer.

This guide uses a small, named model as a learning example. It does not rank models by speed or claim that we benchmarked your hardware. The useful result is a working local setup and a record of what it can and cannot do.

Before installing: decide what should stay local

Write down one task, such as extracting dates from meeting notes or rewriting a short paragraph. Use made-up data for the first trial. If the task involves workplace documents, check your organization’s rules before copying them into any AI application.

Ollama supports both local and cloud models. Its FAQ explains the difference and the cloud-disable setting. For this local-only exercise, add the Windows user environment variable OLLAMA_NO_CLOUD with value 1, quit Ollama completely, and reopen it. The documented log message is Ollama cloud disabled: true. Keep its API bound to the default loopback address; exposing a model server to the internet is unnecessary for using it on your PC.

Local processing reduces one route by which prompts leave your computer. It does not protect against malware, shared Windows accounts, a third-party chat frontend that sends data elsewhere, or someone opening saved conversation files. Treat the whole setup as the privacy boundary.

Install Ollama and download one small model

1. Use the installer linked from the official Ollama Windows instructions. The standard Windows installation runs under your user account without requiring administrator access.
2. Open PowerShell after installation. Run ollama --version. If Windows cannot find the command, close and reopen PowerShell before investigating the installation path.
3. Run the command below. The first run downloads the model, so internet access is needed for that step. Let the download finish before judging response speed.

ollama run llama3.2:1b

The official llama3.2:1b entry lists this as a roughly 1.3 GB download at the time of review. It is an older, small text model selected to make the exercise manageable, not a claim about the best model available. Disk size is not total runtime memory: model loading, context and the rest of Windows also need space.

Start with a short prompt. If your PC becomes unresponsive, stop the run and close other memory-heavy applications. Do not purchase a graphics card based on a single failed attempt. Record the problem first: a download error, an out-of-memory error and a slow but successful answer call for different fixes.

Run three checks with answers you can verify yourself

Use the following original mini-test. Start a fresh conversation for each row so earlier answers do not give away the result. The expected answers are derived directly from the sample text, not from an observed model run.

Check	Prompt to try	What a pass requires
Extraction	From this note, return only the event date: “The workshop is on 18 September 2026. Registration closes on 12 September 2026.”	18 September 2026, without confusing it with registration.
Missing information	Use only this note: “The workshop is on 18 September 2026.” What room is it in? If absent, say “Not stated.”	Not stated. Any invented room fails.
Simple calculation	A fictional invoice has 3 cables at $8 each and shipping of $5. No tax applies. Give the total and the calculation.	$29, with 3 × 8 + 5 shown correctly.

Repeat a failed check once with the same wording and record both answers. Do not count a corrected answer after you supplied the solution as a pass. For an extraction task, one invented field can matter more than an elegant paragraph; choose your acceptance rule before reading the response.

Next, replace the workshop note with a short, non-sensitive example from your intended use. Keep an answer key that you wrote yourself. Three easy checks establish a baseline, not proof that the model is reliable for every document.

Separate performance problems from answer problems

The Ollama CLI reference documents commands for listing downloaded models and active runs. In another PowerShell window, ollama ls shows what is installed and ollama ps shows what is loaded. To unload the example, use ollama stop llama3.2:1b; to remove its downloaded model, use ollama rm llama3.2:1b.

Symptom	Useful next check
First response takes much longer	Separate initial download/loading from subsequent replies before comparing timings.
Short prompts work; large documents fail	Reduce input length and record memory use. Change one variable at a time.
Replies are fast but inaccurate	Review the answer-key failures. Faster hardware does not establish factual correctness.
The PC gets hot or slows during sustained use	Check airflow and resource use; our GPU thermal-throttling guide explains how to distinguish heat symptoms from guesses.

Keep a small evaluation log

Record the date, Ollama version, exact model tag, RAM, GPU if present, prompt, response, pass/fail reason and whether the model was already loaded. If you time replies, state how you measured them and keep the input unchanged. This makes a later model comparison meaningful without pretending that results from one computer apply to all PCs.

For code-related work, add compilation or test results using our AI coding assistant evaluation workflow. For documents where you want to inspect citations back to sources, compare the Gemini Notebook source-checking exercise; that is a separate cloud workflow with a different privacy decision.

The stopping point is practical: keep the local setup if it solves the selected task with acceptable errors and resource use. If it fails your checks, retain the log and change one model or one task assumption, rather than quietly lowering the standard.