How to Run an AI Writing Assistant Offline on a Laptop

What you are setting up

Two pieces are involved. The model runner is an ordinary desktop application that loads a model into memory and gives you a chat window, and usually a local connection that other programs on the same laptop can use. The model is a single large file downloaded once. Several free, open-source runners exist for all three major operating systems; this guide describes the steps they share rather than any one product, because their menus change often and the principles do not.

Set it up step by step

1. 01

   Check memory and storage

   Use the table above. If the laptop has 8 GB, choose the smallest models only (1 to 3 billion parameters) and close the browser while the assistant runs. The laptop specs guide for local AI explains the sizing in detail.

2. 02

   Install a model runner from its official source

   Download the installer only from the project's own website or its official code repository, not from a download portal or a video description. Check that the project is actively maintained: a release in the last few months and an open issue tracker are good signs.

3. 03

   Download a model that fits

   In the runner's model library, pick an instruction-tuned open-weight model of 7 or 8 billion parameters in a 4-bit quantised version; the file will be about 4 to 5 GB. Read the model's licence summary, which says whether commercial use is allowed. Prefer models from developers who publish what the model was trained for.

4. 04

   Test it with Wi-Fi switched off

   Turn Wi-Fi off, or unplug the network cable, and ask the assistant to rewrite a paragraph. If it answers, it is running locally. If it errors, the runner is still reaching the network for something and you should check its settings for any cloud or sync option and switch it off.

5. 05

   Connect it to your writing

   The simplest method is copy and paste. Many runners also open a local connection on the laptop that text editors and note apps can use through a plugin; the address points at the laptop itself, so the text never leaves it. Keep your drafts in your own files rather than in the runner's chat history.

6. 06

   Keep a record of your own work

   If you use the assistant for suggestions, save your draft before and after each session. The guide to keeping version history of drafts shows how to do that automatically.

What local models are good and bad at

Small local models are good at the mechanical parts of writing: tightening a paragraph, suggesting a clearer structure, turning notes into an outline, changing tone, and catching repetition. They are weaker than the largest cloud services at long reasoning and at recalling facts, and they will state wrong facts confidently, so treat anything factual they produce as a lead to check, not an answer. A model does not know anything that happened after its training data was collected.

Troubleshooting

• It is very slow. The model is probably too big for the memory and is spilling to the SSD. Choose a smaller model or a more heavily quantised version, and close other applications.
• The fans are loud and the laptop is hot. That is normal under this load. Use it on a hard surface with clear vents; see the laptop overheating guide for safe temperatures.
• It will not start after an update. Model file formats change occasionally. Update the runner first, then download the model again if it asks.
• Answers are cut off. Increase the maximum response length in the runner's settings, or ask it to continue.

More task guides live in the how-to hub. If a suggestion from the assistant ever needs defending, the guide to reading an AI detector score explains what those tools can and cannot show, and the print screen guide is a quick way to capture a setting before you change it. For keeping the whole setup private, read what AI writing tools send off the laptop.

Common questions

Is an offline AI assistant really private?

Once the model is downloaded and any cloud or sync options are off, the text you type stays on the laptop. Test it with the network disconnected to be sure.

Do I need a graphics card?

No. Runners use the processor and whatever graphics the laptop has. A graphics chip with its own memory makes suitable models faster.

Is it free?

The runners mentioned here are free and open source, and many open-weight models are free to use. Check each model's licence for commercial use.

Can it read my documents?

Only what you paste in or deliberately connect. Some runners can index a folder you choose; leave that off unless you want it.

Does my school or employer allow it?

Policies differ. A local tool avoids sending text to a third party, but it does not change the rules about whether AI help is permitted on a given piece of work.