I tested open source AI alternatives against ChatGPT and Gemini; this local tool proved most reliable

Since I have been studying and using ChatGPT and Google Gemini, I have been interested in finding alternatives, mainly to see how the quality differs between the LLM models.

One issue I sometimes run into is privacy. The more I try different models, the more personal data I potentially give up on the cloud.

But I've explored other open source alternatives to run locally on my computer, since I recently upgraded my desktop.

However, running a local open source model means you need powerful hardware to support it.

So, you can't do this the same way even on a beefy Android phone (the options are limited). You will most likely need a computer with sufficient RAM.

I was able to do this, and since I want to return to learning coding, I'll have an indispensable resource to do it with.

How I set up a local LLM on my desktop computer

Check if your device can handle it

Keep expectations in check when running open source, open-weight LLMs; most are specialized, have focused reasoning, and can only run as well as your hardware can handle.

ChatGPT and Gemini excel at general use because they run on massive data centers (supercomputers in the cloud).

The only costs are the hardware and your electricity bill.

You can download a lighter model that doesn't put your laptop under as much strain. Or, if you are feeling more confident, you can tackle the bigger LLMs.

Still, those will push your VRAM/RAM requirements high, at least needing 24GB of RAM to run, which is usually fine on a desktop since more have gone toward the 32GB mark.

You can set up an open-weight model or an open source one. If you value privacy and transparency most, you'll want to stick to open source ones.

To get started, I used an application to run the model. Since I use Windows 11, I chose LM Studio (about a 2GB install).

Plus, it's easy to navigate because it doesn't require intricate coding knowledge or command prompts (Ollama uses command prompts).

Next, since running LLM-based AI is demanding on RAM, you'll want to determine how much load you can handle. My desktop computer has 32GB, which can comfortably run 8- to 14-billion-parameter models.

Still, if you're using 16GB (like a lot of laptops), you might want to aim for the 7- to 9-billion-parameter range.

For a safe pick on a lightweight model, Gemma 4 by Google is a solid choice for reasoning, image input, and tool calling.

If you aren't sure what your computer can handle, LM Studio will indicate whether it can, with a green indicator next to the files when you add one.

You will also want to make sure there is a blue checkmark badge to ensure you install trustworthy models. The blue checkmark badge indicates that the author has been verified.

The model I picked was DeepSeek-R1-0528-Qwen3-8B,which, according to the author's readme, is considered "a distilled version of the DeepSeek-R1-0528 model, created by continuing the post-training process on the Qwen3 8B Base model using Chain-of-Thought (CoT) from DeepSeek-R1-0528."

I'm interested in DeepSeek and Qwen. I have used models from both, and they have performed well, so I wanted to see if it could rival larger LLMs in efficiency, as the readme claims.

It also specializes in math/coding. As a beginner, I see it as being a good tool (a few of my friends have mentioned that both models are what they use for debugging).

The download size was around 5GB, so it wasn't that big compared to some that can easily stretch to over 16GB.

I also chose a second model because I wanted image inputs to work as well, and a model that worked better for general use, like when you need help with your files or documents. For that, I chose Gemma 4 12B optimized with Quantized Aware Training.

From what I tested, I preferred treating the DeepSeek-R1-0528-Qwen3-8B model like a homework helper, asking it specific questions on how to make assets for a game in Unreal Engine, and then using the Gemma 4 model for its multimodal capabilities.

Why use a local LLM tool over public ones?

Spread the wealth to stay productive

You have to keep your expectations in check with open source and open-weight AIs. But the benefits outweigh the cons as long as you know what you plan to use it for.

Beyond the transparency of those models, you don't have to deal with the hassle of monetization since you host the model on your hardware and can keep your data private.

That's especially good for those who work with sensitive data. Running a model locally keeps the information on your private server and lets you use it offline .

The con is that these LLM tools don't perform at the peak level of popular closed models, and any training needed to improve the tool is your responsibility (fine-tuning).

But for someone like me who wants to use it as a hobby, that's fine, because I want to have my own AI tool as a resource when I am trying to learn coding, as I would like to build a website for my mother, as well as make my own game and app.

As someone who doesn't have much knowledge, I need constant back-and-forth while I am learning, which is where AI comes in.

Eventually, I would like something similar for learning a language without having to pay a subscription.

My experiences with the Gemma 4 and DeepSeek-R1-0528-Qwen3-8B models have been positive. Though I would never use them conversationally, as neither model is up to date.

But for learning fundamentals that have been circulating for the past few decades and a half, it works.

I also thought the way both models showed their reasoning was clear, since it gave me a breakdown in sections or a step-by-step analysis.

I fed the same prompt to Gemini and ChatGPT, and both preferred to spit out tables or a summary as the default answer.

Having the open models give me long-form answers made it easier to see where in the statement it might be hallucinating , since that's a common problem with any model.

Sometimes the way ChatGPT and Gemini portray answers appears very believable on the surface.

Also, with both of these models, I can see their reasoning in real time. Whether the AI believes my statements, checks certain databases, and raises doubts.

That helped me adapt my prompt-writing so I can present my next few prompts properly.

Practice makes perfect even with AI

Save your Gemini tokens for your smartphone.

While options for hosting local LLMs are limited on a phone, using Gemini on your mobile device does wonders. You can use the AI more conversationally or create custom Gems that improve your workflow .