Fully Autonomous Local Agent
I don’t believe in the good intentions of service providers. They pretend to offer win-win scenarios until you have no choice. Then it happens their way. This is why I am obsessed with having truly independent agents that could keep working even if I get banned or otherwise disconnected from popular services.
This direction led me to Chinese models early, before they became a hot thing around the internet (I started with the so-so M2.5 from MiniMax), and now I have finally reached a checkpoint of hosting the entire stack on my own machine.
Before I jump to the meat, here are the prerequisites: there are many trade-offs when talking about running LLMs. Taking into account my current situation, I went with the latest and best device I could afford for my daily work while keeping agents independent. Therefore, the agents received the leftovers.
The Harness
A place where a context and an environment meets a LLM model.
“Harness is everything” is a motto of 2026. It’s an opinionated statement, but it’s hard to disagree with because of the way software surrounding an LLM works with it. From simple chatting, we moved to software that is able to analyze the current context and act within a given environment. This is possible because of a harness — software that is connected to the environment and is able to pass the state of that environment (context) to a model (LLM).
So far, we mostly have four types of harnesses:
- chat — text/voice conversations. Conversation is context, context is conversation.
- coding — tools that are able to craft more condensed and specific context for coding tasks. Read files, gather metrics, use helper tools and common tools.
- work — a harness that is able to use office tools, mostly text, presentations, spreadsheets, etc.
- general purpose — a generic pipeline where you can craft your own environment and context, set the rules, directions, etc.
OpenClaw belongs to the general-purpose harness category. It can be a coding agent, PR agent, or office worker. It can do voice, text, or images. All you need is an idea, patience, and enough credits on your API account.
As a side note, I must mention that the DeepSeek harness looks very promising in terms of its design, but for me it was broken in too many ways, and I found it unusable for my scenario.
Home For an Agent
That’s why OpenClaw became my default go-to tool for experimenting and building pipelines. I already shared the Bob and Stew story, where I described the differences between OpenClaw and Hermes.
Eli Lap
Since then, one major change has happened: OpenClaw is actually running five agents. That’s always been the main advantage of OpenClaw — the ability to set up multiple isolated agents. Other than that, not much has changed. It’s still a VM. It still has full access. But the skills have been split, and personalities had to be reinvented.
So now there are Stew, Dave, Kevin, Tim and… Sparx. All living on the same floor. And Sparx is the one I want to tell you more about.
Privacy Island
Discussions about privacy in the context of LLM usage have always been present. The fact that everything has been parsed and analyzed doesn’t mean new things can’t be born. It doesn’t mean some relationships between different pieces of data can’t change. And it means one thing: we still have some data we want to protect from being exposed.
This often becomes the main argument in discussions about investing in hardware that’s capable of running LLMs on-premises. For exactly that reason, I gathered a small budget, rebuilt and upgraded an older PC to build a local LLM server.
It’s not much: 16 GB of RAM. i7-4770. RTX 2070. The cheapest possible option. That’s the way economics has always worked. Get the absolute minimum that does the job. And in September 2026, even this setup makes sense.
2 Aspects of Privacy
The first obvious aspect of privacy is removing calls to an external API for the LLM itself. For agents, it’s mostly about decomposing tasks and arranging tool calls. The smarter the model, the higher the level of abstraction it can swallow and determine the direction from. Not all tasks require such a level of “intelligence”. I am used to decomposing tasks myself to keep high-level control over the direction.
Another aspect came up during my latest OpenClaw upgrade. It’s memory and search. You see, OpenClaw runs a background process from time to time that processes all conversations that have happened and creates metadata for them.
That metadata consists of vectors that allow it to search for similar topics and terms and recall what was discussed previously. This allows it to “gather” knowledge and prevents the blank-page issue, when context must be rebuilt entirely from scratch.
During the last upgrade, I recalled that I had a plan to use self-hosted embeddings, which are much cheaper compared with running LLMs. At the end of the day, both the LLM and embedding models surprised me.
Model Choice
The RTX 2070 is a good starting point, yet providing only 8 GB of VRAM makes it super tiny. My goal was to have a usable setup in terms of processing speed vs. response quality. This led me to Qwen3.5-9B, which I’ve been testing locally on multiple machines. And it wasn’t good. The first run felt like it took ages to provide a response. Memory consumption was around 7 GB, leaving very little room for experiments.
For embeddings, I tried qwen3-embedding-0.6b. I thought such a tiny model should fit. And I was wrong. It was trying to allocate just under 5 GB of VRAM! So I thought it might be small enough to run on the CPU. Actually, it was, but processing around 1k elements across my agents was taking hours. Slow. Too slow!
And then I spotted a suggestion: Spark-X2.5-4B. A quick search looked promising for my needs. Great tool-calling and agent support. I switched with no regrets, and so far I am sure this is the best possible setup for an 8 GB system. The only downside is that it requires the latest (dev) llama.cpp at this point because it has its own architecture, which is unsupported in earlier releases. This made my setup a bit uglier, but I hope I can get back to it next month and that the latest releases will make it possible to run it directly. I ended up running it with a 256K context and Q4 quantization.
But embeddings still weren’t working. Therefore, I had to go back to nomic-embed-text-v1.5, which I’d experimented with earlier, running it CPU-only. The downside of this model is that it’s English-only, but it’s fast, reliable, and seems sufficient for my needs.
The RTX 2070 is enough to fit both models, and I ended up with speeds that made me happy and a fully autonomous agent.
Inference
For inference, I chose LocalAI, which provides a more convenient way to manage and maintain an LLM server. Although at this point it has a proxy for the Spark model, which is running on a llama.cpp server directly, it still provides a unified entry point for OpenClaw and other clients I may want to use in the future.
A brief introduction to LocalAI: it provides a ChatGPT-compatible API and has a modular architecture, so models and backends are modular and can be configured and run directly from the WebUI. Backends are containers with an engine + gRPC wrapper + YAML manifest file. It’s open source and written in Go, so I may get my hands dirty one day.
Final Thoughts
Sparx is now a separate agent. It’s running 24/7 and doing regular search/summary tasks on demand for me. It gets slower at some points. It has issues misinterpreting tool responses; for example, it broke the SearXNG config the moment I asked it to use it for search. Nevertheless, I know smaller models are still capable once they have specific instructions. Within fully controllable environments, that’s not a major problem.
Finally, I feel I’m one step closer to having a setup that can route and scale whenever needed, keeping tiny things local when tasks are clear. That’s the model I believe is sustainable in the upcoming future.
