Thoughts and Experiences on Developing Lightweight AI Agents

Recently, the popular Pi Agent gained attention for being small, lightweight, and highly scalable. Inspired by this, I explored its basic approach and developed a similarly lightweight agent named Zlite using Golang (https://github.com/helloxz/zlite). This article shares my thoughts and experiences regarding AI agent development.

Note: The following content is based on my personal development experience and understanding of Agent frameworks. It may not be entirely accurate; please correct me if there are any errors.

Why Develop Zlite?

I have been searching for a lightweight and simple agent for less complex scenarios (such as managing Nginx). Although Pi Agent meets these conditions, it relies on the Node.js environment, which still feels too heavy for server deployment in my case. Thus, I decided to develop a lightweight agent called Zlite using Golang.

From concept to implementation, with AI assistance, it took about two days to build Zlite. During this process, I learned a lot, so I organized my knowledge and experiences into the following content.

What is an AI Agent?

We can think of AI as a brain: it can think but lacks limbs, meaning it has no ability to manipulate objects. Initially, it could only perform basic Q&A. When we equip AI with "limbs" to execute specific operations—such as reading web pages, sending emails, or placing orders—it becomes a complete agent.

Typical examples include comprehensive agents like Hermes Agent, OpenClaw, and WorkBuddy, which assist with office work, computer operation, and coding. There are also domain-specific agents optimized for coding, such as Codex CLI, Claude Code, and Deepseek Harness.

Essentially, an agent is "Brain + Limbs." The Large Language Model (LLM) serves as the "brain," while the "limbs" are Tool Calls. OpenAI supported Function Calling early on, enabling tool invocation capabilities that have now become an industry standard. We can view Function Calling as the hands and feet that empower the AI.

Once equipped with limbs, the AI becomes an agent capable of more tasks, forming a complete loop:

1. User inputs content.
2. AI brain thinks and sends instructions.
3. Limbs (Function Calling) execute actions.
4. Specific tools/functions complete the task.
5. Results are returned to the AI brain.
6. The AI brain organizes and outputs the result to the user.

Tool Functions

A basic agent typically implements the following tool functions for AI invocation:

• Read file content
• Modify files
• Delete files
• Execute shell commands

Expanding further, we can implement additional tools:

• Web Fetch: Allows AI to read content from a specific link directly.
• Web Search: Enables real-time search for latest content based on keywords, usually via third-party APIs.

In a single conversation turn, the AI infers user intent and calls necessary tools from the injected set, then organizes the execution results before outputting them to the user.

Note: A single conversation turn often involves multiple sequential tool function calls.

Once you understand tool function invocation, you can build a primitive, simple agent. Many AI SDKs now have built-in tool calling capabilities, so deep technical understanding isn't always required.

Content Storage

Since LLMs are stateless and do not remember previous conversations, we must store dialogue content for each window. When continuing a conversation, we manually concatenate historical messages and send them back to the AI.

Where should we store this data? Common approaches include:

• Memory: Fast and efficient, but data is lost upon restart. Unsuitable for long-term storage.
• Local Files: For small projects, storing dialogue locally is simple and reliable. Many CLI agents use the .jsonl format.
• Database: For medium-to-large projects, local files may be limiting. Professional databases like SQLite or PostgreSQL are preferred.
• Hybrid Storage: Some special scenarios may combine local storage with database storage.

For a lightweight agent, local file storage is usually sufficient. The recommended format is .jsonl. I hadn't encountered .jsonl until developing AI agents.

Simply put, jsonl is a collection of multi-line JSON data. If each conversation turn is organized as a JSON object, multiple turns become multiple JSON objects saved line-by-line in a .jsonl file.

How to Implement Modes?

If you've used coding tools like OpenCode, you might notice modes like plan and build. While naming conventions vary, their essence is the same:

• Plan: Read-only mode; cannot modify, edit, or delete files.
• Build: Execution mode; higher permissions, allowing direct file operations without user consent.

Initially, I didn't understand how these modes were implemented. After discussing with AI, I realized it's essentially "Prompts + Conditional Tool Exposure."

For example, if the frontend passes a parameter value of plan, the backend injects a prompt telling the AI it is in plan mode and can only perform read-only operations. However, relying solely on prompts is unreliable as some models may ignore them. Therefore, a fallback mechanism is needed.

We can wrap the basic tools (read, modify, delete, shell) into a tool set. In plan mode, only read operations and limited shell commands are exposed. In build mode, the full set of functions and more shell commands are available. Consequently, in plan mode, the AI simply cannot call a delete function because it doesn't know the function exists.

Interaction Entry Points

To interact with AI via natural language, we need an entry point. Common methods include:

• WEB: Send content directly via a web page.
• Client: Send content through a desktop/mobile client.
• TUI: Interact via a terminal interface.

For instance, Deepseek's recent Deepseek Harness uses a WEB interface, Tencent's WorkBuddy uses a client, and Claude Code initially used a TUI interface (now also offering a desktop version). Entry point forms are diverse.

Context Compression

LLM context windows are finite (e.g., 512K, 1M tokens). While technology improves, true infinite length is impossible. To achieve "fake infinite" conversations, we use context compression: after many turns, the AI summarizes the conversation, and the summary is carried forward. However, this risks losing key information, sometimes causing the AI to appear less intelligent when the window nears capacity.

Common Context Compression Strategies:

• No Compression: Simply truncate old messages once a threshold (turns or tokens) is reached. Simple but violent; the AI forgets initial content.
• Smart Compression: After a certain number of turns (e.g., 60), summarize the first half (e.g., 30 turns) using AI. The new context becomes "Summary + Remaining Turns." More complex but better quality, though summarization may still lose details.
• Semantic Recall: Uses vector models to embed content into a vector database. During conversation, the AI retrieves relevant past content via semantic search. Complex but effective for long dialogues, though quality may still drop with very large windows.
• Others: There may be more advanced methods; feel free to share them.

Implementing Skills (Skills)

You may have heard of AI "skills" or skills. Many desktop agents feature a skill center for one-click installation. Essentially, skills are prompt injections that help AI understand and regulate behavior.

However, injecting entire skill contents into prompts makes them overly verbose and wastes tokens. A clever design uses "Metadata + Full Content":

• Metadata: A brief description of what the skill does. Only this is injected into the prompt.
• Full Content: Includes usage scenarios, steps, and examples.

The system injects metadata to let the AI know the skill exists. When the conversation actually requires a specific skill, the AI retrieves the full content.

More skills do not always mean better capability. Each skill consumes context windows (tokens) and increases "selection confusion" during invocation. Too many skills can interfere with each other, reducing response speed and accuracy.

Design Optimization:

Consider controlling skills via "Global Skills + Project Skills + Limit Total Count" rather than allowing unlimited installations.

AI Inquiry Interaction

Using Coding CLI, you might notice AI pop-ups asking for choices. I was curious how AI handles user waiting given HTTP timeouts.

It turns out this is a facade. When AI sends an inquiry, it follows a specific format, ending that turn. The frontend captures this format, displays a popup for user selection, and then resumes by sending a new concatenated turn back to the AI. The frontend makes it look like a continuous interaction within one turn.

ACP Protocol

Agent Client Protocol (ACP) is an open protocol designed to standardize communication between AI programming assistants (Agents) and clients. Supporting ACP allows any compatible client (e.g., Zed, VS Code, Zacp, Codex) to connect and communicate with your agent.

ACP support is optional. For greater universality, consider implementing it. SDKs exist for various languages, making integration straightforward without needing deep initial understanding.

MCP Protocol

MCP (Model Context Protocol) is an open standard aiming to unify connections between AI models and external data sources/tools, allowing AI assistants to access any resource via a standardized interface. Think of it as the "USB-C of AI"—a universal protocol for plug-and-play connectivity. — Content cited from AI

Currently, I primarily use the MCP framework to call MCP servers and don't have a deep understanding of the internals. Generally, developing an Agent requires only the ability to call MCP, not to implement MCP capabilities (which belong to the server side). For a lightweight agent, MCP calling is also optional.

Conclusion

After understanding the general framework and ideas of agents from this article, you should be able to command AI to develop your own lightweight agent. However, to make it truly useful rather than just functional, continuous optimization of details and bug fixes are necessary.

Additionally, the lightweight agent Zlite developed by me is open-sourced on GitHub: https://github.com/helloxz/zlite. Interested readers are encouraged to explore and deconstruct it with AI assistance. I hope this helps.