Thoughts and Experiences on Developing Lightweight AI Agents

Recently, the popular Pi Agent gained attention for being small, lightweight, and highly scalable. Inspired by this, I explored its basic approach and developed a similarly lightweight agent named Zlite using Golang. This article shares my thoughts and experiences in AI agent development.

Note: The following content is based on my personal development experience and understanding of Agent frameworks. It may not be entirely accurate; please correct any errors.

Why Develop Zlite?

I have been looking for a lightweight and simple agent for less complex scenarios (such as managing Nginx). Although Pi Agent meets these conditions, it relies on Node.js, which I consider too heavy for server deployment. Thus, I decided to develop a lightweight agent called Zlite using Golang.

From idea to implementation, with AI assistance, it took about two days to create Zlite. During this process, I learned a lot, which I have organized into the following content.

What is an AI Agent?

We can think of AI as a brain: it can think but lacks limbs, meaning it cannot perform physical actions. Initially, it could only engage in basic Q&A. When we equip AI with "limbs" to execute specific operations—such as reading web pages, sending emails, or placing orders—it becomes a complete agent.

Typical examples include comprehensive agents like Hermes Agent, OpenClaw, and WorkBuddy, which assist with office tasks, computer operations, and coding. There are also domain-specific agents optimized for coding, such as Codex CLI, Claude Code, and Deepseek Harness.

Essentially, an agent is "Brain + Limbs." The large language model (LLM) serves as the brain, while the limbs are tool calls. OpenAI introduced Function Calling early on, enabling tool usage, which has now become an industry standard. We can view Function Calling as the limbs that give AI hands and feet.

Once equipped with limbs, an agent can perform more tasks, such as reading web pages, sending emails, or shopping. This forms a complete loop:

1. User inputs content.
2. AI brain thinks and sends instructions.
3. Limbs (Function Calling) execute the task.
4. Specific tools/functions complete the action.
5. Results are returned to the AI brain.
6. AI brain organizes and outputs the result to the user.

Tool Functions

A basic agent typically implements the following tool functions for AI invocation:

• Read file content
• Modify files
• Delete files
• Execute shell commands

Expanding further, we can add more tool functions:

• Web Fetch: Allows AI to read content from a specific URL directly.
• Web Search: Enables real-time search based on keywords, usually implemented via third-party APIs.

In a single conversation session, the AI infers user intent and invokes necessary tool functions from the injected set, then organizes the execution results before outputting them to the user.

Note: A single conversation session often involves multiple sequential tool function calls.

Once you understand tool function calling, you can build a primitive, simple agent. Many AI SDKs now have built-in support for this, so deep knowledge of the underlying mechanics isn't always required.

Content Storage

Since LLMs are stateless and do not remember past conversations, we must store dialogue content for each window. When continuing a conversation, we manually concatenate historical messages and send them back to the AI.

Where should we store this data? Common approaches include:

• Memory: Fast and efficient, but data is lost upon restart; unsuitable for long-term storage.
• Local Files: Simple and reliable for small projects. Many CLI agents store data locally in .jsonl format.
• Database: For medium-to-large projects, local files may be limiting. Professional databases like SQLite or PostgreSQL are preferred.
• Hybrid Storage: Some special scenarios may use a combination of local storage and databases.

For a lightweight agent, local file storage is usually sufficient. The recommended format is .jsonl.

Simply put, jsonl is a collection of multiple lines of JSON data. If each conversation turn is organized as a JSON object, multiple turns form multiple JSON objects saved line-by-line in a jsonl file.

How to Implement Modes?

If you've used coding tools like OpenCode, you might notice modes like plan and build. While naming conventions vary, their essence remains the same:

• Plan: Read-only operation; no file modification, editing, or deletion.
• Build: Execution mode with higher privileges, allowing direct file operations without user consent.

Initially, I didn't understand how these modes were implemented. After discussing with AI, I realized it's essentially "Prompt Engineering + Conditional Tool Exposure."

For example, if the frontend passes a parameter value of plan, the backend injects a prompt telling the AI it is in plan mode and can only perform read-only operations. However, relying solely on prompts is unreliable as some models may ignore them. Therefore, a fallback mechanism is needed.

We can wrap the basic tool functions (read, modify, delete, shell) into a set. In plan mode, only read operations and limited shell commands are exposed. In build mode, the full set of functions and more shell commands are available. Consequently, in plan mode, the AI simply cannot call a delete function because it doesn't know it exists.

Interaction Entry Points

To interact with AI via natural language, we need an entry point. Common methods include:

• WEB: Send content to AI directly through a webpage.
• Client: Send content via a desktop or mobile client application.
• TUI: Interact through a terminal interface.

For instance, Deepseek's recent release, Deepseek Harness, uses a WEB interface. Tencent's WorkBuddy uses a client interface. Claude Code initially used a TUI interface (though it now offers a desktop version). Entry points vary widely.

Context Compression

LLM context windows are finite (e.g., 512K, 1M tokens). While technology is advancing, true infinite length is impossible. To achieve "fake infinite" conversations, we use context compression.

What is fake infinite conversation? It involves compressing dialogue content. As conversation rounds increase, the AI summarizes previous interactions, and the summary is carried forward. However, this risks losing key information, potentially causing the AI to appear less intelligent when the window nears capacity. Thus, context compression is not foolproof but necessary due to window limits.

Common Context Compression Strategies:

• No Compression: A brute-force approach where initial dialogues are truncated once a certain number of rounds or token count is reached. Simple but ineffective as the AI forgets early content.
• Smart Compression: After a set number of rounds (e.g., 60), the program logic summarizes the first half (e.g., 30 rounds) using AI. The new context becomes "Summary + Remaining Rounds." This improves quality slightly but still risks information loss.
• Semantic Retrieval: Uses vector models to embed input content into a vector database. During conversation, the AI retrieves relevant matches via semantic search alongside the context window. This is complex but effective for long conversations, though quality may still drop with very large windows.
• Others: More advanced methods exist; feel free to share them.

Implementing Skills (Skills)

You may have heard of AI skills (skills). Many desktop agents feature a skill center for one-click installation. Essentially, skills are prompt injections designed to help AI understand and regulate behavior.

However, injecting entire skill descriptions into prompts wastes tokens and makes prompts overly verbose. A clever design uses "Metadata + Full Content":

• Metadata: A brief description of what the skill does. Only this is injected into the prompt.
• Full Content: Includes usage scenarios, steps, and examples. This is loaded only when the AI actually needs the skill.

More skills do not necessarily mean better performance. Each skill consumes valuable context windows (tokens) and increases "selection confusion" during invocation. Too many skills can interfere with each other, crowd out core reasoning space, and reduce response speed and accuracy.

Design Optimization:

Control skills by implementing "Global Skills + Project-Specific Skills" and limiting the total number, rather than allowing unlimited installations.

AI Inquiry Interaction

Using Coding CLI, you might notice AI sometimes pops up asking for choices. Initially, I wondered how AI handles user waiting given HTTP timeouts.

The reality is a facade. When AI sends an inquiry, it follows a specific format, ending that round of conversation. The frontend captures this format, displays a popup for user selection, and then resumes by sending a new concatenated dialogue back to the AI. The frontend makes it look like a continuous interaction within a single round.

ACP Protocol

Agent Client Protocol (ACP) is an open protocol designed to standardize communication between AI programming assistants (Agents) and clients. Supporting ACP allows any compatible client (e.g., Zed, VS Code, Zacp, Codex) to connect and communicate with your agent.

ACP support is optional. If you want stronger universality for your agent, consider supporting it. SDKs are available for various languages.

MCP Protocol

MCP (Model Context Protocol) is an open standard aimed at unifying connections between AI models and external data sources/tools, allowing AI assistants to access any resource via a standardized interface. Think of it as the "USB-C" of the AI world—a universal protocol for plug-and-play connectivity. — Content cited from AI

Currently, I primarily use the MCP framework to call MCP servers and haven't delved deeply into implementation details. Generally, developing an Agent requires only the ability to call MCP, not to implement MCP capabilities, which belong to the server side. For lightweight agents, MCP calling is optional.

Conclusion

After understanding the general framework and ideas of agents from this article, you should be well-equipped to command AI to develop your own lightweight agent. However, to make it truly useful rather than just functional, continuous optimization of details and bug fixes are necessary.

Additionally, the lightweight agent Zlite developed by me is open-sourced on GitHub: https://github.com/helloxz/zlite. Feel free to explore and analyze it with AI assistance. I hope this helps.