How does AI agent work?
I picked up this book right after setting up my first AI assistant—Leila—right
here on the Blog . She’s an AI assistant and thinking partner I trained using
Jotform’s no-code AI agent platform. Once I had Leila as my thinking partner, I
felt compelled to dive deeper into the world of AI agents—and that’s when this
remarkable book came into my hands. Although it’s written in Chinese, it offers
rich insights into American technology and deployment strategies, with a strong
focus on OpenAI’s LLMs. Let’s explore it together!
I tried to link between the book and Jotform no-code AI agent to support
Non-technical business to build their own agent and solve any business problem
they experienced in their daily work life. Jotform no code AI agent is not
mentioned in the book, I just tried to bridge between no-code AI agent and the
book.
A Practical Guide to AI Agents: From Concept to Multi-Agent Systems
Introduction
The dawn of large language models (LLMs) has ushered in a new era of artificial
intelligence, moving beyond simple question-answering chatbots to the creation
of autonomous, problem-solving "Agents." This book serves as a
comprehensive, project-based guide for anyone looking to understand and build
these powerful AI systems. Written in an engaging, accessible style, it
demystifies complex concepts through a narrative dialogue between Xiao Xue (小雪) and
Brother Ka (咖哥 offer clear, hands-on approach.
The book's central idea is that AI Agents, powered by LLMs, are not just a
novelty but a fundamental shift in how we interact with technology. They can
plan, remember, use tools, and take action, acting as digital teammates that
automate complex workflows. However, the book's approach, while excellent for
learning, is heavily tied to the OpenAI ecosystem, a choice that presents both
strengths and limitations for business users.
Book Summary:
The book is structured into three main parts, guiding the reader from foundational
theory to advanced, multi-agent collaboration.
Part 1: Foundations and Theory (Chapters 1-3)
The journey begins by defining what an AI Agent is and why it matters,
positioning it within the context of the "Life 3.0" concept—a
technological life form that can design its own software and hardware. Key
characteristics like autonomy, adaptability, and interactivity are introduced.
The book
then builds the core technical architecture, breaking an Agent down into four
essential components: Planning, Memory, Tools, and Action. It introduces the
critical ReAct (Reasoning and Acting) framework, where an agent alternates
between thinking about a task and taking concrete steps to solve it. The final
foundational chapter provides an overview of the three primary development
tools used throughout the book: the OpenAI API, the LangChain framework, and
the LlamaIndex framework.
Part 2: Practical Agent Development (Chapters 4-8)
This is the heart of the book, presenting a series of sevenp progressively complex,
hands-on Agent projects.
· Agent 1 (Chapter 4): Automated Office Work uses the OpenAI Assistants API and
DALL-E 3 to automatically create a PowerPoint presentation from sales data.
· Agent 2 (Chapter 5): Versatile Selection Engine focuses on OpenAI's Function
Calling, enabling an agent to intelligently choose and execute custom
functions.
· Agent 3 (Chapter 6): Synergizing Reasoning and Action implements a ReAct
Agent using LangChain to automatically research prices and calculate markups.
· Agent 4 (Chapter 7): Decoupling Planning and Execution introduces a
"Plan-and-Execute" framework in LangChain for handling complex,
multi-step tasks like inventory management.
· Agent 5 (Chapter 8): Knowledge Extraction and Integration builds a
Retrieval-Augmented Generation (RAG) agent using LlamaIndex to answer questions
based on private financial documents.
Part 3: Advanced Agents and Future Directions (Chapters 9-10 & Appendix)
The final part explores the broader ecosystem and future of Agent technology.
· Agent 6 (Chapter 9): GitHub's Star Projects surveys influential open-source
projects like AutoGPT, BabyAGI, and CAMEL, each demonstrating different
paradigms of autonomous and multi-agent collaboration.
· Agent 7 (Chapter 10): Multi-Agent Frameworks delves into advanced frameworks
like Microsoft's AutoGen and MetaGPT, which enable the creation of systems of
multiple, specialized Agents working together to simulate complex
organizations.
· The Appendix looks ahead to the research frontier, discussing concepts like
multi-modal Agents, human-AI collaboration, and embodied AI.
A Critical Perspective for the Business User
While the book is an excellent resource for learning the concepts and getting
hands-on with code, it is crucial for business developers to recognize its
limitations. The most significant is its almost exclusive focus on the OpenAI
ecosystem. This creates several challenges:
· Cost Barrier: Every experiment requires paying for OpenAI API calls, which
can become expensive, discouraging iterative exploration.
· Vendor Lock-in: The book's approach ties you to a single provider, creating a
dependency that could be risky if OpenAI changes its pricing, policies, or
suffers outages.
· Rapid Obsolescence: The code examples and specific API usages are likely to
become outdated quickly as the technology evolves.
The book can help you build solid knowledge about AI agents. But, it is heavily
focus on coding and programming skills. Therefore, I understand that some
business developers may lack programming and coding skills. Therefore, I tried
to implement some knowledge in the book using no code AI agent platform. In
this context, however, as a business developer, you can leverage the book's
core concepts and avoid these pitfalls by using user-friendly platforms like
Jotform, which abstract away the technical complexity and cost concerns,
allowing you to focus purely on business value.
Key Concepts Explained simply:
To build and understand AI Agents, you need to grasp a few fundamental
concepts. Here’s a breakdown of the most essential ideas from the book.
1. The Core Architecture of an AI Agent
An AI Agent is a system that can autonomously perform tasks, make decisions,
and interact with its environment. It is built on four key components.
A. Planning:
This is the Agent's ability to think ahead and break down a complex goal into
smaller, manageable steps. For example, if you ask it to "research
competitors and write a report," the Agent will plan to first search for
competitors, then analyze their websites, then compare them to your company,
and finally write up the findings.
B.Memory:
Agents need to remember things.
· Short-Term Memory: This is the memory of the ongoing conversation. It allows
the Agent to maintain a coherent back-and-forth dialogue with you, just like
your Jotform Agent remembers the history of a chat with a lead.
· Long-Term Memory: This is a persistent store of information, like a knowledge
base. Your Jotform Agent uses your uploaded documents (FAQs, product
information) as its long-term memory.
C.Tools:
This is what makes Agents powerful. A language model by itself can only
generate text. Tools allow an Agent to take action in the real world. Tools can
include a calculator, a search engine, a calendar, or an API to your CRM. Your
Jotform Agent (I used myself because I am neither a programmer nor a
developer) uses tools through
integrations to schedule meetings, send emails, or update your CRM.
D.Action:
This is the final step of the cycle where the Agent executes a decision. It
might be sending a response, updating a database, calling a function, or any
other concrete output based on its planning and use of tools.
2. The ReAct Framework (Reasoning + Acting)
ReAct is one of the most important cognitive frameworks for how Agents work. It
stands for Reasoning and Acting, and it describes a cycle where the Agent
alternates between these two modes.
The cycle has three key steps:
1. Thought: The Agent considers the current situation and decides what to do
next. This is the reasoning part.
2. Action: The Agent does something. This might be calling a tool, sending a
message, or making a decision.
3. Observation: The Agent looks at what happened as a result of its action.
This could be the response from a tool or the user's reply.
This Thought-Action-Observation cycle continues until the Agent achieves its
goal. This framework allows the Agent to be flexible and adapt to new
information, much like how a human solves a problem.
3. RAG (Retrieval-Augmented Generation)
RAG is a technique that solves a fundamental limitation of LLMs: they don't know
your specific business data. RAG works in two steps:
1. Retrieval: When a user asks a question, the Agent first searches your
private knowledge base (like internal documents, manuals, or FAQs) to find the
most relevant information.
2. Generation: The Agent then takes this retrieved information and feeds it to
the LLM along with the user's question. The LLM generates a final, accurate
answer based on the provided context.
RAG is a cost-effective and powerful way to give an Agent "expertise"
in your business, as it doesn't require retraining the expensive LLM. This is
the exact technology your Jotform Agent uses when it answers questions from
your Knowledge Base.
4. The AI Maturity Curve
This concept describes how an organization's ability to use AI evolves over
time. It's a useful framework for planning your AI strategy.
· Awareness: Learning about AI.
· Experimentation: Running small, isolated pilot projects.
· Adoption: Using AI in regular, recurring business processes.
· Integration: Connecting AI to core business systems (CRM, ERP, etc.).
· Transformation: Redesigning business models and operations around AI, where
AI agents automate complex workflows.
5. Function Calling
Function Calling is a specific tool that allows an Agent to bridge the gap between
understanding natural language and executing code. Instead of the Agent
generating a text response, it can generate a structured JSON object that
describes which function to call and with what parameters. This allows a
developer to write the actual code for the function (e.g.,
get_inventory(product_id)) and have the Agent call it automatically, making the
Agent capable of performing concrete actions.
6. The Gartner Hype Cycle for Technology
This is a well-known model for understanding how expectations for a new
technology change over time. Understanding it helps you remain realistic about
AI Agents.
1. Innovation Trigger: The technology is born (e.g., release of ChatGPT).
2. Peak of Inflated Expectations: Media buzz and hype are at a maximum.
3. Trough of Disillusionment: The technology fails to meet the high
expectations, and interest wanes.
4. Slope of Enlightenment: People begin to understand the technology's true,
practical use cases.
5. Plateau of Productivity: The technology becomes mainstream and integrated
into society.
Case Studies & Business Applications
The book provides seven detailed case studies (called "Agents") that
demonstrate the application of these concepts. Here, we break them down and
connect them to practical business uses, including how you can build them with
both code (as per the book) and without code (using platforms like Jotform).
Agent 1: Automated Office Work — Creating PPTs with Assistants API and
DALL-E 3
· Goal: Automatically generate a professional PowerPoint
presentation from sales data, including visualizations, insights, and a cover
image.
· Technology: OpenAI Assistants API, GPT-4, DALL-E 3, Code
Interpreter.
· Key Concepts: Multi-modal generation (text + images),
AI-powered data analysis, automated workflow.
· Step-by-Step Workflow:
1. Load sales data (CSV).
2. Create an Assistant with the Code Interpreter tool.
3. Send a message to the Assistant to calculate quarterly sales and
generate a line chart.
4. Run the Assistant and retrieve the generated chart image.
5. Ask the Assistant for insights and a compelling title for the PPT.
6. Use DALL-E 3 to generate a cover image based on a prompt.
7. Assemble the slides (title and content) into a .pptx file using a
Python library.
· Business Application:
Streamlines
the creation of reports, monthly business reviews, and investor presentations.
Saves hours of manual data analysis and slide creation.
· Jotform Integration:
You could trigger this workflow through a
Jotform submission. For example, when a sales rep submits a quarterly report
form, a Jotform integration (like Zapier or Make) could trigger a custom Python
script (using the same Assistants API) to generate and email the presentation.
Agent 2: A Versatile Selection Engine — Calling Functions with Function Calling
· Goal: Enable an Agent to intelligently choose and call a custom
function to provide a personalized response.
· Technology: OpenAI Function Calling.
· Key Concepts: Tool metadata, structured output (requires_action
state).
· Step-by-Step Workflow:
1. Define a custom Python function (e.g., get_encouragement(name,
mood)).
2. Create an Assistant with the function defined as a tool (JSON
schema).
3. User sends a message (e.g., "Please cheer up sad Xiao
Xue.").
4. The Run enters a requires_action state, providing the function name
and arguments.
5. Your code executes the local Python function and submits the output
back.
6. The Run completes, and the Assistant provides the final, empathetic
response.
· Business Application:
This pattern
is crucial for building advanced decision-making agents. For example, an agent
could decide to check inventory, look up a customer's account balance, or
escalate a complex issue to a human based on the user's intent. It bridges the
gap between conversation and action.
· Jotform Integration: In Jotform, this is analogous to using
conditional logic or an Agent's ability to decide which integration to call.
For example, if a lead is "hot," the Agent can trigger a
high-priority email. If the lead is "warm," it can schedule a
follow-up task.
Agent 3: Synergizing Reasoning and Action — Automatic Pricing with LangChain's
ReAct
· Goal: Build an Agent that can answer a multi-step question by
using reasoning and external tools.
· Technology: LangChain ReAct Agent, SerpAPI (web search),
llm-math (calculator).
· Key Concepts: ReAct framework, Agent Executor,
Thought-Action-Observation loop.
· Step-by-Step Workflow:
1. Define tools: "serpapi" for search, "llm-math"
for calculations.
2. Create a ReAct prompt template guiding the Agent to use Thought,
Action, Action Input, Observation.
3. Invoke the Agent with a query (e.g., "What is the wholesale
price of roses, and what is a 5% markup?").
4. The Agent loops: Thought: I need to search for the wholesale price.
-> Action: Search -> Observation: Average price is 25 yuan. -> Thought:
Now I need to calculate the markup. -> Action: Calculator with 25*1.05.
-> Final Answer.
· Business Application:
Automates
research and analysis that combines external data with internal business rules.
Perfect for dynamic pricing, competitive analysis, and market research reports.
· Jotform Integration: A Jotform Agent could use this pattern via
a custom integration or through Jotform's own logic. When a lead asks for a
price, the Agent could automatically check live market prices and apply a markup
before responding, ensuring the quote is always current and competitive.
Agent 4: Decoupling Planning and Execution — Smart Inventory Scheduling with
Plan-and-Execute
· Goal: Enable an Agent to handle complex, multi-step tasks by first creating a
detailed plan and then executing it.
· Technology: LangChain Plan-and-Execute, ReAct Agent (as executor).
· Key Concepts: Task decomposition, separation of concerns.
· Step-by-Step Workflow:
1. Define custom tools: check_inventory, calculate_price, schedule_delivery.
2. The Agent receives a request: "Check rose inventory and give a
shipping plan."
3. Planning Phase: The Planner generates a list of steps (e.g., Step 1:
Check inventory. Step 2: Analyze demand...).
4. Execution Phase: The Executor (a ReAct Agent) carries out each step.
If it lacks data (demand info), it asks for clarification.
5. On receiving a clear request, it executes all steps and provides the
final plan.
· Business Application:
This is crucial for automating complex business workflows
like project management, order processing, supply chain coordination, and lead
qualification. The Agent won't get lost in a long reasoning loop but will
execute a structured plan.
· Jotform Integration
Your Jotform
Agent is a form of a Plan-and-Execute Agent, just without the code. The
"Planning" is your Agent's instructions and conversational flow. The
"Execution" is following that flow to capture lead data, check
conditions, and trigger integrations (tools).
Agent 5: Knowledge Extraction and Integration — Implementing RAG with
LlamaIndex
· Goal: Build a conversational Agent that can answer questions
based on a private, external knowledge base (financial reports).
· Technology: LlamaIndex, ReActAgent, VectorStoreIndex,
QueryEngineTool.
· Key Concepts: RAG, vector embeddings, Agentic RAG.
· Step-by-Step Workflow:
1. Load financial report PDFs.
2. Create searchable indexes (vector stores) for each company.
3. Create Query Engines for each company's index.
4. Wrap these engines in tools, giving them names and descriptions
(e.g., "A_Finance").
5. Create a ReAct Agent with access to these tools.
6. User asks: "Compare the sales revenue of Company A and Company
B."
7. The Agent thinks and calls the "A_Finance" tool, observes
the result, then calls the "B_Finance" tool, and finally synthesizes
the information into a comparative summary.
· Business Application:
Creates a "copilot" for your business. This can be used for internal knowledge management (HR policies, employee handbooks), customer support (product FAQs, troubleshooting), and competitive intelligence (analyzing market reports and competitor data).
· Jotform Integration:
This is
exactly how your Jotform Agent works. You upload documents to your Knowledge
Base (the vector store), and when a lead asks a question, the Agent searches
the Knowledge Base, retrieves the relevant section, and uses it to generate an
answer. This is the foundation of modern, context-aware AI assistants.
Agent 6: GitHub's Star Projects — AutoGPT, BabyAGI, and CAMEL
· Goal: Explore the broader ecosystem and understand different
paradigms of Agent design.
· Projects: AutoGPT (autonomous, internet-connected agent),
BabyAGI (task-driven, brainstorming agent), CAMEL (role-playing multi-agent
framework).
· Key Concepts: Autonomous task completion, task prioritization, multi-agent
role-playing, "inception prompting."
· Business Application:
These projects are not for immediate production but are
powerful examples of future possibilities. They demonstrate the potential for
agents to act as autonomous researchers, creative brainstorming partners, and
collaborative teams.
· Jotform Integration:
Jotform
provides a more controlled and user-friendly platform. While you can't deploy
AutoGPT directly in Jotform, you can use the concepts—like breaking down tasks
or having agents with different "roles"—to design more sophisticated
Agent workflows using conditional logic and integrations.
Agent 7: Multi-Agent Frameworks — AutoGen and MetaGPT
· Goal: Explore advanced frameworks that orchestrate multiple
Agents working together.
· Frameworks: AutoGen (flexible, customizable agent
conversations), MetaGPT (simulates a software company with SOPs).
· Key Concepts: Multi-agent collaboration, specialized roles
(Product Manager, Engineer, QA), hierarchical and structured workflows.
· Business Application:
This is the future of business process automation. Imagine an AI team where one agent researches a lead, another drafts a personalized outreach email, a third schedules a meeting, and a fourth updates the CRM—all without human intervention. This can automate entire departments or business units.
· Jotform Integration:
Currently, Jotform Agents are single entities, but the
multi-agent concept is the next frontier. You could potentially design multiple
Jotform forms, each representing a different "agent" or step in a
process, and connect them via integrations to create a similar effect of a
multi-step, collaborative workflow.
Practical Recommendations for Business Developers:
As a business developer, you have a unique advantage: you know what problems to
solve. You don't need to become a software engineer to benefit from this
technology.
· Use No-Code/Low-Code Platforms: This is your most effective path. Build a
functional Agent with Jotform within hours. It handles all the underlying
technical complexity, costs, and vendor management for you.
· Leverage the Concepts: Understand the book's concepts (RAG, ReAct,
Plan-and-Execute) to design better Agent instructions and workflows, even if
you aren't writing code. Your business knowledge is more valuable than your
coding skills.
· Focus on Data: The quality of your Agent is determined by the quality of your
instructions (prompts) and the data in your Knowledge Base. Invest time in creating
excellent, clear content for your Agent to use.
· Start Small, Think Big: Begin with a simple, high-value use case, like a lead
qualification agent. Learn from it, measure its success, and then gradually
increase its complexity and capabilities.
By combining the conceptual knowledge from the book with practical,
user-friendly tools, you can build powerful, business-critical AI Agents today,
without getting lost in the code.
Recommended
Reading mentioned in the book:
https://hai.stanford.edu/news/
References:
I hope you
enjoy today’s Moment, and looking forward to hearing about what you build next.
Do you have any problem you want to solve? I am eager to hear your Agent story
in the comment below!
Comments
Post a Comment