Designing a smarter, more powerful Agent

Hello, I’m Dave, a Backend Engineer at QANDA.

Some of you may have been surprised by the recent changes on QANDA. In response to the era of Generative AI, we’ve gone beyond simple feature upgrades to conduct a full redesign — ranging from the main screen of the app to the overall user journey.

In this blog post, we’ll walk you through:

The new direction of QANDA’s AI product in the Gen AI era
The technical approaches our backend team took to bring that vision to life

We hope this provides practical insights for anyone interested in the design and architecture of AI-powered services.

Why and How Has QANDA Changed?

The previous version of the QANDA app was best known for its core feature: allowing students to take a photo of a question they didn’t understand and receive an AI-generated solution.

Press enter or click to view image in full size

However, this feature was heavily focused on a specific use case — providing solutions when a student encounters a difficult problem.
As a result, it had limitations when it came to supporting a broader range of learning goals, such as concept explanations, recommending similar problems, or generating quizzes.

🗺 A Newly Defined Direction

So, we set a new goal:

“Understand the user’s learning intent in real time, and leverage LLMs to deliver optimal UI/UX accordingly.”

With this new direction, QANDA has been transformed from a simple problem-solving assistant into a proactive AI learning assistant capable of supporting a wide range of educational needs.

Press enter or click to view image in full size

A Completely New Engineering Paradigm

At its core, the new system we built was what’s commonly referred to as an “Agent.”
Instead of simply reacting based on predefined logic, we needed a structure that could understand a wide range of user requests and intelligently select the appropriate tool to solve the problem.

Initially, we followed a typical product development process: listing out user needs, defining features for each, and then implementing them in the UI. For example:

“I don’t know how to solve this problem.” → Provide step-by-step solution
“I solved it myself — can you check if it’s correct?” → Review and give feedback on handwritten answers
“I don’t understand the concept.” → Explain the concept with examples
“I want to practice similar questions.” → Search and recommend similar problems
“I think I’m starting to get it.” → Generate a quiz to check understanding

In a traditional setup, each of these requests would have resulted in a separate API and dedicated UI components to match.

Press enter or click to view image in full size

However, we chose a different approach: a unified Agent that drives the interaction with students through a single chat-based interface.

Press enter or click to view image in full size

In other words, our goal was to build a structure capable of handling diverse user needs within a single conversational flow. To make this possible, there were three key technical challenges we had to carefully consider:

✅ 1. Defining Chat Interface Components
→ The system needed to dynamically combine different blocks based on user requests.

✅ 2. Agent Intent Analysis
→ The Agent had to understand the user’s learning intent from natural language input.

✅ 3. Designing a Tool Framework for the Agent
→ The Agent had to be capable of solving complex tasks by combining multiple tools.

With these principles in place, we were able to abstract the entire system into the following three core concepts:

Agent (Intent Understanding) → Tool (Problem Solving) → Block (Visual Response)

Understanding Student Intent

As mentioned earlier, accurately identifying the student’s intent is critical for QANDA to flexibly support a wide range of learning use cases.

For example, even when a student uploads the same photo of a problem, their intent can vary greatly:

They may simply want a solution
They might want feedback on their own answer
They could be looking for a concept explanation
They may want to practice with similar problems
They might want a quiz generated based on the content

These diverse needs make intent recognition essential. However, in practice, it’s challenging for current LLMs to perfectly classify all of these intents in a single shot.

Moreover, as new learning needs and use cases will continue to emerge, it was essential to design the system from the beginning with scalability and flexibility in mind.

🤖 A Multi-Agent Strategy for Intent Analysis and Execution

To address this, we first divided user needs into two main flows:

Problem-Solving-Oriented: Flows focused on helping students solve problems, such as feedback, explanations, and concept clarifications
Content-Creation-Oriented: Flows focused on generating learning materials, such as quizzes, flashcards, and summary notes

Based on this separation, we decided to build independent Agents responsible for each flow. This led us to the following architecture:

Press enter or click to view image in full size

🕸️ Multi-Agent Swarm Architecture

This architecture resembles what is commonly known as a Swarm Architecture. Each agent focuses on its specific domain of expertise, and requests that fall outside its scope are handed off to another appropriate agent. (reference)

The initial request is processed by the determine_active_agent, which delegates the responsibility to either the Answer Question Agent (focused on problem solving) or the Generate Learning Material Agent (focused on content creation).
From that point on, the active agent handles the conversation until the context changes, at which point it hands off the session to another agent as needed.

(For reference: the info_agent handles QANDA policy-related questions, while the vanilla_agent is a free-tier agent for users who have exhausted their quota.)

Key advantages of this approach include:

✅ Clearer context management
→ Each agent only needs to know its own role, which minimizes prompt complexity and history tracking.
✅ Use-case-specific evaluation and testing
→ Evaluation metrics and test scenarios can be independently designed for each use case.
✅ Separation of concerns for focused development
→ Developers can work with clear goals and responsibilities, as each agent has a well-defined purpose.
✅ High scalability and reusability of agents
→ New use cases can be supported by adding or combining independent agents.
→ The same agent can be reused across different scenarios.

Of course, like any architectural decision, there are trade-offs. In this case, the Swarm architecture can increase time and cost to reach the final output. We’ll cover how we addressed that challenge in a future post.

Next, we’ll dive into the specific roles and characteristics of each agent.

Problem-Solving Agent: Planning and Execution

The first component we began developing was the Problem-Solving Agent. This agent needed to address a wide range of learning needs, including:

Step-by-step solutions for questions students don’t understand
Similar question recommendations by analyzing question types and retrieving relevant problems from QANDA’s database
Feedback on handwritten answers, reviewing and commenting on students’ own solutions
Graph generation, to visually represent functions or geometric concepts when needed
Handling multiple questions in a single image, separating and solving them one by one
Terminology explanations for concepts that appear during the solution process

There were also system-level goals and constraints to consider:

Maintain a minimum answer accuracy rate (e.g., n%)
Optimize cost by increasing the use of database-based question resources

🧠 Initial Approach: ReAct-Based Architecture

Our initial design imagined the system as a set of modular Tools, with the Agent selecting and calling the appropriate Tool depending on the context of the user’s query. This approach closely resembled the ReAct (Reasoning + Acting) strategy (reference), and followed a flow like this:

Press enter or click to view image in full size

Simple ReAct Agent

However, once we began actual evaluations, several limitations became apparent:

There was a lack of overall coherence in responses, and in particular, the system often lost context or mixed up the order when handling multiple questions within a single request.

For example, when a user submitted an image with multiple problems and asked the agent to solve two of them, we frequently observed issues like the one below:

Press enter or click to view image in full size

The expected behavior was for the agent to solve questions a) and b) in the first turn, then continue with questions c) and d) in the next. However, we often saw issues such as the agent re-solving questions a) and b), or skipping question c) entirely and jumping to d).

Additionally, the agent was supposed to first consult the QANDA database and return a matching problem if one existed. But in practice, it often displayed a greedy tendency — focusing only on the immediate problem in front of it, without considering the broader context.

Of course, if the task were purely solving a single question, the agent would have performed much better. However, since the Problem-Solving Agent is expected to handle a variety of complex and layered requests, we needed a different approach.

👉 Strategy Shift: Introducing a Plan-and-Execute Architecture

As a result, we moved away from the ReAct-style approach that relied on reactive tool invocation, and instead adopted a Plan → Execute architecture. (reference)

Press enter or click to view image in full size

In the Plan phase, the system analyzes the overall structure of the request and outlines the necessary tasks in order. Then, in the Execute phase, it follows that plan step by step — calling the appropriate tools and updating the system state along the way.

The Replan phase is optional and comes into play after task execution, where the system can adjust the plan based on the intermediate results.

Press enter or click to view image in full size

The diagram may look a bit complex, but it reflects the actual architecture of our production Problem-Solving Agent. As described above, the agent completes the problem-solving process by going through the Plan, Execute, and optionally Replan phases.

This approach has shown much more stable performance when handling multi-step problems, sequential requests, and generating complex responses. Because the agent is now able to think in a more structured way, it can deliver responses that are more coherent and consistent.
The previous issue of “greedy” behavior was also resolved effectively — by introducing the Replan phase, the agent can now check for duplicate or already-answered questions before proceeding.

🦾 QANDA’s Problem-Solving Assistant in Action
Let’s take a look at how the newly built Problem-Solving Agent powers real features in our product, supporting students in learning through problem data.

Press enter or click to view image in full size

The image above shows the result of asking a question about a geometric sequence. The agent first created a plan to provide feedback based on the user’s handwritten solution, and then used the feedback tool to execute that plan — delivering a corrected and annotated response.

Press enter or click to view image in full size

Looking at the flowchart above may make things easier to understand — you can see how the Planner analyzes the user’s request, decides which Tool to use, and passes that plan to the Executor.

In contrast,

Press enter or click to view image in full size

If the user, using the same image, also wants to search for similar solutions,

Press enter or click to view image in full size

As shown in the flowchart above, if that’s the case, the Planner can include the search tool — which queries the QANDA database — as part of its plan. Then, the Executor follows through by executing the search and returning a matching problem.

In reality, the Planner handles a wide variety of use cases, making its role even more critical. To support this, we’ve implemented a number of carefully designed guideline prompts within QANDA to help steer the Planner’s behavior effectively.

In this way, the Problem-Solving Agent was the first to be designed within the QANDA system — and also the one that handles the broadest range of scenarios. As such, it served not only as a core product feature but also as a testbed for validating key Agent design concepts such as reasoning, tool usage, and handling user requests.

While building the Problem-Solving Agent, the QANDA Backend Team explored and evaluated a variety of architectural approaches, including ReAct and Plan-and-Execute frameworks. Through this, we accumulated valuable knowledge about which structures are best suited for different types of tasks.

We also addressed many practical challenges — such as how to handle failures in the agent flow, how to separate internal messages to the model from the user-facing responses, and more.

The insights we gained through this process became a solid foundation when we moved on to develop the Generator Agent, which powers content creation workflows.
Next, we’ll introduce how this agent was designed to go beyond solving problems and proactively assist students in their learning journey.

Content Generation Agent: Creating Learning Materials

While the Problem-Solving Agent focuses on the process of finding correct answers, the Content Generation Agent is responsible for shaping the learning journey — helping learners engage actively and enhance their understanding through interactive experiences.

For example, the process of generating quizzes or flashcards doesn’t end with a single response:

It takes more time to generate the learning content itself,
And it involves multiple follow-up interactions such as solving the quiz, grading, regenerating, or editing,
All of which require a dedicated UI and flow that goes beyond a traditional chat interface.

When designing for this kind of interaction, many LLM-based product ecosystems refer to these elements as Canvas or Artifact — distinct components defined outside the standard chat paradigm.

Press enter or click to view image in full size

We also adopted the term “Artifact” within QANDA to clearly define and separate these content-centric experiences from standard chat flows at the system level.

🎛️ What is an Artifact?

An Artifact is not just a single response block. It represents an independent learning experience that takes place in a separate viewer interface, where the user can interact directly — and those interactions influence both system state and future AI responses.

Key characteristics of an Artifact include:

Users can interact within a dedicated viewer and directly modify information.
These changes affect the context of subsequent AI responses.
For example: the feedback and suggestions an LLM can give will differ greatly depending on whether the user hasn’t taken a quiz yet, or has already completed and reviewed it.
To support this behavior, an Artifact — unlike a normal response — requires backend logic for state management, data persistence, change tracking, and sometimes even needs to be managed by its own dedicated service.

❓When Does Something Qualify as an Artifact?

From a development perspective, we typically classify a feature as an Artifact if it meets one or more of the following criteria:

Events occur beyond simple Read operations — e.g., the user’s actions result in changes to system state (Create/Update/Delete).
Session state or results need to be persistently stored.
The interaction cannot be handled purely through chat, and requires a dedicated UI or task flow.
Examples include: Taking a quiz and getting graded, viewing and editing handwritten notes, etc.

An Artifact is not just a visual element — it’s a branching point in the user experience where data changes, system context shifts, and interactive learning happens.

The Generator Agent is designed to lead these Artifact-driven experiences. It serves as a proactive learning assistant that encourages students to participate more actively in their learning journey.

🧮 How QANDA Generates Artifacts

With this definition in place, we developed the Generator Agent and the architecture for its interaction patterns, which are shown below:

Press enter or click to view image in full size

First, regarding the structure of the agent — we also used a Plan-and-Execute approach here, since the agent needed to support a variety of interactions such as creating and modifying artifacts.

One key difference, however, is that we did not include a Replan phase, as there were no complex conditional flows or branching logic requiring dynamic replanning.

Additionally, since the agent within the chat cannot keep waiting for an artifact to be generated, once the generation tool is triggered, a new flow is launched asynchronously within the artifact domain (in this case, a quiz), as shown below.

Press enter or click to view image in full size

The following diagram provides a high-level view of the overall structure —
showing how the system responds dynamically to user requests and how the artifact domain operates independently once triggered.

Press enter or click to view image in full size

The Quiz Agent is the Artifact Agent that operates within the domain described above.

While the routing and request handling for creation, modification, and retrieval are managed by the Agents Swarm and the Content Generation Agent, the actual logic is executed within the Artifact Agent itself. (Details about the Planner and related components are omitted here.)

For example, as shown in the image below:

Press enter or click to view image in full size

When a user says “Create a quiz for me,” the Agents Swarm detects the quiz generation intent and delegates the task to the Content Generation Agent. That agent then sends a creation request to the Quiz Backend, and the resulting Artifact is rendered within the chat interface.

Each Artifact follows domain-specific logic — such as creation, modification, or progress tracking — and in this case, the Quiz Artifact is shown loading and then successfully completing its generation.

Press enter or click to view image in full size

This Artifact comes with its own dedicated view, allowing users to interact with it independently — whether through direct interaction within the viewer or by creating or modifying new artifacts through chat, as shown above.

So far, we’ve covered the two core agents. Now, let’s talk about how we designed the system to efficiently manage and operate these agents.

Separation of Agent and Chat Logic

In a system that handles complex learning flows and diverse user interactions, it’s crucial to clearly separate the responsibilities between the Agent logic and the chat interface flow.

Initially, our imagined structure had the agent handling the entire process: receiving the user’s message, generating a response, converting it into blocks, and rendering it directly in the chat window.

However, this tightly coupled structure led to several problems:

The agent became too heavy, increasing complexity and making testing more difficult
The chat UI/UX logic was entangled with agent logic, making reuse and scalability harder
Any change in chat flow or UI had unintended side effects on the agent’s behavior

👷 Strategy for Architectural Improvement

To solve this, we redesigned the structure to decouple the agent from the chat system and clearly define their individual responsibilities:

The Agent focuses solely on its core role: Understanding user intent → Selecting and using tools → Generating a response
Everything else — such as checking user quota, transforming responses into UI components, managing conversation flow, and rendering — is handled by a separate backend chat orchestration layer

Here’s a visual representation we used in our design process to help make this separation easier to understand:

Press enter or click to view image in full size

The User label represents the chat interface, the Backend (DB) label refers to the flow handled and stored by the Chat Backend server, and the Agent label corresponds to the Agents Swarm.

Through this architectural separation, we gained several key benefits:

✅ Modularity: Agents can now be developed independently and reused across different parts of the system. For example, the Generator Agent can be released as a standalone product or offered as an API, without being tightly coupled to QANDA’s user interface.
✅ Scalability: The chat interface can now be flexibly adapted to support new flows and UI designs.
✅ Stability: Critical functions tied to system reliability — such as quota management and message flow control — have been moved outside the agent, improving overall robustness.
✅ Maintainability: By ensuring each layer manages only its own responsibilities, we’ve minimized the blast radius of changes and made the system easier to maintain.

This structure has ultimately laid the foundation for QANDA to expand across a broader range of platforms and channels in the future.

Future Directions

But there’s still so much more to talk about.

In our upcoming blog posts, we’ll be diving deeper into topics like:

🔍 Evaluation

While the backend engineers were busy designing and implementing the system architecture, our AI engineers focused on future-oriented research — such as Agent Memory and Document Parsers — while also significantly improving our evaluation framework.

Press enter or click to view image in full size

In Gen AI systems, defining and measuring output quality remains a critical challenge. To address this, QANDA has developed an in-house evaluation dataset that we’ve been continuously refining.

It includes:

Evaluation scenarios with diverse input/output cases
Assessments ranging from individual Agent/Tool units to end-to-end evaluations across complete learning flows
Other elements that influence chat performance, such as LaTeX rendering, next-response suggestions, and more

In an upcoming post, we’ll share how this evaluation framework helps accelerate Agent development and refine quality standards in meaningful ways.

📈 Monitoring: Latency & Cost Optimization

As mentioned earlier, one of the key challenges in Gen AI systems is striking the right balance between rich, accurate responses and fast, cost-effective execution.

We continuously monitor several key metrics:

Latency

TTFT (Time to First Token): Time from user input to the appearance of the first token
TPOT (Time per Output Token): Average time between generated tokens
Total Latency: Time to generate the complete response

Cost

Total cost of LLM calls per response
Cost breakdown per Agent / per Tool
Cached prompt ratio

In a future post, we’ll share the optimization strategies we’ve applied based on these metrics, the improvements we’ve achieved, and the guardrail systems we plan to introduce moving forward.

🧠 Managing Agent Memory

To become a true learning partner, an Agent must go beyond simple context tracking — it must offer personalized experiences by deeply understanding the user.

For instance, knowing what subject the student is currently studying, which national curriculum they follow, their learning habits, or even personal interests — all help generate more relevant learning strategies.
This also enhances CRM capabilities such as push notifications, reminders, content recommendations, and marketing messages.

Agent personalization is not a minor feature — it’s a core strategy for increasing product engagement and long-term retention.

Currently, QANDA’s backend team is actively collaborating with AI engineers to explore multiple approaches for building a memory system that retains long-term user context.
We’re exploring technical areas such as:

Storing and retrieving user state with vector databases
Designing short-term and long-term memory entities
Experiments with prompt injection for personalized context
Encoding strategies

This goes far beyond just “storing data.” It’s about observing behavior, accumulating meaningful context, and delivering better conversations and smarter recommendations.

We’ll provide a deep dive into the design, structure, and impact of this memory system in a future article.

🧪 Prompt Management and A/B Testing

QANDA’s Agent architecture uses hundreds of prompts, all of which are systematically managed for use in both A/B testing and evaluation strategies.

We believe future AI products will shift more toward agent-centric designs, and testing these natural-language-driven systems will look very different from traditional QA and UI testing.

That’s why we’re building a new kind of testing infrastructure — designed for evaluating conversational intelligence.

In a future post, we’ll explain:

How we structure, test, and version our prompts
What kinds of evaluation and behavior data we collect
How we apply insights to optimize performance and consistency

🔧 How We Built It with LangGraph

Some of you may have already noticed this from the diagrams above:
QANDA’s multi-agent architecture — including intent branching, Tool calling logic, Plan-and-Execute strategy, and Artifact-based content generation — has all been implemented using LangGraph.

We’ll share detailed implementation stories, key architectural decisions, and lessons learned from working with LangGraph in an upcoming technical deep dive.

Press enter or click to view image in full size

In fact, as shown in the image above, even evaluation is fully graph-based and visualized, allowing us to inspect and debug behaviors in an intuitive way.

LangGraph has proven to be a powerful framework for building the kind of architecture we envisioned — handling complex concerns such as inter-agent messaging, state-driven flow control, conditional branching, and session persistence, all with ease and flexibility.

At QANDA, we’ve been using LangChain since its early days, and through numerous use cases, we’ve developed a deep understanding of various implementation patterns — many of which we applied to our current Agent system.

Through our experience with LangChain and LangGraph, we’ve gained the following key insights:

How to define and execute collaborative workflows between agents
How to manage non-deterministic responses and state transitions in a robust way
Why we chose LangGraph over LangChain, and what challenges we encountered in taking it to production

We plan to share these implementation stories in a future post and we believe they’ll be especially valuable for anyone interested in LangChain, LangGraph, or building graph-based reasoning systems in real-world environments.

In our next blog post, we’ll dive deeper into the topics above — focusing on the practical challenges and hard-earned insights needed to bring a Gen AI system to a production-ready state.

Thanks for reading, and we hope you’ll stay tuned! 🙌🏻

Join our team and shape the future of education with us! Check out our careers page here.

How Was the New QANDA Agent Built?

댓글