<div id="readability-page-1" class="page"><div>‍Coding and code debugging have been the most common use case of AI agents, just some of the many examples including <a href="https://github.com/smol-ai/developer" target="_blank" rel="noopener noreferrer" data-external="true">Smol Developer</a>, <a href="https://github.com/AntonOsika/gpt-engineer" target="_blank" rel="noopener noreferrer" data-external="true">GPT Engineer</a>, <a href="https://github.com/irgolic/AutoPR" target="_blank" rel="noopener noreferrer" data-external="true">AutoPR</a>, ReactAgent, and <a href="https://bloop.ai/" target="_blank" rel="noopener noreferrer" data-external="true">Bloop</a>. <a href="https://github.com/sweepai" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a> represents this category of agents in <a href="https://www.ycombinator.com/companies/sweep" target="_blank" rel="noopener noreferrer" data-external="true">YC S23</a>.
<a href="https://sweep.dev/" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a> works by generating code on GitHub issue submissions, e.g. your bugs and feature requests. It generates code in the form of pull requests, which the user can comment on and iterate until reaching their desired result. The following diagram shows the pipeline of turning an issue into a pull request.
<figure><figcaption>Source: docs.sweep.dev (https://github.com/sweepai/sweep)</figcaption></figure>
We asked the <a href="https://sweep.dev/" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a> founders - <a href="https://twitter.com/wwzeng1" target="_blank" rel="noopener noreferrer" data-external="true">William Zeng</a> and <a href="https://twitter.com/kevinlu1248" target="_blank" rel="noopener noreferrer" data-external="true">Kevin Lu</a> - for their view on the current agents' space, and the challenges they are working on to solve for the agents.
<h2 id="users-and-use-cases">Users and Use Cases</h2>
<a href="https://github.com/sweepai" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a> works with Github issues. Given an issue, the <a href="https://sweep.dev/" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a> agent plans how to solve it, writes code, and turns the issues directly into a pull request (without an IDE). It uses embedding-based code search.
The initial startup time usually takes 3-5 minutes, depending on the codebase. Users can iterate on results by adding comments to pull requests.
<figure></figure>
<figure><figcaption>Source: github.com/sweepai (https://github.com/sweepai/sweep/blob/main/docs/installation.md)</figcaption></figure>
‍
As many agents for developers, it is open-source, and getting increasingly popular, with over <a href="https://github.com/orgs/sweepai/repositories" target="_blank" rel="noopener noreferrer" data-external="true">5000 stars on GitHub</a>.
“We target really strong developers and try to help be as efficient as possible” say the founders.
The current total usage is around 100 pull requests a day. What the end users build with <a href="https://sweep.dev/" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a> differs, since the developers are utilizing it for their existing projects.
<h2 id="current-challenges">Current Challenges</h2>
Kevin and William share their experience with agent instances failing. “When an agent fails, it is in approximately 20% cases caused by prompts, and 80% all kinds of other issues.” &nbsp;The bug has often a combination of root causes.
Sometimes the agent straight away gets the wrong idea or uses the wrong format for the result of the task. If there is a bug, they can go and twist a prompt a little bit, and fix it.
“Given that the agent operates in a linear manner, it is usually easy to determine what caused the issue and decompose the process into steps,” said Kevin and William.
The agent’s workflow is pretty constrained and so, there is no need to stop it in the middle. Instead, when a bug happens, restarting the whole process is the most pragmatic way.

Debugging has been a struggle for most agents’ developers, and there is no strong consensus about agent-specific <a href="https://github.com/e2b-dev/awesome-sdks-for-ai-agents" target="_blank" rel="noopener noreferrer" data-external="true">tools or frameworks</a> used for debugging agents' prompts and errors in general.
“With Sweep, the process of debugging usually involves looking through the history of the conversation with the agent,” summarize the Sweep founders, who created their own internal tool to visualize the chats.
“We built this chat visualizer in 2 hours,” say Kevin and William. “If a user contacts us, with their permission to view their logs for debugging purposes, we can visualize what happened and fix what needs to be fixed, then click redeliver."
End users cannot access the visualization, so they usually contact the Sweep team. “This happens about once a day”, William and Kevin say.
The following image shows <a href="https://sweep.dev/" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a>’s internal visualization of agents’ bugs.
<figure><figcaption>Source: Sweep (https://sweep.dev/), internal</figcaption></figure>
<h2 id="communication-with-users">Communication with users</h2>
The user experience is very server-centralized, as Kevin and William comment: “If the servers have an issue, everyone has an issue.” The <a href="https://sweep.dev/" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a> team often learns about issues directly from the end users and consequently assists in fixing them.
The Sweep team aspires for users to engage their agents in meaningful work-related tasks, not just playing with <a href="https://github.com/sweepai/create-react-app" target="_blank" rel="noopener noreferrer" data-external="true">test repositories</a> or experimenting. &nbsp;“Since we found at the start that developers are still using AI agents for the sake of exploring new technologies and having fun, we focused our strategy on limiting the usage of Sweep. This way, the users are mindful and use Sweep for the actual work.”
They hence limit to 5 uses a month of <a href="https://openai.com/gpt-4" target="_blank" rel="noopener noreferrer" data-external="true">GPT-4</a>, the rest of GPT-3.5.
<figure><figcaption>Source: &nbsp;Testing Sweep on E2B team GitHub (https://github.com/e2b-dev)</figcaption></figure>
<h2 id="conclusion">Conclusion</h2>
Users can work with <a href="https://sweep.dev/" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a> independently, without assistance. William and Kevin never stop developing their agent and making even bigger progress toward agents becoming regular work tools. &nbsp;“Currently, we are reading a lot into the old style of indexing code. The goal is to become proficient in both code understanding and agent understanding.”
If you want to try <a href="https://github.com/sweepai/sweep" target="_blank" rel="noopener noreferrer" data-external="true">Sweep</a>, you can install it via <a href="https://github.com/sweepai/sweep" target="_blank" rel="noopener noreferrer" data-external="true">https://github.com/sweepai/sweep</a>. After installation, you can add the repository you want the agent to work on, make a ticket (e.g. writing tests), add the label "sweep" and watch the AI agent do the work.
If you are interested in open-source agents for developers and coding, check the whole <a href="https://github.com/e2b-dev/awesome-ai-agents" target="_blank" rel="noopener noreferrer" data-external="true">AI agents database</a>.</div></div>

E2B Blog & Customer Stories

Vaclav Mlejnsky

We asked the Sweep founders for their view on the current agents' space, and the challenges they are working on to solve for the agents.

Sweep Founders Share Learnings from Building an AI Coding Assistant

댓글