Why AI Agents are Good System Design [System Design Sundays]
How Agentic AI can help us implement good principles of Software Design.
Hey, it’s your favorite cult leader here 🦹♂️🦹♂️
On Sundays, I will go over various Systems Design topics⚙⚙. These can be mock interviews, writeups by various organizations, or overviews of topics that you need to design better systems. 📝📝
I put a lot of effort into creating work that is informative, useful, and independent from undue influence. If you’d like to support my writing, please consider becoming a paid subscriber to this newsletter. Doing so helps me put more effort into writing/research, reach more people, and supports my crippling chocolate milk addiction. Help me democratize the most important ideas in Tech to over 100K readers weekly.
PS- We follow a “pay what you can” model, which allows you to support within your means, and support my mission of providing high-quality technical education to everyone for less than the price of a cup of coffee. Check out this post for more details and to find a plan that works for you. Many companies have a learning budget, and you can expense your subscription through that budget. You can use the following for an email template.
I’ve been a huge advocate for Agentic Design for a long time. In my mind, Agentic design can help us implement some software development best principles in modern LLM pipelines, which is a huge plus. I will combine my experience over the last 2.5 years- from when I actively started looking into Agentic AI (for English to SQL translation) to our current experiments with IQIDIS AI, the best Legal AI currently in the market (come say hi- info@iqidis.ai).
In this article, I want to go back to the basics to answer a fundamental question- why is Agentic AI so powerful, especially in sensitive fields like Legal AI? To me, this boils down to how Agentic AI allows us to address the limitations of pure LLM/Deep Learning Systems and implement great software design principles. That is exactly what I want to explore here today-
Executive Highlights (TL;DR of the article)
To fully understand things, let’s first clarify the terminology. Let’s first hone in on what makes a system Agentic.
Agentic AI is a way to ensure your stuff works by making the AI take auditable steps that we can control and correct. This gives us better accuracy, fewer uncontrollable errors, and the ability to tack on more features. For example, we can use the following setup to answer Sales Related Questions-
This will allow us to monitor the decision-making, allowing us to catch where things are going wrong. Using the specific columns, a separate SQL generation tool, etc, also improves performance and efficiency.
More concretely, an agentic AI system doesn’t rely on a single LLM to solve a complex problem. Instead, it breaks down the user’s request into a series of smaller steps, each handled by a specialized agent. These agents can be diverse:
LLMs for specific tasks: One agent might summarize text, while another translates it. This is generally better than giving multiple instructions to one instance of an LLM.
Traditional algorithms: An agent could perform calculations, access databases, or interact with APIs. This is highly recommended for things like search, computations query generation, etc.
Rule-based systems: Agents can enforce constraints or handle specific scenarios based on predefined rules.
The system orchestrates these agents, coordinating their actions and combining their outputs to produce the final result. This approach contrasts with the monolithic approach of using a single LLM for everything,
This “agent-oriented” approach naturally leads to systems that are:
Modular: Agents are self-contained units that are meant to handle specific tasks. This allows us to implement separation of concerns, ensuring that we know exactly where each failure originates. This modularity also promotes code reusability — build an agent once and deploy it anywhere.
This has been incredibly helpful when building IQIDIS’s Legal AI helpers. So far, people have relied on powerful Foundational Models (ChatGPT) + Fine-Tuning to improve the capacity of their systems. This is incredibly inefficient and introduces lots of security + stability issues. Being lawyer-led, IQIDIS understands the tasks that lawyers must engage with- which we have been able to decompose into Legal AI workflow that can be augmented or automated with specific AI agents.
Scalable: Agents can work in parallel, distributing the workload like a team of ants carrying a crumb. This distributed processing power allows systems to handle massive data volumes and user traffic without breaking a sweat. Better modularity also leads to a more transparent system, which allows for another kind of scalability: the scalability of functionality. Transparent systems are easier to change, allowing us to build more complex core functionalities into the system- capturing more value in the market.
Robust: Agents can help us implement Safety and Robustness into solutions fairly easily. Since we know how/when agents interact, we can monitor our system for breaks and address issues even before they occur (eg. spinning up more resources for worker agents that are about to hit capacity or have specific agents to detect bad inputs) and build specific flows to account for the inevitable case that something goes down (for example using placeholders or other Databases if our case-search functionality goes down).
However, this powerful approach comes with its own set of challenges: good design becomes key as mistakes can compound over time much more quickly. Debugging interactions between multiple agents can be tricky, and there’s a risk of unintended consequences if agents aren’t properly designed and tested. It’s also critical to properly model the interactions between agents, as Agents can get stuck in a never-ending “conversation” with/ each other- which leads to weird outputs and a worse user experience and will make you AWS’s favorite customer.
Despite these challenges, the potential benefits of agent-based systems are immense (and this is why we’ve made Agents such a focus on AI Made Simple in recent weeks).
Let’s explore these ideas in more detail now.
I provide various consulting and advisory services. If you‘d like to explore how we can work together, reach out to me through any of my socials over here or reply to this email.
1. How Agentic AI Improves Modularity
What is Modularity:
Modularity involves breaking down a complex system into smaller, independent modules. This is crucial for managing complexity, promoting code reusability, simplifying development and maintenance, and isolating errors. Non-modular systems can have increasingly complex connections, as non-specialized modules tend to take on larger and larger responsibilities, making the code-base harder to iterate on.
How AI Agents Help:
You’re smart people, so instead of repeating the how from the tl;dr, let’s explore two use cases.
Imagine building a customer support chatbot. Instead of a single LLM trying to handle everything, you could have specialized agents:
Natural Language Understanding (NLU) Agent: Processes user input to extract intent and entities (e.g., “I want to track my order” — intent: track order, entity: order ID).
Order Tracking Agent: Queries the database for order status using the extracted order ID.
Response Generation Agent: Constructs a natural language response based on the order status.
Escalation Agent: If the order status is problematic or the user expresses dissatisfaction, this agent routes the conversation to a human operator.
Each agent focuses on a specific task, making the system easier to understand, debug, and extend. The NLU agent could be reused in a different application, like a voice assistant. If the Order Tracking Agent fails, the other agents can still function, offering a generic apology and suggesting alternative contact methods. And most importantly, this improves the performance of your system in many key ways for example-
Things like Order Tracking and large parts of of response generation can be done with standard techniques reducing costs (no calling expensive LLMs).
It also wipes out any uncontrollable errors introduced by LLMs.
It improves the transparency of your system since you’re relying on more well-understood setups.
So much cleaner than LLMs for everything.
A good example of this is IQIDIS’s unique infinite context search + legally refined generations. Instead of forcing everything on one LLM (which is what the current people in the market try to do), we are able to ensure that the best tools for each task are applied to every desired task. For example, we can apply agents to restrict the jurisdictions/kinds of laws we want to look through (reducing unneeded hallucinations such as when the existence of an existing Texas Law confuses an LLM when giving suggestions regarding Tennessee Law). We also have different tools for formatting, generation, document review, multi-modal information extraction, etc.- allowing us to better iterate and build powerful capabilities that no one else in the space can provide.
2. Scalability
Principle
Scalability is a system’s ability to handle increasing workloads or data volumes without performance degradation. This is crucial in today’s data-driven world, where systems must adapt to growing demands.
How AI Agents Help with Scalability:
Consider a system processing large volumes of scientific literature. An agent-based approach could employ:
Ingestion Agents: Multiple agents working in parallel to download and pre-process articles from different sources (this is something that we do a lot with our clients that bots to pull information from specific sources).
Analysis Agents: Specialized agents performing different analyses, like topic extraction, sentiment analysis, or named entity recognition.
Aggregation Agent: Combines the results from the analysis agents.
This distributed processing allows the system to handle massive datasets efficiently. If the volume of literature increases, more Ingestion and Analysis agents can be deployed without modifying the core system.
As mentioned earlier, the use of Agents over a more pure LLM-based system also allows us to have much more control over a system, which let’s us build improvements w/o worrying about the inherent randomness of LLMs. This is a huge plus.
3. Robustness
Understanding Software Robustness
Robustness is a system’s ability to withstand failures and continue operating correctly. Robustness can be accidental (someone makes a mistake when using your platform and uploads too much), expected (ESPN expects huge spikes in usage during UFC PPVs), or malicious (someone injects corrupted input or DDoS’s your platforms).
Robustness is key to ensuring that your users can confidently use your product, especially in sensitive domains where small mistakes might cause lasting user churn (minor bugs won’t cause me to stop using IG, they might stop a doctor from using a Medical Diagnosis AI).
How AI Agents Help Software Robustness:
In self-driving vehicles, agent-based systems can enhance safety. Different agents could control braking, steering, and obstacle avoidance. If one agent malfunctions, the others can take over or initiate emergency procedures. This redundancy is crucial for ensuring passenger safety (there is also a much more obvious idea of having backup agents in-case your primary agent fails).
This redundancy and error handling ensures the system continues to function even with partial failures.
Agents are also much safer than relying on one Model to do everything since small errors in one area of the model (even if unrelated) can creep into model behaviors and completely mess up the functionality. An example of this is Gemini infamously refusing to help an under-18 user with C++ b/c it would be unsafe (here the Safety concepts for minors caused conflicts w/ Programming Safety, even though they are semantically very different)
Such mistakes happening when performing legal analysis or building a case argument can have life-long consequences. IQIDIS’s use of Agents allows us to minimize these issues by having much more control than the ChatGPT wrapper that gets pitched as a Legal co-pilot these days.
These are some of the many benefits of building Agentic AI. We will be exploring them in a lot more in-depth in our upcoming articles in our AI Agents Series over here (sign up for free). Also, before you leave, I have an exciting announcement about IQIDIS.
We’re now accepting a limited number of Launch Partners who will receive:
Lifetime locked-in rate of $199/month
Priority access to new features
Early access to Tier 1 (full AI associate suite)
Early access to Tier 2 (practice management and collaboration tools)
We built IQIDIS to transform how law is practiced. If you’re interested in joining us as a Launch Partner and helping shape the future of legal practice, contact us at info@iqidis.ai.
Look forward to hearing from you.
Thank you for reading and hope you have a wonderful day.
Dev <3.
I put a lot of work into writing this newsletter. To do so, I rely on you for support. If a few more people choose to become paid subscribers, the Chocolate Milk Cult can continue to provide high-quality and accessible education and opportunities to anyone who needs it. If you think this mission is worth contributing to, please consider a premium subscription. You can do so for less than the cost of a Netflix Subscription (pay what you want here).
If you liked this article and wish to share it, please refer to the following guidelines.
That is it for this piece. I appreciate your time. As always, if you’re interested in working with me or checking out my other work, my links will be at the end of this email/post. And if you found value in this write-up, I would appreciate you sharing it with more people. It is word-of-mouth referrals like yours that help me grow. You can share your testimonials over here.
Reach out to me
Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.
Small Snippets about Tech, AI and Machine Learning over here
AI Newsletter- https://artificialintelligencemadesimple.substack.com/
My grandma’s favorite Tech Newsletter- https://codinginterviewsmadesimple.substack.com/
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819