AI agent tools aren’t ready for the real world yet, new study finds

Recent research underscores the rapid advancement of software frameworks that support AI agents—systems designed for independent task execution. Despite their improvement, a new study highlights that these frameworks remain largely unfit for extensive real-world deployment. The research, titled "Agentic AI Frameworks Under The Microscope," was conducted by Karthik Vaidhyanathan from the International Institute of Information Technology Hyderabad and Davide Taibi, head of software architecture at the University of Southern Denmark, and was published in IEEE Software, a respected peer-reviewed journal. The paper investigates some of the most widely used agentic AI frameworks, which enable developers to create systems capable of planning, utilizing tools, and collaborating with other AI agents. These frameworks are increasingly being employed to automate various tasks, including coding, email management, data analysis, and workflow coordination. However, the authors caution that the technology still displays significant fragility despite the growing interest. The study points out that many agentic AI frameworks are still in their developmental stages, exhibiting limitations in memory capabilities, basic planning functions, and inconsistent ecosystem support. The authors warn companies to approach these tools with care, as "scalability and reliability remain open challenges" in their implementation. One notable area of improvement is modularity, with most modern frameworks now allowing for the integration of external tools such as databases and APIs. This enhancement facilitates the connection between AI agents and real-world systems. However, the frameworks struggle with memory retention. Many utilize vector storage methods, which save information as numerical data, enabling AI models to recall past interactions. Unfortunately, this memory is often basic and transient. Some frameworks allow memory sharing among multiple agents, promoting collaboration, while others are limited to single-session memory retention. While this makes testing easier, it hampers their effectiveness for long-term projects that require sustained continuity. Few systems support a unified memory, referred to as global state, that synchronizes information across agents. Planning capabilities also emerged as a weak point in the study. Current AI agents predominantly rely on simple trial-and-error methods or reactive strategies, lacking the deeper reasoning abilities that combine logic, rules, and learning. The research also examined the landscape of developer adoption. Several open-source projects have garnered substantial contributor and follower bases, while newer frameworks are quickly gaining traction. Most tools are primarily developed in Python, the leading programming language for AI, with some also accommodating JavaScript and enterprise languages like C# and Java to appeal to both startups and larger corporations. However, the authors argue that essential enterprise features are either missing or inadequately developed, indicating that today's agentic AI platforms are still more akin to experimental frameworks than market-ready products. To advance beyond prototype stages, the study calls for significant enhancements in persistent memory, hybrid reasoning capabilities, improved security measures, and better monitoring systems.