loader

Your App Journey Starts Here! Read a FREE chapter and create magic with Power Platform!

Get on Kindle

AI Agents

LLM-based AI agents are applications where the outputs from large language models drive and manage the entire workflow.

flowchart TD A[User Input/Request] --> B[Agent Core LLM] B --> C[Abstract Reasoning & Acting ReAct] C --> D[Planning & Decision Making] D --> E[Tool & Resource Selection] E --> F{Task Type?} F -- Standard Tasks --> G[External APIs / Data Sources] F -- Code Execution --> H[CodeAgent Execution Environment] G --> I[Data & Observations] H --> I I --> J[Memory Update & Contextualization] J --> K[Response Generation] K --> L{Human-in-the-Loop?} L -- Yes --> M[Human Review & Feedback] L -- No --> N[User Output] M --> N I -- Feedback Loop --> D
  • User Input/Request (A): The process begins with the user's query or command.
  • Agent Core (LLM) (B): The language model serves as the central coordinator.
  • Abstract Reasoning & Acting (ReAct) (C): Employs the ReAct framework to combine abstract reasoning with immediate actions.
  • Planning & Decision Making (D): The agent formulates a plan based on the reasoning outputs.
  • Tool & Resource Selection (E): Determines which external resources or code execution paths to use.
  • Task Type Decision (F): A branching point:
    • Standard Tasks (G): Interactions with external APIs or data sources.
    • Code Execution (H): Handling tasks that require code execution via CodeAgents.
  • Data & Observations (I): Collects information from the chosen tools or APIs.
  • Memory Update & Contextualization (J): Updates internal memory to keep track of the context and past interactions.
  • Response Generation (K): Produces a preliminary response based on the aggregated data and context.
  • Human-in-the-Loop Decision (L): An optional branch where a human can review and provide feedback:
    • Human Review & Feedback (M): Allows human intervention to validate or adjust the generated response.
    • User Output (N): Final output delivered to the user after review (or directly if no human review is needed).
  • Feedback Loop: Observations feed back into planning for dynamic adjustments as needed.
# Framework/Platform/Tool Key Focus Strengths Use Cases Notable Features
1 LangChain Empowering the development of LLM-based applications Highly modular architecture, extensive integration capabilities, and vibrant community support Developing chatbots, document analytics, and retrieval-augmented generation systems Robust chain and agent abstractions with a comprehensive third-party ecosystem
2 Microsoft Research AutoGen Advanced multi-agent orchestration and autonomous agent innovation Event-driven design, sophisticated inter-agent communication, and a research-backed framework Experimental task automation, complex autonomous workflows, and cutting-edge research projects Dynamic task delegation, continuous innovation, and an open-source experimental ecosystem
3 AG2 (AgentOS), from AutoGen's original creators Streamline production-ready multi-agent system development through rapid agent orchestration Flexible agent construction, powerful multi-agent conversation framework, and seamless human-AI collaboration Customer service automation, travel planning, game design, and complex problem-solving for both MVPs and enterprise solutions Specialized agent roles (assistant, executor, critic, chat managers), automated dialogue routing, AG2 Studio for visual design, plus integrated scaling and marketplace tools
4 Smolagents Deliver lightweight, collaborative AI agents Optimized for minimal overhead with high customizability and modular design Rapid prototyping and deployments in resource-constrained environments Lean architecture for quick experiments and flexible integration options
5 Microsoft's Agentic AI Frameworks Enterprise-grade agentic AI for scalable, secure solutions Robust security, regulatory compliance, and seamless Azure integration Production applications requiring strong enterprise support Unified runtime combining AutoGen with Semantic Kernel for integrated multi-agent management
6 LangGraph Graph-based orchestration for stateful multi-agent systems Precise control with visual workflow mapping and explicit state management Complex, adaptive AI applications that require detailed process control Nodes-and-edges model extending LangChain for intricate workflow design
7 CrewAI Facilitating role-based collaboration among AI agents Well-defined roles, hierarchical task delegation, and integrated human oversight Simulating organizational workflows and enhancing collaborative problem-solving Role-centric architecture featuring detailed backstories and goal alignment
8 AutoGPT Autonomous AI agents with self-planning capabilities Adaptive learning, high flexibility, and minimal human intervention Automated content creation and task management through autonomous decision-making Iterative task decomposition with built-in self-improvement mechanisms
9 OpenAI Swarm Experimental, lightweight multi-agent coordination Simplicity with minimal orchestration overhead Educational experiments and simple integrations where production-grade robustness is not critical An "anti-framework" leveraging model reasoning for agent handoffs
10 OpenAI Agents SDK Building agents with OpenAI's new tools - Agent development with native AI tools Simplifies the development of agents with built-in tools - Tight GPT-4/5 integration, Pre-configured toolchain, Low-code workflows Autonomous customer support, Complex workflow automation, Data synthesis Responses API: Unified chat + tool execution, Smart Orchestration: Auto-tool selection, Session Memory: Persistent context tracking
11 LlamaIndex Efficient data ingestion and indexing for LLM applications Effective data organization with diverse indexing strategies Enhancing LLM applications via retrieval-augmented generation from various data sources Supports multiple indexing methods including list, vector, tree, keyword, and knowledge graph approaches
12 Langflow Visual interface for building LangChain applications User-friendly design with interactive prototyping and rapid workflow visualization Quick design, testing, and visualization of LLM-based workflows Drag-and-drop UI coupled with visual debugging tailored for LangChain environments
13 Lyzr.ai Agent Studio Streamlined AI orchestration platform Ease of use and strong integration with existing systems Plug-and-play multi-agent solutions for diverse business applications Focus on simplicity, seamless integration, and efficient deployment
14 RASA Conversational AI framework for building chatbots Robust dialogue management and customizable natural language understanding pipelines Developing context-aware, enterprise-grade chatbot solutions Open-source flexibility with strong customization and support for on-premise deployment
15 Atomic Agents Lightweight, atomic agents for fine-grained operations Minimalistic design focused on individual task execution Experimental micro-agent implementations for specific low-level functions Emphasis on modular, atomic operations that enable granular control
16 Phidata (now Agno) Data orchestration combined with AI agent integration Streamlined data pipelines and robust workflow management End-to-end integration of data and AI workflows for enterprise environments A mature platform evolved from Phidata, offering advanced integration and enterprise readiness
17 MetaGPT Hierarchical multi-agent framework for meta-level orchestration Advanced coordination that efficiently manages complex agent hierarchies Applications requiring layered agent coordination and strategic meta-planning Sophisticated hierarchical orchestration for comprehensive multi-agent control
18 SuperAGI Autonomous agents for complex task automation High autonomy, scalability, and production-grade robustness Advanced automation for multi-step projects and enterprise-level tasks Robust self-sufficient design enabling fully autonomous agent operations
19 TaskWeaver Streamlined task orchestration in multi-agent environments Simplifies task management through seamless integration with agent workflows Automating repetitive tasks in enterprise settings Visual workflow builder with tight integration with Microsoft tools
20 AgentGPT Autonomous agent orchestration with goal decomposition Easy setup and an intuitive interface for managing autonomous tasks Small-scale autonomous applications and rapid prototyping Web-based interface that facilitates efficient creation and monitoring of agent tasks
21 ChatDev AI Chat-centric platform for AI development Enhances developer productivity by integrating chat and coding workflows Assisting software development with conversational AI and automated code generation Conversational interface specifically designed for developer-centric workflows
22 Copilot Studio Low-code, graphical agent builder for Microsoft 365 Intuitive design, rich integration with Microsoft 365, and robust security features Building customized AI agents for internal/external use and extending Microsoft 365 Copilot Graphical UI with connectors for over 1,500 data sources and versatile deployment options
23 Salesforce Agentforce 2dx Embedding proactive, autonomous agentic AI into any workflow to anticipate business needs and dynamically act without human intervention Combines advanced natural language processing with proactive data triggers and seamless integration into the Salesforce ecosystem; enhanced by low-code/pro-code tools, real-time analytics, and robust scalability Optimizes customer and employee workflows across industries—from automating customer service inquiries, processing orders, and managing recruitment to streamlining healthcare support and smart home services Features include the Agentforce API, Invocable Actions for embedding in business logic, MuleSoft integration for unified API access, Slack Workflow Builder integration, Agentforce Developer Edition, AgentExchange marketplace, and advanced analytics tools (e.g., Interaction Explorer) for continuous performance optimization
24 AutoAgent Empowering fully autonomous agent orchestration with minimal human oversight Streamlined self-planning, iterative task refinement, and lightweight architecture for seamless integration Automated task management, content creation, workflow orchestration, and rapid prototyping of autonomous systems Plug-and-play modules, self-improvement mechanisms, a user-friendly API, and robust external service connectivity
25 SAP Joule - Agent builder in Joule studio AI-driven business process automation Integration with SAP systems, low-code development, and scalable workflows Supply chain optimization, financial process automation, and HR management Pre-built connectors for SAP system integrations and AI-driven workflows with automated decision-making
26 ServiceNow AI Agents Autonomous workflow management Cross-enterprise data access, advanced reasoning capabilities, and governance & analytics IT service management, customer service automation, and HR process optimization AI Agent Orchestrator for multi-agent collaboration and Now Assist Skill Kit for custom GenAI skills
27 Oracle AI Agents Enterprise AI solution integration Integration with Oracle Cloud, AI-driven insights, and scalable architecture CRM automation, ERP process optimization, and supply chain management Oracle Cloud Integration with unified AI services and AI-driven analytics for real-time insights
28 Vertex AI Agent Builder Custom AI agent development Flexible model training, integration with Google Cloud, and rapid deployment Custom AI model creation, real-time data processing, and edge AI applications AutoML capabilities for simplified model training and Google Cloud Integration for scalable infrastructure
29 Intelligent Agents with WatsonX.ai Cognitive AI solutions for business Advanced NLP capabilities, integration with IBM systems, and AI-driven decision-making Customer service chatbots, business process automation, and data analysis Watson NLP for advanced text analysis and IBM Cloud Integration for unified AI services

Learn Retrieval Augmented Generation (RAG)

RAG

Agentic programs are the conduit that links LLMs to the external world, enabling dynamic interactions with diverse systems and data sources.

When to Use Agents When to Avoid Agents
When the workflow isn't easily determined in advance, requiring dynamic planning and iterative decision-making. When the workflow is well-defined and deterministic, allowing a fixed, rule-based approach.
For handling complex user requests that involve multiple, interacting factors and evolving criteria. When predefined, structured workflows are sufficient to cover all use cases, ensuring simplicity and reliability.
When you need to integrate multiple external data sources (APIs, dashboards, databases) or real-time information. When the overhead of dynamic agent behavior may introduce unnecessary complexity or potential errors.
When leveraging multi-step agent workflows with planning, memory, and tool usage can enhance problem-solving in real-world tasks. When strict control, determinism, and auditability are critical, such as in regulated environments or tasks with low tolerance for unpredictability.
When multi-agent collaboration is beneficial to tackle tasks requiring cooperative decision-making and adaptive control flow. When a simple, linear process is adequate and additional agent orchestration could complicate the system.

Check out updates from AI influencers

"To solve really hard problems, we'll have to use several different representations."
Marvin Minsky
"Artificial intelligence will be the last invention humanity will ever need to make."
Mo Gawdat

Agentic AI glossary

Accuracy

"The correctness of decisions and actions taken by AI agents, validated through continuous learning and feedback mechanisms."

Agent Customization

"Tailoring agents to specific tasks through parameter adjustments or specialized training."

Agent Development

"The process of creating agents with modules for perception, cognition, and action execution."

Agent Interaction

"Communication between agents via shared memory or protocols to coordinate actions."

Agent Memory

"A repository storing short-term (immediate context) and long-term (historical data) information for decision-making."

Agent Prompt

"Instructions guiding an agent’s behavior within specific contexts or tasks."

Agentic AI

"Autonomous systems that perform tasks with minimal human intervention by integrating perception, planning, and action."

Agentic Framework

"A structured architecture enabling agents to autonomously interact with environments and tools."

Agentic Patterns

"Reusable design strategies for building goal-oriented agents, such as multi-step reasoning or collaboration."

Agentic RAG

"Combines retrieval-augmented generation (RAG) with autonomous decision-making for context-aware responses."

Agents

"Autonomous entities that perceive environments, set goals, and execute actions."

AI Agent Collaboration

"Coordination among multiple agents via shared memory or communication protocols to achieve common objectives."

Alignment

"Ensuring agent behavior aligns with ethical guidelines or predefined objectives."

Autonomous Operation

"Goal-driven execution of tasks without constant human oversight."

Cognitive Architecture

"A blueprint for agent design, integrating perception, reasoning, and action modules."

Collaboration

"Agents working together through shared goals and coordinated plans."

Concept-CoT Agent

"An agent using chain-of-thought reasoning to break down abstract concepts into actionable steps."

Continual Pretraining

"Ongoing training of models on new data to maintain relevance and adaptability."

CoT (Chain-of-Thought)

"A reasoning method where agents decompose problems into sequential steps."

Design Patterns

"Reusable solutions for common challenges in agent architecture, like coordination or error handling."

Distillation

"Compressing complex models into smaller, efficient versions while retaining core capabilities."

Functional Calling

"The ability of agents to invoke external tools or APIs during task execution."

Goal

"The objective an agent aims to achieve, guiding its planning and actions."

HITL (Human-in-the-Loop)

"Human oversight for validation, correction, or ethical compliance in agent operations."

Improvement Over Time

"Agents refining performance through learning algorithms like RLHF or supervised fine-tuning."

Logicality

"Coherent and consistent reasoning processes within agents."

Long-term Memory

"Persistent storage of historical data for informed decision-making."

LRM

"Language Reasoning Model (context-specific term; possibly a variant of LLM)."

MAS (Multi-Agent Systems)

"Networks of agents collaborating to solve complex problems."

MCP

"The Model Context Protocol (MCP) is an open-source standard developed by Anthropic to simplify and standardize how large language models (LLMs) interact with external data sources and tools. MCP enables seamless integration by providing a universal interface, eliminating the need for custom integrations, and allowing AI applications to access context-rich data efficiently through a client-server architecture using JSON-RPC communication"

Model Outputs

"Structured or unstructured results generated by agents, such as decisions or data."

MoE (Mixture of Experts)

"Architecture where specialized submodels handle distinct tasks."

Multi-Agent CoT Prompting

"Coordinated chain-of-thought reasoning across multiple agents."

Multi-Agent Conversations

"Interactions between agents using natural language to negotiate or collaborate."

Multi-Agents

"Systems where multiple agents interact, each with specialized roles."

Multi-step Processes

"Tasks requiring sequential planning and execution across interdependent steps."

Open-Ended Problems

"Challenges without predefined solutions, requiring adaptive reasoning and creativity."

Orchestration

"Managing agent workflows, tool usage, and resource allocation."

Post-Training

"Techniques like fine-tuning applied after initial model training to enhance performance."

Procedural Memory

"Storage of learned skills or processes for task execution."

Prompt Template

"Predefined structures guiding agent responses or actions in specific scenarios."

RAG (Retrieval-Augmented Generation)

"Enhancing responses with external data retrieval for accuracy."

RAG-powered Contextual Understanding

"Using retrieved data to inform real-time decisions."

ReAct (Reasoning and Acting)

"A framework where agents alternate between reasoning and taking actions."

Reasoning

"Processing information to derive insights, often using LLMs for logical inference."

Reflection

"Agents analyzing past actions to improve future decisions."

Reinforcement Learning

"Training agents via rewards/penalties to optimize behavior."

RLHF (Reinforcement Learning from Human Feedback)

"Aligning agent behavior with human preferences through feedback."

Short-term Memory

"Temporary storage of immediate context for real-time decision-making."

Structured Outputs

"Formatted results (e.g., JSON or tables) ensuring consistency in agent responses."

Supervised Fine-Tuning

"Refining pre-trained models using labeled data for specific tasks."

System Prompt

"High-level directives defining an agent’s role or operational boundaries."

Tools

"External resources (APIs, databases) agents use to execute tasks."

Workflows

"Sequences of automated steps agents follow to accomplish complex tasks."