TechSetupGuides
IntermediateAutoGenMicrosoftAIMulti-AgentPython.NETLLMAgent FrameworkMCPOpenAIAzureDockerGradio

microsoft/autogen: Multi-Agent AI Framework

Complete setup guide for Microsoft AutoGen - a programming framework for creating multi-agent AI applications. Includes installation of core components (autogen-core, autogen-agentchat, autogen-ext), AutoGen Studio GUI, model integration with OpenAI/Azure/Ollama/Anthropic, MCP server support, code execution, and production deployment patterns.

  1. Step 1

    Important: AutoGen is in Maintenance Mode

    Before starting, note that AutoGen is now in maintenance mode and will not receive new features. For new projects, Microsoft recommends starting with Microsoft Agent Framework (MAF), which is the enterprise-ready successor. This guide is for existing AutoGen users and those learning from the framework.

    # For new projects, consider Microsoft Agent Framework instead:
    # https://github.com/microsoft/agent-framework
    # Migration guide: https://learn.microsoft.com/en-us/agent-framework/migration-guide/from-autogen/
    ⚠ Heads up: AutoGen is community-managed going forward. Use this guide to maintain existing AutoGen projects or learn multi-agent patterns.
  2. Step 2

    System Prerequisites

    AutoGen requires Python 3.10 or later. You'll need pip or uv for package management. Optional tools include Docker for code execution sandboxing and Node.js for AutoGen Studio web interface.

    # Check Python version (3.10+ required)
    python3 --version
    
    # Check pip (or install uv as modern alternative)
    pip --version
    uv --version  # Optional: faster package manager
    
    # Optional: Docker for code execution sandboxing
    docker --version
    
    # Optional: Node.js for AutoGen Studio
    node --version  # For frontend components
    npm --version
  3. Step 3

    Understand the AutoGen Architecture

    AutoGen uses a layered architecture with three main components:

    • autogen-core: Event-driven programming framework for scalable multi-agent systems
    • autogen-agentchat: Higher-level API for rapid prototyping (similar to v0.2)
    • autogen-ext: Extensions library with integrations (OpenAI, Azure, Anthropic, MCP, etc.)
    AutoGen Package Structure:
    ├── autogen-core (0.7.5)
    │   ├── Event-driven agents
    │   ├── Message passing
    │   ├── Local/distributed runtime
    │   └── Cross-language support (.NET/Python)
    ├── autogen-agentchat (0.7.5)
    │   ├── Simpler API
    │   ├── Two-agent chat
    │   ├── Group chats
    │   └── Built on Core
    └── autogen-ext (0.7.5)
        ├── LLM clients
        ├── MCP support
        ├── Code execution
        └── External integrations
    ⚠ Heads up: Unlike older v0.2, AutoGen 0.7+ requires understanding async programming and event patterns. Review the migration guide if upgrading.
  4. Step 4

    Install Core Packages

    Install the three main AutoGen packages: autogen-core (foundation), autogen-agentchat (high-level API), and autogen-ext with optional dependencies for your target LLM provider.

    # Create a project directory
    mkdir autogen-demo
    cd autogen-demo
    
    # Create virtual environment (recommended)
    python3 -m venv .venv
    source .venv/bin/activate  # Linux/macOS
    
    # Windows: .venv\Scripts\activate
    
    # Install core packages with OpenAI support
    pip install -U "autogen-agentchat" "autogen-ext[openai]"
    
    # Verify installation
    python -c "import autogen_agentchat; print('AutoGen AgentChat installed')"
    python -c "import autogen_core; print('AutoGen Core installed')"
  5. Step 5

    Set Up LLM API Key

    AutoGen requires an LLM provider. The most common is OpenAI GPT-4o. Alternative providers include Azure OpenAI, Anthropic, Ollama (local), and Google Gemini. Set your API key as an environment variable.

    # OpenAI (GPT-4o, GPT-3.5)
    export OPENAI_API_KEY="your-api-key-here"
    
    # Azure OpenAI (enterprise)
    export AZURE_API_KEY="your-azure-key"
    export AZURE_API_VERSION="2024-02-01"
    export AZURE_DEPLOYMENT_NAME="gpt-4o"
    
    # Anthropic (Claude models)
    export ANTHROPIC_API_KEY="your-anthropic-key"
    
    # Ollama (local models - no API key needed)
    # Download from https://ollama.ai
    # Then run: ollama pull llama3.2
    
    # For development, use a .env file (install python-dotenv first)
    echo "OPENAI_API_KEY=your-api-key-here" > .env
    pip install python-dotenv
  6. Step 6

    Quick Start: Hello World

    Create a minimal AutoGen agent to test your setup. This uses the AgentChat API to create an assistant agent with OpenAI's GPT-4o model.

    import asyncio
    from autogen_agentchat.agents import AssistantAgent
    from autogen_ext.models.openai import OpenAIChatCompletionClient
    
    async def main() -> None:
        # Create model client
        model_client = OpenAIChatCompletionClient(model="gpt-4o")
        
        # Create assistant agent
        agent = AssistantAgent("assistant", model_client=model_client)
        
        # Run a task
        result = await agent.run(task="Say 'Hello World!'")
        print(result)
        
        # Close the connection
        await model_client.close()
    
    # Run the async function
    asyncio.run(main())
  7. Step 7

    Multi-Agent Orchestration

    AutoGen's strength is multi-agent collaboration. Create specialized agents (math, chemistry, web browsing) and orchestrate them using AgentTool for complex workflows.

    import asyncio
    from autogen_agentchat.agents import AssistantAgent
    from autogen_agentchat.tools import AgentTool
    from autogen_agentchat.ui import Console
    from autogen_ext.models.openai import OpenAIChatCompletionClient
    
    async def main() -> None:
        model_client = OpenAIChatCompletionClient(model="gpt-4o")
    
        # Create specialized agents
        math_agent = AssistantAgent(
            "math_expert",
            model_client=model_client,
            system_message="You are a math expert.",
            description="A math expert assistant.",
            model_client_stream=True,
        )
        
        chemistry_agent = AssistantAgent(
            "chemistry_expert",
            model_client=model_client,
            system_message="You are a chemistry expert.",
            description="A chemistry expert assistant.",
            model_client_stream=True,
        )
        
        # Create tools for the agents
        math_tool = AgentTool(math_agent, return_value_as_last_message=True)
        chemistry_tool = AgentTool(chemistry_agent, return_value_as_last_message=True)
        
        # Create a coordinator agent with tools
        coordinator = AssistantAgent(
            "coordinator",
            system_message="You are a general assistant. Use expert tools when needed.",
            model_client=model_client,
            tools=[math_tool, chemistry_tool],
            max_tool_iterations=10,
        )
        
        # Run tasks with multi-agent collaboration
        await Console(coordinator.run_stream(task="What is the integral of x^2?"))
        await Console(coordinator.run_stream(task="What is the molecular weight of water?"))
        
        # Clean up connections
        await model_client.close()
    
    asyncio.run(main())
  8. Step 8

    Install AutoGen Studio (Optional GUI)

    AutoGen Studio provides a no-code web-based UI for prototyping multi-agent workflows. Install and run it separately from your coding projects.

    # Install AutoGen Studio
    pip install -U autogenstudio
    
    # Launch the web UI on port 8080
    autogenstudio ui --port 8080 --appdir ./my-app
    
    # Open browser to http://localhost:8080
    
    # Note: AutoGen Studio is for prototyping, not production
    # It does not include authentication or security features
    ⚠ Heads up: AutoGen Studio is NOT production-ready. It's a rapid prototyping tool without authentication, security, or error handling. Use the AutoGen framework to build production applications.
  9. Step 9

    Integrate with Model Context Protocol (MCP)

    Then use MCP in your agent code:

    import asyncio
    from autogen_agentchat.agents import AssistantAgent
    from autogen_agentchat.ui import Console
    from autogen_ext.models.openai import OpenAIChatCompletionClient
    from autogen_ext.tools.mcp import McpWorkbench, StdioServerParams
    
    async def main() -> None:
        model_client = OpenAIChatCompletionClient(model="gpt-4o")
        
        # Configure MCP server (Playwright for web browsing)
        server_params = StdioServerParams(
            command="npx",
            args=["@playwright/mcp@latest", "--headless"],
        )
        
        async with McpWorkbench(server_params) as mcp:
            agent = AssistantAgent(
                "web_browsing_assistant",
                model_client=model_client,
                workbench=mcp,
                model_client_stream=True,
                max_tool_iterations=10,
            )
            await Console(
                agent.run_stream(task="Count stars in microsoft/autogen repository on GitHub")
            )
    
    asyncio.run(main())
  10. Step 10

    Code Execution and Sandboxing

    AutoGen can execute code dynamically using Docker-based sandboxing or local execution. This enables agents to write and run Python code, perform data analysis, and more. Always use Docker for untrusted code execution.

    from autogen_core.code_executor import DockerCommandLineCodeExecutor
    
    # Using Docker for safe code execution (RECOMMENDED)
    async with DockerCommandLineCodeExecutor(
        image_name="python:3.11",
        work_dir="/tmp",
    ) as code_executor:
        result = await code_executor.execute_code_block(
            code="print('Hello from sandboxed environment')"
        )
        print(f"Output: {result.output}")
    
    # Using local execution (NOT recommended for production)
    from autogen_core.code_executor import CommandLineCodeExecutor
    async with CommandLineCodeExecutor() as local_executor:
        result = await local_executor.execute_code_block(code="print('Local execution')")
    ⚠ Heads up: Never execute untrusted code locally. Always use Docker sandboxing for production. Install docker-py library for Docker execution.
  11. Step 11

    Production Configuration

    For production deployments, set up proper environment management, logging, error handling, and resource limits. Use Gradio or FastAPI for web interfaces.

    import asyncio
    import os
    from dotenv import load_dotenv
    from autogen_agentchat.agents import AssistantAgent
    from autogen_ext.models.openai import OpenAIChatCompletionClient
    
    # Load environment variables
    load_dotenv()
    
    async def create_agent() -> AssistantAgent:
        api_key = os.getenv("OPENAI_API_KEY")
        if not api_key:
            raise ValueError("OPENAI_API_KEY not set")
        
        model_client = OpenAIChatCompletionClient(
            model="gpt-4o",
            api_key=api_key,
            max_tokens=1024
        )
        
        agent = AssistantAgent(
            "production_assistant",
            model_client=model_client,
            system_message="You are a helpful, safe AI assistant.",
            description="Production-grade assistant agent",
            max_tool_iterations=10,
            model_client_stream=True,
        )
        return agent, model_client
    
    async def main():
        try:
            agent, model_client = await create_agent()
            result = await agent.run(task="Create a simple Python function")
            print(f"Result: {result}")
        except Exception as e:
            print(f"Error: {e}")
        finally:
            if model_client:
                await model_client.close()
  12. Step 12

    Test the Installation

    Run a comprehensive test to verify all components are working: core packages, model integration, and optional features.

    # Test 1: Verify packages are installed
    python3 << 'EOF'
    import autogen_core
    import autogen_agentchat
    print("Core packages installed")
    
    try:
        from autogen_ext.models.openai import OpenAIChatCompletionClient
        print("OpenAI integration available")
    except ImportError:
        print("OpenAI integration missing")
        
    try:
        from autogen_core.code_executor import DockerCommandLineCodeExecutor
        print("Docker code execution available")
    except ImportError:
        print("Docker execution missing")
    EOF
  13. Step 13

    Deployment Options

    Create a requirements.txt file with:

    autogen-core==0.7.5
    autogen-agentchat==0.7.5
    autogen-ext[openai,docker]==0.7.5
    python-dotenv
    fastapi
    uvicorn
  14. Step 14

    Common Integrations

    AutoGen supports various LLM providers and tools through autogen-ext. Here's how to install key integrations.

    # Azure (enterprise deployment)
    pip install "autogen-ext[azure]"
    
    # Anthropic (Claude models)
    pip install "autogen-ext[anthropic]"
    
    # Google Gemini
    pip install "autogen-ext[gemini]"
    
    # Ollama (local models)
    pip install "autogen-ext[ollama]"
    
    # Llama.cpp (local, low-latency)
    pip install "autogen-ext[llama-cpp]"
    
    # File processing and analysis
    pip install "autogen-ext[file-surfer] autogen-ext[web-surfer]"
    
    # Code execution
    pip install "autogen-ext[docker]"
    
    # Memory and knowledge graphs
    pip install "autogen-ext[chromadb] autogen-ext[graphrag]"
    
    # Semantic Kernel integration
    pip install "autogen-ext[semantic-kernel-core]"
  15. Step 15

    Advanced Features: Memory and Knowledge Graphs

    AutoGen supports memory systems and knowledge graphs through ChromaDB, Mem0, and GraphRAG extensions for persistent agent memory and retrieval.

    # Memory with ChromaDB
    from autogen_ext.memory.chromadb import ChromaDBMemory
    from autogen_agentchat.agents import AssistantAgent
    
    # Install: pip install "autogen-ext[chromadb]"
    
    chroma_memory = ChromaDBMemory(collection_name="agent_memory")
    
    agent = AssistantAgent(
        "remembering_agent",
        model_client=model_client,
        memory=chroma_memory,
        system_message="Remember user preferences and past conversations."
    )
    
    # Knowledge Graph with GraphRAG
    # Install: pip install "autogen-ext[graphrag]"
    from autogen_ext.graphrag import GraphRAGIndexer, GraphRAGQueryEngine
    from autogen_agentchat.tools import AgentTool
    
    indexer = GraphRAGIndexer(graphrag_config_path="config.yaml")
    query_engine = GraphRAGQueryEngine(indexer=indexer)
    
    agent = AssistantAgent(
        "knowledge_agent",
        model_client=model_client,
        tools=[AgentTool(query_engine)]
    )
  16. Step 16

    Troubleshooting Common Issues

    Here are common issues and their solutions when setting up AutoGen.

    Issue: API key errors
    Solution: Check OPENAI_API_KEY is set and valid:
      export OPENAI_API_KEY="sk-..."
    
    Issue: Docker execution fails
    Solution: Ensure Docker is running and python-docker is installed:
      pip install docker
      docker ps  # Verify Docker daemon
    
    Issue: Async programming errors
    Solution: AutoGen 0.7+ uses async/await. Ensure you run:
      asyncio.run(main())
      NOT: main()
    
    Issue: Missing optional dependencies
    Solution: Install specific extras:
      pip install "autogen-ext[openai] autogen-ext[docker]"
    
    Issue: AutoGen Studio won't start
    Solution: Clear appdir and reinstall:
      rm -rf ./myapp
      pip install --upgrade --force-reinstall autogenstudio
    
    Issue: Model compatibility errors
    Solution: Check supported models in documentation. GPT-4o is recommended.
    
    Issue: Network timeouts
    Solution: Increase connection timeout:
      OpenAIChatCompletionClient(
          model="gpt-4o",
          timeout=30
      )
    
    Issue: Migration from v0.2
    Solution: Review migration guide:
      https://github.com/microsoft/autogen/blob/main/MIGRATION-GUIDE.md
  17. Step 17

    Additional Resources

    For more information on AutoGen and related tools:

    Official Resources:

    • GitHub: https://github.com/microsoft/autogen
    • Documentation: https://microsoft.github.io/autogen/
    • Discord Community: https://discord.gg/autogen

    Migration Guide:

    • Microsoft Agent Framework Migration: https://learn.microsoft.com/en-us/agent-framework/migration-guide/from-autogen/
    • Microsoft Agent Framework: https://github.com/microsoft/agent-framework

    PyPI Packages:

    • autogen-core: https://pypi.org/project/autogen-core/
    • autogen-agentchat: https://pypi.org/project/autogen-agentchat/
    • autogen-ext: https://pypi.org/project/autogen-ext/

Feature requests

Sign in to suggest features or vote on existing ones.

No feature requests yet.

Discussion

0 people marked this as worked·Sign in to mark your own.

Sign in to join the discussion.

No comments yet.