# Architecture Specification: Dual-Mode Agent System **Date:** November 27, 2025 **Status:** SPECIFICATION **Goal:** Graceful degradation from full multi-agent orchestration to simple single-agent mode --- ## 1. Core Concept: Two Operating Modes ```text ┌─────────────────────────────────────────────────────────────────────┐ │ USER REQUEST │ │ │ │ │ ▼ │ │ ┌─────────────────┐ │ │ │ Mode Selection │ │ │ │ (Auto-detect) │ │ │ └────────┬────────┘ │ │ │ │ │ ┌───────────────┴───────────────┐ │ │ │ │ │ │ ▼ ▼ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ SIMPLE MODE │ │ ADVANCED MODE │ │ │ │ (Free Tier) │ │ (Paid Tier) │ │ │ │ │ │ │ │ │ │ pydantic-ai │ │ MS Agent Fwk │ │ │ │ single-agent │ │ + pydantic-ai │ │ │ │ loop │ │ multi-agent │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ └───────────────┬───────────────┘ │ │ ▼ │ │ ┌─────────────────┐ │ │ │ Research Report │ │ │ │ with Citations │ │ │ └─────────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ ``` --- ## 2. Mode Comparison | Aspect | Simple Mode | Advanced Mode | |--------|-------------|---------------| | **Trigger** | No API key OR `LLM_PROVIDER=huggingface` | OpenAI API key present (currently OpenAI only) | | **Framework** | pydantic-ai only | Microsoft Agent Framework + pydantic-ai | | **Architecture** | Single orchestrator loop | Multi-agent coordination | | **Agents** | One agent does Search→Judge→Report | SearchAgent, JudgeAgent, ReportAgent, AnalysisAgent | | **State Management** | Simple dict | Thread-safe `MagenticState` with context vars | | **Quality** | Good (functional) | Better (specialized agents, coordination) | | **Cost** | Free (HuggingFace Inference) | Paid (OpenAI/Anthropic) | | **Use Case** | Demos, hackathon, budget-constrained | Production, research quality | --- ## 3. Simple Mode Architecture (pydantic-ai Only) ```text ┌─────────────────────────────────────────────────────┐ │ Orchestrator │ │ │ │ while not sufficient and iteration < max: │ │ 1. SearchHandler.execute(query) │ │ 2. JudgeHandler.assess(evidence) ◄── pydantic-ai Agent │ │ 3. if sufficient: break │ │ 4. query = judge.next_queries │ │ │ │ return ReportGenerator.generate(evidence) │ └─────────────────────────────────────────────────────┘ ``` **Components:** - `src/orchestrator.py` - Simple loop orchestrator - `src/agent_factory/judges.py` - JudgeHandler with pydantic-ai - `src/tools/search_handler.py` - Scatter-gather search - `src/tools/pubmed.py`, `clinicaltrials.py`, `europepmc.py` - Search tools --- ## 4. Advanced Mode Architecture (MS Agent Framework + pydantic-ai) ```text ┌─────────────────────────────────────────────────────────────────────┐ │ Microsoft Agent Framework Orchestrator │ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ SearchAgent │───▶│ JudgeAgent │───▶│ ReportAgent │ │ │ │ (BaseAgent) │ │ (BaseAgent) │ │ (BaseAgent) │ │ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ │ │ │ │ ▼ ▼ ▼ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ pydantic-ai │ │ pydantic-ai │ │ pydantic-ai │ │ │ │ Agent() │ │ Agent() │ │ Agent() │ │ │ │ output_type=│ │ output_type=│ │ output_type=│ │ │ │ SearchResult│ │ JudgeAssess │ │ Report │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ │ Shared State: MagenticState (thread-safe via contextvars) │ │ - evidence: list[Evidence] │ │ - embedding_service: EmbeddingService │ └─────────────────────────────────────────────────────────────────────┘ ``` **Components:** - `src/orchestrator_magentic.py` - Multi-agent orchestrator - `src/agents/search_agent.py` - SearchAgent (BaseAgent) - `src/agents/judge_agent.py` - JudgeAgent (BaseAgent) - `src/agents/report_agent.py` - ReportAgent (BaseAgent) - `src/agents/analysis_agent.py` - AnalysisAgent (BaseAgent) - `src/agents/state.py` - Thread-safe state management - `src/agents/tools.py` - @ai_function decorated tools --- ## 5. Mode Selection Logic ```python # src/orchestrator_factory.py (actual implementation) def create_orchestrator( search_handler: SearchHandlerProtocol | None = None, judge_handler: JudgeHandlerProtocol | None = None, config: OrchestratorConfig | None = None, mode: Literal["simple", "magentic", "advanced"] | None = None, ) -> Any: """ Auto-select orchestrator based on available credentials. Priority: 1. If mode explicitly set, use that 2. If OpenAI key available -> Advanced Mode (currently OpenAI only) 3. Otherwise -> Simple Mode (HuggingFace free tier) """ effective_mode = _determine_mode(mode) if effective_mode == "advanced": orchestrator_cls = _get_magentic_orchestrator_class() return orchestrator_cls(max_rounds=config.max_iterations if config else 10) # Simple mode requires handlers if search_handler is None or judge_handler is None: raise ValueError("Simple mode requires search_handler and judge_handler") return Orchestrator( search_handler=search_handler, judge_handler=judge_handler, config=config, ) ``` --- ## 6. Shared Components (Both Modes Use) These components work in both modes: | Component | Purpose | |-----------|---------| | `src/tools/pubmed.py` | PubMed search | | `src/tools/clinicaltrials.py` | ClinicalTrials.gov search | | `src/tools/europepmc.py` | Europe PMC search | | `src/tools/search_handler.py` | Scatter-gather orchestration | | `src/tools/rate_limiter.py` | Rate limiting | | `src/utils/models.py` | Evidence, Citation, JudgeAssessment | | `src/utils/config.py` | Settings | | `src/services/embeddings.py` | Vector search (optional) | --- ## 7. pydantic-ai Integration Points Both modes use pydantic-ai for structured LLM outputs: ```python # In JudgeHandler (both modes) from pydantic_ai import Agent from pydantic_ai.models.huggingface import HuggingFaceModel from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.models.anthropic import AnthropicModel class JudgeHandler: def __init__(self, model: Any = None): self.model = model or get_model() # Auto-selects based on config self.agent = Agent( model=self.model, output_type=JudgeAssessment, # Structured output! system_prompt=SYSTEM_PROMPT, ) async def assess(self, question: str, evidence: list[Evidence]) -> JudgeAssessment: result = await self.agent.run(format_prompt(question, evidence)) return result.output # Guaranteed to be JudgeAssessment ``` --- ## 8. Microsoft Agent Framework Integration Points Advanced mode wraps pydantic-ai agents in BaseAgent: ```python # In JudgeAgent (advanced mode only) from agent_framework import BaseAgent, AgentRunResponse, ChatMessage, Role class JudgeAgent(BaseAgent): def __init__(self, judge_handler: JudgeHandlerProtocol): super().__init__( name="JudgeAgent", description="Evaluates evidence quality", ) self._handler = judge_handler # Uses pydantic-ai internally async def run(self, messages, **kwargs) -> AgentRunResponse: question = extract_question(messages) evidence = self._evidence_store.get("current", []) # Delegate to pydantic-ai powered handler assessment = await self._handler.assess(question, evidence) return AgentRunResponse( messages=[ChatMessage(role=Role.ASSISTANT, text=format_response(assessment))], additional_properties={"assessment": assessment.model_dump()}, ) ``` --- ## 9. Benefits of This Architecture 1. **Graceful Degradation**: Works without API keys (free tier) 2. **Progressive Enhancement**: Better with API keys (orchestration) 3. **Code Reuse**: pydantic-ai handlers shared between modes 4. **Hackathon Ready**: Demo works without requiring paid keys 5. **Production Ready**: Full orchestration available when needed 6. **Future Proof**: Can add more agents to advanced mode 7. **Testable**: Simple mode is easier to unit test --- ## 10. Known Risks and Mitigations > **From Senior Agent Review** ### 10.1 Bridge Complexity (MEDIUM) **Risk:** In Advanced Mode, agents (Agent Framework) wrap handlers (pydantic-ai). Both are async. Context variables (`MagenticState`) must propagate correctly through the pydantic-ai call stack. **Mitigation:** - pydantic-ai uses standard Python `contextvars`, which naturally propagate through `await` chains - Test context propagation explicitly in integration tests - If issues arise, pass state explicitly rather than via context vars ### 10.2 Integration Drift (MEDIUM) **Risk:** Simple Mode and Advanced Mode might diverge in behavior over time (e.g., Simple Mode uses logic A, Advanced Mode uses logic B). **Mitigation:** - Both modes MUST call the exact same underlying Tools (`src/tools/*`) and Handlers (`src/agent_factory/*`) - Handlers are the single source of truth for business logic - Agents are thin wrappers that delegate to handlers ### 10.3 Testing Burden (LOW-MEDIUM) **Risk:** Two distinct orchestrators (`src/orchestrator.py` and `src/orchestrator_magentic.py`) doubles integration testing surface area. **Mitigation:** - Unit test handlers independently (shared code) - Integration tests for each mode separately - End-to-end tests verify same output for same input (determinism permitting) ### 10.4 Dependency Conflicts (LOW) **Risk:** `agent-framework-core` might conflict with `pydantic-ai`'s dependencies (e.g., different pydantic versions). **Status:** Both use `pydantic>=2.x`. Should be compatible. --- ## 11. Naming Clarification > See `00_SITUATION_AND_PLAN.md` Section 4 for full details. **Important:** The codebase uses "magentic" in file names (`orchestrator_magentic.py`, `magentic_agents.py`) but this refers to our internal naming for Microsoft Agent Framework integration, **NOT** the `magentic` PyPI package. **Future action:** Rename to `orchestrator_advanced.py` to eliminate confusion.