implements documentation improvements

AUDIO_INPUT_FIX.md

@@ -1,94 +0,0 @@
-# Audio Input Display Fix
-
-## Issue
-The audio input (microphone button) was not displaying in the ChatInterface multimodal textbox.
-
-## Root Cause
-When `multimodal=True` is set on `gr.ChatInterface`, it should automatically show image and audio buttons. However:
-1. The buttons might be hidden in a dropdown menu
-2. Browser permissions might be blocking microphone access
-3. The `file_types` parameter might not have been explicitly set
-
-## Fix Applied
-
-### 1. Added `file_types` Parameter
-Explicitly specified which file types are accepted to ensure audio is enabled:
-
-```python
-gr.ChatInterface(
-    fn=research_agent,
-    multimodal=True,
-    file_types=["image", "audio", "video"],  # Explicitly enable image, audio, and video
-    ...
-)
-```
-
-**File:** `src/app.py` (line 929)
-
-### 2. Enhanced UI Description
-Updated the description to make it clearer where to find the audio input:
-
-- Added explicit instructions about clicking the 📷 and 🎤 icons
-- Added a tip about looking for icons in the text input box
-- Clarified drag & drop functionality
-
-**File:** `src/app.py` (lines 942-948)
-
-## How It Works Now
-
-1. **Audio Recording Button**: The 🎤 microphone icon should appear in the textbox toolbar when `multimodal=True` is set
-2. **File Upload**: Users can drag & drop audio files or click to upload
-3. **Browser Permissions**: Browser will prompt for microphone access when user clicks the audio button
-
-## Testing
-
-To verify the fix:
-1. Look for the 🎤 microphone icon in the text input box
-2. Click it to start recording (browser will ask for microphone permission)
-3. Alternatively, drag & drop an audio file into the textbox
-4. Check browser console for any permission errors
-
-## Browser Requirements
-
-- **Chrome/Edge**: Should work with microphone permissions
-- **Firefox**: Should work with microphone permissions  
-- **Safari**: May require additional configuration
-- **HTTPS Required**: Microphone access typically requires HTTPS (or localhost)
-
-## Troubleshooting
-
-If audio input still doesn't appear:
-
-1. **Check Browser Permissions**:
-   - Open browser settings
-   - Check microphone permissions for the site
-   - Ensure microphone is not blocked
-
-2. **Check Browser Console**:
-   - Open Developer Tools (F12)
-   - Look for permission errors or warnings
-   - Check for any JavaScript errors
-
-3. **Try Different Browser**:
-   - Some browsers have stricter permission policies
-   - Try Chrome or Firefox if Safari doesn't work
-
-4. **Check Gradio Version**:
-   - Ensure `gradio>=6.0.0` is installed
-   - Update if needed: `pip install --upgrade gradio`
-
-5. **HTTPS Requirement**:
-   - Microphone access requires HTTPS (or localhost)
-   - If deploying, ensure SSL is configured
-
-## Additional Notes
-
-- The audio button is part of the MultimodalTextbox component
-- It should appear as an icon in the textbox toolbar
-- If it's still not visible, it might be in a dropdown menu (click the "+" or "..." button)
-- The `file_types` parameter ensures audio files are accepted for upload
-
-
-
-
-

docs/contributing.md → CONTRIBUTING.md

@@ -12,7 +12,14 @@ Thank you for your interest in contributing to The DETERMINATOR! This guide will
 - [Key Principles](#key-principles)
 - [Pull Request Process](#pull-request-process)
 
-> **Note**: Additional sections (Code Style, Error Handling, Testing, Implementation Patterns, Code Quality, and Prompt Engineering) are available as separate pages in the navigation sidebar.
+> **Note**: Additional sections (Code Style, Error Handling, Testing, Implementation Patterns, Code Quality, and Prompt Engineering) are available as separate pages in the [documentation](https://deepcritical.github.io/GradioDemo/contributing/).  
+> **Note on Project Names**: "The DETERMINATOR" is the product name, "DeepCritical" is the organization/project name, and "determinator" is the Python package name.
+
+## Repository Information
+
+- **GitHub Repository**: [`DeepCritical/GradioDemo`](https://github.com/DeepCritical/GradioDemo) (source of truth, PRs, code review)
+- **HuggingFace Space**: [`DataQuests/DeepCritical`](https://huggingface.co/spaces/DataQuests/DeepCritical) (deployment/demo)
+- **Package Name**: `determinator` (Python package name in `pyproject.toml`)
 
 ## Git Workflow
 
@@ -22,9 +29,31 @@ Thank you for your interest in contributing to The DETERMINATOR! This guide will
 - **NEVER** push directly to `main` or `dev` on HuggingFace
 - GitHub is source of truth; HuggingFace is for deployment
 
+### Dual Repository Setup
+
+This project uses a dual repository setup:
+
+- **GitHub (`DeepCritical/GradioDemo`)**: Source of truth for code, PRs, and code review
+- **HuggingFace (`DataQuests/DeepCritical`)**: Deployment target for the Gradio demo
+
+#### Remote Configuration
+
+When cloning, set up remotes as follows:
+
+```bash
+# Clone from GitHub
+git clone https://github.com/DeepCritical/GradioDemo.git
+cd GradioDemo
+
+# Add HuggingFace remote (optional, for deployment)
+git remote add huggingface-upstream https://huggingface.co/spaces/DataQuests/DeepCritical
+```
+
+**Important**: Never push directly to `main` or `dev` on HuggingFace. Always work through GitHub PRs. GitHub is the source of truth; HuggingFace is for deployment/demo only.
+
 ## Getting Started
 
-1. **Fork the repository** on GitHub
+1. **Fork the repository** on GitHub: [`DeepCritical/GradioDemo`](https://github.com/DeepCritical/GradioDemo)
 2. **Clone your fork**:
 
    ```bash
@@ -35,7 +64,8 @@ Thank you for your interest in contributing to The DETERMINATOR! This guide will
 3. **Install dependencies**:
 
    ```bash
-   make install
+   uv sync --all-extras
+   uv run pre-commit install
    ```
 
 4. **Create a feature branch**:
@@ -48,7 +78,9 @@ Thank you for your interest in contributing to The DETERMINATOR! This guide will
 6. **Run checks**:
 
    ```bash
-   make check
+   uv run ruff check src tests
+   uv run mypy src
+   uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire
    ```
 
 7. **Commit and push**:
@@ -57,22 +89,72 @@ Thank you for your interest in contributing to The DETERMINATOR! This guide will
    git commit -m "Description of changes"
    git push origin yourname-feature-name
    ```
+
 8. **Create a pull request** on GitHub
 
+## Package Manager
+
+This project uses [`uv`](https://github.com/astral-sh/uv) as the package manager. All commands should be prefixed with `uv run` to ensure they run in the correct environment.
+
+### Installation
+
+```bash
+# Install uv if you haven't already (recommended: standalone installer)
+# Unix/macOS/Linux:
+curl -LsSf https://astral.sh/uv/install.sh | sh
+
+# Windows (PowerShell):
+powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
+
+# Alternative: pipx install uv
+# Or: pip install uv
+
+# Sync all dependencies including dev extras
+uv sync --all-extras
+
+# Install pre-commit hooks
+uv run pre-commit install
+```
+
 ## Development Commands
 
 ```bash
-make install      # Install dependencies + pre-commit
-make check        # Lint + typecheck + test (MUST PASS)
-make test         # Run unit tests
-make lint         # Run ruff
-make format       # Format with ruff
-make typecheck    # Run mypy
-make test-cov     # Test with coverage
-make docs-build  # Build documentation
-make docs-serve  # Serve documentation locally
+# Installation
+uv sync --all-extras              # Install all dependencies including dev
+uv run pre-commit install          # Install pre-commit hooks
+
+# Code Quality Checks (run all before committing)
+uv run ruff check src tests       # Lint with ruff
+uv run ruff format src tests      # Format with ruff
+uv run mypy src                   # Type checking
+uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire  # Tests with coverage
+
+# Testing Commands
+uv run pytest tests/unit/ -v -m "not openai" -p no:logfire              # Run unit tests (excludes OpenAI tests)
+uv run pytest tests/ -v -m "huggingface" -p no:logfire                 # Run HuggingFace tests
+uv run pytest tests/ -v -p no:logfire                                  # Run all tests
+uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire  # Tests with terminal coverage
+uv run pytest --cov=src --cov-report=html -p no:logfire                # Generate HTML coverage report (opens htmlcov/index.html)
+
+# Documentation Commands
+uv run mkdocs build                # Build documentation
+uv run mkdocs serve                # Serve documentation locally (http://127.0.0.1:8000)
 ```
 
+### Test Markers
+
+The project uses pytest markers to categorize tests. See [Testing Guidelines](docs/contributing/testing.md) for details:
+
+- `unit`: Unit tests (mocked, fast)
+- `integration`: Integration tests (real APIs)
+- `slow`: Slow tests
+- `openai`: Tests requiring OpenAI API key
+- `huggingface`: Tests requiring HuggingFace API key
+- `embedding_provider`: Tests requiring API-based embedding providers
+- `local_embeddings`: Tests using local embeddings
+
+**Note**: The `-p no:logfire` flag disables the logfire plugin to avoid conflicts during testing.
+
 ## Code Style & Conventions
 
 ### Type Safety
@@ -118,10 +200,10 @@ result = await loop.run_in_executor(None, cpu_bound_function, args)
 
 ### Pre-commit
 
-- Run `make check` before committing
+- Pre-commit hooks run automatically on commit
 - Must pass: lint + typecheck + test-cov
-- Pre-commit hooks installed via `make install`
-- **CRITICAL**: Make sure you run the full pre-commit checks before opening a PR (not draft), otherwise Obstacle is the Way will lose his mind
+- Install hooks with: `uv run pre-commit install`
+- Note: `uv sync --all-extras` installs the pre-commit package, but you must run `uv run pre-commit install` separately to set up the git hooks
 
 ## Error Handling & Logging
 
@@ -191,7 +273,7 @@ except httpx.HTTPError as e:
 1. Write failing test in `tests/unit/`
 2. Implement in `src/`
 3. Ensure test passes
-4. Run `make check` (lint + typecheck + test)
+4. Run checks: `uv run ruff check src tests && uv run mypy src && uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire`
 
 ### Test Examples
 
@@ -212,7 +294,8 @@ async def test_real_pubmed_search():
 
 ### Test Coverage
 
-- Run `make test-cov` for coverage report
+- Run `uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire` for coverage report
+- Run `uv run pytest --cov=src --cov-report=html -p no:logfire` for HTML coverage report (opens `htmlcov/index.html`)
 - Aim for >80% coverage on critical paths
 - Exclude: `__init__.py`, `TYPE_CHECKING` blocks
 
@@ -385,7 +468,7 @@ Example:
 
 ## Pull Request Process
 
-1. Ensure all checks pass: `make check`
+1. Ensure all checks pass: `uv run ruff check src tests && uv run mypy src && uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire`
 2. Update documentation if needed
 3. Add tests for new features
 4. Update CHANGELOG if applicable
@@ -393,11 +476,19 @@ Example:
 6. Address review feedback
 7. Wait for approval before merging
 
+## Project Structure
+
+- `src/`: Main source code
+- `tests/`: Test files (`unit/` and `integration/`)
+- `docs/`: Documentation source files (MkDocs)
+- `examples/`: Example usage scripts
+- `pyproject.toml`: Project configuration and dependencies
+- `.pre-commit-config.yaml`: Pre-commit hook configuration
+
 ## Questions?
 
-- Open an issue on GitHub
-- Check existing documentation
+- Open an issue on [GitHub](https://github.com/DeepCritical/GradioDemo)
+- Check existing [documentation](https://deepcritical.github.io/GradioDemo/)
 - Review code examples in the codebase
 
-Thank you for contributing to DeepCritical!
-
+Thank you for contributing to The DETERMINATOR!

ERROR_FIXES_SUMMARY.md

@@ -1,156 +0,0 @@
-# Error Fixes Summary
-
-## Issues Identified and Fixed
-
-### 1. ✅ `'Settings' object has no attribute 'ocr_api_url'`
-
-**Error Location:** `src/services/image_ocr.py:33`
-
-**Root Cause:** 
-The code was trying to access `settings.ocr_api_url` which doesn't exist in older versions of the config. This happens when running a previous version of the app where `ocr_api_url` wasn't added to the Settings class yet.
-
-**Fix Applied:**
-- Added defensive coding using `getattr()` with a fallback default URL
-- Default URL: `"https://prithivmlmods-multimodal-ocr3.hf.space"`
-
-**Code Change:**
-```python
-# Before:
-self.api_url = api_url or settings.ocr_api_url
-
-# After:
-default_url = getattr(settings, "ocr_api_url", None) or "https://prithivmlmods-multimodal-ocr3.hf.space"
-self.api_url = api_url or default_url
-```
-
-**File:** `src/services/image_ocr.py`
-
----
-
-### 2. ✅ `Expected code to be unreachable, but got: ('research_complete', False)`
-
-**Error Location:** `src/orchestrator/graph_orchestrator.py` (decision node execution)
-
-**Root Cause:**
-When Pydantic AI encounters a validation error or returns an unexpected format, it may return a tuple like `('research_complete', False)` instead of the expected `KnowledgeGapOutput` object. The decision function was trying to access `result.research_complete` on a tuple, causing the error.
-
-**Fix Applied:**
-1. **Enhanced decision function** in `graph_builder.py` to handle tuple formats
-2. **Improved tuple handling** in `graph_orchestrator.py` decision node execution
-3. **Better reconstruction** of `KnowledgeGapOutput` from validation error tuples
-
-**Code Changes:**
-
-**File: `src/agent_factory/graph_builder.py`**
-- Replaced lambda with named function `_decision_function()` that handles tuples
-- Added logic to extract `research_complete` from various tuple formats
-- Handles: `('research_complete', False)`, dicts in tuples, boolean values in tuples
-
-**File: `src/orchestrator/graph_orchestrator.py`**
-- Enhanced tuple detection and reconstruction in `_execute_decision_node()`
-- Added specific handling for `('research_complete', False)` format
-- Improved fallback logic for unexpected tuple formats
-- Better error messages and logging
-
-**File: `src/orchestrator/graph_orchestrator.py` (agent node execution)**
-- Improved handling of tuple outputs in `_execute_agent_node()`
-- Better reconstruction of `KnowledgeGapOutput` from validation errors
-- More graceful fallback for non-knowledge_gap nodes
-
----
-
-### 3. ⚠️ `Local state is not initialized - app is not locally available` (Modal TTS)
-
-**Error Location:** Modal TTS service
-
-**Root Cause:**
-This is expected behavior when Modal credentials are not configured or the app is not running in a Modal environment. It's not a critical error - TTS will simply be unavailable.
-
-**Status:** 
-- This is **not an error** - it's expected when Modal isn't configured
-- The app gracefully degrades and continues without TTS
-- Users can still use the app, just without audio output
-
-**No Fix Needed:** This is working as designed with graceful degradation.
-
----
-
-### 4. ⚠️ `Invalid file descriptor: -1` (Asyncio cleanup)
-
-**Error Location:** Python asyncio event loop cleanup
-
-**Root Cause:**
-This is a Python asyncio cleanup warning that occurs during shutdown. It's not critical and doesn't affect functionality.
-
-**Status:**
-- This is a **warning**, not an error
-- Occurs during application shutdown
-- Doesn't affect runtime functionality
-- Common in Python 3.11+ with certain asyncio patterns
-
-**No Fix Needed:** This is a known Python asyncio quirk and doesn't impact functionality.
-
----
-
-### 5. ⚠️ MCP Server Warning: `gr.State input will not be updated between tool calls`
-
-**Error Location:** Gradio MCP server setup
-
-**Root Cause:**
-Some MCP tools use `gr.State` inputs, which Gradio warns won't update between tool calls. This is a limitation of how MCP tools interact with Gradio state.
-
-**Status:**
-- This is a **warning**, not an error
-- MCP tools will still work, but state won't persist between calls
-- This is a known Gradio MCP limitation
-
-**No Fix Needed:** This is a Gradio limitation, not a bug in our code.
-
----
-
-## Summary of Fixes
-
-### Critical Fixes (Applied):
-1. ✅ **OCR API URL Attribute Error** - Fixed with defensive coding
-2. ✅ **Graph Orchestrator Tuple Handling** - Fixed with enhanced tuple detection and reconstruction
-
-### Non-Critical (Expected Behavior):
-3. ⚠️ **Modal TTS Error** - Expected when Modal not configured (graceful degradation)
-4. ⚠️ **Asyncio Cleanup Warning** - Python asyncio quirk (non-critical)
-5. ⚠️ **MCP State Warning** - Gradio limitation (non-critical)
-
-## Testing Recommendations
-
-1. **Test OCR functionality:**
-   - Upload an image with text
-   - Verify OCR processing works
-   - Check logs for any remaining errors
-
-2. **Test graph execution:**
-   - Run a research query
-   - Verify knowledge gap evaluation works
-   - Check that decision nodes route correctly
-   - Monitor logs for tuple handling warnings
-
-3. **Test with/without Modal:**
-   - Verify app works without Modal credentials
-   - Test TTS if Modal is configured
-   - Verify graceful degradation
-
-## Files Modified
-
-1. `src/services/image_ocr.py` - Added defensive `ocr_api_url` access
-2. `src/orchestrator/graph_orchestrator.py` - Enhanced tuple handling in decision and agent nodes
-3. `src/agent_factory/graph_builder.py` - Improved decision function to handle tuples
-
-## Next Steps
-
-1. Test the fixes with the reported error scenarios
-2. Monitor logs for any remaining issues
-3. Consider adding unit tests for tuple handling edge cases
-4. Document the tuple format handling for future reference
-
-
-
-
-

FILE_OUTPUT_IMPLEMENTATION_PLAN.md

@@ -1,241 +0,0 @@
-# File Output Implementation Plan
-
-## Overview
-
-This plan implements file writing and return functionality for report-writing agents, enabling reports to be saved as files and returned through the Gradio ChatInterface.
-
-## Current State Analysis
-
-✅ **Report Generation**: All agents generate markdown strings  
-✅ **File Output Integration**: `event_to_chat_message()` supports file paths  
-✅ **Graph Orchestrator**: Can handle file paths in results  
-❌ **File Writing**: No agents write files to disk  
-❌ **File Service**: No utility service for saving reports  
-
----
-
-## Implementation Plan
-
-### PROJECT 1: File Writing Service
-**Goal**: Create a reusable service for saving reports to files
-
-#### Activity 1.1: Create Report File Service
-**File**: `src/services/report_file_service.py` (NEW)
-
-**Tasks**:
-1. Create `ReportFileService` class
-2. Implement `save_report()` method
-   - Accepts: report content (str), filename (optional), output_dir (optional)
-   - Returns: file path (str)
-   - Uses temp directory by default
-   - Supports custom output directory
-   - Handles file naming with timestamps
-3. Implement `save_report_multiple_formats()` method
-   - Save as .md (always)
-   - Optionally save as .html, .pdf (future)
-4. Add configuration support
-   - Read from settings
-   - Enable/disable file saving
-   - Configurable output directory
-5. Add error handling and logging
-6. Add file cleanup utilities (optional)
-
-**Line-level subtasks**:
-- Line 1-20: Imports and class definition
-- Line 21-40: `__init__()` method with settings
-- Line 41-80: `save_report()` method
-  - Line 41-50: Input validation
-  - Line 51-60: Directory creation
-  - Line 61-70: File writing
-  - Line 71-80: Error handling
-- Line 81-100: `save_report_multiple_formats()` method
-- Line 101-120: Helper methods (filename generation, cleanup)
-
----
-
-### PROJECT 2: Configuration Updates
-**Goal**: Add settings for file output functionality
-
-#### Activity 2.1: Update Settings Model
-**File**: `src/utils/config.py`
-
-**Tasks**:
-1. Add `save_reports_to_file: bool` field (default: True)
-2. Add `report_output_directory: str | None` field (default: None, uses temp)
-3. Add `report_file_format: Literal["md", "md_html", "md_pdf"]` field (default: "md")
-4. Add `report_filename_template: str` field (default: "report_{timestamp}_{query_hash}.md")
-
-**Line-level subtasks**:
-- Line 166-170: Add `save_reports_to_file` field after TTS config
-- Line 171-175: Add `report_output_directory` field
-- Line 176-180: Add `report_file_format` field
-- Line 181-185: Add `report_filename_template` field
-
----
-
-### PROJECT 3: Graph Orchestrator Integration
-**Goal**: Integrate file writing into graph execution
-
-#### Activity 3.1: Update Graph Orchestrator
-**File**: `src/orchestrator/graph_orchestrator.py`
-
-**Tasks**:
-1. Import `ReportFileService` at top
-2. Initialize service in `__init__()` (optional, can be lazy)
-3. Modify `_execute_agent_node()` for synthesizer node
-   - After `long_writer_agent.write_report()`, save to file
-   - Return dict with `{"message": report, "file": file_path}`
-4. Update final event generation to handle file paths
-   - Already implemented, verify it works correctly
-
-**Line-level subtasks**:
-- Line 1-35: Add import for `ReportFileService`
-- Line 119-148: Update `__init__()` to accept optional file service
-- Line 589-650: Modify `_execute_agent_node()` synthesizer handling
-  - Line 642-645: After `write_report()`, add file saving
-  - Line 646-650: Return dict with file path
-- Line 534-564: Verify final event generation handles file paths (already done)
-
----
-
-### PROJECT 4: Research Flow Integration
-**Goal**: Integrate file writing into research flows
-
-#### Activity 4.1: Update IterativeResearchFlow
-**File**: `src/orchestrator/research_flow.py`
-
-**Tasks**:
-1. Import `ReportFileService` at top
-2. Add optional file service to `__init__()`
-3. Modify `_create_final_report()` method
-   - After `writer_agent.write_report()`, save to file if enabled
-   - Return string (backward compatible) OR dict with file path
-
-**Line-level subtasks**:
-- Line 1-50: Add import for `ReportFileService`
-- Line 48-120: Update `__init__()` to accept optional file service
-- Line 622-667: Modify `_create_final_report()` method
-  - Line 647-652: After `write_report()`, add file saving
-  - Line 653-667: Return report string (keep backward compatible for now)
-
-#### Activity 4.2: Update DeepResearchFlow
-**File**: `src/orchestrator/research_flow.py`
-
-**Tasks**:
-1. Add optional file service to `__init__()` (if not already)
-2. Modify `_create_final_report()` method
-   - After `long_writer_agent.write_report()` or `proofreader_agent.proofread()`, save to file
-   - Return string (backward compatible) OR dict with file path
-
-**Line-level subtasks**:
-- Line 670-750: Update `DeepResearchFlow.__init__()` to accept optional file service
-- Line 954-1005: Modify `_create_final_report()` method
-  - Line 979-983: After `write_report()`, add file saving
-  - Line 984-989: After `proofread()`, add file saving
-  - Line 990-1005: Return report string (keep backward compatible)
-
----
-
-### PROJECT 5: Agent Factory Integration
-**Goal**: Make file service available to agents if needed
-
-#### Activity 5.1: Update Agent Factory (Optional)
-**File**: `src/agent_factory/agents.py`
-
-**Tasks**:
-1. Add optional file service parameter to agent creation functions (if needed)
-2. Pass file service to agents that need it (currently not needed, agents return strings)
-
-**Line-level subtasks**:
-- Not required - agents return strings, file writing happens at orchestrator level
-
----
-
-### PROJECT 6: Testing & Validation
-**Goal**: Ensure file output works end-to-end
-
-#### Activity 6.1: Unit Tests
-**File**: `tests/unit/services/test_report_file_service.py` (NEW)
-
-**Tasks**:
-1. Test `save_report()` with default settings
-2. Test `save_report()` with custom directory
-3. Test `save_report()` with custom filename
-4. Test error handling (permission errors, disk full, etc.)
-5. Test file cleanup
-
-**Line-level subtasks**:
-- Line 1-30: Test fixtures and setup
-- Line 31-60: Test basic save functionality
-- Line 61-90: Test custom directory
-- Line 91-120: Test error handling
-
-#### Activity 6.2: Integration Tests
-**File**: `tests/integration/test_file_output_integration.py` (NEW)
-
-**Tasks**:
-1. Test graph orchestrator with file output
-2. Test research flows with file output
-3. Test Gradio ChatInterface receives file paths
-4. Test file download in Gradio UI
-
-**Line-level subtasks**:
-- Line 1-40: Test setup with mock orchestrator
-- Line 41-80: Test file generation in graph execution
-- Line 81-120: Test file paths in AgentEvent
-- Line 121-160: Test Gradio message conversion
-
----
-
-## Implementation Order
-
-1. **PROJECT 2** (Configuration) - Foundation
-2. **PROJECT 1** (File Service) - Core functionality
-3. **PROJECT 3** (Graph Orchestrator) - Primary integration point
-4. **PROJECT 4** (Research Flows) - Secondary integration points
-5. **PROJECT 6** (Testing) - Validation
-6. **PROJECT 5** (Agent Factory) - Not needed, skip
-
----
-
-## File Changes Summary
-
-### New Files
-- `src/services/report_file_service.py` - File writing service
-- `tests/unit/services/test_report_file_service.py` - Unit tests
-- `tests/integration/test_file_output_integration.py` - Integration tests
-
-### Modified Files
-- `src/utils/config.py` - Add file output settings
-- `src/orchestrator/graph_orchestrator.py` - Add file saving after report generation
-- `src/orchestrator/research_flow.py` - Add file saving in both flows
-
----
-
-## Gradio Integration Notes
-
-According to Gradio ChatInterface documentation:
-- File paths in chat message content are automatically converted to download links
-- Markdown links like `[Download: filename](file_path)` work
-- Files must be accessible from the Gradio server
-- Temp files are fine as long as they exist during the session
-
-Current implementation in `event_to_chat_message()` already handles this correctly.
-
----
-
-## Success Criteria
-
-✅ Reports are saved to files when generated  
-✅ File paths are included in AgentEvent data  
-✅ File paths appear as download links in Gradio ChatInterface  
-✅ File saving is configurable (can be disabled)  
-✅ Backward compatible (existing code still works)  
-✅ Error handling prevents crashes if file writing fails  
-
-
-
-
-
-
-

FILE_OUTPUT_VERIFICATION.md

@@ -1,224 +0,0 @@
-# File Output Implementation Verification
-
-## Status: ✅ ALL CHANGES RETAINED
-
-All file output functionality has been successfully implemented and retained in the codebase.
-
----
-
-## Verification Checklist
-
-### ✅ PROJECT 1: File Writing Service
-- **File**: `src/services/report_file_service.py`
-- **Status**: ✅ EXISTS
-- **Key Components**:
-  - `ReportFileService` class
-  - `save_report()` method
-  - `save_report_multiple_formats()` method
-  - `_generate_filename()` helper
-  - `_sanitize_filename()` helper
-  - `cleanup_old_files()` method
-  - `get_report_file_service()` singleton function
-
-### ✅ PROJECT 2: Configuration Updates
-- **File**: `src/utils/config.py`
-- **Status**: ✅ ALL SETTINGS PRESENT
-- **Settings Added** (lines 181-195):
-  - ✅ `save_reports_to_file: bool = True`
-  - ✅ `report_output_directory: str | None = None`
-  - ✅ `report_file_format: Literal["md", "md_html", "md_pdf"] = "md"`
-  - ✅ `report_filename_template: str = "report_{timestamp}_{query_hash}.md"`
-
-### ✅ PROJECT 3: Graph Orchestrator Integration
-- **File**: `src/orchestrator/graph_orchestrator.py`
-- **Status**: ✅ FULLY INTEGRATED
-
-#### Imports (Line 35)
-```python
-from src.services.report_file_service import ReportFileService, get_report_file_service
-```
-✅ Present
-
-#### File Service Initialization (Line 152)
-```python
-self._file_service: ReportFileService | None = None
-```
-✅ Present
-
-#### Helper Method (Lines 162-175)
-```python
-def _get_file_service(self) -> ReportFileService | None:
-    """Get file service instance (lazy initialization)."""
-    ...
-```
-✅ Present
-
-#### Synthesizer Node File Saving (Lines 673-694)
-- ✅ Saves report after `long_writer_agent.write_report()`
-- ✅ Returns dict with `{"message": report, "file": file_path}` if file saved
-- ✅ Returns string if file saving fails (backward compatible)
-- ✅ Error handling with logging
-
-#### Writer Node File Saving (Lines 729-748)
-- ✅ Saves report after `writer_agent.write_report()`
-- ✅ Returns dict with `{"message": report, "file": file_path}` if file saved
-- ✅ Returns string if file saving fails (backward compatible)
-- ✅ Error handling with logging
-
-#### Final Event Handling (Lines 558-585)
-- ✅ Extracts file path from final result dict
-- ✅ Adds file path to `event_data["file"]` or `event_data["files"]`
-- ✅ Handles both single file and multiple files
-- ✅ Sets appropriate message
-
-### ✅ PROJECT 4: Research Flow Integration
-- **File**: `src/orchestrator/research_flow.py`
-- **Status**: ✅ FULLY INTEGRATED
-
-#### Imports (Line 28)
-```python
-from src.services.report_file_service import ReportFileService, get_report_file_service
-```
-✅ Present
-
-#### IterativeResearchFlow
-- **File Service Initialization** (Line 117): ✅ Present
-- **Helper Method** (Lines 119-132): ✅ Present
-- **File Saving in `_create_final_report()`** (Lines 683-692): ✅ Present
-  - Saves after `writer_agent.write_report()`
-  - Logs file path
-  - Error handling with logging
-
-#### DeepResearchFlow
-- **File Service Initialization** (Line 761): ✅ Present
-- **Helper Method** (Lines 763-776): ✅ Present
-- **File Saving in `_create_final_report()`** (Lines 1055-1064): ✅ Present
-  - Saves after `long_writer_agent.write_report()` or `proofreader_agent.proofread()`
-  - Logs file path
-  - Error handling with logging
-
-### ✅ PROJECT 5: Gradio Integration
-- **File**: `src/app.py`
-- **Status**: ✅ ALREADY IMPLEMENTED (from previous work)
-- **Function**: `event_to_chat_message()` (Lines 209-350)
-- **Features**:
-  - ✅ Detects file paths in `event.data["file"]` or `event.data["files"]`
-  - ✅ Formats files as markdown download links
-  - ✅ Handles both single and multiple files
-  - ✅ Validates file paths with `_is_file_path()` helper
-
----
-
-## Implementation Summary
-
-### File Saving Locations
-
-1. **Graph Orchestrator - Synthesizer Node** (Deep Research)
-   - Location: `src/orchestrator/graph_orchestrator.py:673-694`
-   - Trigger: After `long_writer_agent.write_report()`
-   - Returns: Dict with file path or string (backward compatible)
-
-2. **Graph Orchestrator - Writer Node** (Iterative Research)
-   - Location: `src/orchestrator/graph_orchestrator.py:729-748`
-   - Trigger: After `writer_agent.write_report()`
-   - Returns: Dict with file path or string (backward compatible)
-
-3. **IterativeResearchFlow**
-   - Location: `src/orchestrator/research_flow.py:683-692`
-   - Trigger: After `writer_agent.write_report()` in `_create_final_report()`
-   - Returns: String (file path logged, not returned)
-
-4. **DeepResearchFlow**
-   - Location: `src/orchestrator/research_flow.py:1055-1064`
-   - Trigger: After `long_writer_agent.write_report()` or `proofreader_agent.proofread()`
-   - Returns: String (file path logged, not returned)
-
-### File Path Flow
-
-```
-Report Generation
-    ↓
-ReportFileService.save_report()
-    ↓
-File saved to disk (temp directory or configured directory)
-    ↓
-File path returned to orchestrator
-    ↓
-File path included in result dict: {"message": report, "file": file_path}
-    ↓
-Result dict stored in GraphExecutionContext
-    ↓
-Final event extraction (lines 558-585)
-    ↓
-File path added to AgentEvent.data["file"]
-    ↓
-event_to_chat_message() (src/app.py)
-    ↓
-File path formatted as markdown download link
-    ↓
-Gradio ChatInterface displays download link
-```
-
----
-
-## Testing Recommendations
-
-### Unit Tests
-- [ ] Test `ReportFileService.save_report()` with various inputs
-- [ ] Test filename generation with templates
-- [ ] Test file sanitization
-- [ ] Test error handling (permission errors, disk full, etc.)
-
-### Integration Tests
-- [ ] Test graph orchestrator file saving for synthesizer node
-- [ ] Test graph orchestrator file saving for writer node
-- [ ] Test file path inclusion in AgentEvent
-- [ ] Test Gradio message conversion with file paths
-- [ ] Test file download in Gradio UI
-
-### Manual Testing
-- [ ] Run iterative research flow and verify file is created
-- [ ] Run deep research flow and verify file is created
-- [ ] Verify file appears as download link in Gradio ChatInterface
-- [ ] Test with file saving disabled (`save_reports_to_file=False`)
-- [ ] Test with custom output directory
-
----
-
-## Configuration Options
-
-All settings are in `src/utils/config.py`:
-
-```python
-# Enable/disable file saving
-save_reports_to_file: bool = True
-
-# Custom output directory (None = use temp directory)
-report_output_directory: str | None = None
-
-# File format (currently only "md" is fully implemented)
-report_file_format: Literal["md", "md_html", "md_pdf"] = "md"
-
-# Filename template with placeholders
-report_filename_template: str = "report_{timestamp}_{query_hash}.md"
-```
-
----
-
-## Conclusion
-
-✅ **All file output functionality has been successfully implemented and retained.**
-
-The implementation is:
-- ✅ Complete (all planned features implemented)
-- ✅ Backward compatible (existing code continues to work)
-- ✅ Error resilient (file saving failures don't crash workflows)
-- ✅ Configurable (can be enabled/disabled via settings)
-- ✅ Integrated with Gradio (file paths appear as download links)
-
-No reimplementation needed. All changes are present and correct.
-
-
-
-
-

FIX_SUMMARY.md

@@ -1,102 +0,0 @@
-# Fix Summary: Research Results Not Returned to User
-
-## Problem
-The application was returning "Research completed" instead of the actual research report content to users. Reports were being generated and saved to files, but the final result wasn't being properly extracted and returned to the Gradio interface.
-
-## Root Causes
-
-1. **Incomplete Result Extraction**: The `_execute_graph` method in `graph_orchestrator.py` only checked the last executed node (`current_node_id`) for the final result. If the graph execution broke early due to budget/time limits, or if the last node wasn't the synthesizer/writer exit node, the result wouldn't be found.
-
-2. **Incomplete Dict Handling**: When the synthesizer or writer nodes returned a dict with `{"message": final_report, "file": file_path}`, the code only extracted the message if the dict had a "file" key. If the dict had a "message" key but no "file" key, the message wouldn't be extracted.
-
-3. **No Fallback Logic**: There was no fallback to check all exit nodes for results if the current node wasn't an exit node.
-
-## Solution
-
-### Changes Made to `src/orchestrator/graph_orchestrator.py`
-
-1. **Enhanced Result Extraction** (lines 555-600):
-   - First checks if `current_node_id` is an exit node and gets its result
-   - If no result, prioritizes checking "synthesizer" and "writer" exit nodes
-   - Falls back to checking all exit nodes if still no result
-   - Added comprehensive logging to help debug result extraction
-
-2. **Improved Dict Handling** (lines 602-640):
-   - Now checks for "message" key first (most important)
-   - Extracts message from dict even if "file" key is missing
-   - Only uses default messages if "message" key is not present
-   - Added logging for result type and extraction process
-
-3. **Better Error Handling**:
-   - Logs warnings when no result is found, including all available node results
-   - Logs unexpected result types for debugging
-
-## Key Code Changes
-
-### Before:
-```python
-final_result = context.get_node_result(current_node_id) if current_node_id else None
-message: str = "Research completed"
-
-if isinstance(final_result, str):
-    message = final_result
-elif isinstance(final_result, dict):
-    if "file" in final_result:
-        # Only extracts message if file exists
-        message = final_result.get("message", "Report generated. Download available.")
-```
-
-### After:
-```python
-# Check all exit nodes with priority
-final_result = None
-if current_node_id and current_node_id in self._graph.exit_nodes:
-    final_result = context.get_node_result(current_node_id)
-
-if not final_result:
-    # Prioritize synthesizer/writer nodes
-    for exit_node_id in ["synthesizer", "writer"]:
-        if exit_node_id in self._graph.exit_nodes:
-            result = context.get_node_result(exit_node_id)
-            if result:
-                final_result = result
-                break
-
-# Extract message from dict first
-if isinstance(final_result, dict):
-    if "message" in final_result:
-        message = final_result["message"]  # Extract message regardless of file
-```
-
-## Testing Recommendations
-
-1. **Test Deep Research Flow**:
-   - Run a query that triggers deep research mode
-   - Verify the full report is returned, not just "Research completed"
-   - Check that reports are properly displayed in the UI
-
-2. **Test Iterative Research Flow**:
-   - Run a query that triggers iterative research mode
-   - Verify the report is returned correctly
-
-3. **Test Budget/Time Limits**:
-   - Run queries that exceed budget or time limits
-   - Verify that partial results are still returned if available
-
-4. **Test File Saving**:
-   - Verify reports are saved to files
-   - Verify file paths are included in event data when available
-
-## Files Modified
-
-- `src/orchestrator/graph_orchestrator.py`: Enhanced result extraction and message handling logic
-
-## Expected Behavior After Fix
-
-- Users will see the full research report content in the chat interface
-- Reports will be properly extracted from synthesizer/writer nodes
-- File paths will be included in event data when reports are saved
-- Better logging will help debug any future issues with result extraction
-
-
-

MULTIMODAL_SETTINGS_IMPLEMENTATION_PLAN.md

@@ -1,386 +0,0 @@
-# Multimodal Settings & File Rendering - Implementation Plan
-
-## Executive Summary
-
-This document provides a comprehensive analysis of the current settings implementation, multimodal input handling, and file rendering in `src/app.py`, along with a detailed implementation plan to improve the user experience.
-
-## 1. Current Settings Analysis
-
-### 1.1 Settings Structure in `src/app.py`
-
-**Current Implementation (Lines 741-887):**
-
-1. **Sidebar Structure:**
-   - Authentication section (lines 745-750)
-   - About section (lines 752-764)
-   - Settings section (lines 767-850):
-     - Research Configuration Accordion (lines 771-796):
-       - `mode_radio`: Orchestrator mode selector
-       - `graph_mode_radio`: Graph research mode selector
-       - `use_graph_checkbox`: Graph execution toggle
-     - Audio Output Accordion (lines 798-850):
-       - `enable_audio_output_checkbox`: TTS enable/disable
-       - `tts_voice_dropdown`: Voice selection
-       - `tts_speed_slider`: Speech speed control
-       - `tts_gpu_dropdown`: GPU type (non-interactive, visible only if Modal available)
-
-2. **Hidden Components (Lines 852-865):**
-   - `hf_model_dropdown`: Hidden Textbox for model selection
-   - `hf_provider_dropdown`: Hidden Textbox for provider selection
-
-3. **Main Area Components (Lines 867-887):**
-   - `audio_output`: Audio output component (visible based on `settings.enable_audio_output`)
-   - Visibility update function for TTS components
-
-### 1.2 Settings Flow
-
-**Settings → Function Parameters:**
-- Settings from sidebar accordions are passed via `additional_inputs` to `research_agent()` function
-- Hidden textboxes are also passed but use empty strings (converted to None)
-- OAuth token/profile are automatically passed by Gradio
-
-**Function Signature (Lines 535-546):**
-```python
-async def research_agent(
-    message: str | MultimodalPostprocess,
-    history: list[dict[str, Any]],
-    mode: str = "simple",
-    hf_model: str | None = None,
-    hf_provider: str | None = None,
-    graph_mode: str = "auto",
-    use_graph: bool = True,
-    tts_voice: str = "af_heart",
-    tts_speed: float = 1.0,
-    oauth_token: gr.OAuthToken | None = None,
-    oauth_profile: gr.OAuthProfile | None = None,
-)
-```
-
-### 1.3 Issues Identified
-
-1. **Settings Organization:**
-   - Audio output component is in main area, not sidebar
-   - Hidden components (hf_model, hf_provider) should be visible or removed
-   - No image input enable/disable setting (only audio input has this)
-
-2. **Visibility:**
-   - Audio output visibility is controlled by checkbox, but component placement is suboptimal
-   - TTS settings visibility is controlled by checkbox change event
-
-3. **Configuration Gaps:**
-   - No `enable_image_input` setting in config (only `enable_audio_input` exists)
-   - Image processing always happens if files are present (line 626 comment says "not just when enable_image_input is True" but setting doesn't exist)
-
-## 2. Multimodal Input Analysis
-
-### 2.1 Current Implementation
-
-**ChatInterface Configuration (Line 892-958):**
-- `multimodal=True`: Enables MultimodalTextbox component
-- MultimodalTextbox automatically provides:
-  - Text input
-  - Image upload button
-  - Audio recording button
-  - File upload support
-
-**Input Processing (Lines 613-642):**
-- Message can be `str` or `MultimodalPostprocess` (dict format)
-- MultimodalPostprocess format: `{"text": str, "files": list[FileData], "audio": tuple | None}`
-- Processing happens in `research_agent()` function:
-  - Extracts text, files, and audio from message
-  - Calls `multimodal_service.process_multimodal_input()`
-  - Condition: `if files or (audio_input_data is not None and settings.enable_audio_input)`
-
-**Multimodal Service (src/services/multimodal_processing.py):**
-- Processes audio input if `settings.enable_audio_input` is True
-- Processes image files (no enable/disable check - always processes if files present)
-- Extracts text from images using OCR service
-- Transcribes audio using STT service
-
-### 2.2 Gradio Documentation Findings
-
-**MultimodalTextbox (ChatInterface with multimodal=True):**
-- Automatically provides image and audio input capabilities
-- Inputs are always visible when ChatInterface is rendered
-- No explicit visibility control needed - it's part of the textbox component
-- Files are handled via `files` array in MultimodalPostprocess
-- Audio recording is handled via `audio` tuple in MultimodalPostprocess
-
-**Reference Implementation Pattern:**
-```python
-gr.ChatInterface(
-    fn=chat_function,
-    multimodal=True,  # Enables image/audio inputs
-    # ... other parameters
-)
-```
-
-### 2.3 Issues Identified
-
-1. **Visibility:**
-   - Multimodal inputs ARE always visible (they're part of MultimodalTextbox)
-   - No explicit control needed - this is working correctly
-   - However, users may not realize image/audio inputs are available
-
-2. **Configuration:**
-   - No `enable_image_input` setting to disable image processing
-   - Image processing always happens if files are present
-   - Audio processing respects `settings.enable_audio_input`
-
-3. **User Experience:**
-   - No visual indication that multimodal inputs are available
-   - Description mentions "🎤 Multimodal Support" but could be more prominent
-
-## 3. File Rendering Analysis
-
-### 3.1 Current Implementation
-
-**File Detection (Lines 168-195):**
-- `_is_file_path()`: Checks if text looks like a file path
-- Checks for file extensions and path separators
-
-**File Rendering in Events (Lines 242-298):**
-- For "complete" events, checks `event.data` for "files" or "file" keys
-- Validates files exist using `os.path.exists()`
-- Formats files as markdown download links: `📎 [Download: filename](filepath)`
-- Stores files in metadata for potential future use
-
-**File Links Format:**
-```python
-file_links = "\n\n".join([
-    f"📎 [Download: {_get_file_name(f)}]({f})" 
-    for f in valid_files
-])
-result["content"] = f"{content}\n\n{file_links}"
-```
-
-### 3.2 Issues Identified
-
-1. **Rendering Method:**
-   - Uses markdown links in content string
-   - May not work reliably in all Gradio versions
-   - Better approach: Use Gradio's native file components or File component
-
-2. **File Validation:**
-   - Only checks if file exists
-   - Doesn't validate file type or size
-   - No error handling for inaccessible files
-
-3. **User Experience:**
-   - Files appear as text links, not as proper file components
-   - No preview for images/PDFs
-   - No file size information
-
-## 4. Implementation Plan
-
-### Activity 1: Settings Reorganization
-
-**Goal:** Move all settings to sidebar with better organization
-
-**File:** `src/app.py`
-
-**Tasks:**
-
-1. **Move Audio Output Component to Sidebar (Lines 867-887)**
-   - Move `audio_output` component into sidebar
-   - Place it in Audio Output accordion or create separate section
-   - Update visibility logic to work within sidebar
-
-2. **Add Image Input Settings (New)**
-   - Add `enable_image_input` checkbox to sidebar
-   - Create "Image Input" accordion or add to existing "Multimodal Input" accordion
-   - Update config to include `enable_image_input` setting
-
-3. **Organize Settings Accordions**
-   - Research Configuration (existing)
-   - Multimodal Input (new - combine image and audio input settings)
-   - Audio Output (existing - move component here)
-   - Model Configuration (new - for hf_model, hf_provider if we make them visible)
-
-**Subtasks:**
-- [ ] Line 867-871: Move `audio_output` component definition into sidebar
-- [ ] Line 873-887: Update visibility update function to work with sidebar placement
-- [ ] Line 798-850: Reorganize Audio Output accordion to include audio_output component
-- [ ] Line 767-796: Keep Research Configuration as-is
-- [ ] After line 796: Add new "Multimodal Input" accordion with enable_image_input and enable_audio_input checkboxes
-- [ ] Line 852-865: Consider making hf_model and hf_provider visible or remove them
-
-### Activity 2: Multimodal Input Visibility
-
-**Goal:** Ensure multimodal inputs are always visible and well-documented
-
-**File:** `src/app.py`
-
-**Tasks:**
-
-1. **Verify Multimodal Inputs Are Visible**
-   - Confirm `multimodal=True` in ChatInterface (already done - line 894)
-   - Add visual indicators in description
-   - Add tooltips or help text
-
-2. **Add Image Input Configuration**
-   - Add `enable_image_input` to config (src/utils/config.py)
-   - Update multimodal processing to respect this setting
-   - Add UI control in sidebar
-
-**Subtasks:**
-- [ ] Line 894: Verify `multimodal=True` is set (already correct)
-- [ ] Line 908: Enhance description to highlight multimodal capabilities
-- [ ] src/utils/config.py: Add `enable_image_input: bool = Field(default=True, ...)`
-- [ ] src/services/multimodal_processing.py: Add check for `settings.enable_image_input` before processing images
-- [ ] src/app.py: Add enable_image_input checkbox to sidebar
-
-### Activity 3: File Rendering Improvements
-
-**Goal:** Improve file rendering using proper Gradio components
-
-**File:** `src/app.py`
-
-**Tasks:**
-
-1. **Improve File Rendering Method**
-   - Use Gradio File component or proper file handling
-   - Add file previews for images
-   - Show file size and type information
-
-2. **Enhance File Validation**
-   - Validate file types
-   - Check file accessibility
-   - Handle errors gracefully
-
-**Subtasks:**
-- [ ] Line 280-296: Replace markdown link approach with proper file component rendering
-- [ ] Line 168-195: Enhance `_is_file_path()` to validate file types
-- [ ] Line 242-298: Update `event_to_chat_message()` to use Gradio File components
-- [ ] Add file preview functionality for images
-- [ ] Add error handling for inaccessible files
-
-### Activity 4: Configuration Updates
-
-**Goal:** Add missing configuration settings
-
-**File:** `src/utils/config.py`
-
-**Tasks:**
-
-1. **Add Image Input Setting**
-   - Add `enable_image_input` field
-   - Add `ocr_api_url` field if missing
-   - Add property methods for availability checks
-
-**Subtasks:**
-- [ ] After line 147: Add `enable_image_input: bool = Field(default=True, description="Enable image input (OCR) in multimodal interface")`
-- [ ] Check if `ocr_api_url` exists (should be in config)
-- [ ] Add `image_ocr_available` property if missing
-
-### Activity 5: Multimodal Service Updates
-
-**Goal:** Respect image input enable/disable setting
-
-**File:** `src/services/multimodal_processing.py`
-
-**Tasks:**
-
-1. **Add Image Input Check**
-   - Check `settings.enable_image_input` before processing images
-   - Log when image processing is skipped due to setting
-
-**Subtasks:**
-- [ ] Line 66-77: Add check for `settings.enable_image_input` before processing image files
-- [ ] Add logging when image processing is skipped
-
-## 5. Detailed File-Level Tasks
-
-### File: `src/app.py`
-
-**Line-Level Subtasks:**
-
-1. **Lines 741-850: Sidebar Reorganization**
-   - [ ] 741-765: Keep authentication and about sections
-   - [ ] 767-796: Keep Research Configuration accordion
-   - [ ] 797: Add new "Multimodal Input" accordion after Research Configuration
-   - [ ] 798-850: Reorganize Audio Output accordion, move audio_output component here
-   - [ ] 852-865: Review hidden components - make visible or remove
-
-2. **Lines 867-887: Audio Output Component**
-   - [ ] 867-871: Move `audio_output` definition into sidebar (Audio Output accordion)
-   - [ ] 873-887: Update visibility function to work with sidebar placement
-
-3. **Lines 892-958: ChatInterface Configuration**
-   - [ ] 894: Verify `multimodal=True` (already correct)
-   - [ ] 908: Enhance description with multimodal capabilities
-   - [ ] 946-956: Review `additional_inputs` - ensure all settings are included
-
-4. **Lines 242-298: File Rendering**
-   - [ ] 280-296: Replace markdown links with proper file component rendering
-   - [ ] Add file preview for images
-   - [ ] Add file size/type information
-
-5. **Lines 613-642: Multimodal Input Processing**
-   - [ ] 626: Update condition to check `settings.enable_image_input` for files
-   - [ ] Add logging for when image processing is skipped
-
-### File: `src/utils/config.py`
-
-**Line-Level Subtasks:**
-
-1. **Lines 143-180: Audio/Image Configuration**
-   - [ ] 144-147: `enable_audio_input` exists (keep as-is)
-   - [ ] After 147: Add `enable_image_input: bool = Field(default=True, description="Enable image input (OCR) in multimodal interface")`
-   - [ ] Check if `ocr_api_url` exists (add if missing)
-   - [ ] Add `image_ocr_available` property method
-
-### File: `src/services/multimodal_processing.py`
-
-**Line-Level Subtasks:**
-
-1. **Lines 65-77: Image Processing**
-   - [ ] 66: Add check: `if files and settings.enable_image_input:`
-   - [ ] 71-77: Keep image processing logic inside the new condition
-   - [ ] Add logging when image processing is skipped
-
-## 6. Testing Checklist
-
-- [ ] Verify all settings are in sidebar
-- [ ] Test multimodal inputs (image upload, audio recording)
-- [ ] Test file rendering (markdown, PDF, images)
-- [ ] Test enable/disable toggles for image and audio inputs
-- [ ] Test audio output generation and display
-- [ ] Test file download links
-- [ ] Verify settings persist across chat sessions
-- [ ] Test on different screen sizes (responsive design)
-
-## 7. Implementation Order
-
-1. **Phase 1: Configuration** (Foundation)
-   - Add `enable_image_input` to config
-   - Update multimodal service to respect setting
-
-2. **Phase 2: Settings Reorganization** (UI)
-   - Move audio output to sidebar
-   - Add image input settings to sidebar
-   - Organize accordions
-
-3. **Phase 3: File Rendering** (Enhancement)
-   - Improve file rendering method
-   - Add file previews
-   - Enhance validation
-
-4. **Phase 4: Testing & Refinement** (Quality)
-   - Test all functionality
-   - Fix any issues
-   - Refine UI/UX
-
-## 8. Success Criteria
-
-- ✅ All settings are in sidebar
-- ✅ Multimodal inputs are always visible and functional
-- ✅ Files are rendered properly with previews
-- ✅ Image and audio input can be enabled/disabled
-- ✅ Settings are well-organized and intuitive
-- ✅ No regressions in existing functionality
-
-
-
-
-

MULTIMODAL_SETTINGS_IMPLEMENTATION_SUMMARY.md

@@ -1,157 +0,0 @@
-# Multimodal Settings & File Rendering - Implementation Summary
-
-## ✅ Completed Implementation
-
-### 1. Configuration Updates (`src/utils/config.py`)
-
-**Added Settings:**
-- ✅ `enable_image_input: bool = Field(default=True, ...)` - Enable/disable image OCR processing
-- ✅ `ocr_api_url: str | None = Field(default="https://prithivmlmods-multimodal-ocr3.hf.space", ...)` - OCR service URL
-
-**Location:** Lines 148-156 (after `enable_audio_output`)
-
-### 2. Multimodal Service Updates (`src/services/multimodal_processing.py`)
-
-**Changes:**
-- ✅ Added check for `settings.enable_image_input` before processing image files
-- ✅ Image processing now respects the enable/disable setting (similar to audio input)
-
-**Location:** Line 66 - Added condition: `if files and settings.enable_image_input:`
-
-### 3. Sidebar Reorganization (`src/app.py`)
-
-**New Accordion: "📷 Multimodal Input"**
-- ✅ Added `enable_image_input_checkbox` - Control image OCR processing
-- ✅ Added `enable_audio_input_checkbox` - Control audio STT processing
-- ✅ Located after "Research Configuration" accordion
-
-**Updated Accordion: "🔊 Audio Output"**
-- ✅ Moved `audio_output` component into this accordion (was in main area)
-- ✅ Component now appears in sidebar with other audio settings
-- ✅ Visibility controlled by `enable_audio_output_checkbox`
-
-**Settings Organization:**
-1. 🔬 Research Configuration (existing)
-2. 📷 Multimodal Input (NEW)
-3. 🔊 Audio Output (updated - now includes audio_output component)
-
-**Location:** Lines 770-850
-
-### 4. Function Signature Updates (`src/app.py`)
-
-**Updated `research_agent()` function:**
-- ✅ Added `enable_image_input: bool = True` parameter
-- ✅ Added `enable_audio_input: bool = True` parameter
-- ✅ Function now accepts UI settings directly from sidebar checkboxes
-
-**Location:** Lines 535-547
-
-### 5. Multimodal Input Processing (`src/app.py`)
-
-**Updates:**
-- ✅ Uses function parameters (`enable_image_input`, `enable_audio_input`) instead of only config settings
-- ✅ Filters files and audio based on UI settings before processing
-- ✅ More responsive to user changes (no need to restart app)
-
-**Location:** Lines 624-636
-
-### 6. File Rendering Improvements (`src/app.py`)
-
-**Enhancements:**
-- ✅ Added file size display in download links
-- ✅ Better error handling for file size retrieval
-- ✅ Improved formatting with file size information (B, KB, MB)
-
-**Location:** Lines 286-300
-
-### 7. UI Description Updates (`src/app.py`)
-
-**Enhanced Description:**
-- ✅ Better explanation of multimodal capabilities
-- ✅ Clear list of supported input types (Images, Audio, Text)
-- ✅ Reference to sidebar settings for configuration
-
-**Location:** Lines 907-912
-
-## 📋 Current Settings Structure
-
-### Sidebar Layout:
-
-```
-🔐 Authentication
-  - Login button
-  - About section
-
-⚙️ Settings
-  ├─ 🔬 Research Configuration
-  │   ├─ Orchestrator Mode
-  │   ├─ Graph Research Mode
-  │   └─ Use Graph Execution
-  │
-  ├─ 📷 Multimodal Input (NEW)
-  │   ├─ Enable Image Input (OCR)
-  │   └─ Enable Audio Input (STT)
-  │
-  └─ 🔊 Audio Output
-      ├─ Enable Audio Output
-      ├─ TTS Voice
-      ├─ TTS Speech Speed
-      ├─ TTS GPU Type (if Modal available)
-      └─ 🔊 Audio Response (moved from main area)
-```
-
-## 🔍 Key Features
-
-### Multimodal Inputs (Always Visible)
-- **Image Upload**: Available in ChatInterface textbox (multimodal=True)
-- **Audio Recording**: Available in ChatInterface textbox (multimodal=True)
-- **File Upload**: Supported via MultimodalTextbox
-- **Visibility**: Always visible - part of ChatInterface component
-- **Control**: Can be enabled/disabled via sidebar settings
-
-### File Rendering
-- **Method**: Markdown download links in chat content
-- **Format**: `📎 [Download: filename (size)](filepath)`
-- **Validation**: Checks file existence before rendering
-- **Metadata**: Files stored in message metadata for future use
-
-### Settings Flow
-1. User changes settings in sidebar checkboxes
-2. Settings passed to `research_agent()` via `additional_inputs`
-3. Function uses UI settings (with config defaults as fallback)
-4. Multimodal processing respects enable/disable flags
-5. Settings persist during chat session
-
-## 🧪 Testing Checklist
-
-- [ ] Verify all settings are in sidebar
-- [ ] Test image upload with OCR enabled/disabled
-- [ ] Test audio recording with STT enabled/disabled
-- [ ] Test file rendering (markdown, PDF, images)
-- [ ] Test audio output generation and display in sidebar
-- [ ] Test file download links
-- [ ] Verify settings work without requiring app restart
-- [ ] Test on different screen sizes (responsive design)
-
-## 📝 Notes
-
-1. **Multimodal Inputs Visibility**: The inputs are always visible because they're part of the `MultimodalTextbox` component when `multimodal=True` is set in ChatInterface. No additional visibility control is needed.
-
-2. **Settings Persistence**: Settings are passed via function parameters, so they persist during the chat session but reset when the app restarts. For persistent settings across sessions, consider using Gradio's state management or session storage.
-
-3. **File Rendering**: Gradio ChatInterface automatically handles markdown file links. The current implementation with file size information should work well. For more advanced file previews, consider using Gradio's File component in a custom Blocks layout.
-
-4. **Hidden Components**: The `hf_model_dropdown` and `hf_provider_dropdown` are still hidden. Consider making them visible in a "Model Configuration" accordion if needed, or remove them if not used.
-
-## 🚀 Next Steps (Optional Enhancements)
-
-1. **Model Configuration Accordion**: Make hf_model and hf_provider visible in sidebar
-2. **File Previews**: Add image previews for uploaded images in chat
-3. **Settings Persistence**: Implement session-based settings storage
-4. **Advanced File Rendering**: Use Gradio File component for better file handling
-5. **Error Handling**: Add better error messages for failed file operations
-
-
-
-
-

Makefile

@@ -1,42 +0,0 @@
-.PHONY: install test lint format typecheck check clean all cov cov-html
-
-# Default target
-all: check
-
-install:
-	uv sync --all-extras
-	uv run pre-commit install
-
-test:
-	uv run pytest tests/unit/ -v -m "not openai" -p no:logfire
-
-test-hf:
-	uv run pytest tests/ -v -m "huggingface" -p no:logfire
-
-test-all:
-	uv run pytest tests/ -v -p no:logfire
-
-# Coverage aliases
-cov: test-cov
-test-cov:
-	uv run pytest --cov=src --cov-report=term-missing -m "not openai" -p no:logfire
-
-cov-html:
-	uv run pytest --cov=src --cov-report=html -p no:logfire
-	@echo "Coverage report: open htmlcov/index.html"
-
-lint:
-	uv run ruff check src tests
-
-format:
-	uv run ruff format src tests
-
-typecheck:
-	uv run mypy src
-
-check: lint typecheck test-cov
-	@echo "All checks passed!"
-
-clean:
-	rm -rf .pytest_cache .mypy_cache .ruff_cache __pycache__ .coverage htmlcov
-	find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true

PDF_REPORT_INTEGRATION.md

@@ -1,134 +0,0 @@
-# PDF Report Generation Integration
-
-## Summary
-
-Integrated PDF generation functionality into the report file service using utilities from `folder/utils copy`. Reports can now be automatically converted to PDF format as a final step.
-
-## Changes Made
-
-### 1. Added PDF Conversion Utilities
-
-**Files Created:**
-- `src/utils/md_to_pdf.py` - Markdown to PDF conversion utility
-- `src/utils/markdown.css` - CSS styling for PDF output
-
-**Features:**
-- Uses `md2pdf` library for conversion
-- Includes error handling and graceful fallback
-- Supports custom CSS styling
-- Logs conversion status
-
-### 2. Enhanced ReportFileService
-
-**File:** `src/services/report_file_service.py`
-
-**Changes:**
-- Added `_save_pdf()` method to generate PDF from markdown
-- Updated `save_report_multiple_formats()` to implement PDF generation
-- PDF is generated when `report_file_format` is set to `"md_pdf"`
-- Both markdown and PDF files are saved and returned
-
-**Method Signature:**
-```python
-def _save_pdf(
-    self,
-    report_content: str,
-    query: str | None = None,
-) -> str:
-    """Save report as PDF. Returns path to PDF file."""
-```
-
-### 3. Updated Graph Orchestrator
-
-**File:** `src/orchestrator/graph_orchestrator.py`
-
-**Changes:**
-- Updated synthesizer node to use `save_report_multiple_formats()`
-- Updated writer node to use `save_report_multiple_formats()`
-- Both nodes now return PDF paths in result dict when available
-- Result includes both `file` (markdown) and `files` (both formats) keys
-
-**Result Format:**
-```python
-{
-    "message": final_report,  # Report content
-    "file": "/path/to/report.md",  # Markdown file
-    "files": ["/path/to/report.md", "/path/to/report.pdf"]  # Both formats
-}
-```
-
-## Configuration
-
-PDF generation is controlled by the `report_file_format` setting in `src/utils/config.py`:
-
-```python
-report_file_format: Literal["md", "md_html", "md_pdf"] = Field(
-    default="md",
-    description="File format(s) to save reports in."
-)
-```
-
-**Options:**
-- `"md"` - Save only markdown (default)
-- `"md_html"` - Save markdown + HTML (not yet implemented)
-- `"md_pdf"` - Save markdown + PDF ✅ **Now implemented**
-
-## Usage
-
-### Enable PDF Generation
-
-Set the environment variable or update settings:
-```bash
-REPORT_FILE_FORMAT=md_pdf
-```
-
-Or in code:
-```python
-from src.utils.config import settings
-settings.report_file_format = "md_pdf"
-```
-
-### Dependencies
-
-PDF generation requires the `md2pdf` library:
-```bash
-pip install md2pdf
-```
-
-If `md2pdf` is not installed, the system will:
-- Log a warning
-- Continue with markdown-only saving
-- Not fail the report generation
-
-## File Output
-
-When PDF generation is enabled:
-1. Markdown file is always saved first
-2. PDF is generated from the markdown content
-3. Both file paths are returned in the result
-4. Gradio interface can display/download both files
-
-## Error Handling
-
-- If PDF generation fails, markdown file is still saved
-- Errors are logged but don't interrupt report generation
-- Graceful fallback ensures reports are always available
-
-## Integration Points
-
-PDF generation is automatically triggered when:
-1. Graph orchestrator synthesizer node completes
-2. Graph orchestrator writer node completes
-3. `save_report_multiple_formats()` is called
-4. `report_file_format` is set to `"md_pdf"`
-
-## Future Enhancements
-
-- HTML format support (`md_html`)
-- Custom PDF templates
-- PDF metadata (title, author, keywords)
-- PDF compression options
-- Batch PDF generation
-
-
-

README.md

@@ -34,16 +34,15 @@ tags:
 > [!IMPORTANT]
 > **You are reading the Gradio Demo README!**
 > 
-> - 📚 **Documentation**: See our [technical documentation](deepcritical.github.io/GradioDemo/) for detailed information
-> - 📖 **Complete README**: Check out the [full README](.github/README.md) for setup, configuration, and contribution guidelines
-> - 🏆 **Hackathon Submission**: Keep reading below for more information about our MCP Hackathon submission
+> - 📚 **Documentation**: See our [technical documentation](https://deepcritical.github.io/GradioDemo/) for detailed information
+> - 📖 **Complete README**: Check out the [Github README](.github/README.md) for setup, configuration, and contribution guidelines
+> - ⚠️**This README is for our Gradio Demo Only !** 
 
 <div align="center">
 
 [![GitHub](https://img.shields.io/github/stars/DeepCritical/GradioDemo?style=for-the-badge&logo=github&logoColor=white&label=GitHub&labelColor=181717&color=181717)](https://github.com/DeepCritical/GradioDemo)
 [![Documentation](https://img.shields.io/badge/Docs-0080FF?style=for-the-badge&logo=readthedocs&logoColor=white&labelColor=0080FF&color=0080FF)](deepcritical.github.io/GradioDemo/)
 [![Demo](https://img.shields.io/badge/Demo-FFD21E?style=for-the-badge&logo=huggingface&logoColor=white&labelColor=FFD21E&color=FFD21E)](https://huggingface.co/spaces/DataQuests/DeepCritical)
-[![YouTube](https://img.shields.io/badge/YouTube-FF0000?style=for-the-badge&logo=youtube&logoColor=white&label=Watch%20Demo&labelColor=FF0000&color=FF0000)](https://www.youtube.com/watch?v=https://youtu.be/Mb0M83BqgOw)
 [![codecov](https://codecov.io/gh/DeepCritical/GradioDemo/graph/badge.svg?token=B1f05RCGpz)](https://codecov.io/gh/DeepCritical/GradioDemo)
 [![Join us on Discord](https://img.shields.io/discord/1109943800132010065?label=Discord&logo=discord&style=flat-square)](https://discord.gg/qdfnvSPcqP) 
 
@@ -55,147 +54,3 @@ tags:
 ## About
 
 The DETERMINATOR is a powerful generalist deep research agent system that stops at nothing until finding precise answers to complex questions. It uses iterative search-and-judge loops to comprehensively investigate any research question from any domain.
-
-
-> For this hackathon we're proposing a simple yet powerful Deep Research Agent that iteratively looks for the answer until it finds it using general purpose websearch and special purpose retrievers for technical retrievers.
-
-## Who We Are & Motivation
-
-We're a group from the `DeepCritical` Group that met in the `hugging-science` discord.
-
-We're enthusiastic about strongly typed and robust pythonic agentic frameworks , currently building ai-assisted multi-agent systems for research automations , like critical literature reviews , clinical data retrival , and bio informatics and computational medicine applications . 
-
-Starting from Magentic Design Patterns for agentic systems , we discovered we could get better results with iterative graphs , orchestrators and planners with magentic agentics as single tools inside iterations.
-
-## Do You Like This App ? 
-
-Please join us @ https://hf.co/spaces/DataQuests/DeepCritical where we will keep maintaining it !
-
-## The DETERMINATOR is Lightweight and POWERFUL
-
-- very accessible (multimodal inputs , audio and text out)
-- fully local embeddings 
-- configurable providers (local/hosted) for websearch
-- all data stays local
-- fully configurable models and huggingface providers with login
-- easily extensible and hackable
-- uses Gradio a lot (clients, mcp , third party huggingface tools)
-- Modal for text-to-speech (remote gpu for "local model")
-- Braxel for statistical analysis (code execution sandbox)
-- Open Source Models from around the 🌐World
-- Using Google Gemma , Qwen , Zai , Llama , Mistral Reasoning Models
-- Nebius , Together , Scaleway , Hyperbolic, Novita , nscale ,  Sambanova , ovh , fireworks, all supported and configurable.
-- 💖 made with love
-
-
-## What Can It Do ? 
-
-- long running tasks (potentially millions of tokens over hours and hours)
-- data processing and rendering
-- statistical analyses 
-- literature reviews 
-- websearch
-- synthetize complex information
-- find hard to find information
-
-## Deep Critical In the Media 
-
-- Social Medial Posts about Deep Critical :
-  - 𝕏 [![X](https://x.com/marioaderman/status/1995247432444133471)]
-  - 💼 [![LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:7400984658496081920/)]
-  - 𝕏 [![X](https://x.com/viratzzs/status/1995258812165664942)]
-  
-  -💼 [![LinkedIn](https://www.linkedin.com/in/ana-bossler-07304717?utm_source=share&utm_campaign=share_via&utm_content=profile&utm_medium=ios_app)
-  -
-  -
-
-> [!IMPORTANT]
-> **IF YOU ARE A JUDGE**
-> 
-> This project was produced with passion by a group of volunteers please check out or documentation and readmes and please do keep reading below for our story
->
-> - 📚 **Documentation**: See our [technical documentation](https://deepcritical.github.io/GradioDemo/) for detailed information
-> - 📖 **Complete README**: Check out the Github [full README](.github/README.md) for setup, configuration, and contribution guidelines
-> - 🏆 **Hackathon Submission**: Keep reading below for more information about our MCP Hackathon submission
-
-
-**Key Features**:
-- **Generalist**: Handles queries from any domain (medical, technical, business, scientific, etc.)
-- **Automatic Medical Detection**: Automatically determines if medical knowledge sources (PubMed, ClinicalTrials.gov) are needed
-- **Multi-Source Search**: Web search, PubMed, ClinicalTrials.gov, Europe PMC, RAG
-- **Stops at Nothing**: Only stops at configured limits (budget, time, iterations), otherwise continues until finding precise answers
-- **Evidence Synthesis**: Comprehensive reports with proper citations
-
-**Important**: The DETERMINATOR is a research tool that synthesizes evidence. It cannot provide medical advice or answer medical questions directly.
-
-## Important information
-
-- **[readme](.github\README.md)**: configure, deploy , contribute and learn more here.
-- **[docs](deepcritical.github.io/GradioDemo/)**: want to know how all this works ? read our detailed technical documentation here.
-- **[demo](https://huggingface/spaces/DataQuests/DeepCritical)**: Try our demo on huggingface
-- **[team](### Team)**: Join us , or follow us !
-- **[video]**: See our demo video
-
-## Future Developments
-
-- [] Apply Deep Research Systems To Generate Short Form Video (up to 5 minutes)
-- [] Visualize Pydantic Graphs as Loading Screens in the UI
-- [] Improve Data Science with more Complex Graph Agents
-- [] Create Deep Critical Drug Reporposing / Discovery Demo
-- [] Create Deep Critical Literal Review
-- [] Create Deep Critical Hypothesis Generator
-- [] Create PyPi Package 
-
-## Completed
-
-- [x] **Multi-Source Search**: PubMed, ClinicalTrials.gov, bioRxiv/medRxiv
-- [x] **MCP Integration**: Use our tools from Claude Desktop or any MCP client
-- [x] **HuggingFace OAuth**: Sign in with HuggingFace 
-- [x] **Modal Sandbox**: Secure execution of AI-generated statistical code
-- [x] **LlamaIndex RAG**: Semantic search and evidence synthesis
-- [x] **HuggingfaceInference**: 
-- [x] **HuggingfaceMCP Custom Config To Use Community Tools**:
-- [x] **Strongly Typed Composable Graphs**:
-- [x] **Specialized Research Teams of Agents**: 
-
-### Team
-- **ZJ**
-    - 💼 [LinkedIn](https://www.linkedin.com/in/josephpollack/)
-- **Mario Aderman**
-    - 🤗 [HuggingFace](https://huggingface.co/SeasonalFall84)
-    - 💼 [LinkedIn](https://www.linkedin.com/in/mario-aderman/)
-    - 𝕏 [X](https://x.com/marioaderman)
-- **Joseph Pollack**
-    - 🤗 [HuggingFace](https://huggingface.co/Tonic)
-    - 💼 [LinkedIn](https://www.linkedin.com/in/josephpollack/)
-    - 𝕏 [X](https://x.com/josephpollack)
-- **Virat Chauran**
-    - 𝕏 [X](https://x.com/viratzzs/)
-    - 💼 [LinkedIn](https://www.linkedin.com/in/viratchauhan/)
-    - 🤗 [HuggingFace](https://huggingface.co/ViratChauhan)
-- **Anna Bossler**
-    -  💼 [LinkedIn](https://www.linkedin.com/in/ana-bossler-07304717)
-
-
-## Acknowledgements
-
-- [DeepBoner](https://hf.co/spaces/mcp-1st-birthday/deepboner)
-- Magentic Paper
-- [Huggingface](https://hf.co)
-- [Gradio](https://gradio.app)
-- [DeepCritical](https://github.com/DeepCritical)
-- [Modal](https://modal.com)
-- Microsoft
-- Pydantic
-- Llama-index
-- Anthhropic/MCP
-- All our Tool Providers
-
-
-## Links
-
-[![GitHub](https://img.shields.io/github/stars/DeepCritical/GradioDemo?style=for-the-badge&logo=github&logoColor=white&label=GitHub&labelColor=181717&color=181717)](https://github.com/DeepCritical/GradioDemo)
-[![Documentation](https://img.shields.io/badge/Docs-0080FF?style=for-the-badge&logo=readthedocs&logoColor=white&labelColor=0080FF&color=0080FF)](deepcritical.github.io/GradioDemo/)
-[![Demo](https://img.shields.io/badge/Demo-FFD21E?style=for-the-badge&logo=huggingface&logoColor=white&labelColor=FFD21E&color=FFD21E)](https://huggingface.co/spaces/DataQuests/DeepCritical)
-[![codecov](https://codecov.io/gh/DeepCritical/GradioDemo/graph/badge.svg?token=B1f05RCGpz)](https://codecov.io/gh/DeepCritical/GradioDemo)
-[![Join us on Discord](https://img.shields.io/discord/1109943800132010065?label=Discord&logo=discord&style=flat-square)](https://discord.gg/qdfnvSPcqP) 

REPORT_WRITING_AGENTS_ANALYSIS.md

@@ -1,189 +0,0 @@
-# Report Writing Agents Analysis
-
-## Summary
-
-This document identifies all agents and methods in the repository that generate reports or write to files.
-
-## Key Finding
-
-**All report-writing agents return strings (markdown) - NONE write directly to files.**
-
-The agents generate report content but do not save it to disk. File writing would need to be added as a separate step.
-
----
-
-## Report Writing Agents
-
-### 1. WriterAgent
-**File**: `src/agents/writer.py`
-
-**Method**: `async def write_report(query, findings, output_length, output_instructions) -> str`
-
-**Returns**: Markdown formatted report string
-
-**Purpose**: Generates final reports from research findings with numbered citations
-
-**File Writing**: ❌ **NO** - Returns string only
-
-**Key Features**:
-- Validates inputs
-- Truncates very long findings (max 50,000 chars)
-- Retry logic (3 retries)
-- Returns markdown with numbered citations
-
----
-
-### 2. LongWriterAgent
-**File**: `src/agents/long_writer.py`
-
-**Methods**:
-- `async def write_next_section(original_query, report_draft, next_section_title, next_section_draft) -> LongWriterOutput`
-- `async def write_report(original_query, report_title, report_draft) -> str`
-
-**Returns**: 
-- `write_next_section()`: `LongWriterOutput` object (structured output)
-- `write_report()`: Complete markdown report string
-
-**Purpose**: Iteratively writes report sections with proper citations and reference management
-
-**File Writing**: ❌ **NO** - Returns string only
-
-**Key Features**:
-- Writes sections iteratively
-- Reformats and deduplicates references
-- Adjusts heading levels
-- Aggregates references across sections
-
----
-
-### 3. ProofreaderAgent
-**File**: `src/agents/proofreader.py`
-
-**Method**: `async def proofread(query, report_draft) -> str`
-
-**Returns**: Final polished markdown report string
-
-**Purpose**: Proofreads and finalizes report drafts
-
-**File Writing**: ❌ **NO** - Returns string only
-
-**Key Features**:
-- Combines sections
-- Removes duplicates
-- Adds summary
-- Preserves references
-- Polishes wording
-
----
-
-### 4. ReportAgent
-**File**: `src/agents/report_agent.py`
-
-**Method**: `async def run(messages, thread, **kwargs) -> AgentRunResponse`
-
-**Returns**: `AgentRunResponse` with markdown text in `messages[0].text`
-
-**Purpose**: Generates structured scientific reports from evidence and hypotheses
-
-**File Writing**: ❌ **NO** - Returns `AgentRunResponse` object
-
-**Key Features**:
-- Uses structured `ResearchReport` model
-- Validates citations
-- Returns markdown via `report.to_markdown()`
-
----
-
-## File Writing Operations Found
-
-### Temporary File Writing (Not Reports)
-
-1. **ImageOCRService** (`src/services/image_ocr.py`)
-   - `_save_image_temp(image) -> str`
-   - Saves temporary images for OCR processing
-   - Returns temp file path
-
-2. **STTService** (`src/services/stt_gradio.py`)
-   - `_save_audio_temp(audio_array, sample_rate) -> str`
-   - Saves temporary audio files for transcription
-   - Returns temp file path
-
----
-
-## Where Reports Are Used
-
-### Graph Orchestrator
-**File**: `src/orchestrator/graph_orchestrator.py`
-
-- Line 642: `final_report = await long_writer_agent.write_report(...)`
-- Returns string result, stored in graph context
-- Final result passed through `AgentEvent` with `message` field
-
-### Research Flows
-**File**: `src/orchestrator/research_flow.py`
-
-- `IterativeResearchFlow._create_final_report()`: Calls `writer_agent.write_report()`
-- `DeepResearchFlow._create_final_report()`: Calls `long_writer_agent.write_report()`
-- Both return strings
-
----
-
-## Integration Points for File Writing
-
-To add file writing capability, you would need to:
-
-1. **After report generation**: Save the returned string to a file
-2. **In graph orchestrator**: After `write_report()`, save to file and include path in result
-3. **In research flows**: After `_create_final_report()`, save to file
-
-### Example Implementation Pattern
-
-```python
-import tempfile
-from pathlib import Path
-
-# After report generation
-report_content = await writer_agent.write_report(...)
-
-# Save to file
-output_dir = Path("/tmp/reports")  # or configurable
-output_dir.mkdir(exist_ok=True)
-file_path = output_dir / f"report_{timestamp}.md"
-
-with open(file_path, "w", encoding="utf-8") as f:
-    f.write(report_content)
-
-# Return both content and file path
-return {
-    "message": "Report generated successfully",
-    "file": str(file_path)
-}
-```
-
----
-
-## Recommendations
-
-1. **Add file writing utility**: Create a helper function to save reports to files
-2. **Make it optional**: Add configuration flag to enable/disable file saving
-3. **Use temp directory**: Save to temp directory by default, allow custom path
-4. **Include in graph results**: Modify graph orchestrator to optionally save and return file paths
-5. **Support multiple formats**: Consider saving as both `.md` and potentially `.pdf` or `.html`
-
----
-
-## Current State
-
-✅ **Report Generation**: Fully implemented  
-❌ **File Writing**: Not implemented  
-✅ **File Output Integration**: Recently added (see previous work on `event_to_chat_message`)  
-
-The infrastructure to handle file outputs in Gradio is in place, but the agents themselves do not yet write files. They would need to be enhanced or wrapped to add file writing capability.
-
-
-
-
-
-
-
-

SERPER_WEBSEARCH_IMPLEMENTATION_PLAN.md

@@ -1,403 +0,0 @@
-# SERPER Web Search Implementation Plan
-
-## Executive Summary
-
-This plan details the implementation of SERPER-based web search by vendoring code from `folder/tools/web_search.py` into `src/tools/`, creating a protocol-compliant `SerperWebSearchTool`, fixing the existing `WebSearchTool`, and integrating both into the main search flow.
-
-## Project Structure
-
-### Project 1: Vendor and Refactor Core Web Search Components
-**Goal**: Extract and vendor Serper/SearchXNG search logic from `folder/tools/web_search.py` into `src/tools/`
-
-### Project 2: Create Protocol-Compliant SerperWebSearchTool
-**Goal**: Implement `SerperWebSearchTool` class that fully complies with `SearchTool` protocol
-
-### Project 3: Fix Existing WebSearchTool Protocol Compliance
-**Goal**: Make existing `WebSearchTool` (DuckDuckGo) protocol-compliant
-
-### Project 4: Integrate Web Search into SearchHandler
-**Goal**: Add web search tools to main search flow in `src/app.py`
-
-### Project 5: Update Callers and Dependencies
-**Goal**: Update all code that uses web search to work with new implementation
-
-### Project 6: Testing and Validation
-**Goal**: Add comprehensive tests for all web search implementations
-
----
-
-## Detailed Implementation Plan
-
-### PROJECT 1: Vendor and Refactor Core Web Search Components
-
-#### Activity 1.1: Create Vendor Module Structure
-**File**: `src/tools/vendored/__init__.py`
-- **Task 1.1.1**: Create `src/tools/vendored/` directory
-- **Task 1.1.2**: Create `__init__.py` with exports
-
-**File**: `src/tools/vendored/web_search_core.py`
-- **Task 1.1.3**: Vendor `ScrapeResult`, `WebpageSnippet`, `SearchResults` models from `folder/tools/web_search.py` (lines 23-37)
-- **Task 1.1.4**: Vendor `scrape_urls()` function (lines 274-299)
-- **Task 1.1.5**: Vendor `fetch_and_process_url()` function (lines 302-348)
-- **Task 1.1.6**: Vendor `html_to_text()` function (lines 351-368)
-- **Task 1.1.7**: Vendor `is_valid_url()` function (lines 371-410)
-- **Task 1.1.8**: Vendor `ssl_context` setup (lines 115-120)
-- **Task 1.1.9**: Add imports: `aiohttp`, `asyncio`, `BeautifulSoup`, `ssl`
-- **Task 1.1.10**: Add `CONTENT_LENGTH_LIMIT = 10000` constant
-- **Task 1.1.11**: Add type hints following project standards
-- **Task 1.1.12**: Add structlog logging
-- **Task 1.1.13**: Replace `print()` statements with `logger` calls
-
-**File**: `src/tools/vendored/serper_client.py`
-- **Task 1.1.14**: Vendor `SerperClient` class from `folder/tools/web_search.py` (lines 123-196)
-- **Task 1.1.15**: Remove dependency on `ResearchAgent` and `ResearchRunner`
-- **Task 1.1.16**: Replace filter agent with simple relevance filtering or remove it
-- **Task 1.1.17**: Add `__init__` that takes `api_key: str | None` parameter
-- **Task 1.1.18**: Update `search()` method to return `list[WebpageSnippet]` without filtering
-- **Task 1.1.19**: Remove `_filter_results()` method (or make it optional)
-- **Task 1.1.20**: Add error handling with `SearchError` and `RateLimitError`
-- **Task 1.1.21**: Add structlog logging
-- **Task 1.1.22**: Add type hints
-
-**File**: `src/tools/vendored/searchxng_client.py`
-- **Task 1.1.23**: Vendor `SearchXNGClient` class from `folder/tools/web_search.py` (lines 199-271)
-- **Task 1.1.24**: Remove dependency on `ResearchAgent` and `ResearchRunner`
-- **Task 1.1.25**: Replace filter agent with simple relevance filtering or remove it
-- **Task 1.1.26**: Add `__init__` that takes `host: str` parameter
-- **Task 1.1.27**: Update `search()` method to return `list[WebpageSnippet]` without filtering
-- **Task 1.1.28**: Remove `_filter_results()` method (or make it optional)
-- **Task 1.1.29**: Add error handling with `SearchError` and `RateLimitError`
-- **Task 1.1.30**: Add structlog logging
-- **Task 1.1.31**: Add type hints
-
-#### Activity 1.2: Create Rate Limiting for Web Search
-**File**: `src/tools/rate_limiter.py`
-- **Task 1.2.1**: Add `get_serper_limiter()` function (rate: "10/second" with API key)
-- **Task 1.2.2**: Add `get_searchxng_limiter()` function (rate: "5/second")
-- **Task 1.2.3**: Use `RateLimiterFactory.get()` pattern
-
----
-
-### PROJECT 2: Create Protocol-Compliant SerperWebSearchTool
-
-#### Activity 2.1: Implement SerperWebSearchTool Class
-**File**: `src/tools/serper_web_search.py`
-- **Task 2.1.1**: Create new file `src/tools/serper_web_search.py`
-- **Task 2.1.2**: Add imports:
-  - `from src.tools.base import SearchTool`
-  - `from src.tools.vendored.serper_client import SerperClient`
-  - `from src.tools.vendored.web_search_core import scrape_urls, WebpageSnippet`
-  - `from src.tools.rate_limiter import get_serper_limiter`
-  - `from src.tools.query_utils import preprocess_query`
-  - `from src.utils.config import settings`
-  - `from src.utils.exceptions import SearchError, RateLimitError`
-  - `from src.utils.models import Citation, Evidence`
-  - `import structlog`
-  - `from tenacity import retry, stop_after_attempt, wait_exponential`
-
-- **Task 2.1.3**: Create `SerperWebSearchTool` class
-- **Task 2.1.4**: Add `__init__(self, api_key: str | None = None)` method
-  - Line 2.1.4.1: Get API key from parameter or `settings.serper_api_key`
-  - Line 2.1.4.2: Validate API key is not None, raise `ConfigurationError` if missing
-  - Line 2.1.4.3: Initialize `SerperClient(api_key=self.api_key)`
-  - Line 2.1.4.4: Get rate limiter: `self._limiter = get_serper_limiter(self.api_key)`
-
-- **Task 2.1.5**: Add `@property def name(self) -> str:` returning `"serper"`
-
-- **Task 2.1.6**: Add `async def _rate_limit(self) -> None:` method
-  - Line 2.1.6.1: Call `await self._limiter.acquire()`
-
-- **Task 2.1.7**: Add `@retry(...)` decorator with exponential backoff
-
-- **Task 2.1.8**: Add `async def search(self, query: str, max_results: int = 10) -> list[Evidence]:` method
-  - Line 2.1.8.1: Call `await self._rate_limit()`
-  - Line 2.1.8.2: Preprocess query: `clean_query = preprocess_query(query)`
-  - Line 2.1.8.3: Use `clean_query if clean_query else query`
-  - Line 2.1.8.4: Call `search_results = await self._client.search(query, filter_for_relevance=False, max_results=max_results)`
-  - Line 2.1.8.5: Call `scraped = await scrape_urls(search_results)`
-  - Line 2.1.8.6: Convert `ScrapeResult` to `Evidence` objects:
-    - Line 2.1.8.6.1: Create `Citation` with `title`, `url`, `source="serper"`, `date="Unknown"`, `authors=[]`
-    - Line 2.1.8.6.2: Create `Evidence` with `content=scraped.text`, `citation`, `relevance=0.0`
-  - Line 2.1.8.7: Return `list[Evidence]`
-  - Line 2.1.8.8: Add try/except for `httpx.HTTPStatusError`:
-    - Line 2.1.8.8.1: Check for 429 status, raise `RateLimitError`
-    - Line 2.1.8.8.2: Otherwise raise `SearchError`
-  - Line 2.1.8.9: Add try/except for `httpx.TimeoutException`, raise `SearchError`
-  - Line 2.1.8.10: Add generic exception handler, log and raise `SearchError`
-
-#### Activity 2.2: Implement SearchXNGWebSearchTool Class
-**File**: `src/tools/searchxng_web_search.py`
-- **Task 2.2.1**: Create new file `src/tools/searchxng_web_search.py`
-- **Task 2.2.2**: Add imports (similar to SerperWebSearchTool)
-- **Task 2.2.3**: Create `SearchXNGWebSearchTool` class
-- **Task 2.2.4**: Add `__init__(self, host: str | None = None)` method
-  - Line 2.2.4.1: Get host from parameter or `settings.searchxng_host`
-  - Line 2.2.4.2: Validate host is not None, raise `ConfigurationError` if missing
-  - Line 2.2.4.3: Initialize `SearchXNGClient(host=self.host)`
-  - Line 2.2.4.4: Get rate limiter: `self._limiter = get_searchxng_limiter()`
-
-- **Task 2.2.5**: Add `@property def name(self) -> str:` returning `"searchxng"`
-
-- **Task 2.2.6**: Add `async def _rate_limit(self) -> None:` method
-
-- **Task 2.2.7**: Add `@retry(...)` decorator
-
-- **Task 2.2.8**: Add `async def search(self, query: str, max_results: int = 10) -> list[Evidence]:` method
-  - Line 2.2.8.1-2.2.8.10: Similar structure to SerperWebSearchTool
-
----
-
-### PROJECT 3: Fix Existing WebSearchTool Protocol Compliance
-
-#### Activity 3.1: Update WebSearchTool Class
-**File**: `src/tools/web_search.py`
-- **Task 3.1.1**: Add `@property def name(self) -> str:` method returning `"duckduckgo"` (after line 17)
-
-- **Task 3.1.2**: Change `search()` return type from `SearchResult` to `list[Evidence]` (line 19)
-
-- **Task 3.1.3**: Update `search()` method body:
-  - Line 3.1.3.1: Keep existing search logic (lines 21-43)
-  - Line 3.1.3.2: Instead of returning `SearchResult`, return `evidence` list directly (line 44)
-  - Line 3.1.3.3: Update exception handler to return empty list `[]` instead of `SearchResult` (line 51)
-
-- **Task 3.1.4**: Add imports if needed:
-  - Line 3.1.4.1: `from src.utils.exceptions import SearchError`
-  - Line 3.1.4.2: Update exception handling to raise `SearchError` instead of returning error `SearchResult`
-
-- **Task 3.1.5**: Add query preprocessing:
-  - Line 3.1.5.1: Import `from src.tools.query_utils import preprocess_query`
-  - Line 3.1.5.2: Add `clean_query = preprocess_query(query)` before search
-  - Line 3.1.5.3: Use `clean_query if clean_query else query`
-
-#### Activity 3.2: Update Retrieval Agent Caller
-**File**: `src/agents/retrieval_agent.py`
-- **Task 3.2.1**: Update `search_web()` function (line 31):
-  - Line 3.2.1.1: Change `results = await _web_search.search(query, max_results)` 
-  - Line 3.2.1.2: Change to `evidence = await _web_search.search(query, max_results)`
-  - Line 3.2.1.3: Update check: `if not evidence:` instead of `if not results.evidence:`
-  - Line 3.2.1.4: Update state update: `new_count = state.add_evidence(evidence)` instead of `results.evidence`
-  - Line 3.2.1.5: Update logging: `results_found=len(evidence)` instead of `len(results.evidence)`
-  - Line 3.2.1.6: Update output formatting: `for i, r in enumerate(evidence[:max_results], 1):` instead of `results.evidence[:max_results]`
-  - Line 3.2.1.7: Update deduplication: `await state.embedding_service.deduplicate(evidence)` instead of `results.evidence`
-  - Line 3.2.1.8: Update output message: `Found {len(evidence)} web results` instead of `len(results.evidence)`
-
----
-
-### PROJECT 4: Integrate Web Search into SearchHandler
-
-#### Activity 4.1: Create Web Search Tool Factory
-**File**: `src/tools/web_search_factory.py`
-- **Task 4.1.1**: Create new file `src/tools/web_search_factory.py`
-- **Task 4.1.2**: Add imports:
-  - `from src.tools.web_search import WebSearchTool`
-  - `from src.tools.serper_web_search import SerperWebSearchTool`
-  - `from src.tools.searchxng_web_search import SearchXNGWebSearchTool`
-  - `from src.utils.config import settings`
-  - `from src.utils.exceptions import ConfigurationError`
-  - `import structlog`
-
-- **Task 4.1.3**: Add `logger = structlog.get_logger()`
-
-- **Task 4.1.4**: Create `def create_web_search_tool() -> SearchTool | None:` function
-  - Line 4.1.4.1: Check `settings.web_search_provider`
-  - Line 4.1.4.2: If `"serper"`:
-    - Line 4.1.4.2.1: Check `settings.serper_api_key` or `settings.web_search_available()`
-    - Line 4.1.4.2.2: If available, return `SerperWebSearchTool()`
-    - Line 4.1.4.2.3: Else log warning and return `None`
-  - Line 4.1.4.3: If `"searchxng"`:
-    - Line 4.1.4.3.1: Check `settings.searchxng_host` or `settings.web_search_available()`
-    - Line 4.1.4.3.2: If available, return `SearchXNGWebSearchTool()`
-    - Line 4.1.4.3.3: Else log warning and return `None`
-  - Line 4.1.4.4: If `"duckduckgo"`:
-    - Line 4.1.4.4.1: Return `WebSearchTool()` (always available)
-  - Line 4.1.4.5: If `"brave"` or `"tavily"`:
-    - Line 4.1.4.5.1: Log warning "Not yet implemented"
-    - Line 4.1.4.5.2: Return `None`
-  - Line 4.1.4.6: Default: return `WebSearchTool()` (fallback to DuckDuckGo)
-
-#### Activity 4.2: Update SearchHandler Initialization
-**File**: `src/app.py`
-- **Task 4.2.1**: Add import: `from src.tools.web_search_factory import create_web_search_tool`
-
-- **Task 4.2.2**: Update `configure_orchestrator()` function (around line 73):
-  - Line 4.2.2.1: Before creating `SearchHandler`, call `web_search_tool = create_web_search_tool()`
-  - Line 4.2.2.2: Create tools list: `tools = [PubMedTool(), ClinicalTrialsTool(), EuropePMCTool()]`
-  - Line 4.2.2.3: If `web_search_tool is not None`:
-    - Line 4.2.2.3.1: Append `web_search_tool` to tools list
-    - Line 4.2.2.3.2: Log info: "Web search tool added to search handler"
-  - Line 4.2.2.4: Update `SearchHandler` initialization to use `tools` list
-
----
-
-### PROJECT 5: Update Callers and Dependencies
-
-#### Activity 5.1: Update web_search_adapter
-**File**: `src/tools/web_search_adapter.py`
-- **Task 5.1.1**: Update `web_search()` function to use new implementation:
-  - Line 5.1.1.1: Import `from src.tools.web_search_factory import create_web_search_tool`
-  - Line 5.1.1.2: Remove dependency on `folder.tools.web_search`
-  - Line 5.1.1.3: Get tool: `tool = create_web_search_tool()`
-  - Line 5.1.1.4: If `tool is None`, return error message
-  - Line 5.1.1.5: Call `evidence = await tool.search(query, max_results=5)`
-  - Line 5.1.1.6: Convert `Evidence` objects to formatted string:
-    - Line 5.1.1.6.1: Format each evidence with title, URL, content preview
-  - Line 5.1.1.7: Return formatted string
-
-#### Activity 5.2: Update Tool Executor
-**File**: `src/tools/tool_executor.py`
-- **Task 5.2.1**: Verify `web_search_adapter.web_search()` usage (line 86) still works
-- **Task 5.2.2**: No changes needed if adapter is updated correctly
-
-#### Activity 5.3: Update Planner Agent
-**File**: `src/orchestrator/planner_agent.py`
-- **Task 5.3.1**: Verify `web_search_adapter.web_search()` usage (line 14) still works
-- **Task 5.3.2**: No changes needed if adapter is updated correctly
-
-#### Activity 5.4: Remove Legacy Dependencies
-**File**: `src/tools/web_search_adapter.py`
-- **Task 5.4.1**: Remove import of `folder.llm_config` and `folder.tools.web_search`
-- **Task 5.4.2**: Update error messages to reflect new implementation
-
----
-
-### PROJECT 6: Testing and Validation
-
-#### Activity 6.1: Unit Tests for Vendored Components
-**File**: `tests/unit/tools/test_vendored_web_search_core.py`
-- **Task 6.1.1**: Test `scrape_urls()` function
-- **Task 6.1.2**: Test `fetch_and_process_url()` function
-- **Task 6.1.3**: Test `html_to_text()` function
-- **Task 6.1.4**: Test `is_valid_url()` function
-
-**File**: `tests/unit/tools/test_vendored_serper_client.py`
-- **Task 6.1.5**: Mock SerperClient API calls
-- **Task 6.1.6**: Test successful search
-- **Task 6.1.7**: Test error handling
-- **Task 6.1.8**: Test rate limiting
-
-**File**: `tests/unit/tools/test_vendored_searchxng_client.py`
-- **Task 6.1.9**: Mock SearchXNGClient API calls
-- **Task 6.1.10**: Test successful search
-- **Task 6.1.11**: Test error handling
-- **Task 6.1.12**: Test rate limiting
-
-#### Activity 6.2: Unit Tests for Web Search Tools
-**File**: `tests/unit/tools/test_serper_web_search.py`
-- **Task 6.2.1**: Test `SerperWebSearchTool.__init__()` with valid API key
-- **Task 6.2.2**: Test `SerperWebSearchTool.__init__()` without API key (should raise)
-- **Task 6.2.3**: Test `name` property returns `"serper"`
-- **Task 6.2.4**: Test `search()` returns `list[Evidence]`
-- **Task 6.2.5**: Test `search()` with mocked SerperClient
-- **Task 6.2.6**: Test error handling (SearchError, RateLimitError)
-- **Task 6.2.7**: Test query preprocessing
-- **Task 6.2.8**: Test rate limiting
-
-**File**: `tests/unit/tools/test_searchxng_web_search.py`
-- **Task 6.2.9**: Similar tests for SearchXNGWebSearchTool
-
-**File**: `tests/unit/tools/test_web_search.py`
-- **Task 6.2.10**: Test `WebSearchTool.name` property returns `"duckduckgo"`
-- **Task 6.2.11**: Test `WebSearchTool.search()` returns `list[Evidence]`
-- **Task 6.2.12**: Test `WebSearchTool.search()` with mocked DDGS
-- **Task 6.2.13**: Test error handling
-- **Task 6.2.14**: Test query preprocessing
-
-#### Activity 6.3: Integration Tests
-**File**: `tests/integration/test_web_search_integration.py`
-- **Task 6.3.1**: Test `SerperWebSearchTool` with real API (marked `@pytest.mark.integration`)
-- **Task 6.3.2**: Test `SearchXNGWebSearchTool` with real API (marked `@pytest.mark.integration`)
-- **Task 6.3.3**: Test `WebSearchTool` with real DuckDuckGo (marked `@pytest.mark.integration`)
-- **Task 6.3.4**: Test `create_web_search_tool()` factory function
-- **Task 6.3.5**: Test SearchHandler with web search tool
-
-#### Activity 6.4: Update Existing Tests
-**File**: `tests/unit/agents/test_retrieval_agent.py`
-- **Task 6.4.1**: Update tests to expect `list[Evidence]` instead of `SearchResult`
-- **Task 6.4.2**: Mock `WebSearchTool.search()` to return `list[Evidence]`
-
-**File**: `tests/unit/tools/test_tool_executor.py`
-- **Task 6.4.3**: Verify tests still pass with updated `web_search_adapter`
-
----
-
-## Implementation Order
-
-1. **PROJECT 1**: Vendor core components (foundation)
-2. **PROJECT 3**: Fix existing WebSearchTool (quick win, unblocks retrieval agent)
-3. **PROJECT 2**: Create SerperWebSearchTool (new functionality)
-4. **PROJECT 4**: Integrate into SearchHandler (main integration)
-5. **PROJECT 5**: Update callers (cleanup dependencies)
-6. **PROJECT 6**: Testing (validation)
-
----
-
-## Dependencies and Prerequisites
-
-### External Dependencies
-- `aiohttp` - Already in requirements
-- `beautifulsoup4` - Already in requirements
-- `duckduckgo-search` - Already in requirements
-- `tenacity` - Already in requirements
-- `structlog` - Already in requirements
-
-### Internal Dependencies
-- `src/tools/base.py` - SearchTool protocol
-- `src/tools/rate_limiter.py` - Rate limiting utilities
-- `src/tools/query_utils.py` - Query preprocessing
-- `src/utils/config.py` - Settings and configuration
-- `src/utils/exceptions.py` - Custom exceptions
-- `src/utils/models.py` - Evidence, Citation models
-
-### Configuration Requirements
-- `SERPER_API_KEY` - For Serper provider
-- `SEARCHXNG_HOST` - For SearchXNG provider
-- `WEB_SEARCH_PROVIDER` - Environment variable (default: "duckduckgo")
-
----
-
-## Risk Assessment
-
-### High Risk
-- **Breaking changes to retrieval_agent.py**: Must update carefully to handle `list[Evidence]` instead of `SearchResult`
-- **Legacy folder dependencies**: Need to ensure all code is properly vendored
-
-### Medium Risk
-- **Rate limiting**: Serper API may have different limits than expected
-- **Error handling**: Need to handle API failures gracefully
-
-### Low Risk
-- **Query preprocessing**: May need adjustment for web search vs PubMed
-- **Testing**: Integration tests require API keys
-
----
-
-## Success Criteria
-
-1. ✅ `SerperWebSearchTool` implements `SearchTool` protocol correctly
-2. ✅ `WebSearchTool` implements `SearchTool` protocol correctly
-3. ✅ Both tools can be added to `SearchHandler`
-4. ✅ `web_search_adapter` works with new implementation
-5. ✅ `retrieval_agent` works with updated `WebSearchTool`
-6. ✅ All unit tests pass
-7. ✅ Integration tests pass (with API keys)
-8. ✅ No dependencies on `folder/tools/web_search.py` in `src/` code
-9. ✅ Configuration supports multiple providers
-10. ✅ Error handling is robust
-
----
-
-## Notes
-
-- The vendored code should be self-contained and not depend on `folder/` modules
-- Filter agent functionality from original code is removed (can be added later if needed)
-- Rate limiting follows the same pattern as PubMed tool
-- Query preprocessing may need web-specific adjustments (less aggressive than PubMed)
-- Consider adding relevance scoring in the future
-
-
-
-
-
-
-
-

WEB_SEARCH_TOOL_ASSESSMENT.md

@@ -1,239 +0,0 @@
-# Web Search Tool Assessment
-
-## Executive Summary
-
-The application has **two separate web search implementations** with different readiness levels:
-
-1. **`WebSearchTool`** (`src/tools/web_search.py`) - **Partially Ready** ⚠️
-   - Functional but **NOT compliant** with `SearchTool` protocol
-   - **NOT integrated** into main search handler
-   - Only used in magentic orchestrator's retrieval agent
-
-2. **`web_search_adapter`** (`src/tools/web_search_adapter.py`) - **Functional** ✅
-   - Used by tool executor for WebSearchAgent tasks
-   - Relies on legacy `folder/tools/web_search.py` implementation
-
-## Detailed Analysis
-
-### 1. WebSearchTool (`src/tools/web_search.py`)
-
-#### Current Implementation
-- **Location**: `src/tools/web_search.py`
-- **Provider**: DuckDuckGo (no API key required)
-- **Status**: ⚠️ **Partially Ready**
-
-#### Issues Identified
-
-**❌ Protocol Non-Compliance:**
-```python
-# Missing required 'name' property
-class WebSearchTool:
-    # Should have: @property def name(self) -> str: return "web"
-    
-    # Wrong return type - should return list[Evidence], not SearchResult
-    async def search(self, query: str, max_results: int = 10) -> SearchResult:
-        # Returns SearchResult instead of list[Evidence]
-```
-
-**Comparison with other tools:**
-- `PubMedTool` has `@property def name(self) -> str: return "pubmed"`
-- `PubMedTool.search()` returns `list[Evidence]`
-- `WebSearchTool` returns `SearchResult` (contains `evidence` list inside)
-
-**❌ Not Integrated:**
-- **NOT** included in `SearchHandler` initialization in `src/app.py`:
-  ```python
-  search_handler = SearchHandler(
-      tools=[PubMedTool(), ClinicalTrialsTool(), EuropePMCTool()],
-      # WebSearchTool() is missing!
-  )
-  ```
-
-**✅ Current Usage:**
-- Used in `src/agents/retrieval_agent.py` (magentic orchestrator):
-  ```python
-  from src.tools.web_search import WebSearchTool
-  _web_search = WebSearchTool()
-  ```
-
-#### Fix Required
-To make `WebSearchTool` compliant and usable:
-
-1. **Add `name` property:**
-   ```python
-   @property
-   def name(self) -> str:
-       return "web"
-   ```
-
-2. **Fix return type:**
-   ```python
-   async def search(self, query: str, max_results: int = 10) -> list[Evidence]:
-       # ... existing code ...
-       return evidence  # Return list[Evidence] directly, not SearchResult
-   ```
-
-3. **Register in SearchHandler:**
-   ```python
-   from src.tools.web_search import WebSearchTool
-   
-   search_handler = SearchHandler(
-       tools=[
-           PubMedTool(), 
-           ClinicalTrialsTool(), 
-           EuropePMCTool(),
-           WebSearchTool()  # Add this
-       ],
-   )
-   ```
-
----
-
-### 2. web_search_adapter (`src/tools/web_search_adapter.py`)
-
-#### Current Implementation
-- **Location**: `src/tools/web_search_adapter.py`
-- **Status**: ✅ **Functional**
-- **Provider**: Uses legacy `folder/tools/web_search.py` (Serper/SearchXNG)
-
-#### Usage
-- Used by `src/tools/tool_executor.py` for `WebSearchAgent` tasks:
-  ```python
-  if task.agent == "WebSearchAgent":
-      result_text = await web_search(task.query)
-  ```
-
-- Used by `src/orchestrator/planner_agent.py` for background context
-
-#### Dependencies
-- Requires `folder/tools/web_search.py` (legacy implementation)
-- Supports Serper API (requires `SERPER_API_KEY`)
-- Supports SearchXNG API (requires `SEARCHXNG_HOST`)
-
-#### Limitations
-- Returns formatted string (not `Evidence` objects)
-- Not integrated with `SearchHandler` (different execution path)
-- Depends on legacy folder structure
-
----
-
-## Integration Status
-
-### SearchHandler Integration
-**Current State**: ❌ **NOT Integrated**
-
-The main `SearchHandler` in `src/app.py` only includes:
-- `PubMedTool()`
-- `ClinicalTrialsTool()`
-- `EuropePMCTool()`
-
-**WebSearchTool is missing from the main search flow.**
-
-### Tool Executor Integration
-**Current State**: ✅ **Integrated**
-
-`web_search_adapter` is used via `tool_executor.py`:
-- Executes when `AgentTask.agent == "WebSearchAgent"`
-- Used in iterative/deep research flows
-- Returns formatted text (not Evidence objects)
-
-### Magentic Orchestrator Integration
-**Current State**: ✅ **Integrated**
-
-`WebSearchTool` is used in `retrieval_agent.py`:
-- Direct instantiation: `_web_search = WebSearchTool()`
-- Used via `search_web()` function
-- Updates workflow state with evidence
-
----
-
-## Can It Be Used?
-
-### WebSearchTool (`src/tools/web_search.py`)
-**Status**: ⚠️ **Can be used, but with limitations**
-
-**Can be used:**
-- ✅ In magentic orchestrator (already working)
-- ✅ As standalone tool (functional)
-
-**Cannot be used:**
-- ❌ In `SearchHandler` (protocol non-compliance)
-- ❌ In parallel search flows (not registered)
-
-**To make fully usable:**
-1. Fix protocol compliance (add `name`, fix return type)
-2. Register in `SearchHandler`
-3. Test integration
-
-### web_search_adapter
-**Status**: ✅ **Can be used**
-
-**Can be used:**
-- ✅ Via `tool_executor` for WebSearchAgent tasks
-- ✅ In planner agent for background context
-- ✅ In iterative/deep research flows
-
-**Limitations:**
-- Returns string format (not Evidence objects)
-- Requires legacy folder dependencies
-- Different execution path than SearchHandler
-
----
-
-## Recommendations
-
-### Priority 1: Fix WebSearchTool Protocol Compliance
-Make `WebSearchTool` fully compliant with `SearchTool` protocol:
-
-1. Add `name` property
-2. Change return type from `SearchResult` to `list[Evidence]`
-3. Update all callers if needed
-
-### Priority 2: Integrate into SearchHandler
-Add `WebSearchTool` to main search flow:
-
-```python
-from src.tools.web_search import WebSearchTool
-
-search_handler = SearchHandler(
-    tools=[
-        PubMedTool(), 
-        ClinicalTrialsTool(), 
-        EuropePMCTool(),
-        WebSearchTool()  # Add web search
-    ],
-)
-```
-
-### Priority 3: Consolidate Implementations
-Consider consolidating the two implementations:
-- Keep `WebSearchTool` as the main implementation
-- Deprecate or migrate `web_search_adapter` usage
-- Remove dependency on `folder/tools/web_search.py`
-
-### Priority 4: Testing
-Add tests for:
-- Protocol compliance
-- SearchHandler integration
-- Error handling
-- Rate limiting (if needed)
-
----
-
-## Summary Table
-
-| Component | Status | Protocol Compliant | Integrated | Can Be Used |
-|-----------|--------|-------------------|------------|-------------|
-| `WebSearchTool` | ⚠️ Partial | ❌ No | ❌ No | ⚠️ Limited |
-| `web_search_adapter` | ✅ Functional | N/A | ✅ Yes (tool_executor) | ✅ Yes |
-
----
-
-## Conclusion
-
-The web search functionality exists in two forms:
-1. **`WebSearchTool`** is functional but needs protocol fixes to be fully integrated
-2. **`web_search_adapter`** is working but uses a different execution path
-
-**Recommendation**: Fix `WebSearchTool` protocol compliance and integrate it into `SearchHandler` for unified search capabilities across all orchestrators.
-

dev/Makefile

@@ -1,51 +0,0 @@
-.PHONY: install test lint format typecheck check clean all cov cov-html
-
-# Default target
-all: check
-
-install:
-	uv sync --all-extras
-	uv run pre-commit install
-
-test:
-	uv run pytest tests/unit/ -v -m "not openai" -p no:logfire
-
-test-hf:
-	uv run pytest tests/ -v -m "huggingface" -p no:logfire
-
-test-all:
-	uv run pytest tests/ -v -p no:logfire
-
-# Coverage aliases
-cov: test-cov
-test-cov:
-	uv run pytest --cov=src --cov-report=term-missing -m "not openai" -p no:logfire
-
-cov-html:
-	uv run pytest --cov=src --cov-report=html -p no:logfire
-	@echo "Coverage report: open htmlcov/index.html"
-
-lint:
-	uv run ruff check src tests
-
-format:
-	uv run ruff format src tests
-
-typecheck:
-	uv run mypy src
-
-check: lint typecheck test-cov
-	@echo "All checks passed!"
-
-docs-build:
-	uv run mkdocs build
-
-docs-serve:
-	uv run mkdocs serve
-
-docs-clean:
-	rm -rf site/
-
-clean:
-	rm -rf .pytest_cache .mypy_cache .ruff_cache __pycache__ .coverage htmlcov
-	find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true

dev/__init__.py

@@ -2,3 +2,4 @@
 
 
 
+

docs/MKDOCS_IMPROVEMENTS_ASSESSMENT.md

@@ -0,0 +1,642 @@
+# MkDocs & Material UI Improvement Assessment
+
+## Current Configuration Analysis
+
+Your current `mkdocs.yml` already includes many excellent features:
+- ✅ Material theme with light/dark mode toggle
+- ✅ Navigation tabs, sections, expand, and top navigation
+- ✅ Search with suggestions and highlighting
+- ✅ Code annotation and copy buttons
+- ✅ Mermaid diagram support
+- ✅ Code include plugin
+- ✅ Minification for performance
+- ✅ Comprehensive markdown extensions
+
+## Recommended Improvements
+
+### 1. **Versioning & Multi-Version Documentation** ⭐ High Priority
+
+If you plan to maintain multiple versions or branches:
+
+```yaml
+plugins:
+  - search
+  - mermaid2
+  - codeinclude
+  - minify:
+      minify_html: true
+      minify_js: true
+      minify_css: true
+  - git-revision-date-localized:
+      enable_creation_date: true
+      type: timeago
+  # Optional: For versioning
+  # - versioning:
+  #     version: ['dev', 'main']
+```
+
+**Benefits**: Shows when pages were last updated, helps users understand document freshness.
+
+### 2. **Git Integration & Revision Information** ⭐ High Priority
+
+Add revision dates and authors to pages:
+
+```yaml
+plugins:
+  - git-revision-date-localized:
+      enable_creation_date: true
+      type: timeago
+      fallback_to_build_date: true
+  - git-committers:
+      repository: DeepCritical/GradioDemo
+      branch: dev
+```
+
+**Benefits**: Users see when content was last updated, builds trust in documentation freshness.
+
+### 3. **Enhanced Navigation Features** ⭐ High Priority
+
+Add breadcrumbs and improve navigation:
+
+```yaml
+theme:
+  features:
+    - navigation.tabs
+    - navigation.sections
+    - navigation.expand
+    - navigation.top
+    - navigation.indexes  # Add index pages
+    - navigation.instant  # Instant page loads
+    - navigation.tracking  # Track scroll position
+    - navigation.smooth  # Smooth scrolling
+    - search.suggest
+    - search.highlight
+    - content.code.annotate
+    - content.code.copy
+    - content.tabs.link  # Link to specific tabs
+    - content.tooltips  # Tooltips for abbreviations
+```
+
+**Benefits**: Better UX, easier navigation, professional feel.
+
+### 4. **Content Tabs for Code Examples** ⭐ High Priority
+
+Perfect for showing multiple code examples (Python, TypeScript, etc.):
+
+```yaml
+markdown_extensions:
+  - pymdownx.tabbed:
+      alternate_style: true
+      combine_header_slug: true  # Add this
+```
+
+**Usage in docs**:
+````markdown
+=== "Python"
+    ```python
+    def example():
+        pass
+    ```
+
+=== "TypeScript"
+    ```typescript
+    function example() {}
+    ```
+````
+
+**Benefits**: Clean way to show multiple implementations without cluttering pages.
+
+### 5. **Enhanced Admonitions** ⭐ Medium Priority
+
+Add more admonition types and better styling:
+
+```yaml
+markdown_extensions:
+  - admonition
+  - pymdownx.details
+  - pymdownx.superfences:
+      custom_fences:
+        - name: mermaid
+          class: mermaid
+          format: !!python/name:pymdownx.superfences.fence_code_format
+        # Add custom admonition fences
+        - name: danger
+          class: danger
+          format: !!python/name:pymdownx.superfences.fence_code_format
+```
+
+**Usage**:
+```markdown
+!!! danger "Important"
+    This is a critical warning.
+```
+
+**Benefits**: Better visual hierarchy for warnings, tips, and important information.
+
+### 6. **Math Formula Support** ⭐ Medium Priority (if needed)
+
+If your documentation includes mathematical formulas:
+
+```yaml
+markdown_extensions:
+  - pymdownx.arithmatex:
+      generic: true
+  - pymdownx.superfences:
+      custom_fences:
+        - name: math
+          class: arithmetic
+          format: !!python/name:pymdownx.superfences.fence_code_format
+
+extra_javascript:
+  - javascripts/mathjax.js
+  - https://polyfill.io/v3/polyfill.min.js?features=es6
+  - https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js
+```
+
+**Benefits**: Essential for scientific/technical documentation with formulas.
+
+### 7. **Better Code Highlighting** ⭐ Medium Priority
+
+Add more language support and better themes:
+
+```yaml
+markdown_extensions:
+  - pymdownx.highlight:
+      anchor_linenums: true
+      line_spans: __span
+      pygments_lang_class: true
+      use_pygments: true
+      noclasses: false  # Use CSS classes instead of inline styles
+```
+
+**Benefits**: Better syntax highlighting, more language support.
+
+### 8. **Social Links Enhancement** ⭐ Low Priority
+
+Add more social platforms and better icons:
+
+```yaml
+extra:
+  social:
+    - icon: fontawesome/brands/github
+      link: https://github.com/DeepCritical/GradioDemo
+      name: GitHub
+    - icon: fontawesome/brands/twitter
+      link: https://twitter.com/yourhandle
+      name: Twitter
+    - icon: material/web
+      link: https://huggingface.co/spaces/DataQuests/DeepCritical
+      name: HuggingFace Space
+    - icon: fontawesome/brands/discord
+      link: https://discord.gg/yourserver
+      name: Discord
+```
+
+**Benefits**: Better community engagement, more ways to connect.
+
+### 9. **Analytics Integration** ⭐ Medium Priority
+
+Add privacy-respecting analytics:
+
+```yaml
+extra:
+  analytics:
+    provider: google
+    property: G-XXXXXXXXXX
+  # Or use privacy-focused alternative:
+  # analytics:
+  #   provider: plausible
+  #   domain: yourdomain.com
+```
+
+**Benefits**: Understand how users interact with your documentation.
+
+### 10. **Custom CSS/JS for Branding** ⭐ Low Priority
+
+Add custom styling:
+
+```yaml
+extra_css:
+  - stylesheets/extra.css
+
+extra_javascript:
+  - javascripts/extra.js
+```
+
+**Benefits**: Customize appearance, add interactive features.
+
+### 11. **Better Table of Contents** ⭐ Medium Priority
+
+Enhance TOC with more options:
+
+```yaml
+markdown_extensions:
+  - toc:
+      permalink: true
+      permalink_title: "Anchor link to this section"
+      baselevel: 1
+      toc_depth: 3
+      slugify: !!python/object/apply:pymdownx.slugs.slugify
+        kwds:
+          case: lower
+```
+
+**Benefits**: Better navigation within long pages, SEO-friendly anchor links.
+
+### 12. **Image Optimization** ⭐ Medium Priority
+
+Add image handling plugin:
+
+```yaml
+plugins:
+  - search
+  - mermaid2
+  - codeinclude
+  - minify:
+      minify_html: true
+      minify_js: true
+      minify_css: true
+  - git-revision-date-localized:
+      enable_creation_date: true
+      type: timeago
+  # Optional: Image optimization
+  # - awesome-pages  # For better page organization
+```
+
+**Benefits**: Faster page loads, better mobile experience.
+
+### 13. **Keyboard Shortcuts** ⭐ Low Priority
+
+Enable keyboard navigation:
+
+```yaml
+theme:
+  keyboard_shortcuts:
+    search: true
+    previous: true
+    next: true
+```
+
+**Benefits**: Power users can navigate faster.
+
+### 14. **Print Styles** ⭐ Low Priority
+
+Better printing experience:
+
+```yaml
+theme:
+  features:
+    - navigation.tabs
+    - navigation.sections
+    - navigation.expand
+    - navigation.top
+    - navigation.indexes
+    - navigation.instant
+    - navigation.tracking
+    - navigation.smooth
+    - search.suggest
+    - search.highlight
+    - content.code.annotate
+    - content.code.copy
+    - content.tabs.link
+    - content.tooltips
+    - content.action.edit  # Edit button
+    - content.action.view  # View source
+```
+
+**Benefits**: Users can print documentation cleanly.
+
+### 15. **Better Search Configuration** ⭐ Medium Priority
+
+Enhance search capabilities:
+
+```yaml
+plugins:
+  - search:
+      lang:
+        - en
+      separator: '[\s\-,:!=\[\]()"`/]+|\.(?!\d)|&[lg]t;|&'
+      prebuild_index: true  # For faster search
+      indexing: full  # Full-text indexing
+```
+
+**Benefits**: Faster, more accurate search results.
+
+### 16. **API Documentation Enhancements** ⭐ High Priority (for your API docs)
+
+Since you have extensive API documentation, consider:
+
+```yaml
+markdown_extensions:
+  - pymdownx.superfences:
+      custom_fences:
+        - name: mermaid
+          class: mermaid
+          format: !!python/name:pymdownx.superfences.fence_code_format
+      preserve_tabs: true
+  # Add API-specific features
+  - attr_list
+  - md_in_html
+  - pymdownx.caret
+  - pymdownx.tilde
+```
+
+**Benefits**: Better formatting for API endpoints, parameters, responses.
+
+### 17. **Blog/News Section** ⭐ Low Priority (if needed)
+
+If you want to add a blog:
+
+```yaml
+plugins:
+  - blog:
+      blog_dir: blog
+      blog_description: "News and updates"
+      post_date_format: full
+      post_url_format: '{slug}'
+      archive: true
+```
+
+**Benefits**: Keep users updated with changelog, announcements.
+
+### 18. **Tags and Categories** ⭐ Low Priority
+
+Organize content with tags:
+
+```yaml
+markdown_extensions:
+  - meta
+```
+
+Then in frontmatter:
+```markdown
+---
+tags:
+  - api
+  - agents
+  - getting-started
+---
+```
+
+**Benefits**: Better content organization, related content discovery.
+
+### 19. **Better Mobile Experience** ⭐ High Priority
+
+Ensure mobile optimization:
+
+```yaml
+theme:
+  features:
+    - navigation.tabs
+    - navigation.sections
+    - navigation.expand
+    - navigation.top
+    - navigation.instant  # Helps on mobile
+    - navigation.tracking
+    - navigation.smooth
+    - search.suggest
+    - search.highlight
+    - content.code.annotate
+    - content.code.copy
+    - content.tabs.link
+    - content.tooltips
+    - toc.integrate  # Better mobile TOC
+```
+
+**Benefits**: Better experience for mobile users (growing segment).
+
+### 20. **Feedback Mechanism** ⭐ Medium Priority
+
+Add feedback buttons:
+
+```yaml
+extra:
+  feedback:
+    title: "Was this page helpful?"
+    ratings:
+      - icon: material/thumb-up-outline
+        name: "This page was helpful"
+      - icon: material/thumb-down-outline
+        name: "This page could be improved"
+```
+
+**Benefits**: Understand what content needs improvement.
+
+## Priority Recommendations
+
+### Immediate (High Impact, Easy Implementation)
+1. ✅ **Git revision dates** - Shows content freshness
+2. ✅ **Enhanced navigation features** - Better UX
+3. ✅ **Content tabs** - Perfect for code examples
+4. ✅ **Better search configuration** - Faster search
+
+### Short-term (High Impact, Medium Effort)
+5. ✅ **API documentation enhancements** - Better API docs
+6. ✅ **Enhanced admonitions** - Better visual hierarchy
+7. ✅ **Mobile optimization** - Better mobile experience
+8. ✅ **Analytics** - Understand user behavior
+
+### Long-term (Nice to Have)
+9. ⚠️ **Versioning** - If you need multiple versions
+10. ⚠️ **Math formulas** - If you have mathematical content
+11. ⚠️ **Blog section** - If you want to publish updates
+12. ⚠️ **Custom CSS/JS** - For advanced customization
+
+## Implementation Example
+
+Here's an enhanced `mkdocs.yml` with the high-priority improvements:
+
+```yaml
+site_name: The DETERMINATOR
+site_description: Generalist Deep Research Agent that Stops at Nothing
+site_author: The DETERMINATOR Team
+site_url: https://deepcritical.github.io/GradioDemo/
+
+repo_name: DeepCritical/GradioDemo
+repo_url: https://github.com/DeepCritical/GradioDemo
+edit_uri: edit/dev/docs/
+
+strict: false
+
+theme:
+  name: material
+  palette:
+    - scheme: default
+      primary: orange
+      accent: red
+      toggle:
+        icon: material/brightness-7
+        name: Switch to dark mode
+    - scheme: slate
+      primary: orange
+      accent: red
+      toggle:
+        icon: material/brightness-4
+        name: Switch to light mode
+  features:
+    - navigation.tabs
+    - navigation.sections
+    - navigation.expand
+    - navigation.top
+    - navigation.indexes
+    - navigation.instant
+    - navigation.tracking
+    - navigation.smooth
+    - search.suggest
+    - search.highlight
+    - content.code.annotate
+    - content.code.copy
+    - content.tabs.link
+    - content.tooltips
+    - toc.integrate
+  icon:
+    repo: fontawesome/brands/github
+  language: en
+
+plugins:
+  - search:
+      lang:
+        - en
+      separator: '[\s\-,:!=\[\]()"`/]+|\.(?!\d)|&[lg]t;|&'
+      prebuild_index: true
+      indexing: full
+  - mermaid2
+  - codeinclude
+  - git-revision-date-localized:
+      enable_creation_date: true
+      type: timeago
+      fallback_to_build_date: true
+  - minify:
+      minify_html: true
+      minify_js: true
+      minify_css: true
+
+markdown_extensions:
+  - dev.docs_plugins:
+      base_path: "."
+  - pymdownx.highlight:
+      anchor_linenums: true
+      line_spans: __span
+      pygments_lang_class: true
+      use_pygments: true
+      noclasses: false
+  - pymdownx.inlinehilite
+  - pymdownx.superfences:
+      custom_fences:
+        - name: mermaid
+          class: mermaid
+          format: !!python/name:pymdownx.superfences.fence_code_format
+      preserve_tabs: true
+  - pymdownx.tabbed:
+      alternate_style: true
+      combine_header_slug: true
+  - pymdownx.tasklist:
+      custom_checkbox: true
+  - pymdownx.emoji:
+      emoji_generator: !!python/name:pymdownx.emoji.to_svg
+      emoji_index: !!python/name:pymdownx.emoji.twemoji
+  - pymdownx.snippets
+  - admonition
+  - pymdownx.details
+  - attr_list
+  - md_in_html
+  - tables
+  - meta
+  - toc:
+      permalink: true
+      permalink_title: "Anchor link to this section"
+      baselevel: 1
+      toc_depth: 3
+      slugify: !!python/object/apply:pymdownx.slugs.slugify
+        kwds:
+          case: lower
+
+nav:
+  - Home: index.md
+  - Overview:
+    - overview/architecture.md
+    - overview/features.md
+  - Getting Started:
+    - getting-started/installation.md
+    - getting-started/quick-start.md
+    - getting-started/mcp-integration.md
+    - getting-started/examples.md
+  - Configuration:
+    - configuration/index.md
+  - Architecture:
+    - "Graph Orchestration": architecture/graph_orchestration.md
+    - "Workflow Diagrams": architecture/workflow-diagrams.md
+    - "Agents": architecture/agents.md
+    - "Orchestrators": architecture/orchestrators.md
+    - "Tools": architecture/tools.md
+    - "Middleware": architecture/middleware.md
+    - "Services": architecture/services.md
+  - API Reference:
+    - api/agents.md
+    - api/tools.md
+    - api/orchestrators.md
+    - api/services.md
+    - api/models.md
+  - Contributing:
+    - contributing/index.md
+    - contributing/code-quality.md
+    - contributing/code-style.md
+    - contributing/error-handling.md
+    - contributing/implementation-patterns.md
+    - contributing/prompt-engineering.md
+    - contributing/testing.md
+  - License: LICENSE.md
+  - Team: team.md
+
+extra:
+  social:
+    - icon: fontawesome/brands/github
+      link: https://github.com/DeepCritical/GradioDemo
+      name: GitHub
+    - icon: material/web
+      link: https://huggingface.co/spaces/DataQuests/DeepCritical
+      name: HuggingFace Space
+  version:
+    provider: mike
+  generator:
+    enabled: false
+
+copyright: Copyright © 2024 DeepCritical Team
+```
+
+## Additional Documentation Improvements
+
+### Content Structure
+1. **Add a changelog page** - Keep users informed of updates
+2. **Add a FAQ section** - Address common questions
+3. **Add a glossary** - Define technical terms
+4. **Add a troubleshooting guide** - Help users solve common issues
+5. **Add video tutorials** - Embed videos for complex topics
+
+### Visual Enhancements
+1. **Add diagrams** - Use more Mermaid diagrams for complex flows
+2. **Add screenshots** - Visual guides for UI features
+3. **Add code examples** - More practical examples
+4. **Add comparison tables** - Compare different approaches/options
+
+### SEO & Discoverability
+1. **Add meta descriptions** - Better search engine results
+2. **Add Open Graph tags** - Better social media sharing
+3. **Add sitemap** - Help search engines index your docs
+4. **Add robots.txt** - Control search engine crawling
+
+## Next Steps
+
+1. Review this assessment
+2. Prioritize features based on your needs
+3. Test changes in a branch
+4. Gather user feedback
+5. Iterate and improve
+
+## Resources
+
+- [MkDocs User Guide](https://www.mkdocs.org/user-guide/)
+- [Material for MkDocs Documentation](https://squidfunk.github.io/mkdocs-material/)
+- [Material for MkDocs Reference](https://squidfunk.github.io/mkdocs-material/reference/)
+- [MkDocs Plugins](https://github.com/mkdocs/mkdocs/wiki/MkDocs-Plugins)
+

docs/api/agents.md

@@ -12,27 +12,19 @@ This page documents the API for DeepCritical agents.
 
 #### `evaluate`
 
-```python
-async def evaluate(
-    self,
-    query: str,
-    background_context: str,
-    conversation_history: Conversation,
-    iteration: int,
-    time_elapsed_minutes: float,
-    max_time_minutes: float
-) -> KnowledgeGapOutput
-```
+<!--codeinclude-->
+[KnowledgeGapAgent.evaluate](../src/agents/knowledge_gap.py) start_line:66 end_line:74
+<!--/codeinclude-->
 
 Evaluates research completeness and identifies outstanding knowledge gaps.
 
 **Parameters**:
 - `query`: Research query string
-- `background_context`: Background context for the query
-- `conversation_history`: Conversation history with previous iterations
-- `iteration`: Current iteration number
-- `time_elapsed_minutes`: Elapsed time in minutes
-- `max_time_minutes`: Maximum time limit in minutes
+- `background_context`: Background context for the query (default: "")
+- `conversation_history`: History of actions, findings, and thoughts as string (default: "")
+- `iteration`: Current iteration number (default: 0)
+- `time_elapsed_minutes`: Elapsed time in minutes (default: 0.0)
+- `max_time_minutes`: Maximum time limit in minutes (default: 10)
 
 **Returns**: `KnowledgeGapOutput` with:
 - `research_complete`: Boolean indicating if research is complete
@@ -48,21 +40,17 @@ Evaluates research completeness and identifies outstanding knowledge gaps.
 
 #### `select_tools`
 
-```python
-async def select_tools(
-    self,
-    query: str,
-    knowledge_gaps: list[str],
-    available_tools: list[str]
-) -> AgentSelectionPlan
-```
+<!--codeinclude-->
+[ToolSelectorAgent.select_tools](../src/agents/tool_selector.py) start_line:78 end_line:84
+<!--/codeinclude-->
 
-Selects tools for addressing knowledge gaps.
+Selects tools for addressing a knowledge gap.
 
 **Parameters**:
+- `gap`: The knowledge gap to address
 - `query`: Research query string
-- `knowledge_gaps`: List of knowledge gaps to address
-- `available_tools`: List of available tool names
+- `background_context`: Optional background context (default: "")
+- `conversation_history`: History of actions, findings, and thoughts as string (default: "")
 
 **Returns**: `AgentSelectionPlan` with list of `AgentTask` objects.
 
@@ -76,23 +64,17 @@ Selects tools for addressing knowledge gaps.
 
 #### `write_report`
 
-```python
-async def write_report(
-    self,
-    query: str,
-    findings: str,
-    output_length: str = "medium",
-    output_instructions: str | None = None
-) -> str
-```
+<!--codeinclude-->
+[WriterAgent.write_report](../src/agents/writer.py) start_line:67 end_line:73
+<!--/codeinclude-->
 
 Generates a markdown report from research findings.
 
 **Parameters**:
 - `query`: Research query string
 - `findings`: Research findings to include in report
-- `output_length`: Desired output length ("short", "medium", "long")
-- `output_instructions`: Additional instructions for report generation
+- `output_length`: Optional description of desired output length (default: "")
+- `output_instructions`: Optional additional instructions for report generation (default: "")
 
 **Returns**: Markdown string with numbered citations.
 
@@ -106,36 +88,25 @@ Generates a markdown report from research findings.
 
 #### `write_next_section`
 
-```python
-async def write_next_section(
-    self,
-    query: str,
-    draft: ReportDraft,
-    section_title: str,
-    section_content: str
-) -> LongWriterOutput
-```
+<!--codeinclude-->
+[LongWriterAgent.write_next_section](../src/agents/long_writer.py) start_line:94 end_line:100
+<!--/codeinclude-->
 
 Writes the next section of a long-form report.
 
 **Parameters**:
-- `query`: Research query string
-- `draft`: Current report draft
-- `section_title`: Title of the section to write
-- `section_content`: Content/guidance for the section
+- `original_query`: The original research query
+- `report_draft`: Current report draft as string (all sections written so far)
+- `next_section_title`: Title of the section to write
+- `next_section_draft`: Draft content for the next section
 
-**Returns**: `LongWriterOutput` with updated draft.
+**Returns**: `LongWriterOutput` with formatted section and references.
 
 #### `write_report`
 
-```python
-async def write_report(
-    self,
-    query: str,
-    report_title: str,
-    report_draft: ReportDraft
-) -> str
-```
+<!--codeinclude-->
+[LongWriterAgent.write_report](../src/agents/long_writer.py) start_line:263 end_line:268
+<!--/codeinclude-->
 
 Generates final report from draft.
 
@@ -156,14 +127,9 @@ Generates final report from draft.
 
 #### `proofread`
 
-```python
-async def proofread(
-    self,
-    query: str,
-    report_title: str,
-    report_draft: ReportDraft
-) -> str
-```
+<!--codeinclude-->
+[ProofreaderAgent.proofread](../src/agents/proofreader.py) start_line:72 end_line:76
+<!--/codeinclude-->
 
 Proofreads and polishes a report draft.
 
@@ -184,21 +150,17 @@ Proofreads and polishes a report draft.
 
 #### `generate_observations`
 
-```python
-async def generate_observations(
-    self,
-    query: str,
-    background_context: str,
-    conversation_history: Conversation
-) -> str
-```
+<!--codeinclude-->
+[ThinkingAgent.generate_observations](../src/agents/thinking.py) start_line:70 end_line:76
+<!--/codeinclude-->
 
 Generates observations from conversation history.
 
 **Parameters**:
 - `query`: Research query string
-- `background_context`: Background context
-- `conversation_history`: Conversation history
+- `background_context`: Optional background context (default: "")
+- `conversation_history`: History of actions, findings, and thoughts as string (default: "")
+- `iteration`: Current iteration number (default: 1)
 
 **Returns**: Observation string.
 
@@ -210,14 +172,11 @@ Generates observations from conversation history.
 
 ### Methods
 
-#### `parse_query`
+#### `parse`
 
-```python
-async def parse_query(
-    self,
-    query: str
-) -> ParsedQuery
-```
+<!--codeinclude-->
+[InputParserAgent.parse](../src/agents/input_parser.py) start_line:82 end_line:82
+<!--/codeinclude-->
 
 Parses and improves a user query.
 
@@ -241,6 +200,7 @@ All agents have factory functions in `src.agent_factory.agents`:
 
 **Parameters**:
 - `model`: Optional Pydantic AI model. If None, uses `get_model()` from settings.
+- `oauth_token`: Optional OAuth token from HuggingFace login (takes priority over env vars)
 
 **Returns**: Agent instance.
 

docs/api/models.md

@@ -15,7 +15,7 @@ This page documents the Pydantic models used throughout DeepCritical.
 **Fields**:
 - `citation`: Citation information (title, URL, date, authors)
 - `content`: Evidence text content
-- `relevance_score`: Relevance score (0.0-1.0)
+- `relevance`: Relevance score (0.0-1.0)
 - `metadata`: Additional metadata dictionary
 
 ## Citation
@@ -29,9 +29,10 @@ This page documents the Pydantic models used throughout DeepCritical.
 <!--/codeinclude-->
 
 **Fields**:
+- `source`: Source name (e.g., "pubmed", "clinicaltrials", "europepmc", "web", "rag")
 - `title`: Article/trial title
 - `url`: Source URL
-- `date`: Publication date (optional)
+- `date`: Publication date (YYYY-MM-DD or "Unknown")
 - `authors`: List of authors (optional)
 
 ## KnowledgeGapOutput
@@ -72,9 +73,10 @@ This page documents the Pydantic models used throughout DeepCritical.
 <!--/codeinclude-->
 
 **Fields**:
-- `agent_name`: Name of agent to use
-- `query`: Task query
-- `context`: Additional context dictionary
+- `gap`: The knowledge gap being addressed (optional)
+- `agent`: Name of agent to use
+- `query`: The specific query for the agent
+- `entity_website`: The website of the entity being researched, if known (optional)
 
 ## ReportDraft
 
@@ -87,9 +89,7 @@ This page documents the Pydantic models used throughout DeepCritical.
 <!--/codeinclude-->
 
 **Fields**:
-- `title`: Report title
 - `sections`: List of report sections
-- `references`: List of citations
 
 ## ReportSection
 
@@ -102,9 +102,8 @@ This page documents the Pydantic models used throughout DeepCritical.
 <!--/codeinclude-->
 
 **Fields**:
-- `title`: Section title
-- `content`: Section content
-- `order`: Section order number
+- `section_title`: The title of the section
+- `section_content`: The content of the section
 
 ## ParsedQuery
 
@@ -134,7 +133,7 @@ This page documents the Pydantic models used throughout DeepCritical.
 <!--/codeinclude-->
 
 **Fields**:
-- `iterations`: List of iteration data
+- `history`: List of iteration data
 
 ## IterationData
 
@@ -147,12 +146,10 @@ This page documents the Pydantic models used throughout DeepCritical.
 <!--/codeinclude-->
 
 **Fields**:
-- `iteration`: Iteration number
-- `observations`: Generated observations
-- `knowledge_gaps`: Identified knowledge gaps
-- `tool_calls`: Tool calls made
-- `findings`: Findings from tools
-- `thoughts`: Agent thoughts
+- `gap`: The gap addressed in the iteration
+- `tool_calls`: The tool calls made
+- `findings`: The findings collected from tool calls
+- `thought`: The thinking done to reflect on the success of the iteration and next steps
 
 ## AgentEvent
 
@@ -180,12 +177,13 @@ This page documents the Pydantic models used throughout DeepCritical.
 <!--/codeinclude-->
 
 **Fields**:
-- `tokens_used`: Tokens used so far
-- `tokens_limit`: Token limit
-- `time_elapsed_seconds`: Elapsed time in seconds
-- `time_limit_seconds`: Time limit in seconds
-- `iterations`: Current iteration count
-- `iterations_limit`: Iteration limit
+- `tokens_used`: Total tokens used
+- `tokens_limit`: Token budget limit
+- `time_elapsed_seconds`: Time elapsed in seconds
+- `time_limit_seconds`: Time budget limit (default: 600.0 seconds / 10 minutes)
+- `iterations`: Number of iterations completed
+- `iterations_limit`: Maximum iterations (default: 10)
+- `iteration_tokens`: Tokens used per iteration (iteration number -> token count)
 
 ## See Also
 

docs/api/orchestrators.md

@@ -12,33 +12,21 @@ This page documents the API for DeepCritical orchestrators.
 
 #### `run`
 
-```python
-async def run(
-    self,
-    query: str,
-    background_context: str = "",
-    max_iterations: int | None = None,
-    max_time_minutes: float | None = None,
-    token_budget: int | None = None
-) -> AsyncGenerator[AgentEvent, None]
-```
+<!--codeinclude-->
+[IterativeResearchFlow.run](../src/orchestrator/research_flow.py) start_line:134 end_line:140
+<!--/codeinclude-->
 
 Runs iterative research flow.
 
 **Parameters**:
 - `query`: Research query string
 - `background_context`: Background context (default: "")
-- `max_iterations`: Maximum iterations (default: from settings)
-- `max_time_minutes`: Maximum time in minutes (default: from settings)
-- `token_budget`: Token budget (default: from settings)
-
-**Yields**: `AgentEvent` objects for:
-- `started`: Research started
-- `search_complete`: Search completed
-- `judge_complete`: Evidence evaluation completed
-- `synthesizing`: Generating report
-- `complete`: Research completed
-- `error`: Error occurred
+- `output_length`: Optional description of desired output length (default: "")
+- `output_instructions`: Optional additional instructions for report generation (default: "")
+
+**Returns**: Final report string.
+
+**Note**: `max_iterations`, `max_time_minutes`, and `token_budget` are constructor parameters, not `run()` parameters.
 
 ## DeepResearchFlow
 
@@ -50,33 +38,18 @@ Runs iterative research flow.
 
 #### `run`
 
-```python
-async def run(
-    self,
-    query: str,
-    background_context: str = "",
-    max_iterations_per_section: int | None = None,
-    max_time_minutes: float | None = None,
-    token_budget: int | None = None
-) -> AsyncGenerator[AgentEvent, None]
-```
+<!--codeinclude-->
+[DeepResearchFlow.run](../src/orchestrator/research_flow.py) start_line:778 end_line:778
+<!--/codeinclude-->
 
 Runs deep research flow.
 
 **Parameters**:
 - `query`: Research query string
-- `background_context`: Background context (default: "")
-- `max_iterations_per_section`: Maximum iterations per section (default: from settings)
-- `max_time_minutes`: Maximum time in minutes (default: from settings)
-- `token_budget`: Token budget (default: from settings)
-
-**Yields**: `AgentEvent` objects for:
-- `started`: Research started
-- `planning`: Creating research plan
-- `looping`: Running parallel research loops
-- `synthesizing`: Synthesizing results
-- `complete`: Research completed
-- `error`: Error occurred
+
+**Returns**: Final report string.
+
+**Note**: `max_iterations_per_section`, `max_time_minutes`, and `token_budget` are constructor parameters, not `run()` parameters.
 
 ## GraphOrchestrator
 
@@ -88,24 +61,19 @@ Runs deep research flow.
 
 #### `run`
 
-```python
-async def run(
-    self,
-    query: str,
-    research_mode: str = "auto",
-    use_graph: bool = True
-) -> AsyncGenerator[AgentEvent, None]
-```
+<!--codeinclude-->
+[GraphOrchestrator.run](../src/orchestrator/graph_orchestrator.py) start_line:177 end_line:177
+<!--/codeinclude-->
 
 Runs graph-based research orchestration.
 
 **Parameters**:
 - `query`: Research query string
-- `research_mode`: Research mode ("iterative", "deep", or "auto")
-- `use_graph`: Whether to use graph execution (default: True)
 
 **Yields**: `AgentEvent` objects during graph execution.
 
+**Note**: `research_mode` and `use_graph` are constructor parameters, not `run()` parameters.
+
 ## Orchestrator Factory
 
 **Module**: `src.orchestrator_factory`
@@ -116,22 +84,18 @@ Runs graph-based research orchestration.
 
 #### `create_orchestrator`
 
-```python
-def create_orchestrator(
-    search_handler: SearchHandlerProtocol,
-    judge_handler: JudgeHandlerProtocol,
-    config: dict[str, Any],
-    mode: str | None = None
-) -> Any
-```
+<!--codeinclude-->
+[create_orchestrator](../src/orchestrator_factory.py) start_line:44 end_line:50
+<!--/codeinclude-->
 
 Creates an orchestrator instance.
 
 **Parameters**:
-- `search_handler`: Search handler protocol implementation
-- `judge_handler`: Judge handler protocol implementation
-- `config`: Configuration dictionary
-- `mode`: Orchestrator mode ("simple", "advanced", "magentic", or None for auto-detect)
+- `search_handler`: Search handler protocol implementation (optional, required for simple mode)
+- `judge_handler`: Judge handler protocol implementation (optional, required for simple mode)
+- `config`: Configuration object (optional)
+- `mode`: Orchestrator mode ("simple", "advanced", "magentic", "iterative", "deep", "auto", or None for auto-detect)
+- `oauth_token`: Optional OAuth token from HuggingFace login (takes priority over env vars)
 
 **Returns**: Orchestrator instance.
 
@@ -153,24 +117,19 @@ Creates an orchestrator instance.
 
 #### `run`
 
-```python
-async def run(
-    self,
-    query: str,
-    max_rounds: int = 15,
-    max_stalls: int = 3
-) -> AsyncGenerator[AgentEvent, None]
-```
+<!--codeinclude-->
+[MagenticOrchestrator.run](../src/orchestrator_magentic.py) start_line:101 end_line:101
+<!--/codeinclude-->
 
 Runs Magentic orchestration.
 
 **Parameters**:
 - `query`: Research query string
-- `max_rounds`: Maximum rounds (default: 15)
-- `max_stalls`: Maximum stalls before reset (default: 3)
 
 **Yields**: `AgentEvent` objects converted from Magentic events.
 
+**Note**: `max_rounds` and `max_stalls` are constructor parameters, not `run()` parameters.
+
 **Requirements**:
 - `agent-framework-core` package
 - OpenAI API key

docs/api/services.md

@@ -12,9 +12,9 @@ This page documents the API for DeepCritical services.
 
 #### `embed`
 
-```python
-async def embed(self, text: str) -> list[float]
-```
+<!--codeinclude-->
+[EmbeddingService.embed](../src/services/embeddings.py) start_line:55 end_line:55
+<!--/codeinclude-->
 
 Generates embedding for a text string.
 
@@ -68,6 +68,60 @@ Finds duplicate texts based on similarity threshold.
 
 **Returns**: List of (index1, index2) tuples for duplicate pairs.
 
+#### `add_evidence`
+
+```python
+async def add_evidence(
+    self,
+    evidence_id: str,
+    content: str,
+    metadata: dict[str, Any]
+) -> None
+```
+
+Adds evidence to vector store for semantic search.
+
+**Parameters**:
+- `evidence_id`: Unique identifier for the evidence
+- `content`: Evidence text content
+- `metadata`: Additional metadata dictionary
+
+#### `search_similar`
+
+```python
+async def search_similar(
+    self,
+    query: str,
+    n_results: int = 5
+) -> list[dict[str, Any]]
+```
+
+Finds semantically similar evidence.
+
+**Parameters**:
+- `query`: Search query string
+- `n_results`: Number of results to return (default: 5)
+
+**Returns**: List of dictionaries with `id`, `content`, `metadata`, and `distance` keys.
+
+#### `deduplicate`
+
+```python
+async def deduplicate(
+    self,
+    new_evidence: list[Evidence],
+    threshold: float = 0.9
+) -> list[Evidence]
+```
+
+Removes semantically duplicate evidence.
+
+**Parameters**:
+- `new_evidence`: List of evidence items to deduplicate
+- `threshold`: Similarity threshold (default: 0.9, where 0.9 = 90% similar is duplicate)
+
+**Returns**: List of unique evidence items (not already in vector store).
+
 ### Factory Function
 
 #### `get_embedding_service`
@@ -89,63 +143,97 @@ Returns singleton EmbeddingService instance.
 
 #### `ingest_evidence`
 
-```python
-async def ingest_evidence(self, evidence: list[Evidence]) -> None
-```
+<!--codeinclude-->
+[LlamaIndexRAGService.ingest_evidence](../src/services/llamaindex_rag.py) start_line:290 end_line:290
+<!--/codeinclude-->
 
 Ingests evidence into RAG service.
 
 **Parameters**:
-- `evidence`: List of Evidence objects to ingest
+- `evidence_list`: List of Evidence objects to ingest
 
-**Note**: Requires OpenAI API key for embeddings.
+**Note**: Supports multiple embedding providers (OpenAI, local sentence-transformers, Hugging Face).
 
 #### `retrieve`
 
 ```python
-async def retrieve(
+def retrieve(
     self,
     query: str,
-    top_k: int = 5
-) -> list[Document]
+    top_k: int | None = None
+) -> list[dict[str, Any]]
 ```
 
 Retrieves relevant documents for a query.
 
 **Parameters**:
 - `query`: Search query string
-- `top_k`: Number of top results to return (default: 5)
+- `top_k`: Number of top results to return (defaults to `similarity_top_k` from constructor)
 
-**Returns**: List of Document objects with metadata.
+**Returns**: List of dictionaries with `text`, `score`, and `metadata` keys.
 
 #### `query`
 
 ```python
-async def query(
+def query(
     self,
-    query: str,
-    top_k: int = 5
+    query_str: str,
+    top_k: int | None = None
 ) -> str
 ```
 
-Queries RAG service and returns formatted results.
+Queries RAG service and returns synthesized response.
 
 **Parameters**:
-- `query`: Search query string
-- `top_k`: Number of top results to return (default: 5)
+- `query_str`: Query string
+- `top_k`: Number of results to use (defaults to `similarity_top_k` from constructor)
+
+**Returns**: Synthesized response string.
+
+**Raises**:
+- `ConfigurationError`: If no LLM API key is available for query synthesis
+
+#### `ingest_documents`
+
+```python
+def ingest_documents(self, documents: list[Any]) -> None
+```
 
-**Returns**: Formatted query results as string.
+Ingests raw LlamaIndex Documents.
+
+**Parameters**:
+- `documents`: List of LlamaIndex Document objects
+
+#### `clear_collection`
+
+```python
+def clear_collection(self) -> None
+```
+
+Clears all documents from the collection.
 
 ### Factory Function
 
 #### `get_rag_service`
 
 ```python
-@lru_cache(maxsize=1)
-def get_rag_service() -> LlamaIndexRAGService | None
+def get_rag_service(
+    collection_name: str = "deepcritical_evidence",
+    oauth_token: str | None = None,
+    **kwargs: Any
+) -> LlamaIndexRAGService
 ```
 
-Returns singleton LlamaIndexRAGService instance, or None if OpenAI key not available.
+Get or create a RAG service instance.
+
+**Parameters**:
+- `collection_name`: Name of the ChromaDB collection (default: "deepcritical_evidence")
+- `oauth_token`: Optional OAuth token from HuggingFace login (takes priority over env vars)
+- `**kwargs`: Additional arguments for LlamaIndexRAGService (e.g., `use_openai_embeddings=False`)
+
+**Returns**: Configured LlamaIndexRAGService instance.
+
+**Note**: By default, uses local embeddings (sentence-transformers) which require no API keys.
 
 ## StatisticalAnalyzer
 
@@ -160,24 +248,27 @@ Returns singleton LlamaIndexRAGService instance, or None if OpenAI key not avail
 ```python
 async def analyze(
     self,
-    hypothesis: str,
+    query: str,
     evidence: list[Evidence],
-    data_description: str | None = None
+    hypothesis: dict[str, Any] | None = None
 ) -> AnalysisResult
 ```
 
-Analyzes a hypothesis using statistical methods.
+Analyzes a research question using statistical methods.
 
 **Parameters**:
-- `hypothesis`: Hypothesis to analyze
-- `evidence`: List of Evidence objects
-- `data_description`: Optional data description
+- `query`: The research question
+- `evidence`: List of Evidence objects to analyze
+- `hypothesis`: Optional hypothesis dict with `drug`, `target`, `pathway`, `effect`, `confidence` keys
 
 **Returns**: `AnalysisResult` with:
 - `verdict`: SUPPORTED, REFUTED, or INCONCLUSIVE
-- `code`: Generated analysis code
-- `output`: Execution output
-- `error`: Error message if execution failed
+- `confidence`: Confidence in verdict (0.0-1.0)
+- `statistical_evidence`: Summary of statistical findings
+- `code_generated`: Python code that was executed
+- `execution_output`: Output from code execution
+- `key_takeaways`: Key takeaways from analysis
+- `limitations`: List of limitations
 
 **Note**: Requires Modal credentials for sandbox execution.
 

docs/api/tools.md

@@ -56,8 +56,10 @@ Searches PubMed for articles.
 **Returns**: List of `Evidence` objects with PubMed articles.
 
 **Raises**:
-- `SearchError`: If search fails
-- `RateLimitError`: If rate limit is exceeded
+- `SearchError`: If search fails (timeout, HTTP error, XML parsing error)
+- `RateLimitError`: If rate limit is exceeded (429 status code)
+
+**Note**: Uses NCBI E-utilities (ESearch → EFetch). Rate limit: 0.34s between requests. Handles single vs. multiple articles.
 
 ## ClinicalTrialsTool
 
@@ -96,10 +98,10 @@ Searches ClinicalTrials.gov for trials.
 
 **Returns**: List of `Evidence` objects with clinical trials.
 
-**Note**: Only returns interventional studies with status: COMPLETED, ACTIVE_NOT_RECRUITING, RECRUITING, ENROLLING_BY_INVITATION
+**Note**: Only returns interventional studies with status: COMPLETED, ACTIVE_NOT_RECRUITING, RECRUITING, ENROLLING_BY_INVITATION. Uses `requests` library (NOT httpx - WAF blocks httpx). Runs in thread pool for async compatibility.
 
 **Raises**:
-- `SearchError`: If search fails
+- `SearchError`: If search fails (HTTP error, request exception)
 
 ## EuropePMCTool
 
@@ -138,10 +140,10 @@ Searches Europe PMC for articles and preprints.
 
 **Returns**: List of `Evidence` objects with articles/preprints.
 
-**Note**: Includes both preprints (marked with `[PREPRINT - Not peer-reviewed]`) and peer-reviewed articles.
+**Note**: Includes both preprints (marked with `[PREPRINT - Not peer-reviewed]`) and peer-reviewed articles. Handles preprint markers. Builds URLs from DOI or PMID.
 
 **Raises**:
-- `SearchError`: If search fails
+- `SearchError`: If search fails (HTTP error, connection error)
 
 ## RAGTool
 
@@ -149,6 +151,20 @@ Searches Europe PMC for articles and preprints.
 
 **Purpose**: Semantic search within collected evidence.
 
+### Initialization
+
+```python
+def __init__(
+    self,
+    rag_service: LlamaIndexRAGService | None = None,
+    oauth_token: str | None = None
+) -> None
+```
+
+**Parameters**:
+- `rag_service`: Optional RAG service instance. If None, will be lazy-initialized.
+- `oauth_token`: Optional OAuth token from HuggingFace login (for RAG LLM)
+
 ### Properties
 
 #### `name`
@@ -180,7 +196,10 @@ Searches collected evidence using semantic similarity.
 
 **Returns**: List of `Evidence` objects from collected evidence.
 
-**Note**: Requires evidence to be ingested into RAG service first.
+**Raises**:
+- `ConfigurationError`: If RAG service is unavailable
+
+**Note**: Requires evidence to be ingested into RAG service first. Wraps `LlamaIndexRAGService`. Returns Evidence from RAG results.
 
 ## SearchHandler
 
@@ -188,32 +207,51 @@ Searches collected evidence using semantic similarity.
 
 **Purpose**: Orchestrates parallel searches across multiple tools.
 
-### Methods
-
-#### `search`
+### Initialization
 
 ```python
-async def search(
+def __init__(
     self,
-    query: str,
-    tools: list[SearchTool] | None = None,
-    max_results_per_tool: int = 10
-) -> SearchResult
+    tools: list[SearchTool],
+    timeout: float = 30.0,
+    include_rag: bool = False,
+    auto_ingest_to_rag: bool = True,
+    oauth_token: str | None = None
+) -> None
 ```
 
+**Parameters**:
+- `tools`: List of search tools to use
+- `timeout`: Timeout for each search in seconds (default: 30.0)
+- `include_rag`: Whether to include RAG tool in searches (default: False)
+- `auto_ingest_to_rag`: Whether to automatically ingest results into RAG (default: True)
+- `oauth_token`: Optional OAuth token from HuggingFace login (for RAG LLM)
+
+### Methods
+
+#### `execute`
+
+<!--codeinclude-->
+[SearchHandler.execute](../src/tools/search_handler.py) start_line:86 end_line:86
+<!--/codeinclude-->
+
 Searches multiple tools in parallel.
 
 **Parameters**:
 - `query`: Search query string
-- `tools`: List of tools to use (default: all available tools)
 - `max_results_per_tool`: Maximum results per tool (default: 10)
 
 **Returns**: `SearchResult` with:
+- `query`: The search query
 - `evidence`: Aggregated list of evidence
-- `tool_results`: Results per tool
-- `total_count`: Total number of results
+- `sources_searched`: List of source names searched
+- `total_found`: Total number of results
+- `errors`: List of error messages from failed tools
+
+**Raises**:
+- `SearchError`: If search times out
 
-**Note**: Uses `asyncio.gather()` for parallel execution. Handles tool failures gracefully.
+**Note**: Uses `asyncio.gather()` for parallel execution. Handles tool failures gracefully (returns errors in `SearchResult.errors`). Automatically ingests evidence into RAG if enabled.
 
 ## See Also
 

docs/architecture/agents.md

@@ -4,12 +4,16 @@ DeepCritical uses Pydantic AI agents for all AI-powered operations. All agents f
 
 ## Agent Pattern
 
-All agents use the Pydantic AI `Agent` class with the following structure:
+### Pydantic AI Agents
+
+Pydantic AI agents use the `Agent` class with the following structure:
 
 - **System Prompt**: Module-level constant with date injection
 - **Agent Class**: `__init__(model: Any | None = None)`
 - **Main Method**: Async method (e.g., `async def evaluate()`, `async def write_report()`)
-- **Factory Function**: `def create_agent_name(model: Any | None = None) -> AgentName`
+- **Factory Function**: `def create_agent_name(model: Any | None = None, oauth_token: str | None = None) -> AgentName`
+
+**Note**: Factory functions accept an optional `oauth_token` parameter for HuggingFace authentication, which takes priority over environment variables.
 
 ## Model Initialization
 
@@ -155,43 +159,135 @@ For text output (writer agents), agents return `str` directly.
 - `key_entities`: List of key entities
 - `research_questions`: List of research questions
 
-## Factory Functions
+## Magentic Agents
 
-All agents have factory functions in `src/agent_factory/agents.py`:
+The following agents use the `BaseAgent` pattern from `agent-framework` and are used exclusively with `MagenticOrchestrator`:
 
-<!--codeinclude-->
-[Factory Functions](../src/agent_factory/agents.py) start_line:77 end_line:97
-<!--/codeinclude-->
+### Hypothesis Agent
 
-Factory functions:
-- Use `get_model()` if no model provided
-- Raise `ConfigurationError` if creation fails
-- Log agent creation
+**File**: `src/agents/hypothesis_agent.py`
 
-## See Also
+**Purpose**: Generates mechanistic hypotheses based on evidence.
 
-- [Orchestrators](orchestrators.md) - How agents are orchestrated
-- [API Reference - Agents](../api/agents.md) - API documentation
-- [Contributing - Code Style](../contributing/code-style.md) - Development guidelines
-<<<<<<< HEAD
-=======
+**Pattern**: `BaseAgent` from `agent-framework`
+
+**Methods**:
+- `async def run(messages, thread, **kwargs) -> AgentRunResponse`
+
+**Features**:
+- Uses internal Pydantic AI `Agent` with `HypothesisAssessment` output type
+- Accesses shared `evidence_store` for evidence
+- Uses embedding service for diverse evidence selection (MMR algorithm)
+- Stores hypotheses in shared context
+
+### Search Agent
+
+**File**: `src/agents/search_agent.py`
+
+**Purpose**: Wraps `SearchHandler` as an agent for Magentic orchestrator.
+
+**Pattern**: `BaseAgent` from `agent-framework`
+
+**Methods**:
+- `async def run(messages, thread, **kwargs) -> AgentRunResponse`
+
+**Features**:
+- Executes searches via `SearchHandlerProtocol`
+- Deduplicates evidence using embedding service
+- Searches for semantically related evidence
+- Updates shared evidence store
+
+### Analysis Agent
+
+**File**: `src/agents/analysis_agent.py`
+
+**Purpose**: Performs statistical analysis using Modal sandbox.
 
+**Pattern**: `BaseAgent` from `agent-framework`
 
+**Methods**:
+- `async def run(messages, thread, **kwargs) -> AgentRunResponse`
+
+**Features**:
+- Wraps `StatisticalAnalyzer` service
+- Analyzes evidence and hypotheses
+- Returns verdict (SUPPORTED/REFUTED/INCONCLUSIVE)
+- Stores analysis results in shared context
+
+### Report Agent (Magentic)
 
+**File**: `src/agents/report_agent.py`
 
+**Purpose**: Generates structured scientific reports from evidence and hypotheses.
 
+**Pattern**: `BaseAgent` from `agent-framework`
 
+**Methods**:
+- `async def run(messages, thread, **kwargs) -> AgentRunResponse`
 
+**Features**:
+- Uses internal Pydantic AI `Agent` with `ResearchReport` output type
+- Accesses shared evidence store and hypotheses
+- Validates citations before returning
+- Formats report as markdown
 
+### Judge Agent
 
+**File**: `src/agents/judge_agent.py`
 
+**Purpose**: Evaluates evidence quality and determines if sufficient for synthesis.
 
+**Pattern**: `BaseAgent` from `agent-framework`
 
+**Methods**:
+- `async def run(messages, thread, **kwargs) -> AgentRunResponse`
+- `async def run_stream(messages, thread, **kwargs) -> AsyncIterable[AgentRunResponseUpdate]`
 
+**Features**:
+- Wraps `JudgeHandlerProtocol`
+- Accesses shared evidence store
+- Returns `JudgeAssessment` with sufficient flag, confidence, and recommendation
 
+## Agent Patterns
 
+DeepCritical uses two distinct agent patterns:
 
+### 1. Pydantic AI Agents (Traditional Pattern)
 
+These agents use the Pydantic AI `Agent` class directly and are used in iterative and deep research flows:
 
+- **Pattern**: `Agent(model, output_type, system_prompt)`
+- **Initialization**: `__init__(model: Any | None = None)`
+- **Methods**: Agent-specific async methods (e.g., `async def evaluate()`, `async def write_report()`)
+- **Examples**: `KnowledgeGapAgent`, `ToolSelectorAgent`, `WriterAgent`, `LongWriterAgent`, `ProofreaderAgent`, `ThinkingAgent`, `InputParserAgent`
 
->>>>>>> 8086ce5fefde1c867880661d57e1299029a91ead
+### 2. Magentic Agents (Agent-Framework Pattern)
+
+These agents use the `BaseAgent` class from `agent-framework` and are used in Magentic orchestrator:
+
+- **Pattern**: `BaseAgent` from `agent-framework` with `async def run()` method
+- **Initialization**: `__init__(evidence_store, embedding_service, ...)`
+- **Methods**: `async def run(messages, thread, **kwargs) -> AgentRunResponse`
+- **Examples**: `HypothesisAgent`, `SearchAgent`, `AnalysisAgent`, `ReportAgent`, `JudgeAgent`
+
+**Note**: Magentic agents are used exclusively with the `MagenticOrchestrator` and follow the agent-framework protocol for multi-agent coordination.
+
+## Factory Functions
+
+All agents have factory functions in `src/agent_factory/agents.py`:
+
+<!--codeinclude-->
+[Factory Functions](../src/agent_factory/agents.py) start_line:79 end_line:100
+<!--/codeinclude-->
+
+Factory functions:
+- Use `get_model()` if no model provided
+- Accept `oauth_token` parameter for HuggingFace authentication
+- Raise `ConfigurationError` if creation fails
+- Log agent creation
+
+## See Also
+
+- [Orchestrators](orchestrators.md) - How agents are orchestrated
+- [API Reference - Agents](../api/agents.md) - API documentation
+- [Contributing - Code Style](../contributing/code-style.md) - Development guidelines

docs/architecture/graph-orchestration.md

@@ -1,138 +0,0 @@
-# Graph Orchestration Architecture
-
-## Overview
-
-Phase 4 implements a graph-based orchestration system for research workflows using Pydantic AI agents as nodes. This enables better parallel execution, conditional routing, and state management compared to simple agent chains.
-
-## Graph Structure
-
-### Nodes
-
-Graph nodes represent different stages in the research workflow:
-
-1. **Agent Nodes**: Execute Pydantic AI agents
-   - Input: Prompt/query
-   - Output: Structured or unstructured response
-   - Examples: `KnowledgeGapAgent`, `ToolSelectorAgent`, `ThinkingAgent`
-
-2. **State Nodes**: Update or read workflow state
-   - Input: Current state
-   - Output: Updated state
-   - Examples: Update evidence, update conversation history
-
-3. **Decision Nodes**: Make routing decisions based on conditions
-   - Input: Current state/results
-   - Output: Next node ID
-   - Examples: Continue research vs. complete research
-
-4. **Parallel Nodes**: Execute multiple nodes concurrently
-   - Input: List of node IDs
-   - Output: Aggregated results
-   - Examples: Parallel iterative research loops
-
-### Edges
-
-Edges define transitions between nodes:
-
-1. **Sequential Edges**: Always traversed (no condition)
-   - From: Source node
-   - To: Target node
-   - Condition: None (always True)
-
-2. **Conditional Edges**: Traversed based on condition
-   - From: Source node
-   - To: Target node
-   - Condition: Callable that returns bool
-   - Example: If research complete → go to writer, else → continue loop
-
-3. **Parallel Edges**: Used for parallel execution branches
-   - From: Parallel node
-   - To: Multiple target nodes
-   - Execution: All targets run concurrently
-
-## Graph Patterns
-
-### Iterative Research Graph
-
-```
-[Input] → [Thinking] → [Knowledge Gap] → [Decision: Complete?]
-                                              ↓ No          ↓ Yes
-                                    [Tool Selector]    [Writer]
-                                              ↓
-                                    [Execute Tools] → [Loop Back]
-```
-
-### Deep Research Graph
-
-```
-[Input] → [Planner] → [Parallel Iterative Loops] → [Synthesizer]
-                           ↓         ↓         ↓
-                        [Loop1]  [Loop2]  [Loop3]
-```
-
-## State Management
-
-State is managed via `WorkflowState` using `ContextVar` for thread-safe isolation:
-
-- **Evidence**: Collected evidence from searches
-- **Conversation**: Iteration history (gaps, tool calls, findings, thoughts)
-- **Embedding Service**: For semantic search
-
-State transitions occur at state nodes, which update the global workflow state.
-
-## Execution Flow
-
-1. **Graph Construction**: Build graph from nodes and edges
-2. **Graph Validation**: Ensure graph is valid (no cycles, all nodes reachable)
-3. **Graph Execution**: Traverse graph from entry node
-4. **Node Execution**: Execute each node based on type
-5. **Edge Evaluation**: Determine next node(s) based on edges
-6. **Parallel Execution**: Use `asyncio.gather()` for parallel nodes
-7. **State Updates**: Update state at state nodes
-8. **Event Streaming**: Yield events during execution for UI
-
-## Conditional Routing
-
-Decision nodes evaluate conditions and return next node IDs:
-
-- **Knowledge Gap Decision**: If `research_complete` → writer, else → tool selector
-- **Budget Decision**: If budget exceeded → exit, else → continue
-- **Iteration Decision**: If max iterations → exit, else → continue
-
-## Parallel Execution
-
-Parallel nodes execute multiple nodes concurrently:
-
-- Each parallel branch runs independently
-- Results are aggregated after all branches complete
-- State is synchronized after parallel execution
-- Errors in one branch don't stop other branches
-
-## Budget Enforcement
-
-Budget constraints are enforced at decision nodes:
-
-- **Token Budget**: Track LLM token usage
-- **Time Budget**: Track elapsed time
-- **Iteration Budget**: Track iteration count
-
-If any budget is exceeded, execution routes to exit node.
-
-## Error Handling
-
-Errors are handled at multiple levels:
-
-1. **Node Level**: Catch errors in individual node execution
-2. **Graph Level**: Handle errors during graph traversal
-3. **State Level**: Rollback state changes on error
-
-Errors are logged and yield error events for UI.
-
-## Backward Compatibility
-
-Graph execution is optional via feature flag:
-
-- `USE_GRAPH_EXECUTION=true`: Use graph-based execution
-- `USE_GRAPH_EXECUTION=false`: Use agent chain execution (existing)
-
-This allows gradual migration and fallback if needed.

docs/architecture/graph_orchestration.md

@@ -1,9 +1,15 @@
 # Graph Orchestration Architecture
 
+## Overview
+
+DeepCritical implements a graph-based orchestration system for research workflows using Pydantic AI agents as nodes. This enables better parallel execution, conditional routing, and state management compared to simple agent chains.
+
 ## Graph Patterns
 
 ### Iterative Research Graph
 
+The iterative research graph follows this pattern:
+
 ```
 [Input] → [Thinking] → [Knowledge Gap] → [Decision: Complete?]
                                               ↓ No          ↓ Yes
@@ -12,14 +18,31 @@
                                     [Execute Tools] → [Loop Back]
 ```
 
+**Node IDs**: `thinking` → `knowledge_gap` → `continue_decision` → `tool_selector`/`writer` → `execute_tools` → (loop back to `thinking`)
+
+**Special Node Handling**:
+- `execute_tools`: State node that uses `search_handler` to execute searches and add evidence to workflow state
+- `continue_decision`: Decision node that routes based on `research_complete` flag from `KnowledgeGapOutput`
+
 ### Deep Research Graph
 
+The deep research graph follows this pattern:
+
 ```
-[Input] → [Planner] → [Parallel Iterative Loops] → [Synthesizer]
-                           ↓         ↓         ↓
-                        [Loop1]  [Loop2]  [Loop3]
+[Input] → [Planner] → [Store Plan] → [Parallel Loops] → [Collect Drafts] → [Synthesizer]
+                                        ↓         ↓         ↓
+                                     [Loop1]  [Loop2]  [Loop3]
 ```
 
+**Node IDs**: `planner` → `store_plan` → `parallel_loops` → `collect_drafts` → `synthesizer`
+
+**Special Node Handling**:
+- `planner`: Agent node that creates `ReportPlan` with report outline
+- `store_plan`: State node that stores `ReportPlan` in context for parallel loops
+- `parallel_loops`: Parallel node that executes `IterativeResearchFlow` instances for each section
+- `collect_drafts`: State node that collects section drafts from parallel loops
+- `synthesizer`: Agent node that calls `LongWriterAgent.write_report()` directly with `ReportDraft`
+
 ### Deep Research
 
 ```mermaid
@@ -158,14 +181,35 @@ State transitions occur at state nodes, which update the global workflow state.
 
 ## Execution Flow
 
-1. **Graph Construction**: Build graph from nodes and edges
-2. **Graph Validation**: Ensure graph is valid (no cycles, all nodes reachable)
-3. **Graph Execution**: Traverse graph from entry node
-4. **Node Execution**: Execute each node based on type
-5. **Edge Evaluation**: Determine next node(s) based on edges
+1. **Graph Construction**: Build graph from nodes and edges using `create_iterative_graph()` or `create_deep_graph()`
+2. **Graph Validation**: Ensure graph is valid (no cycles, all nodes reachable) via `ResearchGraph.validate_structure()`
+3. **Graph Execution**: Traverse graph from entry node using `GraphOrchestrator._execute_graph()`
+4. **Node Execution**: Execute each node based on type:
+   - **Agent Nodes**: Call `agent.run()` with transformed input
+   - **State Nodes**: Update workflow state via `state_updater` function
+   - **Decision Nodes**: Evaluate `decision_function` to get next node ID
+   - **Parallel Nodes**: Execute all parallel nodes concurrently via `asyncio.gather()`
+5. **Edge Evaluation**: Determine next node(s) based on edges and conditions
 6. **Parallel Execution**: Use `asyncio.gather()` for parallel nodes
-7. **State Updates**: Update state at state nodes
-8. **Event Streaming**: Yield events during execution for UI
+7. **State Updates**: Update state at state nodes via `GraphExecutionContext.update_state()`
+8. **Event Streaming**: Yield `AgentEvent` objects during execution for UI
+
+### GraphExecutionContext
+
+The `GraphExecutionContext` class manages execution state during graph traversal:
+
+- **State**: Current `WorkflowState` instance
+- **Budget Tracker**: `BudgetTracker` instance for budget enforcement
+- **Node Results**: Dictionary storing results from each node execution
+- **Visited Nodes**: Set of node IDs that have been executed
+- **Current Node**: ID of the node currently being executed
+
+Methods:
+- `set_node_result(node_id, result)`: Store result from node execution
+- `get_node_result(node_id)`: Retrieve stored result
+- `has_visited(node_id)`: Check if node was visited
+- `mark_visited(node_id)`: Mark node as visited
+- `update_state(updater, data)`: Update workflow state
 
 ## Conditional Routing
 

docs/architecture/middleware.md

@@ -18,8 +18,8 @@ DeepCritical uses middleware for state management, budget tracking, and workflow
 - `embedding_service: Any`: Embedding service for semantic search
 
 **Methods**:
-- `add_evidence(evidence: Evidence)`: Adds evidence with URL-based deduplication
-- `async search_related(query: str, top_k: int = 5) -> list[Evidence]`: Semantic search
+- `add_evidence(new_evidence: list[Evidence]) -> int`: Adds evidence with URL-based deduplication. Returns the number of new items added (excluding duplicates).
+- `async search_related(query: str, n_results: int = 5) -> list[Evidence]`: Semantic search for related evidence using embedding service
 
 **Initialization**:
 
@@ -30,7 +30,7 @@ DeepCritical uses middleware for state management, budget tracking, and workflow
 **Access**:
 
 <!--codeinclude-->
-[Get Workflow State](../src/middleware/state_machine.py) start_line:112 end_line:125
+[Get Workflow State](../src/middleware/state_machine.py) start_line:115 end_line:129
 <!--/codeinclude-->
 
 ## Workflow Manager
@@ -40,10 +40,10 @@ DeepCritical uses middleware for state management, budget tracking, and workflow
 **Purpose**: Coordinates parallel research loops
 
 **Methods**:
-- `add_loop(loop: ResearchLoop)`: Add a research loop to manage
-- `async run_loops_parallel() -> list[ResearchLoop]`: Run all loops in parallel
-- `update_loop_status(loop_id: str, status: str)`: Update loop status
-- `sync_loop_evidence_to_state()`: Synchronize evidence from loops to global state
+- `async add_loop(loop_id: str, query: str) -> ResearchLoop`: Add a new research loop to manage
+- `async run_loops_parallel(loop_configs: list[dict], loop_func: Callable, judge_handler: Any | None = None, budget_tracker: Any | None = None) -> list[Any]`: Run multiple research loops in parallel. Takes configuration dicts and a loop function.
+- `async update_loop_status(loop_id: str, status: LoopStatus, error: str | None = None)`: Update loop status
+- `async sync_loop_evidence_to_state(loop_id: str)`: Synchronize evidence from a specific loop to global state
 
 **Features**:
 - Uses `asyncio.gather()` for parallel execution
@@ -56,9 +56,22 @@ DeepCritical uses middleware for state management, budget tracking, and workflow
 from src.middleware.workflow_manager import WorkflowManager
 
 manager = WorkflowManager()
-manager.add_loop(loop1)
-manager.add_loop(loop2)
-completed_loops = await manager.run_loops_parallel()
+await manager.add_loop("loop1", "Research query 1")
+await manager.add_loop("loop2", "Research query 2")
+
+async def run_research(config: dict) -> str:
+    loop_id = config["loop_id"]
+    query = config["query"]
+    # ... research logic ...
+    return "report"
+
+results = await manager.run_loops_parallel(
+    loop_configs=[
+        {"loop_id": "loop1", "query": "Research query 1"},
+        {"loop_id": "loop2", "query": "Research query 2"},
+    ],
+    loop_func=run_research,
+)
 ```
 
 ## Budget Tracker
@@ -73,13 +86,13 @@ completed_loops = await manager.run_loops_parallel()
 - **Iterations**: Number of iterations
 
 **Methods**:
-- `create_budget(token_limit, time_limit_seconds, iterations_limit) -> BudgetStatus`
-- `add_tokens(tokens: int)`: Add token usage
-- `start_timer()`: Start time tracking
-- `update_timer()`: Update elapsed time
-- `increment_iteration()`: Increment iteration count
-- `check_budget() -> BudgetStatus`: Check current budget status
-- `can_continue() -> bool`: Check if research can continue
+- `create_budget(loop_id: str, tokens_limit: int = 100000, time_limit_seconds: float = 600.0, iterations_limit: int = 10) -> BudgetStatus`: Create a budget for a specific loop
+- `add_tokens(loop_id: str, tokens: int)`: Add token usage to a loop's budget
+- `start_timer(loop_id: str)`: Start time tracking for a loop
+- `update_timer(loop_id: str)`: Update elapsed time for a loop
+- `increment_iteration(loop_id: str)`: Increment iteration count for a loop
+- `check_budget(loop_id: str) -> tuple[bool, str]`: Check if a loop's budget has been exceeded. Returns (exceeded: bool, reason: str)
+- `can_continue(loop_id: str) -> bool`: Check if a loop can continue based on budget
 
 **Token Estimation**:
 - `estimate_tokens(text: str) -> int`: ~4 chars per token
@@ -91,13 +104,20 @@ from src.middleware.budget_tracker import BudgetTracker
 
 tracker = BudgetTracker()
 budget = tracker.create_budget(
-    token_limit=100000,
+    loop_id="research_loop",
+    tokens_limit=100000,
     time_limit_seconds=600,
     iterations_limit=10
 )
-tracker.start_timer()
+tracker.start_timer("research_loop")
 # ... research operations ...
-if not tracker.can_continue():
+tracker.add_tokens("research_loop", 5000)
+tracker.update_timer("research_loop")
+exceeded, reason = tracker.check_budget("research_loop")
+if exceeded:
+    # Budget exceeded, stop research
+    pass
+if not tracker.can_continue("research_loop"):
     # Budget exceeded, stop research
     pass
 ```

docs/architecture/orchestrators.md

@@ -25,7 +25,7 @@ DeepCritical supports multiple orchestration patterns for research workflows.
 **Usage**:
 
 <!--codeinclude-->
-[IterativeResearchFlow Initialization](../src/orchestrator/research_flow.py) start_line:56 end_line:77
+[IterativeResearchFlow Initialization](../src/orchestrator/research_flow.py) start_line:57 end_line:80
 <!--/codeinclude-->
 
 ### DeepResearchFlow
@@ -48,7 +48,7 @@ DeepCritical supports multiple orchestration patterns for research workflows.
 **Usage**:
 
 <!--codeinclude-->
-[DeepResearchFlow Initialization](../src/orchestrator/research_flow.py) start_line:674 end_line:697
+[DeepResearchFlow Initialization](../src/orchestrator/research_flow.py) start_line:709 end_line:728
 <!--/codeinclude-->
 
 ## Graph Orchestrator
@@ -58,9 +58,10 @@ DeepCritical supports multiple orchestration patterns for research workflows.
 **Purpose**: Graph-based execution using Pydantic AI agents as nodes
 
 **Features**:
-- Uses Pydantic AI Graphs (when available) or agent chains (fallback)
+- Uses graph execution (`use_graph=True`) or agent chains (`use_graph=False`) as fallback
 - Routes based on research mode (iterative/deep/auto)
 - Streams `AgentEvent` objects for UI
+- Uses `GraphExecutionContext` to manage execution state
 
 **Node Types**:
 - **Agent Nodes**: Execute Pydantic AI agents
@@ -73,6 +74,22 @@ DeepCritical supports multiple orchestration patterns for research workflows.
 - **Conditional Edges**: Traversed based on condition
 - **Parallel Edges**: Used for parallel execution branches
 
+**Special Node Handling**:
+
+The `GraphOrchestrator` has special handling for certain nodes:
+
+- **`execute_tools` node**: State node that uses `search_handler` to execute searches and add evidence to workflow state
+- **`parallel_loops` node**: Parallel node that executes `IterativeResearchFlow` instances for each section in deep research mode
+- **`synthesizer` node**: Agent node that calls `LongWriterAgent.write_report()` directly with `ReportDraft` instead of using `agent.run()`
+- **`writer` node**: Agent node that calls `WriterAgent.write_report()` directly with findings instead of using `agent.run()`
+
+**GraphExecutionContext**:
+
+The orchestrator uses `GraphExecutionContext` to manage execution state:
+- Tracks current node, visited nodes, and node results
+- Manages workflow state and budget tracker
+- Provides methods to store and retrieve node execution results
+
 ## Orchestrator Factory
 
 **File**: `src/orchestrator_factory.py`
@@ -99,14 +116,26 @@ DeepCritical supports multiple orchestration patterns for research workflows.
 **Features**:
 - Uses `agent-framework-core`
 - ChatAgent pattern with internal LLMs per agent
-- `MagenticBuilder` with participants: searcher, hypothesizer, judge, reporter
-- Manager orchestrates agents via `OpenAIChatClient`
-- Requires OpenAI API key (function calling support)
-- Event-driven: converts Magentic events to `AgentEvent` for UI streaming
+- `MagenticBuilder` with participants:
+  - `searcher`: SearchAgent (wraps SearchHandler)
+  - `hypothesizer`: HypothesisAgent (generates hypotheses)
+  - `judge`: JudgeAgent (evaluates evidence)
+  - `reporter`: ReportAgent (generates final report)
+- Manager orchestrates agents via chat client (OpenAI or HuggingFace)
+- Event-driven: converts Magentic events to `AgentEvent` for UI streaming via `_process_event()` method
+- Supports max rounds, stall detection, and reset handling
+
+**Event Processing**:
+
+The orchestrator processes Magentic events and converts them to `AgentEvent`:
+- `MagenticOrchestratorMessageEvent` → `AgentEvent` with type based on message content
+- `MagenticAgentMessageEvent` → `AgentEvent` with type based on agent name
+- `MagenticAgentDeltaEvent` → `AgentEvent` for streaming updates
+- `MagenticFinalResultEvent` → `AgentEvent` with type "complete"
 
 **Requirements**:
 - `agent-framework-core` package
-- OpenAI API key
+- OpenAI API key or HuggingFace authentication
 
 ## Hierarchical Orchestrator
 
@@ -136,7 +165,7 @@ DeepCritical supports multiple orchestration patterns for research workflows.
 All orchestrators must initialize workflow state:
 
 <!--codeinclude-->
-[Initialize Workflow State](../src/middleware/state_machine.py) start_line:98 end_line:111
+[Initialize Workflow State](../src/middleware/state_machine.py) start_line:98 end_line:112
 <!--/codeinclude-->
 
 ## Event Streaming
@@ -145,17 +174,23 @@ All orchestrators yield `AgentEvent` objects:
 
 **Event Types**:
 - `started`: Research started
+- `searching`: Search in progress
 - `search_complete`: Search completed
+- `judging`: Evidence evaluation in progress
 - `judge_complete`: Evidence evaluation completed
+- `looping`: Iteration in progress
 - `hypothesizing`: Generating hypotheses
+- `analyzing`: Statistical analysis in progress
+- `analysis_complete`: Statistical analysis completed
 - `synthesizing`: Synthesizing results
 - `complete`: Research completed
 - `error`: Error occurred
+- `streaming`: Streaming update (delta events)
 
 **Event Structure**:
 
 <!--codeinclude-->
-[AgentEvent Model](../src/utils/models.py) start_line:104 end_line:125
+[AgentEvent Model](../src/utils/models.py) start_line:104 end_line:126
 <!--/codeinclude-->
 
 ## See Also

docs/architecture/services.md

@@ -10,17 +10,18 @@ DeepCritical provides several services for embeddings, RAG, and statistical anal
 
 **Features**:
 - **No API Key Required**: Uses local sentence-transformers models
-- **Async-Safe**: All operations use `run_in_executor()` to avoid blocking
-- **ChromaDB Storage**: Vector storage for embeddings
-- **Deduplication**: 0.85 similarity threshold (85% similarity = duplicate)
+- **Async-Safe**: All operations use `run_in_executor()` to avoid blocking the event loop
+- **ChromaDB Storage**: In-memory vector storage for embeddings
+- **Deduplication**: 0.9 similarity threshold by default (90% similarity = duplicate, configurable)
 
 **Model**: Configurable via `settings.local_embedding_model` (default: `all-MiniLM-L6-v2`)
 
 **Methods**:
-- `async def embed(text: str) -> list[float]`: Generate embeddings
-- `async def embed_batch(texts: list[str]) -> list[list[float]]`: Batch embedding
-- `async def similarity(text1: str, text2: str) -> float`: Calculate similarity
-- `async def find_duplicates(texts: list[str], threshold: float = 0.85) -> list[tuple[int, int]]`: Find duplicates
+- `async def embed(text: str) -> list[float]`: Generate embeddings (async-safe via `run_in_executor()`)
+- `async def embed_batch(texts: list[str]) -> list[list[float]]`: Batch embedding (more efficient)
+- `async def add_evidence(evidence_id: str, content: str, metadata: dict[str, Any]) -> None`: Add evidence to vector store
+- `async def search_similar(query: str, n_results: int = 5) -> list[dict[str, Any]]`: Find semantically similar evidence
+- `async def deduplicate(new_evidence: list[Evidence], threshold: float = 0.9) -> list[Evidence]`: Remove semantically duplicate evidence
 
 **Usage**:
 ```python
@@ -32,15 +33,21 @@ embedding = await service.embed("text to embed")
 
 ## LlamaIndex RAG Service
 
-**File**: `src/services/rag.py`
+**File**: `src/services/llamaindex_rag.py`
 
 **Purpose**: Retrieval-Augmented Generation using LlamaIndex
 
 **Features**:
-- **OpenAI Embeddings**: Requires `OPENAI_API_KEY`
-- **ChromaDB Storage**: Vector database for document storage
+- **Multiple Embedding Providers**: OpenAI embeddings (requires `OPENAI_API_KEY`) or local sentence-transformers (no API key)
+- **Multiple LLM Providers**: HuggingFace LLM (preferred) or OpenAI LLM (fallback) for query synthesis
+- **ChromaDB Storage**: Vector database for document storage (supports in-memory mode)
 - **Metadata Preservation**: Preserves source, title, URL, date, authors
-- **Lazy Initialization**: Graceful fallback if OpenAI key not available
+- **Lazy Initialization**: Graceful fallback if dependencies not available
+
+**Initialization Parameters**:
+- `use_openai_embeddings: bool | None`: Force OpenAI embeddings (None = auto-detect)
+- `use_in_memory: bool`: Use in-memory ChromaDB client (useful for tests)
+- `oauth_token: str | None`: Optional OAuth token from HuggingFace login (takes priority over env vars)
 
 **Methods**:
 - `async def ingest_evidence(evidence: list[Evidence]) -> None`: Ingest evidence into RAG
@@ -49,9 +56,13 @@ embedding = await service.embed("text to embed")
 
 **Usage**:
 ```python
-from src.services.rag import get_rag_service
+from src.services.llamaindex_rag import get_rag_service
 
-service = get_rag_service()
+service = get_rag_service(
+    use_openai_embeddings=False,  # Use local embeddings
+    use_in_memory=True,  # Use in-memory ChromaDB
+    oauth_token=token  # Optional HuggingFace token
+)
 if service:
     documents = await service.retrieve("query", top_k=5)
 ```
@@ -92,13 +103,19 @@ result = await analyzer.analyze(
 
 ## Singleton Pattern
 
-All services use the singleton pattern with `@lru_cache(maxsize=1)`:
+Services use singleton patterns for lazy initialization:
 
-```python
-@lru_cache(maxsize=1)
-def get_embedding_service() -> EmbeddingService:
-    return EmbeddingService()
-```
+**EmbeddingService**: Uses a global variable pattern:
+
+<!--codeinclude-->
+[EmbeddingService Singleton](../src/services/embeddings.py) start_line:164 end_line:172
+<!--/codeinclude-->
+
+**LlamaIndexRAGService**: Direct instantiation (no caching):
+
+<!--codeinclude-->
+[LlamaIndexRAGService Factory](../src/services/llamaindex_rag.py) start_line:440 end_line:466
+<!--/codeinclude-->
 
 This ensures:
 - Single instance per process

docs/architecture/tools.md

@@ -14,14 +14,9 @@ All tools implement the `SearchTool` protocol from `src/tools/base.py`:
 
 All tools use the `@retry` decorator from tenacity:
 
-```python
-@retry(
-    stop=stop_after_attempt(3), 
-    wait=wait_exponential(...)
-)
-async def search(self, query: str, max_results: int = 10) -> list[Evidence]:
-    # Implementation
-```
+<!--codeinclude-->
+[Retry Decorator Pattern](../src/tools/pubmed.py) start_line:46 end_line:50
+<!--/codeinclude-->
 
 Tools with API rate limits implement `_rate_limit()` method and use shared rate limiters from `src/tools/rate_limiter.py`.
 
@@ -122,11 +117,23 @@ Missing fields are handled gracefully with defaults.
 
 **Purpose**: Orchestrates parallel searches across multiple tools
 
+**Initialization Parameters**:
+- `tools: list[SearchTool]`: List of search tools to use
+- `timeout: float = 30.0`: Timeout for each search in seconds
+- `include_rag: bool = False`: Whether to include RAG tool in searches
+- `auto_ingest_to_rag: bool = True`: Whether to automatically ingest results into RAG
+- `oauth_token: str | None = None`: Optional OAuth token from HuggingFace login (for RAG LLM)
+
+**Methods**:
+- `async def execute(query: str, max_results_per_tool: int = 10) -> SearchResult`: Execute search across all tools in parallel
+
 **Features**:
-- Uses `asyncio.gather()` with `return_exceptions=True`
-- Aggregates results into `SearchResult`
-- Handles tool failures gracefully
+- Uses `asyncio.gather()` with `return_exceptions=True` for parallel execution
+- Aggregates results into `SearchResult` with evidence and metadata
+- Handles tool failures gracefully (continues with other tools)
 - Deduplicates results by URL
+- Automatically ingests results into RAG if `auto_ingest_to_rag=True`
+- Can add RAG tool dynamically via `add_rag_tool()` method
 
 ## Tool Registration
 
@@ -136,14 +143,21 @@ Tools are registered in the search handler:
 from src.tools.pubmed import PubMedTool
 from src.tools.clinicaltrials import ClinicalTrialsTool
 from src.tools.europepmc import EuropePMCTool
+from src.tools.search_handler import SearchHandler
 
 search_handler = SearchHandler(
     tools=[
         PubMedTool(),
         ClinicalTrialsTool(),
         EuropePMCTool(),
-    ]
+    ],
+    include_rag=True,  # Include RAG tool for semantic search
+    auto_ingest_to_rag=True,  # Automatically ingest results into RAG
+    oauth_token=token  # Optional HuggingFace token for RAG LLM
 )
+
+# Execute search
+result = await search_handler.execute("query", max_results_per_tool=10)
 ```
 
 ## See Also

docs/architecture/workflow-diagrams.md

@@ -627,23 +627,10 @@ gantt
 ## Implementation Highlights
 
 **Simple 4-Agent Setup:**
-```python
-workflow = (
-    MagenticBuilder()
-    .participants(
-        hypothesis=HypothesisAgent(tools=[background_tool]),
-        search=SearchAgent(tools=[web_search, rag_tool]),
-        analysis=AnalysisAgent(tools=[code_execution]),
-        report=ReportAgent(tools=[code_execution, visualization])
-    )
-    .with_standard_manager(
-        chat_client=AnthropicClient(model="claude-sonnet-4"),
-        max_round_count=15,    # Prevent infinite loops
-        max_stall_count=3      # Detect stuck workflows
-    )
-    .build()
-)
-```
+
+<!--codeinclude-->
+[Magentic Workflow Builder](../src/orchestrator_magentic.py) start_line:72 end_line:99
+<!--/codeinclude-->
 
 **Manager handles quality assessment in its instructions:**
 - Checks hypothesis quality (testable, novel, clear)

docs/configuration/CONFIGURATION.md

@@ -1,557 +0,0 @@
-# Configuration Guide
-
-## Overview
-
-DeepCritical uses **Pydantic Settings** for centralized configuration management. All settings are defined in the `Settings` class in `src/utils/config.py` and can be configured via environment variables or a `.env` file.
-
-The configuration system provides:
-
-- **Type Safety**: Strongly-typed fields with Pydantic validation
-- **Environment File Support**: Automatically loads from `.env` file (if present)
-- **Case-Insensitive**: Environment variables are case-insensitive
-- **Singleton Pattern**: Global `settings` instance for easy access throughout the codebase
-- **Validation**: Automatic validation on load with helpful error messages
-
-## Quick Start
-
-1. Create a `.env` file in the project root
-2. Set at least one LLM API key (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `HF_TOKEN`)
-3. Optionally configure other services as needed
-4. The application will automatically load and validate your configuration
-
-## Configuration System Architecture
-
-### Settings Class
-
-The `Settings` class extends `BaseSettings` from `pydantic_settings` and defines all application configuration:
-
-<!--codeinclude-->
-[Settings Class Definition](../src/utils/config.py) start_line:13 end_line:21
-<!--/codeinclude-->
-
-### Singleton Instance
-
-A global `settings` instance is available for import:
-
-<!--codeinclude-->
-[Singleton Instance](../src/utils/config.py) start_line:234 end_line:235
-<!--/codeinclude-->
-
-### Usage Pattern
-
-Access configuration throughout the codebase:
-
-```python
-from src.utils.config import settings
-
-# Check if API keys are available
-if settings.has_openai_key:
-    # Use OpenAI
-    pass
-
-# Access configuration values
-max_iterations = settings.max_iterations
-web_search_provider = settings.web_search_provider
-```
-
-## Required Configuration
-
-### LLM Provider
-
-You must configure at least one LLM provider. The system supports:
-
-- **OpenAI**: Requires `OPENAI_API_KEY`
-- **Anthropic**: Requires `ANTHROPIC_API_KEY`
-- **HuggingFace**: Optional `HF_TOKEN` or `HUGGINGFACE_API_KEY` (can work without key for public models)
-
-#### OpenAI Configuration
-
-```bash
-LLM_PROVIDER=openai
-OPENAI_API_KEY=your_openai_api_key_here
-OPENAI_MODEL=gpt-5.1
-```
-
-The default model is defined in the `Settings` class:
-
-<!--codeinclude-->
-[OpenAI Model Configuration](../src/utils/config.py) start_line:29 end_line:29
-<!--/codeinclude-->
-
-#### Anthropic Configuration
-
-```bash
-LLM_PROVIDER=anthropic
-ANTHROPIC_API_KEY=your_anthropic_api_key_here
-ANTHROPIC_MODEL=claude-sonnet-4-5-20250929
-```
-
-The default model is defined in the `Settings` class:
-
-<!--codeinclude-->
-[Anthropic Model Configuration](../src/utils/config.py) start_line:30 end_line:32
-<!--/codeinclude-->
-
-#### HuggingFace Configuration
-
-HuggingFace can work without an API key for public models, but an API key provides higher rate limits:
-
-```bash
-# Option 1: Using HF_TOKEN (preferred)
-HF_TOKEN=your_huggingface_token_here
-
-# Option 2: Using HUGGINGFACE_API_KEY (alternative)
-HUGGINGFACE_API_KEY=your_huggingface_api_key_here
-
-# Default model
-HUGGINGFACE_MODEL=meta-llama/Llama-3.1-8B-Instruct
-```
-
-The HuggingFace token can be set via either environment variable:
-
-<!--codeinclude-->
-[HuggingFace Token Configuration](../src/utils/config.py) start_line:33 end_line:35
-<!--/codeinclude-->
-
-<!--codeinclude-->
-[HuggingFace API Key Configuration](../src/utils/config.py) start_line:57 end_line:59
-<!--/codeinclude-->
-
-## Optional Configuration
-
-### Embedding Configuration
-
-DeepCritical supports multiple embedding providers for semantic search and RAG:
-
-```bash
-# Embedding Provider: "openai", "local", or "huggingface"
-EMBEDDING_PROVIDER=local
-
-# OpenAI Embedding Model (used by LlamaIndex RAG)
-OPENAI_EMBEDDING_MODEL=text-embedding-3-small
-
-# Local Embedding Model (sentence-transformers, used by EmbeddingService)
-LOCAL_EMBEDDING_MODEL=all-MiniLM-L6-v2
-
-# HuggingFace Embedding Model
-HUGGINGFACE_EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2
-```
-
-The embedding provider configuration:
-
-<!--codeinclude-->
-[Embedding Provider Configuration](../src/utils/config.py) start_line:47 end_line:50
-<!--/codeinclude-->
-
-**Note**: OpenAI embeddings require `OPENAI_API_KEY`. The local provider (default) uses sentence-transformers and requires no API key.
-
-### Web Search Configuration
-
-DeepCritical supports multiple web search providers:
-
-```bash
-# Web Search Provider: "serper", "searchxng", "brave", "tavily", or "duckduckgo"
-# Default: "duckduckgo" (no API key required)
-WEB_SEARCH_PROVIDER=duckduckgo
-
-# Serper API Key (for Google search via Serper)
-SERPER_API_KEY=your_serper_api_key_here
-
-# SearchXNG Host URL (for self-hosted search)
-SEARCHXNG_HOST=http://localhost:8080
-
-# Brave Search API Key
-BRAVE_API_KEY=your_brave_api_key_here
-
-# Tavily API Key
-TAVILY_API_KEY=your_tavily_api_key_here
-```
-
-The web search provider configuration:
-
-<!--codeinclude-->
-[Web Search Provider Configuration](../src/utils/config.py) start_line:71 end_line:74
-<!--/codeinclude-->
-
-**Note**: DuckDuckGo is the default and requires no API key, making it ideal for development and testing.
-
-### PubMed Configuration
-
-PubMed search supports optional NCBI API key for higher rate limits:
-
-```bash
-# NCBI API Key (optional, for higher rate limits: 10 req/sec vs 3 req/sec)
-NCBI_API_KEY=your_ncbi_api_key_here
-```
-
-The PubMed tool uses this configuration:
-
-<!--codeinclude-->
-[PubMed Tool Configuration](../src/tools/pubmed.py) start_line:22 end_line:29
-<!--/codeinclude-->
-
-### Agent Configuration
-
-Control agent behavior and research loop execution:
-
-```bash
-# Maximum iterations per research loop (1-50, default: 10)
-MAX_ITERATIONS=10
-
-# Search timeout in seconds
-SEARCH_TIMEOUT=30
-
-# Use graph-based execution for research flows
-USE_GRAPH_EXECUTION=false
-```
-
-The agent configuration fields:
-
-<!--codeinclude-->
-[Agent Configuration](../src/utils/config.py) start_line:80 end_line:85
-<!--/codeinclude-->
-
-### Budget & Rate Limiting Configuration
-
-Control resource limits for research loops:
-
-```bash
-# Default token budget per research loop (1000-1000000, default: 100000)
-DEFAULT_TOKEN_LIMIT=100000
-
-# Default time limit per research loop in minutes (1-120, default: 10)
-DEFAULT_TIME_LIMIT_MINUTES=10
-
-# Default iterations limit per research loop (1-50, default: 10)
-DEFAULT_ITERATIONS_LIMIT=10
-```
-
-The budget configuration with validation:
-
-<!--codeinclude-->
-[Budget Configuration](../src/utils/config.py) start_line:87 end_line:105
-<!--/codeinclude-->
-
-### RAG Service Configuration
-
-Configure the Retrieval-Augmented Generation service:
-
-```bash
-# ChromaDB collection name for RAG
-RAG_COLLECTION_NAME=deepcritical_evidence
-
-# Number of top results to retrieve from RAG (1-50, default: 5)
-RAG_SIMILARITY_TOP_K=5
-
-# Automatically ingest evidence into RAG
-RAG_AUTO_INGEST=true
-```
-
-The RAG configuration:
-
-<!--codeinclude-->
-[RAG Service Configuration](../src/utils/config.py) start_line:127 end_line:141
-<!--/codeinclude-->
-
-### ChromaDB Configuration
-
-Configure the vector database for embeddings and RAG:
-
-```bash
-# ChromaDB storage path
-CHROMA_DB_PATH=./chroma_db
-
-# Whether to persist ChromaDB to disk
-CHROMA_DB_PERSIST=true
-
-# ChromaDB server host (for remote ChromaDB, optional)
-CHROMA_DB_HOST=localhost
-
-# ChromaDB server port (for remote ChromaDB, optional)
-CHROMA_DB_PORT=8000
-```
-
-The ChromaDB configuration:
-
-<!--codeinclude-->
-[ChromaDB Configuration](../src/utils/config.py) start_line:113 end_line:125
-<!--/codeinclude-->
-
-### External Services
-
-#### Modal Configuration
-
-Modal is used for secure sandbox execution of statistical analysis:
-
-```bash
-# Modal Token ID (for Modal sandbox execution)
-MODAL_TOKEN_ID=your_modal_token_id_here
-
-# Modal Token Secret
-MODAL_TOKEN_SECRET=your_modal_token_secret_here
-```
-
-The Modal configuration:
-
-<!--codeinclude-->
-[Modal Configuration](../src/utils/config.py) start_line:110 end_line:112
-<!--/codeinclude-->
-
-### Logging Configuration
-
-Configure structured logging:
-
-```bash
-# Log Level: "DEBUG", "INFO", "WARNING", or "ERROR"
-LOG_LEVEL=INFO
-```
-
-The logging configuration:
-
-<!--codeinclude-->
-[Logging Configuration](../src/utils/config.py) start_line:107 end_line:108
-<!--/codeinclude-->
-
-Logging is configured via the `configure_logging()` function:
-
-<!--codeinclude-->
-[Configure Logging Function](../src/utils/config.py) start_line:212 end_line:231
-<!--/codeinclude-->
-
-## Configuration Properties
-
-The `Settings` class provides helpful properties for checking configuration state:
-
-### API Key Availability
-
-Check which API keys are available:
-
-<!--codeinclude-->
-[API Key Availability Properties](../src/utils/config.py) start_line:171 end_line:189
-<!--/codeinclude-->
-
-**Usage:**
-
-```python
-from src.utils.config import settings
-
-# Check API key availability
-if settings.has_openai_key:
-    # Use OpenAI
-    pass
-
-if settings.has_anthropic_key:
-    # Use Anthropic
-    pass
-
-if settings.has_huggingface_key:
-    # Use HuggingFace
-    pass
-
-if settings.has_any_llm_key:
-    # At least one LLM is available
-    pass
-```
-
-### Service Availability
-
-Check if external services are configured:
-
-<!--codeinclude-->
-[Modal Availability Property](../src/utils/config.py) start_line:143 end_line:146
-<!--/codeinclude-->
-
-<!--codeinclude-->
-[Web Search Availability Property](../src/utils/config.py) start_line:191 end_line:204
-<!--/codeinclude-->
-
-**Usage:**
-
-```python
-from src.utils.config import settings
-
-# Check service availability
-if settings.modal_available:
-    # Use Modal sandbox
-    pass
-
-if settings.web_search_available:
-    # Web search is configured
-    pass
-```
-
-### API Key Retrieval
-
-Get the API key for the configured provider:
-
-<!--codeinclude-->
-[Get API Key Method](../src/utils/config.py) start_line:148 end_line:160
-<!--/codeinclude-->
-
-For OpenAI-specific operations (e.g., Magentic mode):
-
-<!--codeinclude-->
-[Get OpenAI API Key Method](../src/utils/config.py) start_line:162 end_line:169
-<!--/codeinclude-->
-
-## Configuration Usage in Codebase
-
-The configuration system is used throughout the codebase:
-
-### LLM Factory
-
-The LLM factory uses settings to create appropriate models:
-
-<!--codeinclude-->
-[LLM Factory Usage](../src/utils/llm_factory.py) start_line:129 end_line:144
-<!--/codeinclude-->
-
-### Embedding Service
-
-The embedding service uses local embedding model configuration:
-
-<!--codeinclude-->
-[Embedding Service Usage](../src/services/embeddings.py) start_line:29 end_line:31
-<!--/codeinclude-->
-
-### Orchestrator Factory
-
-The orchestrator factory uses settings to determine mode:
-
-<!--codeinclude-->
-[Orchestrator Factory Mode Detection](../src/orchestrator_factory.py) start_line:97 end_line:110
-<!--/codeinclude-->
-
-## Environment Variables Reference
-
-### Required (at least one LLM)
-
-- `OPENAI_API_KEY` - OpenAI API key (required for OpenAI provider)
-- `ANTHROPIC_API_KEY` - Anthropic API key (required for Anthropic provider)
-- `HF_TOKEN` or `HUGGINGFACE_API_KEY` - HuggingFace API token (optional, can work without for public models)
-
-#### LLM Configuration Variables
-
-- `LLM_PROVIDER` - Provider to use: `"openai"`, `"anthropic"`, or `"huggingface"` (default: `"huggingface"`)
-- `OPENAI_MODEL` - OpenAI model name (default: `"gpt-5.1"`)
-- `ANTHROPIC_MODEL` - Anthropic model name (default: `"claude-sonnet-4-5-20250929"`)
-- `HUGGINGFACE_MODEL` - HuggingFace model ID (default: `"meta-llama/Llama-3.1-8B-Instruct"`)
-
-#### Embedding Configuration Variables
-
-- `EMBEDDING_PROVIDER` - Provider: `"openai"`, `"local"`, or `"huggingface"` (default: `"local"`)
-- `OPENAI_EMBEDDING_MODEL` - OpenAI embedding model (default: `"text-embedding-3-small"`)
-- `LOCAL_EMBEDDING_MODEL` - Local sentence-transformers model (default: `"all-MiniLM-L6-v2"`)
-- `HUGGINGFACE_EMBEDDING_MODEL` - HuggingFace embedding model (default: `"sentence-transformers/all-MiniLM-L6-v2"`)
-
-#### Web Search Configuration Variables
-
-- `WEB_SEARCH_PROVIDER` - Provider: `"serper"`, `"searchxng"`, `"brave"`, `"tavily"`, or `"duckduckgo"` (default: `"duckduckgo"`)
-- `SERPER_API_KEY` - Serper API key (required for Serper provider)
-- `SEARCHXNG_HOST` - SearchXNG host URL (required for SearchXNG provider)
-- `BRAVE_API_KEY` - Brave Search API key (required for Brave provider)
-- `TAVILY_API_KEY` - Tavily API key (required for Tavily provider)
-
-#### PubMed Configuration Variables
-
-- `NCBI_API_KEY` - NCBI API key (optional, increases rate limit from 3 to 10 req/sec)
-
-#### Agent Configuration Variables
-
-- `MAX_ITERATIONS` - Maximum iterations per research loop (1-50, default: `10`)
-- `SEARCH_TIMEOUT` - Search timeout in seconds (default: `30`)
-- `USE_GRAPH_EXECUTION` - Use graph-based execution (default: `false`)
-
-#### Budget Configuration Variables
-
-- `DEFAULT_TOKEN_LIMIT` - Default token budget per research loop (1000-1000000, default: `100000`)
-- `DEFAULT_TIME_LIMIT_MINUTES` - Default time limit in minutes (1-120, default: `10`)
-- `DEFAULT_ITERATIONS_LIMIT` - Default iterations limit (1-50, default: `10`)
-
-#### RAG Configuration Variables
-
-- `RAG_COLLECTION_NAME` - ChromaDB collection name (default: `"deepcritical_evidence"`)
-- `RAG_SIMILARITY_TOP_K` - Number of top results to retrieve (1-50, default: `5`)
-- `RAG_AUTO_INGEST` - Automatically ingest evidence into RAG (default: `true`)
-
-#### ChromaDB Configuration Variables
-
-- `CHROMA_DB_PATH` - ChromaDB storage path (default: `"./chroma_db"`)
-- `CHROMA_DB_PERSIST` - Whether to persist ChromaDB to disk (default: `true`)
-- `CHROMA_DB_HOST` - ChromaDB server host (optional, for remote ChromaDB)
-- `CHROMA_DB_PORT` - ChromaDB server port (optional, for remote ChromaDB)
-
-#### External Services Variables
-
-- `MODAL_TOKEN_ID` - Modal token ID (optional, for Modal sandbox execution)
-- `MODAL_TOKEN_SECRET` - Modal token secret (optional, for Modal sandbox execution)
-
-#### Logging Configuration Variables
-
-- `LOG_LEVEL` - Log level: `"DEBUG"`, `"INFO"`, `"WARNING"`, or `"ERROR"` (default: `"INFO"`)
-
-## Validation
-
-Settings are validated on load using Pydantic validation:
-
-- **Type Checking**: All fields are strongly typed
-- **Range Validation**: Numeric fields have min/max constraints (e.g., `ge=1, le=50` for `max_iterations`)
-- **Literal Validation**: Enum fields only accept specific values (e.g., `Literal["openai", "anthropic", "huggingface"]`)
-- **Required Fields**: API keys are checked when accessed via `get_api_key()` or `get_openai_api_key()`
-
-### Validation Examples
-
-The `max_iterations` field has range validation:
-
-<!--codeinclude-->
-[Max Iterations Validation](../src/utils/config.py) start_line:81 end_line:81
-<!--/codeinclude-->
-
-The `llm_provider` field has literal validation:
-
-<!--codeinclude-->
-[LLM Provider Literal Validation](../src/utils/config.py) start_line:26 end_line:28
-<!--/codeinclude-->
-
-## Error Handling
-
-Configuration errors raise `ConfigurationError` from `src/utils/exceptions.py`:
-
-<!--codeinclude-->
-[ConfigurationError Class](../src/utils/exceptions.py) start_line:22 end_line:25
-<!--/codeinclude-->
-
-### Error Handling Example
-
-```python
-from src.utils.config import settings
-from src.utils.exceptions import ConfigurationError
-
-try:
-    api_key = settings.get_api_key()
-except ConfigurationError as e:
-    print(f"Configuration error: {e}")
-```
-
-### Common Configuration Errors
-
-1. **Missing API Key**: When `get_api_key()` is called but the required API key is not set
-2. **Invalid Provider**: When `llm_provider` is set to an unsupported value
-3. **Out of Range**: When numeric values exceed their min/max constraints
-4. **Invalid Literal**: When enum fields receive unsupported values
-
-## Configuration Best Practices
-
-1. **Use `.env` File**: Store sensitive keys in `.env` file (add to `.gitignore`)
-2. **Check Availability**: Use properties like `has_openai_key` before accessing API keys
-3. **Handle Errors**: Always catch `ConfigurationError` when calling `get_api_key()`
-4. **Validate Early**: Configuration is validated on import, so errors surface immediately
-5. **Use Defaults**: Leverage sensible defaults for optional configuration
-
-## Future Enhancements
-
-The following configurations are planned for future phases:
-
-1. **Additional LLM Providers**: DeepSeek, OpenRouter, Gemini, Perplexity, Azure OpenAI, Local models
-2. **Model Selection**: Reasoning/main/fast model configuration
-3. **Service Integration**: Additional service integrations and configurations

docs/contributing/code-quality.md

@@ -1,6 +1,6 @@
 # Code Quality & Documentation
 
-This document outlines code quality standards and documentation requirements.
+This document outlines code quality standards and documentation requirements for The DETERMINATOR.
 
 ## Linting
 
@@ -12,6 +12,9 @@ This document outlines code quality standards and documentation requirements.
   - `PLR2004`: Magic values (statistical constants)
   - `PLW0603`: Global statement (singleton pattern)
   - `PLC0415`: Lazy imports for optional dependencies
+  - `E402`: Module level import not at top (needed for pytest.importorskip)
+  - `E501`: Line too long (ignore line length violations)
+  - `RUF100`: Unused noqa (version differences between local/CI)
 
 ## Type Checking
 
@@ -22,12 +25,75 @@ This document outlines code quality standards and documentation requirements.
 
 ## Pre-commit
 
-- Run `make check` before committing
-- Must pass: lint + typecheck + test-cov
-- Pre-commit hooks installed via `make install`
+Pre-commit hooks run automatically on commit to ensure code quality. Configuration is in `.pre-commit-config.yaml`.
+
+### Installation
+
+```bash
+# Install dependencies (includes pre-commit package)
+uv sync --all-extras
+
+# Set up git hooks (must be run separately)
+uv run pre-commit install
+```
+
+**Note**: `uv sync --all-extras` installs the pre-commit package, but you must run `uv run pre-commit install` separately to set up the git hooks.
+
+### Pre-commit Hooks
+
+The following hooks run automatically on commit:
+
+1. **ruff**: Lints code and fixes issues automatically
+   - Runs on: `src/` (excludes `tests/`, `reference_repos/`)
+   - Auto-fixes: Yes
+
+2. **ruff-format**: Formats code with ruff
+   - Runs on: `src/` (excludes `tests/`, `reference_repos/`)
+   - Auto-fixes: Yes
+
+3. **mypy**: Type checking
+   - Runs on: `src/` (excludes `folder/`)
+   - Additional dependencies: pydantic, pydantic-settings, tenacity, pydantic-ai
+
+4. **pytest-unit**: Runs unit tests (excludes OpenAI and embedding_provider tests)
+   - Runs: `tests/unit/` with `-m "not openai and not embedding_provider"`
+   - Always runs: Yes (not just on changed files)
+
+5. **pytest-local-embeddings**: Runs local embedding tests
+   - Runs: `tests/` with `-m "local_embeddings"`
+   - Always runs: Yes
+
+### Manual Pre-commit Run
+
+To run pre-commit hooks manually (without committing):
+
+```bash
+uv run pre-commit run --all-files
+```
+
+### Troubleshooting
+
+- **Hooks failing**: Fix the issues shown in the output, then commit again
+- **Skipping hooks**: Use `git commit --no-verify` (not recommended)
+- **Hook not running**: Ensure hooks are installed with `uv run pre-commit install`
+- **Type errors**: Check that all dependencies are installed with `uv sync --all-extras`
 
 ## Documentation
 
+### Building Documentation
+
+Documentation is built using MkDocs. Source files are in `docs/`, and the configuration is in `mkdocs.yml`.
+
+```bash
+# Build documentation
+uv run mkdocs build
+
+# Serve documentation locally (http://127.0.0.1:8000)
+uv run mkdocs serve
+```
+
+The documentation site is published at: <https://deepcritical.github.io/GradioDemo/>
+
 ### Docstrings
 
 - Google-style docstrings for all public functions

docs/contributing/code-style.md

@@ -1,6 +1,44 @@
 # Code Style & Conventions
 
-This document outlines the code style and conventions for DeepCritical.
+This document outlines the code style and conventions for The DETERMINATOR.
+
+## Package Manager
+
+This project uses [`uv`](https://github.com/astral-sh/uv) as the package manager. All commands should be prefixed with `uv run` to ensure they run in the correct environment.
+
+### Installation
+
+```bash
+# Install uv if you haven't already (recommended: standalone installer)
+# Unix/macOS/Linux:
+curl -LsSf https://astral.sh/uv/install.sh | sh
+
+# Windows (PowerShell):
+powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
+
+# Alternative: pipx install uv
+# Or: pip install uv
+
+# Sync all dependencies including dev extras
+uv sync --all-extras
+```
+
+### Running Commands
+
+All development commands should use `uv run` prefix:
+
+```bash
+# Instead of: pytest tests/
+uv run pytest tests/
+
+# Instead of: ruff check src
+uv run ruff check src
+
+# Instead of: mypy src
+uv run mypy src
+```
+
+This ensures commands run in the correct virtual environment managed by `uv`.
 
 ## Type Safety
 

docs/contributing/error-handling.md

@@ -1,6 +1,6 @@
 # Error Handling & Logging
 
-This document outlines error handling and logging conventions for DeepCritical.
+This document outlines error handling and logging conventions for The DETERMINATOR.
 
 ## Exception Hierarchy
 

docs/contributing/implementation-patterns.md

@@ -1,6 +1,6 @@
 # Implementation Patterns
 
-This document outlines common implementation patterns used in DeepCritical.
+This document outlines common implementation patterns used in The DETERMINATOR.
 
 ## Search Tools
 

docs/contributing/index.md

@@ -1,6 +1,8 @@
-# Contributing to DeepCritical
+# Contributing to The DETERMINATOR
 
-Thank you for your interest in contributing to DeepCritical! This guide will help you get started.
+Thank you for your interest in contributing to The DETERMINATOR! This guide will help you get started.
+
+> **Note on Project Names**: "The DETERMINATOR" is the product name, "DeepCritical" is the organization/project name, and "determinator" is the Python package name.
 
 ## Git Workflow
 
@@ -10,44 +12,138 @@ Thank you for your interest in contributing to DeepCritical! This guide will hel
 - **NEVER** push directly to `main` or `dev` on HuggingFace
 - GitHub is source of truth; HuggingFace is for deployment
 
+## Repository Information
+
+- **GitHub Repository**: [`DeepCritical/GradioDemo`](https://github.com/DeepCritical/GradioDemo) (source of truth, PRs, code review)
+- **HuggingFace Space**: [`DataQuests/DeepCritical`](https://huggingface.co/spaces/DataQuests/DeepCritical) (deployment/demo)
+- **Package Name**: `determinator` (Python package name in `pyproject.toml`)
+
+### Dual Repository Setup
+
+This project uses a dual repository setup:
+
+- **GitHub (`DeepCritical/GradioDemo`)**: Source of truth for code, PRs, and code review
+- **HuggingFace (`DataQuests/DeepCritical`)**: Deployment target for the Gradio demo
+
+#### Remote Configuration
+
+When cloning, set up remotes as follows:
+
+```bash
+# Clone from GitHub
+git clone https://github.com/DeepCritical/GradioDemo.git
+cd GradioDemo
+
+# Add HuggingFace remote (optional, for deployment)
+git remote add huggingface-upstream https://huggingface.co/spaces/DataQuests/DeepCritical
+```
+
+**Important**: Never push directly to `main` or `dev` on HuggingFace. Always work through GitHub PRs. GitHub is the source of truth; HuggingFace is for deployment/demo only.
+
+## Package Manager
+
+This project uses [`uv`](https://github.com/astral-sh/uv) as the package manager. All commands should be prefixed with `uv run` to ensure they run in the correct environment.
+
+### Installation
+
+```bash
+# Install uv if you haven't already (recommended: standalone installer)
+# Unix/macOS/Linux:
+curl -LsSf https://astral.sh/uv/install.sh | sh
+
+# Windows (PowerShell):
+powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
+
+# Alternative: pipx install uv
+# Or: pip install uv
+
+# Sync all dependencies including dev extras
+uv sync --all-extras
+
+# Install pre-commit hooks
+uv run pre-commit install
+```
+
 ## Development Commands
 
 ```bash
-make install      # Install dependencies + pre-commit
-make check        # Lint + typecheck + test (MUST PASS)
-make test         # Run unit tests
-make lint         # Run ruff
-make format       # Format with ruff
-make typecheck    # Run mypy
-make test-cov     # Test with coverage
+# Installation
+uv sync --all-extras              # Install all dependencies including dev
+uv run pre-commit install          # Install pre-commit hooks
+
+# Code Quality Checks (run all before committing)
+uv run ruff check src tests       # Lint with ruff
+uv run ruff format src tests      # Format with ruff
+uv run mypy src                   # Type checking
+uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire  # Tests with coverage
+
+# Testing Commands
+uv run pytest tests/unit/ -v -m "not openai" -p no:logfire              # Run unit tests (excludes OpenAI tests)
+uv run pytest tests/ -v -m "huggingface" -p no:logfire                 # Run HuggingFace tests
+uv run pytest tests/ -v -p no:logfire                                  # Run all tests
+uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire  # Tests with terminal coverage
+uv run pytest --cov=src --cov-report=html -p no:logfire                # Generate HTML coverage report (opens htmlcov/index.html)
+
+# Documentation Commands
+uv run mkdocs build                # Build documentation
+uv run mkdocs serve                # Serve documentation locally (http://127.0.0.1:8000)
 ```
 
+### Test Markers
+
+The project uses pytest markers to categorize tests. See [Testing Guidelines](testing.md) for details:
+
+- `unit`: Unit tests (mocked, fast)
+- `integration`: Integration tests (real APIs)
+- `slow`: Slow tests
+- `openai`: Tests requiring OpenAI API key
+- `huggingface`: Tests requiring HuggingFace API key
+- `embedding_provider`: Tests requiring API-based embedding providers
+- `local_embeddings`: Tests using local embeddings
+
+**Note**: The `-p no:logfire` flag disables the logfire plugin to avoid conflicts during testing.
+
 ## Getting Started
 
-1. **Fork the repository** on GitHub
+1. **Fork the repository** on GitHub: [`DeepCritical/GradioDemo`](https://github.com/DeepCritical/GradioDemo)
+
 2. **Clone your fork**:
+
    ```bash
    git clone https://github.com/yourusername/GradioDemo.git
    cd GradioDemo
    ```
+
 3. **Install dependencies**:
+
    ```bash
-   make install
+   uv sync --all-extras
+   uv run pre-commit install
    ```
+
 4. **Create a feature branch**:
+
    ```bash
    git checkout -b yourname-feature-name
    ```
+
 5. **Make your changes** following the guidelines below
+
 6. **Run checks**:
+
    ```bash
-   make check
+   uv run ruff check src tests
+   uv run mypy src
+   uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire
    ```
+
 7. **Commit and push**:
+
    ```bash
    git commit -m "Description of changes"
    git push origin yourname-feature-name
    ```
+
 8. **Create a pull request** on GitHub
 
 ## Development Guidelines
@@ -132,7 +228,7 @@ make test-cov     # Test with coverage
 
 ## Pull Request Process
 
-1. Ensure all checks pass: `make check`
+1. Ensure all checks pass: `uv run ruff check src tests && uv run mypy src && uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire`
 2. Update documentation if needed
 3. Add tests for new features
 4. Update CHANGELOG if applicable
@@ -140,10 +236,19 @@ make test-cov     # Test with coverage
 6. Address review feedback
 7. Wait for approval before merging
 
+## Project Structure
+
+- `src/`: Main source code
+- `tests/`: Test files (`unit/` and `integration/`)
+- `docs/`: Documentation source files (MkDocs)
+- `examples/`: Example usage scripts
+- `pyproject.toml`: Project configuration and dependencies
+- `.pre-commit-config.yaml`: Pre-commit hook configuration
+
 ## Questions?
 
-- Open an issue on GitHub
-- Check existing documentation
+- Open an issue on [GitHub](https://github.com/DeepCritical/GradioDemo)
+- Check existing [documentation](https://deepcritical.github.io/GradioDemo/)
 - Review code examples in the codebase
 
-Thank you for contributing to DeepCritical!
+Thank you for contributing to The DETERMINATOR!

docs/contributing/testing.md

@@ -1,12 +1,45 @@
 # Testing Requirements
 
-This document outlines testing requirements and guidelines for DeepCritical.
+This document outlines testing requirements and guidelines for The DETERMINATOR.
 
 ## Test Structure
 
 - Unit tests in `tests/unit/` (mocked, fast)
 - Integration tests in `tests/integration/` (real APIs, marked `@pytest.mark.integration`)
-- Use markers: `unit`, `integration`, `slow`
+- Use markers: `unit`, `integration`, `slow`, `openai`, `huggingface`, `embedding_provider`, `local_embeddings`
+
+## Test Markers
+
+The project uses pytest markers to categorize tests. These markers are defined in `pyproject.toml`:
+
+- `@pytest.mark.unit`: Unit tests (mocked, fast) - Run with `-m "unit"`
+- `@pytest.mark.integration`: Integration tests (real APIs) - Run with `-m "integration"`
+- `@pytest.mark.slow`: Slow tests - Run with `-m "slow"`
+- `@pytest.mark.openai`: Tests requiring OpenAI API key - Run with `-m "openai"` or exclude with `-m "not openai"`
+- `@pytest.mark.huggingface`: Tests requiring HuggingFace API key or using HuggingFace models - Run with `-m "huggingface"`
+- `@pytest.mark.embedding_provider`: Tests requiring API-based embedding providers (OpenAI, etc.) - Run with `-m "embedding_provider"`
+- `@pytest.mark.local_embeddings`: Tests using local embeddings (sentence-transformers, ChromaDB) - Run with `-m "local_embeddings"`
+
+### Running Tests by Marker
+
+```bash
+# Run only unit tests (excludes OpenAI tests by default)
+uv run pytest tests/unit/ -v -m "not openai" -p no:logfire
+
+# Run HuggingFace tests
+uv run pytest tests/ -v -m "huggingface" -p no:logfire
+
+# Run all tests
+uv run pytest tests/ -v -p no:logfire
+
+# Run only local embedding tests
+uv run pytest tests/ -v -m "local_embeddings" -p no:logfire
+
+# Exclude slow tests
+uv run pytest tests/ -v -m "not slow" -p no:logfire
+```
+
+**Note**: The `-p no:logfire` flag disables the logfire plugin to avoid conflicts during testing.
 
 ## Mocking
 
@@ -20,7 +53,20 @@ This document outlines testing requirements and guidelines for DeepCritical.
 1. Write failing test in `tests/unit/`
 2. Implement in `src/`
 3. Ensure test passes
-4. Run `make check` (lint + typecheck + test)
+4. Run checks: `uv run ruff check src tests && uv run mypy src && uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire`
+
+### Test Command Examples
+
+```bash
+# Run unit tests (default, excludes OpenAI tests)
+uv run pytest tests/unit/ -v -m "not openai" -p no:logfire
+
+# Run HuggingFace tests
+uv run pytest tests/ -v -m "huggingface" -p no:logfire
+
+# Run all tests
+uv run pytest tests/ -v -p no:logfire
+```
 
 ## Test Examples
 
@@ -41,9 +87,27 @@ async def test_real_pubmed_search():
 
 ## Test Coverage
 
-- Run `make test-cov` for coverage report
+### Terminal Coverage Report
+
+```bash
+uv run pytest --cov=src --cov-report=term-missing tests/unit/ -v -m "not openai" -p no:logfire
+```
+
+This shows coverage with missing lines highlighted in the terminal output.
+
+### HTML Coverage Report
+
+```bash
+uv run pytest --cov=src --cov-report=html -p no:logfire
+```
+
+This generates an HTML coverage report in `htmlcov/index.html`. Open this file in your browser to see detailed coverage information.
+
+### Coverage Goals
+
 - Aim for >80% coverage on critical paths
 - Exclude: `__init__.py`, `TYPE_CHECKING` blocks
+- Coverage configuration is in `pyproject.toml` under `[tool.coverage.*]`
 
 ## See Also
 

docs/getting-started/examples.md

@@ -25,6 +25,7 @@ What clinical trials are investigating metformin for cancer prevention?
 ```
 
 **What The DETERMINATOR Does**:
+
 1. Searches ClinicalTrials.gov for relevant trials
 2. Searches PubMed for supporting literature
 3. Provides trial details and status
@@ -35,6 +36,7 @@ What clinical trials are investigating metformin for cancer prevention?
 ### Example 3: Comprehensive Review
 
 **Query**:
+
 ```
 Review the evidence for using metformin as an anti-aging intervention, 
 including clinical trials, mechanisms of action, and safety profile.
@@ -194,25 +196,3 @@ USE_GRAPH_EXECUTION=true
 - Explore the [Architecture Documentation](../architecture/graph_orchestration.md)
 - Check out the [API Reference](../api/agents.md) for programmatic usage
 
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<<<<<<< Updated upstream
-
-
-
-
-
-=======
->>>>>>> Stashed changes

docs/getting-started/installation.md

@@ -12,12 +12,29 @@ This guide will help you install and set up DeepCritical on your system.
 
 ### 1. Install uv (Recommended)
 
-`uv` is a fast Python package installer and resolver. Install it with:
+`uv` is a fast Python package installer and resolver. Install it using the standalone installer (recommended):
 
+**Unix/macOS/Linux:**
 ```bash
+curl -LsSf https://astral.sh/uv/install.sh | sh
+```
+
+**Windows (PowerShell):**
+```powershell
+powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
+```
+
+**Alternative methods:**
+```bash
+# Using pipx (recommended if you have pipx installed)
+pipx install uv
+
+# Or using pip
 pip install uv
 ```
 
+After installation, restart your terminal or add `~/.cargo/bin` to your PATH.
+
 ### 2. Clone the Repository
 
 ```bash
@@ -133,40 +150,3 @@ uv run pre-commit install
 - Learn about [MCP Integration](mcp-integration.md)
 - Explore [Examples](examples.md)
 
-
-
-
-
-
-
-
-
-
-
-
-
-
-<<<<<<< HEAD
-<<<<<<< Updated upstream
-=======
-=======
->>>>>>> 8086ce5fefde1c867880661d57e1299029a91ead
-
-<<<<<<< Updated upstream
-
-
-
-
-
-=======
->>>>>>> Stashed changes
-
-
-
-
-<<<<<<< HEAD
-
->>>>>>> Stashed changes
-
-=======
->>>>>>> 8086ce5fefde1c867880661d57e1299029a91ead

docs/getting-started/mcp-integration.md

@@ -201,16 +201,3 @@ You can configure multiple DeepCritical instances:
 - Read the [Architecture Documentation](../architecture/graph_orchestration.md)
 
 
-
-
-
-
-
-
-
-
-
-
-
-
-

docs/getting-started/quick-start.md

@@ -41,7 +41,7 @@ Get up and running with The DETERMINATOR in minutes.
 ## Start the Application
 
 ```bash
-uv run gradio run src/app.py
+gradio src/app.py
 ```
 
 Open your browser to `http://localhost:7860`.
@@ -140,25 +140,3 @@ What are the active clinical trials investigating Alzheimer's disease treatments
 - Read the [Configuration Guide](../configuration/index.md) for advanced settings
 - Check out the [Architecture Documentation](../architecture/graph_orchestration.md) to understand how it works
 
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<<<<<<< Updated upstream
-
-
-
-
-
-=======
->>>>>>> Stashed changes

docs/implementation/IMPLEMENTATION_SUMMARY.md

@@ -1,188 +0,0 @@
-# Multimodal Audio & Image Integration - Implementation Summary
-
-## ✅ Completed Implementation
-
-### 1. Configuration System (`src/utils/config.py`)
-- ✅ Added audio configuration fields:
-  - `tts_model`, `tts_voice`, `tts_speed`, `tts_gpu`, `tts_timeout`
-  - `stt_api_url`, `stt_source_lang`, `stt_target_lang`
-  - `enable_audio_input`, `enable_audio_output`
-- ✅ Added image OCR configuration:
-  - `ocr_api_url`, `enable_image_input`
-- ✅ Added property methods: `audio_available`, `image_ocr_available`
-
-### 2. STT Service (`src/services/stt_gradio.py`)
-- ✅ Gradio Client integration for nvidia/canary-1b-v2
-- ✅ Supports file and numpy array audio input
-- ✅ Async transcription with error handling
-- ✅ Singleton factory pattern
-
-### 3. TTS Service (`src/services/tts_modal.py`)
-- ✅ **Modal GPU function implementation** following Modal documentation
-- ✅ Kokoro 82M integration via Modal GPU
-- ✅ Module-level function definition with lazy initialization
-- ✅ GPU configuration (T4, A10, A100, L4, L40S)
-- ✅ Async wrapper for TTS synthesis
-- ✅ Error handling and graceful degradation
-
-### 4. Image OCR Service (`src/services/image_ocr.py`)
-- ✅ Gradio Client integration for prithivMLmods/Multimodal-OCR3
-- ✅ Supports image files and PIL/numpy arrays
-- ✅ Text extraction from API results
-- ✅ Singleton factory pattern
-
-### 5. Unified Services
-- ✅ `src/services/audio_processing.py` - Audio service layer
-- ✅ `src/services/multimodal_processing.py` - Multimodal service layer
-
-### 6. ChatInterface Integration (`src/app.py`)
-- ✅ Enabled `multimodal=True` for MultimodalTextbox
-- ✅ Added Audio output component
-- ✅ Integrated STT/TTS/OCR into research flow
-- ✅ Multimodal input processing (text + images + audio)
-- ✅ TTS output generation for final responses
-- ✅ **Configuration UI in Settings Accordion**:
-  - Voice dropdown (20+ Kokoro voices)
-  - Speed slider (0.5x to 2.0x)
-  - GPU dropdown (T4, A10, A100, L4, L40S) - read-only, requires restart
-  - Enable audio output checkbox
-- ✅ Configuration values passed from UI to TTS service
-
-### 7. MCP Integration (`src/mcp_tools.py`)
-- ✅ Added `extract_text_from_image` MCP tool
-- ✅ Added `transcribe_audio_file` MCP tool
-- ✅ Enabled MCP server in app launch
-
-### 8. Dependencies (`pyproject.toml`)
-- ✅ Added audio dependencies (gradio-client, soundfile, Pillow)
-- ✅ Added TTS optional dependencies (torch, transformers)
-- ✅ Installed via `uv add --optional`
-
-## 🔧 Modal GPU Implementation Details
-
-### Function Definition Pattern
-The Modal GPU function is defined using Modal's recommended pattern:
-
-```python
-@app.function(
-    image=tts_image,  # Image with Kokoro dependencies
-    gpu="T4",  # GPU type from settings.tts_gpu
-    timeout=60,  # Timeout from settings.tts_timeout
-)
-def kokoro_tts_function(text: str, voice: str, speed: float) -> tuple[int, np.ndarray]:
-    """Modal GPU function for Kokoro TTS."""
-    from kokoro import KModel, KPipeline
-    import torch
-    
-    model = KModel().to("cuda").eval()
-    pipeline = KPipeline(lang_code=voice[0])
-    pack = pipeline.load_voice(voice)
-    
-    for _, ps, _ in pipeline(text, voice, speed):
-        ref_s = pack[len(ps) - 1]
-        audio = model(ps, ref_s, speed)
-        return (24000, audio.numpy())
-```
-
-### Key Implementation Points
-1. **Module-Level Definition**: Function defined inside `_setup_modal_function()` but attached to app instance
-2. **Lazy Initialization**: Function set up on first use
-3. **GPU Configuration**: Set at function definition time (requires restart to change)
-4. **Runtime Parameters**: Voice and speed can be changed at runtime via UI
-
-## 🔗 Configuration Flow
-
-### Settings → Implementation
-1. `settings.tts_voice` → Default voice (used if UI not configured)
-2. `settings.tts_speed` → Default speed (used if UI not configured)
-3. `settings.tts_gpu` → GPU type (set at function definition, requires restart)
-4. `settings.tts_timeout` → Timeout (set at function definition)
-
-### UI → Implementation
-1. Voice dropdown → `tts_voice` parameter → `AudioService.generate_audio_output()`
-2. Speed slider → `tts_speed` parameter → `AudioService.generate_audio_output()`
-3. GPU dropdown → Informational only (changes require restart)
-4. Enable checkbox → `settings.enable_audio_output` → Controls TTS generation
-
-### Implementation → Modal
-1. `TTSService.synthesize_async()` → Calls Modal GPU function
-2. Modal function executes on GPU → Returns audio tuple
-3. Audio tuple → Gradio Audio component → User hears response
-
-## 📋 Configuration Points in UI
-
-### Settings Accordion Components
-Located in `src/app.py` lines 667-712:
-
-1. **Voice Dropdown** (`tts_voice_dropdown`)
-   - 20+ Kokoro voices
-   - Default: `settings.tts_voice`
-   - Connected to `research_agent()` function
-
-2. **Speed Slider** (`tts_speed_slider`)
-   - Range: 0.5 to 2.0
-   - Step: 0.1
-   - Default: `settings.tts_speed`
-   - Connected to `research_agent()` function
-
-3. **GPU Dropdown** (`tts_gpu_dropdown`)
-   - Choices: T4, A10, A100, L4, L40S
-   - Default: `settings.tts_gpu or "T4"`
-   - Read-only (interactive=False)
-   - Note: Changes require app restart
-
-4. **Enable Audio Output** (`enable_audio_output_checkbox`)
-   - Default: `settings.enable_audio_output`
-   - Controls whether TTS is generated
-
-## 🎯 Usage Flow
-
-1. User opens Settings accordion
-2. Configures TTS voice and speed (optional)
-3. Submits query (text, image, or audio)
-4. Research agent processes query
-5. Final response generated
-6. If audio output enabled:
-   - `AudioService.generate_audio_output()` called
-   - Uses UI-configured voice/speed or settings defaults
-   - Modal GPU function synthesizes audio
-   - Audio displayed in Audio component
-
-## 📝 Notes
-
-- **GPU Changes**: GPU type is set at Modal function definition time. Changes to `settings.tts_gpu` or UI dropdown require app restart.
-- **Voice/Speed Changes**: Can be changed at runtime via UI - no restart required.
-- **Graceful Degradation**: If TTS fails, application continues with text-only response.
-- **Modal Credentials**: Required for TTS. If not configured, TTS service unavailable (graceful fallback).
-
-## ✅ Verification Checklist
-
-- [x] Modal GPU function correctly defined with `@app.function` decorator
-- [x] GPU parameter set from `settings.tts_gpu`
-- [x] Timeout parameter set from `settings.tts_timeout`
-- [x] Voice parameter passed from UI dropdown
-- [x] Speed parameter passed from UI slider
-- [x] Configuration UI elements in Settings accordion
-- [x] Configuration values connected to implementation
-- [x] Dependencies installed via uv
-- [x] Error handling and graceful degradation
-- [x] MCP tools added for audio/image processing
-
-## 🚀 Next Steps
-
-1. Test TTS with Modal credentials configured
-2. Verify GPU function execution on Modal
-3. Test voice and speed changes at runtime
-4. Add unit tests for services
-5. Add integration tests for Modal TTS
-
-
-
-
-
-
-
-
-
-
-

docs/implementation/TOKEN_AUTHENTICATION_REVIEW.md

@@ -1,201 +0,0 @@
-# Token Authentication Review - Gradio & HuggingFace
-
-## Summary
-
-This document reviews the implementation of token authentication for Gradio Client API calls and HuggingFace API usage to ensure tokens are always passed correctly.
-
-## ✅ Implementation Status
-
-### 1. Gradio Client Services
-
-#### STT Service (`src/services/stt_gradio.py`)
-- ✅ **Token Support**: Service accepts `hf_token` parameter in `__init__` and methods
-- ✅ **Client Initialization**: `Client` is created with `hf_token` parameter when token is available
-- ✅ **Token Priority**: Method-level token > instance-level token
-- ✅ **Token Updates**: Client is recreated if token changes
-
-**Implementation Pattern:**
-```python
-async def _get_client(self, hf_token: str | None = None) -> Client:
-    token = hf_token or self.hf_token
-    if token:
-        self.client = Client(self.api_url, hf_token=token)
-    else:
-        self.client = Client(self.api_url)
-```
-
-#### Image OCR Service (`src/services/image_ocr.py`)
-- ✅ **Token Support**: Service accepts `hf_token` parameter in `__init__` and methods
-- ✅ **Client Initialization**: `Client` is created with `hf_token` parameter when token is available
-- ✅ **Token Priority**: Method-level token > instance-level token
-- ✅ **Token Updates**: Client is recreated if token changes
-
-**Same pattern as STT Service**
-
-### 2. Service Layer Integration
-
-#### Audio Service (`src/services/audio_processing.py`)
-- ✅ **Token Passthrough**: `process_audio_input()` accepts `hf_token` and passes to STT service
-- ✅ **Token Flow**: `audio_service.process_audio_input(audio, hf_token=token)`
-
-#### Multimodal Service (`src/services/multimodal_processing.py`)
-- ✅ **Token Passthrough**: `process_multimodal_input()` accepts `hf_token` and passes to both audio and OCR services
-- ✅ **Token Flow**: `multimodal_service.process_multimodal_input(..., hf_token=token)`
-
-### 3. Application Layer (`src/app.py`)
-
-#### Token Extraction
-- ✅ **OAuth Token**: Extracted from `gr.OAuthToken` via `oauth_token.token`
-- ✅ **Fallback**: Uses `HF_TOKEN` or `HUGGINGFACE_API_KEY` from environment
-- ✅ **Token Priority**: `oauth_token > HF_TOKEN > HUGGINGFACE_API_KEY`
-
-**Implementation:**
-```python
-token_value: str | None = None
-if oauth_token is not None:
-    token_value = oauth_token.token if hasattr(oauth_token, "token") else None
-
-# Fallback to env vars
-effective_token = token_value or os.getenv("HF_TOKEN") or os.getenv("HUGGINGFACE_API_KEY")
-```
-
-#### Token Usage in Services
-- ✅ **Multimodal Processing**: Token passed to `process_multimodal_input(..., hf_token=token_value)`
-- ✅ **Consistent Usage**: Token is extracted once and passed through all service layers
-
-### 4. HuggingFace API Integration
-
-#### LLM Factory (`src/utils/llm_factory.py`)
-- ✅ **Token Priority**: `oauth_token > settings.hf_token > settings.huggingface_api_key`
-- ✅ **Provider Usage**: `HuggingFaceProvider(api_key=effective_hf_token)`
-- ✅ **Model Usage**: `HuggingFaceModel(model_name, provider=provider)`
-
-#### Judge Handler (`src/agent_factory/judges.py`)
-- ✅ **Token Priority**: `oauth_token > settings.hf_token > settings.huggingface_api_key`
-- ✅ **InferenceClient**: `InferenceClient(api_key=api_key)` when token provided
-- ✅ **Fallback**: Uses `HF_TOKEN` from environment if no token provided
-
-**Implementation:**
-```python
-effective_hf_token = oauth_token or settings.hf_token or settings.huggingface_api_key
-hf_provider = HuggingFaceProvider(api_key=effective_hf_token)
-```
-
-### 5. MCP Tools (`src/mcp_tools.py`)
-
-#### Image OCR Tool
-- ✅ **Token Support**: `extract_text_from_image()` accepts `hf_token` parameter
-- ✅ **Token Fallback**: Uses `settings.hf_token` or `settings.huggingface_api_key` if not provided
-- ✅ **Service Integration**: Passes token to `ImageOCRService.extract_text()`
-
-#### Audio Transcription Tool
-- ✅ **Token Support**: `transcribe_audio_file()` accepts `hf_token` parameter
-- ✅ **Token Fallback**: Uses `settings.hf_token` or `settings.huggingface_api_key` if not provided
-- ✅ **Service Integration**: Passes token to `STTService.transcribe_file()`
-
-## Token Flow Diagram
-
-```
-User Login (OAuth)
-    ↓
-oauth_token.token
-    ↓
-app.py: token_value
-    ↓
-┌─────────────────────────────────────┐
-│  Service Layer                       │
-├─────────────────────────────────────┤
-│  MultimodalService                   │
-│    ↓ hf_token=token_value            │
-│  AudioService                        │
-│    ↓ hf_token=token_value            │
-│  STTService / ImageOCRService        │
-│    ↓ hf_token=token_value            │
-│  Gradio Client(hf_token=token)       │
-└─────────────────────────────────────┘
-
-Alternative: Environment Variables
-    ↓
-HF_TOKEN or HUGGINGFACE_API_KEY
-    ↓
-settings.hf_token or settings.huggingface_api_key
-    ↓
-Same service flow as above
-```
-
-## Verification Checklist
-
-- [x] STT Service accepts and uses `hf_token` parameter
-- [x] Image OCR Service accepts and uses `hf_token` parameter
-- [x] Audio Service passes token to STT service
-- [x] Multimodal Service passes token to both audio and OCR services
-- [x] App.py extracts OAuth token correctly
-- [x] App.py passes token to multimodal service
-- [x] HuggingFace API calls use token via `HuggingFaceProvider`
-- [x] HuggingFace API calls use token via `InferenceClient`
-- [x] MCP tools accept and use token parameter
-- [x] Token priority is consistent: OAuth > Env Vars
-- [x] Fallback to environment variables when OAuth not available
-
-## Token Parameter Naming
-
-All services consistently use `hf_token` parameter name:
-- `STTService.transcribe_audio(..., hf_token=...)`
-- `STTService.transcribe_file(..., hf_token=...)`
-- `ImageOCRService.extract_text(..., hf_token=...)`
-- `ImageOCRService.extract_text_from_image(..., hf_token=...)`
-- `AudioService.process_audio_input(..., hf_token=...)`
-- `MultimodalService.process_multimodal_input(..., hf_token=...)`
-- `extract_text_from_image(..., hf_token=...)` (MCP tool)
-- `transcribe_audio_file(..., hf_token=...)` (MCP tool)
-
-## Gradio Client API Usage
-
-According to Gradio documentation, the `Client` constructor accepts:
-```python
-Client(space_name, hf_token=None)
-```
-
-Our implementation correctly uses:
-```python
-Client(self.api_url, hf_token=token)  # When token available
-Client(self.api_url)  # When no token (public Space)
-```
-
-## HuggingFace API Usage
-
-### HuggingFaceProvider
-```python
-HuggingFaceProvider(api_key=effective_hf_token)
-```
-✅ Correctly passes token as `api_key` parameter
-
-### InferenceClient
-```python
-InferenceClient(api_key=api_key)  # When token provided
-InferenceClient()  # Falls back to HF_TOKEN env var
-```
-✅ Correctly passes token as `api_key` parameter
-
-## Edge Cases Handled
-
-1. **No Token Available**: Services work without token (public Gradio Spaces)
-2. **Token Changes**: Client is recreated when token changes
-3. **OAuth vs Env**: OAuth token takes priority over environment variables
-4. **Multiple Token Sources**: Consistent priority across all services
-5. **MCP Tools**: Support both explicit token and fallback to settings
-
-## Recommendations
-
-✅ **All implementations are correct and consistent**
-
-The token authentication is properly implemented throughout:
-- Gradio Client services accept and use tokens
-- Service layer passes tokens through correctly
-- Application layer extracts and passes OAuth tokens
-- HuggingFace API calls use tokens via correct parameters
-- MCP tools support token authentication
-- Token priority is consistent across all layers
-
-No changes needed - implementation follows best practices.
-

docs/implementation/TTS_MODAL_IMPLEMENTATION.md

@@ -1,142 +0,0 @@
-# TTS Modal GPU Implementation
-
-## Overview
-
-The TTS (Text-to-Speech) service uses Kokoro 82M model running on Modal's GPU infrastructure. This document describes the implementation details and configuration.
-
-## Implementation Details
-
-### Modal GPU Function Pattern
-
-The implementation follows Modal's recommended pattern for GPU functions:
-
-1. **Module-Level Function Definition**: Modal functions must be defined at module level and attached to an app instance
-2. **Lazy Initialization**: The function is set up on first use via `_setup_modal_function()`
-3. **GPU Configuration**: GPU type is set at function definition time (requires app restart to change)
-
-### Key Files
-
-- `src/services/tts_modal.py` - Modal GPU executor for Kokoro TTS
-- `src/services/audio_processing.py` - Unified audio service wrapper
-- `src/utils/config.py` - Configuration settings
-- `src/app.py` - UI integration with settings accordion
-
-### Configuration Options
-
-All TTS configuration is available in `src/utils/config.py`:
-
-```python
-tts_model: str = "hexgrad/Kokoro-82M"  # Model ID
-tts_voice: str = "af_heart"  # Voice ID
-tts_speed: float = 1.0  # Speed multiplier (0.5-2.0)
-tts_gpu: str = "T4"  # GPU type (T4, A10, A100, etc.)
-tts_timeout: int = 60  # Timeout in seconds
-enable_audio_output: bool = True  # Enable/disable TTS
-```
-
-### UI Configuration
-
-TTS settings are available in the Settings accordion:
-
-- **Voice Dropdown**: Select from 20+ Kokoro voices (af_heart, af_bella, am_michael, etc.)
-- **Speed Slider**: Adjust speech speed (0.5x to 2.0x)
-- **GPU Dropdown**: Select GPU type (T4, A10, A100, L4, L40S) - visible only if Modal credentials configured
-- **Enable Audio Output**: Toggle TTS generation
-
-### Modal Function Implementation
-
-The Modal GPU function is defined as:
-
-```python
-@app.function(
-    image=tts_image,  # Image with Kokoro dependencies
-    gpu="T4",  # GPU type (from settings.tts_gpu)
-    timeout=60,  # Timeout (from settings.tts_timeout)
-)
-def kokoro_tts_function(text: str, voice: str, speed: float) -> tuple[int, np.ndarray]:
-    """Modal GPU function for Kokoro TTS."""
-    from kokoro import KModel, KPipeline
-    import torch
-    
-    model = KModel().to("cuda").eval()
-    pipeline = KPipeline(lang_code=voice[0])
-    pack = pipeline.load_voice(voice)
-    
-    for _, ps, _ in pipeline(text, voice, speed):
-        ref_s = pack[len(ps) - 1]
-        audio = model(ps, ref_s, speed)
-        return (24000, audio.numpy())
-```
-
-### Usage Flow
-
-1. User submits query with audio output enabled
-2. Research agent processes query and generates text response
-3. `AudioService.generate_audio_output()` is called with:
-   - Response text
-   - Voice (from UI dropdown or settings default)
-   - Speed (from UI slider or settings default)
-4. `TTSService.synthesize_async()` calls Modal GPU function
-5. Modal executes Kokoro TTS on GPU
-6. Audio tuple `(sample_rate, audio_array)` is returned
-7. Audio is displayed in Gradio Audio component
-
-### Dependencies
-
-Installed via `uv add --optional`:
-- `gradio-client>=1.0.0` - For STT/OCR API calls
-- `soundfile>=0.12.0` - For audio file I/O
-- `Pillow>=10.0.0` - For image processing
-
-Kokoro is installed in Modal image from source:
-- `git+https://github.com/hexgrad/kokoro.git`
-
-### GPU Types
-
-Modal supports various GPU types:
-- **T4**: Cheapest, good for testing (default)
-- **A10**: Good balance of cost/performance
-- **A100**: Fastest, most expensive
-- **L4**: NVIDIA L4 GPU
-- **L40S**: NVIDIA L40S GPU
-
-**Note**: GPU type is set at function definition time. Changes to `settings.tts_gpu` require app restart.
-
-### Error Handling
-
-- If Modal credentials not configured: TTS service unavailable (graceful degradation)
-- If Kokoro import fails: ConfigurationError raised
-- If synthesis fails: Returns None, logs warning, continues without audio
-- If GPU unavailable: Modal will queue or fail with clear error message
-
-### Configuration Connection
-
-1. **Settings → Implementation**: `settings.tts_voice`, `settings.tts_speed` used as defaults
-2. **UI → Implementation**: UI dropdowns/sliders passed to `research_agent()` function
-3. **Implementation → Modal**: Voice and speed passed to Modal GPU function
-4. **GPU Configuration**: Set at function definition time (requires restart to change)
-
-### Testing
-
-To test TTS:
-1. Ensure Modal credentials configured (`MODAL_TOKEN_ID`, `MODAL_TOKEN_SECRET`)
-2. Enable audio output in settings
-3. Submit a query
-4. Check audio output component for generated speech
-
-### References
-
-- [Kokoro TTS Space](https://huggingface.co/spaces/hexgrad/Kokoro-TTS) - Reference implementation
-- [Modal GPU Documentation](https://modal.com/docs/guide/gpu) - Modal GPU usage
-- [Kokoro GitHub](https://github.com/hexgrad/kokoro) - Source code
-
-
-
-
-
-
-
-
-
-
-

docs/index.md

@@ -30,8 +30,15 @@ The DETERMINATOR is a powerful generalist deep research agent system that uses i
 ## Quick Start
 
 ```bash
-# Install uv if you haven't already
-pip install uv
+# Install uv if you haven't already (recommended: standalone installer)
+# Unix/macOS/Linux:
+curl -LsSf https://astral.sh/uv/install.sh | sh
+
+# Windows (PowerShell):
+powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
+
+# Alternative: pipx install uv
+# Or: pip install uv
 
 # Sync dependencies
 uv sync

docs/overview/architecture.md

@@ -134,10 +134,11 @@ The graph orchestrator (`src/orchestrator/graph_orchestrator.py`) implements a f
 - **Research Flows**: Iterative and deep research patterns (`src/orchestrator/research_flow.py`)
 - **Graph Builder**: Graph construction utilities (`src/agent_factory/graph_builder.py`)
 - **Agents**: Pydantic AI agents (`src/agents/`, `src/agent_factory/agents.py`)
-- **Search Tools**: PubMed, ClinicalTrials.gov, Europe PMC, RAG (`src/tools/`)
+- **Search Tools**: Neo4j knowledge graph, PubMed, ClinicalTrials.gov, Europe PMC, Web search, RAG (`src/tools/`)
 - **Judge Handler**: LLM-based evidence assessment (`src/agent_factory/judges.py`)
 - **Embeddings**: Semantic search & deduplication (`src/services/embeddings.py`)
 - **Statistical Analyzer**: Modal sandbox execution (`src/services/statistical_analyzer.py`)
+- **Multimodal Processing**: Image OCR and audio STT/TTS services (`src/services/multimodal_processing.py`, `src/services/audio_processing.py`)
 - **Middleware**: State management, budget tracking, workflow coordination (`src/middleware/`)
 - **MCP Tools**: Claude Desktop integration (`src/mcp_tools.py`)
 - **Gradio UI**: Web interface with MCP server and streaming (`src/app.py`)
@@ -169,29 +170,25 @@ The system supports complex research workflows through:
 
 - **Orchestrator Factory** (`src/orchestrator_factory.py`):
   - Auto-detects mode: "advanced" if OpenAI key available, else "simple"
-  - Supports explicit mode selection: "simple", "magentic", "advanced"
+  - Supports explicit mode selection: "simple", "magentic" (alias for "advanced"), "advanced", "iterative", "deep", "auto"
   - Lazy imports for optional dependencies
 
-- **Research Modes**:
-  - `iterative`: Single research loop
-  - `deep`: Multi-section parallel research
-  - `auto`: Auto-detect based on query complexity
+- **Orchestrator Modes** (selected in UI or via factory):
+  - `simple`: Legacy linear search-judge loop (Free Tier)
+  - `advanced` or `magentic`: Multi-agent coordination using Microsoft Agent Framework (requires OpenAI API key)
+  - `iterative`: Knowledge-gap-driven research with single loop (Free Tier)
+  - `deep`: Parallel section-based research with planning (Free Tier)
+  - `auto`: Intelligent mode detection based on query complexity (Free Tier)
+
+- **Graph Research Modes** (used within graph orchestrator, separate from orchestrator mode):
+  - `iterative`: Single research loop pattern
+  - `deep`: Multi-section parallel research pattern
+  - `auto`: Auto-detect pattern based on query complexity
 
 - **Execution Modes**:
   - `use_graph=True`: Graph-based execution (parallel, conditional routing)
   - `use_graph=False`: Agent chains (sequential, backward compatible)
 
-
-
-
-
-
-
-
-
-
-
-
-
+**Note**: The UI provides separate controls for orchestrator mode and graph research mode. When using graph-based orchestrators (iterative/deep/auto), the graph research mode determines the specific pattern used within the graph execution.
 
 

docs/overview/features.md

@@ -7,6 +7,7 @@ The DETERMINATOR provides a comprehensive set of features for AI-assisted resear
 ### Multi-Source Search
 
 - **General Web Search**: Search general knowledge sources for any domain
+- **Neo4j Knowledge Graph**: Search structured knowledge graph for papers and disease relationships
 - **PubMed**: Search peer-reviewed biomedical literature via NCBI E-utilities (automatically used when medical knowledge needed)
 - **ClinicalTrials.gov**: Search interventional clinical trials (automatically used when medical knowledge needed)
 - **Europe PMC**: Search preprints and peer-reviewed articles (includes bioRxiv/medRxiv)
@@ -21,9 +22,11 @@ The DETERMINATOR provides a comprehensive set of features for AI-assisted resear
 
 ### Authentication
 
-- **HuggingFace OAuth**: Sign in with HuggingFace account for automatic API token usage
-- **Manual API Keys**: Support for OpenAI, Anthropic, and HuggingFace API keys
-- **Free Tier Support**: Automatic fallback to HuggingFace Inference API
+- **REQUIRED**: Authentication is mandatory before using the application
+- **HuggingFace OAuth**: Sign in with HuggingFace account for automatic API token usage (recommended)
+- **Manual API Keys**: Support for HuggingFace API keys via environment variables (`HF_TOKEN` or `HUGGINGFACE_API_KEY`)
+- **Free Tier Support**: Automatic fallback to HuggingFace Inference API (public models) when no API key is available
+- **Authentication Check**: The application will display an error message if authentication is not provided
 
 ### Secure Code Execution
 
@@ -44,9 +47,25 @@ The DETERMINATOR provides a comprehensive set of features for AI-assisted resear
 - **Parallel Research Loops**: Run multiple research tasks concurrently
 - **Iterative Research**: Single-loop research with search-judge-synthesize cycles that continues until precise answers are found
 - **Deep Research**: Multi-section parallel research with planning and synthesis
-- **Magentic Orchestration**: Multi-agent coordination using Microsoft Agent Framework
+- **Magentic Orchestration**: Multi-agent coordination using Microsoft Agent Framework (alias: "advanced" mode)
 - **Stops at Nothing**: Only stops at configured limits (budget, time, iterations), otherwise continues until finding precise answers
 
+**Orchestrator Modes**:
+- `simple`: Legacy linear search-judge loop
+- `advanced` (or `magentic`): Multi-agent coordination (requires OpenAI API key)
+- `iterative`: Knowledge-gap-driven research with single loop
+- `deep`: Parallel section-based research with planning
+- `auto`: Intelligent mode detection based on query complexity
+
+**Graph Research Modes** (used within graph orchestrator):
+- `iterative`: Single research loop pattern
+- `deep`: Multi-section parallel research pattern
+- `auto`: Auto-detect pattern based on query complexity
+
+**Execution Modes**:
+- `use_graph=True`: Graph-based execution with parallel and conditional routing
+- `use_graph=False`: Agent chains with sequential execution (backward compatible)
+
 ### Real-Time Streaming
 
 - **Event Streaming**: Real-time updates via `AsyncGenerator[AgentEvent]`
@@ -67,6 +86,16 @@ The DETERMINATOR provides a comprehensive set of features for AI-assisted resear
 - **Conversation History**: Track iteration history and agent interactions
 - **State Synchronization**: Share evidence across parallel loops
 
+### Multimodal Input & Output
+
+- **Image Input (OCR)**: Upload images and extract text using optical character recognition
+- **Audio Input (STT)**: Record or upload audio files and transcribe to text using speech-to-text
+- **Audio Output (TTS)**: Generate audio responses with text-to-speech synthesis
+- **Configurable Settings**: Enable/disable multimodal features via sidebar settings
+- **Voice Selection**: Choose from multiple TTS voices (American English: af_*, am_*)
+- **Speech Speed Control**: Adjust TTS speech speed (0.5x to 2.0x)
+- **Multimodal Processing Service**: Integrated service for processing images and audio files
+
 ## Advanced Features
 
 ### Agent System
@@ -108,10 +137,12 @@ The DETERMINATOR provides a comprehensive set of features for AI-assisted resear
 
 ### Gradio Interface
 
-- **Real-Time Chat**: Interactive chat interface
+- **Real-Time Chat**: Interactive chat interface with multimodal support
 - **Streaming Updates**: Live progress updates
 - **Accordion UI**: Organized display of pending/done operations
 - **OAuth Integration**: Seamless HuggingFace authentication
+- **Multimodal Input**: Support for text, images, and audio input in the same interface
+- **Sidebar Settings**: Configuration accordions for research, multimodal, and audio settings
 
 ### MCP Server
 
@@ -136,17 +167,3 @@ The DETERMINATOR provides a comprehensive set of features for AI-assisted resear
 - **Architecture Diagrams**: Visual architecture documentation
 - **API Reference**: Complete API documentation
 
-
-
-
-
-
-
-
-
-
-
-
-
-
-

docs/overview/quick-start.md

@@ -5,8 +5,15 @@ Get started with DeepCritical in minutes.
 ## Installation
 
 ```bash
-# Install uv if you haven't already
-pip install uv
+# Install uv if you haven't already (recommended: standalone installer)
+# Unix/macOS/Linux:
+curl -LsSf https://astral.sh/uv/install.sh | sh
+
+# Windows (PowerShell):
+powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
+
+# Alternative: pipx install uv
+# Or: pip install uv
 
 # Sync dependencies
 uv sync
@@ -23,21 +30,26 @@ Open your browser to `http://localhost:7860`.
 
 ## Basic Usage
 
-### 1. Authentication (Optional)
+### 1. Authentication (REQUIRED)
+
+**Authentication is mandatory** - you must authenticate before using the application. The app will display an error message if you try to use it without authentication.
 
-**HuggingFace OAuth Login**:
+**HuggingFace OAuth Login** (Recommended):
 - Click the "Sign in with HuggingFace" button at the top of the app
 - Your HuggingFace API token will be automatically used for AI inference
 - No need to manually enter API keys when logged in
 
-**Manual API Key (BYOK)**:
-- Provide your own API key in the Settings accordion
-- Supports HuggingFace, OpenAI, or Anthropic API keys
-- Manual keys take priority over OAuth tokens
+**Manual API Key** (Alternative):
+- Set environment variable `HF_TOKEN` or `HUGGINGFACE_API_KEY` before starting the app
+- The app will automatically use these tokens if OAuth login is not available
+- Supports HuggingFace API keys only (OpenAI/Anthropic keys are not used in the current implementation)
 
 ### 2. Start a Research Query
 
 1. Enter your research question in the chat interface
+   - **Text Input**: Type your question directly
+   - **Image Input**: Click the 📷 icon to upload images (OCR will extract text)
+   - **Audio Input**: Click the 🎤 icon to record or upload audio (STT will transcribe to text)
 2. Click "Submit" or press Enter
 3. Watch the real-time progress as the system:
    - Generates observations
@@ -46,6 +58,12 @@ Open your browser to `http://localhost:7860`.
    - Evaluates evidence
    - Synthesizes findings
 4. Review the final research report
+   - **Audio Output**: If enabled, the final response will include audio synthesis (TTS)
+
+**Multimodal Features**:
+- Configure image/audio input and output in the sidebar settings
+- Image OCR and audio STT/TTS can be enabled/disabled independently
+- TTS voice and speed can be customized in the Audio Output settings
 
 ### 3. MCP Integration (Optional)
 
@@ -70,9 +88,12 @@ Connect DeepCritical to Claude Desktop:
 - `search_pubmed`: Search peer-reviewed biomedical literature
 - `search_clinical_trials`: Search ClinicalTrials.gov
 - `search_biorxiv`: Search bioRxiv/medRxiv preprints
+- `search_neo4j`: Search Neo4j knowledge graph for papers and disease relationships
 - `search_all`: Search all sources simultaneously
 - `analyze_hypothesis`: Secure statistical analysis using Modal sandboxes
 
+**Note**: The application automatically uses all available search tools (Neo4j, PubMed, ClinicalTrials.gov, Europe PMC, Web search, RAG) based on query analysis. Neo4j knowledge graph search is included by default for biomedical queries.
+
 ## Next Steps
 
 - Read the [Installation Guide](../getting-started/installation.md) for detailed setup

mkdocs.yml

@@ -88,7 +88,7 @@ nav:
     - getting-started/mcp-integration.md
     - getting-started/examples.md
   - Configuration:
-    - configuration/CONFIGURATION.md
+    - configuration/index.md
   - Architecture:
     - "Graph Orchestration": architecture/graph_orchestration.md
     - "Workflow Diagrams": architecture/workflow-diagrams.md

mkdocs.yml.enhanced

@@ -0,0 +1,166 @@
+site_name: The DETERMINATOR
+site_description: Generalist Deep Research Agent that Stops at Nothing
+site_author: The DeepCritical Team
+site_url: https://deepcritical.github.io/GradioDemo/
+
+repo_name: DeepCritical/GradioDemo
+repo_url: https://github.com/DeepCritical/GradioDemo
+edit_uri: edit/dev/docs/
+
+# Ensure all files are included even if not in nav
+# strict: false
+
+theme:
+  name: material
+  palette:
+    # Light mode
+    - scheme: default
+      primary: orange
+      accent: red
+      toggle:
+        icon: material/brightness-7
+        name: Switch to dark mode
+    # Dark mode
+    - scheme: slate
+      primary: orange
+      accent: red
+      toggle:
+        icon: material/brightness-4
+        name: Switch to light mode
+  features:
+    # Navigation features
+    - navigation.tabs
+    - navigation.sections
+    - navigation.expand
+    - navigation.top
+    - navigation.indexes 
+    - navigation.instant 
+    - navigation.tracking 
+    - navigation.smooth  
+    # Search features
+    - search.suggest
+    - search.highlight
+    # Content features
+    - content.code.annotate
+    - content.code.copy
+    - content.tabs.link 
+    - content.tooltips 
+    - toc.integrate 
+  icon:
+    repo: fontawesome/brands/github
+  language: en
+
+plugins:
+  - search:
+      lang:
+        - en
+      separator: '[\s\-,:!=\[\]()"`/]+|\.(?!\d)|&[lg]t;|&'
+      prebuild_index: true  # Faster search initialization
+      indexing: full  # Full-text indexing for better results
+  - mermaid2
+  - codeinclude
+  - git-revision-date-localized:
+      enable_creation_date: true
+      type: timeago  # Shows "2 days ago" format
+      fallback_to_build_date: true
+  - minify:
+      minify_html: true
+      minify_js: true
+      minify_css: true
+
+markdown_extensions:
+  - dev.docs_plugins:
+      base_path: "."
+  - pymdownx.highlight:
+      anchor_linenums: true
+      line_spans: __span  # Allow line spans for highlighting
+      pygments_lang_class: true  # Add language class to code blocks
+      use_pygments: true
+      noclasses: false  # Use CSS classes for better theming
+  - pymdownx.inlinehilite
+  - pymdownx.superfences:
+      custom_fences:
+        - name: mermaid
+          class: mermaid
+          format: !!python/name:pymdownx.superfences.fence_code_format
+      preserve_tabs: true
+  - pymdownx.tabbed:
+      alternate_style: true
+      combine_header_slug: true  # Better tab linking
+  - pymdownx.tasklist:
+      custom_checkbox: true
+  - pymdownx.emoji:
+      emoji_generator: !!python/name:pymdownx.emoji.to_svg
+      emoji_index: !!python/name:pymdownx.emoji.twemoji
+  - pymdownx.snippets
+  - admonition
+  - pymdownx.details
+  - attr_list
+  - md_in_html
+  - tables
+  - meta  # Frontmatter support for tags, categories, etc.
+  - toc:
+      permalink: true
+      permalink_title: "Anchor link to this section"
+      baselevel: 1
+      toc_depth: 3
+      slugify: !!python/object/apply:pymdownx.slugs.slugify
+        kwds:
+          case: lower
+
+nav:
+  - Home: index.md
+  - Overview:
+    - overview/architecture.md
+    - overview/features.md
+  - Getting Started:
+    - getting-started/installation.md
+    - getting-started/quick-start.md
+    - getting-started/mcp-integration.md
+    - getting-started/examples.md
+  - Configuration:
+    - configuration/index.md
+  - Architecture:
+    - "Graph Orchestration": architecture/graph_orchestration.md
+    - "Workflow Diagrams": architecture/workflow-diagrams.md
+    - "Agents": architecture/agents.md
+    - "Orchestrators": architecture/orchestrators.md
+    - "Tools": architecture/tools.md
+    - "Middleware": architecture/middleware.md
+    - "Services": architecture/services.md
+  - API Reference:
+    - api/agents.md
+    - api/tools.md
+    - api/orchestrators.md
+    - api/services.md
+    - api/models.md
+  - Contributing:
+    - contributing/index.md
+    - contributing/code-quality.md
+    - contributing/code-style.md
+    - contributing/error-handling.md
+    - contributing/implementation-patterns.md
+    - contributing/prompt-engineering.md
+    - contributing/testing.md
+  - License: LICENSE.md
+  - Team: team.md
+
+extra:
+  social:
+    - icon: fontawesome/brands/github
+      link: https://github.com/DeepCritical/GradioDemo
+      name: GitHub
+    - icon: fontawesome/brands/twitter
+      link: https://twitter.com/josephpollack
+      name: Twitter
+    - icon: material/web
+      link: https://huggingface.co/spaces/DataQuests/DeepCritical
+      name: Live Demo
+    - icon: fontawesome/brands/discord
+      link: https://discord.gg/n8ytYeh25n
+      name: Discord
+  generator:
+    enabled: false  # Hide generator meta tag
+
+copyright: Copyright © 2024 DeepCritical Team
+