A multi-agent AI system that analyzes stock analyst recommendations using specialized AI agents and LLM orchestration.
Team Project for Agentic AI, The Data Economy & Fintech
- Overview
- Key Features
- Quick Start
- System Architecture
- Detailed Setup
- Usage Guide
- Troubleshooting
- Project Structure
- Documentation
- Testing
This system leverages specialized AI agents to provide two distinct analytical capabilities for stock recommendations:
Both Explainer and Recommender follow this multi-agent pattern:
┌─────────────────┐
│ Manager Agent │
│ (Synthesizes) │
└────────┬────────┘
│
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Fundamental │ │ Technical │ │ News │
│ Analyst │ │ Analyst │ │ Analyst │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
▼ ▼ ▼
Financial Data Price/Volume Data News Headlines
Execution Flow:
- Data Extraction - Each analyst receives domain-specific filtered data
- Parallel Analysis - Analysts independently evaluate their data sources
- Synthesis - Manager reads all reports and creates final output
- Result Delivery - Comprehensive report with reasoning and confidence
Goal: Understand why a human analyst made their recommendation
Given a historical analyst recommendation (e.g., "SELL" for AMZN on 2008-01-08), the system:
- Analyzes historical data from the period before the recommendation
- Deploys three specialist analysts (Fundamental, Technical, News)
- Synthesizes findings to explain the analyst's reasoning
- Provides confidence assessment and key signal identification
Goal: Generate an independent AI-driven recommendation
For the same stock and date:
- Three specialist analysts independently evaluate the situation
- Each provides their own rating with confidence levels
- A Portfolio Manager intelligently synthesizes (not just votes!) the ratings
- Outputs a final recommendation with detailed reasoning
Key Distinction: Explainer interprets human decisions; Recommender makes independent decisions.
- ✅ Multi-Agent Architecture - Specialized agents with distinct expertise domains
- ✅ Real Financial Data - IBES analyst recommendations, fundamental metrics, technical indicators, news sentiment
- ✅ LLM-Based Synthesis - Intelligent decision-making that considers confidence levels and market context
- ✅ Transparent Reasoning - Full visibility into each agent's analysis and the final synthesis
- ✅ Graceful Degradation - Handles missing or incomplete data appropriately
- ✅ Modern Web Interface - Responsive React UI with real-time status updates
Tech Stack:
- Backend: FastAPI, CrewAI, Google Gemini, Pandas
- Frontend: React, TypeScript, Vite, Tailwind CSS, Shadcn UI
- Data: Feather format for efficient DataFrame storage
Before proceeding, ensure you have:
- Python 3.10 or higher (Download)
- Node.js 18+ and npm (Download)
- Google Gemini API key (Get one free)
- Git (optional, for cloning)
Verify installations:
python --version # Should show 3.10 or higher
node --version # Should show 18 or higher
npm --version # Should show 9 or higher-
Get the code:
git clone <repository-url> cd agentics-project
-
Set up Python environment:
# Create and activate virtual environment python -m venv venv # Windows venv\Scripts\activate # Mac/Linux source venv/bin/activate # Install dependencies pip install --upgrade pip pip install -r requirements.txt
-
Configure API key:
Create a
.envfile in the project root:# Windows PowerShell New-Item -Path .env -ItemType File # Mac/Linux touch .env
Add your API key to
.env:GEMINI_API_KEY=your_actual_api_key_here -
Launch the application:
python run.py
Wait ~30-60 seconds for the system to start. You'll see output like:
✓ Loaded 7804 IBES recommendations ✓ Loaded 128317 FUND rows ✓ Loaded 141418 NEWS items SUCCESS: Backend is ready! VITE v5.4.19 ready in 2236 ms ➜ Local: http://localhost:8080/Open the Local URL shown in your terminal (usually
http://localhost:8080/orhttp://localhost:5173/)
That's it! The script handles frontend dependency installation automatically.
Note: The frontend port may vary (5173, 8080, 5174, etc.). Always use the URL shown in your terminal output under "Local:"
For detailed technical architecture and implementation details, see:
- Explainer Team Documentation - Data flow, agent prompts, synthesis logic
- Recommender Team Documentation - Rating methodology, weighting strategy
Virtual environments isolate project dependencies and prevent conflicts.
Windows:
# Create environment
python -m venv venv
# Activate
venv\Scripts\activate
# You'll see (venv) in your promptMac/Linux:
# Create environment
python3 -m venv venv
# Activate
source venv/bin/activate
# You'll see (venv) in your promptInstall packages:
python -m pip install --upgrade pip
python -m pip install -r requirements.txtpip install --upgrade pip
pip install -r requirements.txtThe application requires a Google Gemini API key for LLM access.
Step 1: Obtain API Key
- Visit Google AI Studio
- Sign in with your Google account
- Click "Create API Key"
- Copy the generated key
Step 2: Create Configuration File
Create .env in the project root (same directory as run.py):
# Unix-based systems (Mac/Linux)
touch .env
# Windows PowerShell
New-Item -Path .env -ItemType File
# Windows Command Prompt
echo. > .envStep 3: Add API Key
Open .env in any text editor and add:
GEMINI_API_KEY=your_actual_api_key_here
Replace your_actual_api_key_here with your actual key.
🔒 Security Note: The .env file is in .gitignore and will never be committed to version control.
Test that everything is configured correctly:
# Test Explainer team
python tests/test_explainer.py
# Test Recommender team
python tests/test_recommender.pyBoth tests should complete in 60-90 seconds without errors.
Standard Launch (Recommended):
python run.pyThis single command:
- Checks Node.js installation
- Installs frontend dependencies (first run only)
- Starts the backend API (port 8000)
- Starts the frontend dev server (port 5173)
- Shows you the access URLs
Manual Launch (Alternative):
If you prefer to run components separately or run.py doesn't work:
# Terminal 1 - Backend
python start_backend.py
# Terminal 2 - Frontend
cd frontend/insight-agent
npm install # First time only
npm run devImportant: Always check the terminal output for the actual URLs. The frontend URL may be different from the defaults shown here.
Open your browser to the Local URL shown in the terminal (e.g., http://localhost:8080/).
Once the app is loaded:
1. Select Analysis Mode
- Explainer: Understand why a human analyst gave their rating
- Recommender: Get an independent AI-generated rating
2. Choose Stock and Date
- Select ticker from dropdown (default: AMZN)
- Pick a recommendation date (grouped by year)
- Explainer shows: date + rating (e.g., "Jan 25 - BUY")
- Recommender shows: date only (to avoid bias)
3. Adjust Time Windows (Optional)
- FUND Window: Days of historical fundamental data (default: 30)
- NEWS Window: Days of news to analyze (default: 7 for Explainer, 30 for Recommender)
4. Run Analysis
- Click "Run Explainer Team" or "Run Recommender Team"
- Wait 30-90 seconds for AI processing
- Progress indicators show current agent activity
5. Review Results
- Explainer: Comprehensive explanation of analyst reasoning
- Recommender: AI rating vs. human rating (click to reveal) + detailed reasoning
- Expand "View detailed work from the 3 analysts" for individual reports
For programmatic access or integration:
Backend API: http://localhost:8000
API Documentation: http://localhost:8000/docs (interactive Swagger UI)
Key Endpoints:
GET /tickers- List available stock tickersGET /recommendations/{ticker}- Get recommendation dates for a tickerPOST /explainer- Run Explainer analysisPOST /recommender- Run Recommender analysisGET /job/{job_id}- Check analysis job status
Explainer Report Structure:
- Key Signals: Most important indicators from each analyst
- Consistency Check: How well the signals align with the rating
- Confidence Assessment: How certain we are about the explanation
- Individual Reports: Full analysis from each specialist
Recommender Report Structure:
- Final Rating: AI's recommendation (StrongBuy/Buy/Hold/UnderPerform/Sell)
- Rating Comparison: AI vs. Human (after reveal)
- Synthesis Rationale: Why the Portfolio Manager chose this rating
- Confidence Levels: How certain each analyst was
- Individual Reports: Full analysis from each specialist
"python: command not found" or "python is not recognized"
-
Windows:
- Reinstall Python and check "Add Python to PATH"
- Try
pyinstead:py -m pip install -r requirements.txt - Use full path:
C:\Python310\python.exe
-
Mac/Linux:
- Try
python3instead:python3 -m pip install -r requirements.txt - Install via homebrew:
brew install python3
- Try
"No module named 'xyz'"
- Ensure virtual environment is activated (look for
(venv)in prompt) - Reinstall dependencies:
pip install -r requirements.txt - Upgrade pip first:
pip install --upgrade pip
"node is not recognized" or "npm is not recognized"
- Install Node.js from nodejs.org (LTS version)
- Close and reopen your terminal (very important!)
- Verify:
node --versionandnpm --version - If still failing, restart your computer
Windows-specific:
- Check if Node.js is in PATH:
C:\Program Files\nodejs - Close and reopen VS Code if using its terminal
"No Gemini API key found"
- Verify
.envexists in project root (same folder asrun.py) - Check filename is exactly
.env(not.env.txt) - Ensure content is:
GEMINI_API_KEY=your_key(no spaces around=) - Virtual environment must be activated when running
Windows Note: Notepad may save as .env.txt. Use "Save As" → "All Files" type.
"API rate limit exceeded"
The free Gemini API has rate limits. Wait 1-2 minutes and retry.
"Port 8000 already in use"
Find and kill the process:
# Windows
netstat -ano | findstr :8000
taskkill /PID <PID_NUMBER> /F
# Mac/Linux
lsof -ti:8000 | xargs kill -9"Port 5173 already in use"
Vite automatically uses the next available port (5174, 5175, etc.). Check terminal output for the actual URL.
"FileNotFoundError: data/..."
You're not in the project root directory. Navigate there:
# Check current directory
pwd # Mac/Linux
Get-Location # Windows PowerShell
# Should end with your project folder name
cd /path/to/agentics-project"venv: command not found" (Mac/Linux)
Try: python3 -m venv venv
PowerShell execution policy error (Windows)
Run PowerShell as Administrator:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUserOr use Command Prompt instead of PowerShell.
"Analysis is taking too long"
This is normal! Each analysis:
- Takes 60-90 seconds (LLM inference is slow)
- First run loads datasets (adds 10-20 seconds)
- Involves 4 agents making sequential LLM calls
Be patient and let it complete.
- Check error messages carefully - they usually indicate the problem
- Review this README's troubleshooting section
- Read the technical documentation in
Docs/ - Search for the error message online
- Verify all prerequisites are correctly installed
agentics-project/
│
├── backend/ # FastAPI backend server
│ ├── api/ # API route handlers
│ │ ├── explainer.py # Explainer endpoint
│ │ ├── recommender.py # Recommender endpoint
│ │ └── tickers.py # Data retrieval endpoints
│ ├── datasets.py # Data loading and caching
│ ├── utils.py # Helper utilities
│ └── main.py # FastAPI application entry
│
├── src/ # Core agent orchestration logic
│ ├── explainer/ # Explainer team implementation
│ │ ├── agents.py # 4 agent definitions + prompts
│ │ ├── tasks.py # CrewAI task definitions
│ │ └── orchestrator.py # Execution coordinator
│ └── recommender/ # Recommender team implementation
│ ├── agents.py # 4 agent definitions + prompts
│ ├── tasks.py # CrewAI task definitions
│ └── orchestrator.py # Execution coordinator
│
├── frontend/ # React web interface
│ └── insight-agent/ # Vite + React + TypeScript app
│ ├── src/
│ │ ├── components/ # React components
│ │ ├── lib/ # Utilities
│ │ └── App.tsx # Main application
│ ├── package.json # Node dependencies
│ └── vite.config.ts # Vite configuration
│
├── data/ # Stock market datasets
│ ├── ibes_dj30_stock_rec_2008_24.feather # Analyst recommendations
│ ├── fund_tech_dj30_stocks_2008_24.feather # Fundamental + technical data
│ └── ciq_dj30_stock_news_2008_24.feather # News headlines + sentiment
│
├── Docs/ # Technical documentation
│ ├── Explainer_Team.md # Explainer architecture deep dive
│ └── Recommender_Team.md # Recommender architecture deep dive
│
├── tests/ # Test scripts
│ ├── test_explainer.py # Explainer integration test
│ └── test_recommender.py # Recommender integration test
│
├── run.py # One-command launcher (recommended)
├── start_backend.py # Backend server script
├── requirements.txt # Python dependencies
├── .env # API keys (create this, not in repo)
├── .gitignore # Git exclusions
└── README.md # This file
- README.md - Setup guide, usage instructions, troubleshooting (this file)
- Docs/Explainer_Team.md - Technical deep dive: architecture, data flow, prompts, synthesis logic
- Docs/Recommender_Team.md - Technical deep dive: rating methodology, weighting strategy, decision-making
Verify the complete agent pipeline:
# Test Explainer workflow
python tests/test_explainer.py
# Test Recommender workflow
python tests/test_recommender.pyExpected results:
- No errors or exceptions
- Completion in 60-90 seconds
- Output showing agent reasoning and final report
Use the web interface to test various scenarios:
- Different stocks: Try multiple tickers to see varied analyses
- Different time periods: Test dates across different market conditions
- Window adjustments: Modify FUND and NEWS windows to see impact
- Edge cases: Try dates with limited data availability
The evaluation system allows you to systematically assess the quality of Explainer outputs through human evaluation. The system prioritizes recommendations with complete data (fewest NAs, good coverage across fundamentals, technicals, and news) to ensure high-quality evaluation samples.
The evaluation process consists of three steps:
- Sampling: Generate a CSV of high-quality recommendations with Explainer outputs
- Human Rating: Teammates fill in rating columns in the CSV
- Aggregation: Compute summary metrics from the ratings
Run the sampling script to generate evaluation samples:
python tests/eval_explainer_sample.py --n_samples 20 --output_path evaluation/explainer_human_study_samples.csvWhat this does:
- Loads all IBES recommendations and computes a data completeness score for each
- The score combines:
fund_non_null_ratio: Ratio of non-null fundamental columns (EPS, ROE, leverage, cash flow, etc.)tech_non_null_ratio: Ratio of non-null technical columns (RSI, MACD, returns, volume, etc.)has_news_indicator: 1 if at least one news item exists in the window, else 0- Total score = fund_ratio + tech_ratio + news_indicator (max 3.0)
- Selects the top N recommendations by data completeness score
- Runs the Explainer on each selected recommendation
- Exports a CSV with:
- Recommendation details (ticker, company, date, rating)
- Data completeness metrics
- Explainer output (manager markdown + individual analyst reports)
- Empty columns for human ratings
Key Options:
--n_samples: Number of samples to generate (default: 20)--output_path: Where to save the CSV (default:evaluation/explainer_human_study_samples.csv)--fund_window_days: Fundamental/technical data window (default: 90)--news_window_days: News data window (default: 30)--max_candidates: Limit candidate pool for faster scoring (default: 1000, use 0 for all)
Sampling Strategy:
The script prioritizes recommendations where:
- Most fundamental columns are non-null (good coverage of financial metrics)
- Most technical columns are non-null (good coverage of price/volume indicators)
- At least some news exists in the time window
This ensures evaluation focuses on cases where the Explainer has rich data to work with, rather than sparse or missing data scenarios.
Open the generated CSV and fill in the rating columns for each sample:
1-5 Scale Ratings:
-
plausibility_1_5: "Does this explanation feel like something a real sell-side analyst could plausibly have written as the reasoning behind the rating?"- 1 = not at all
- 5 = extremely plausible
-
signal_coverage_1_5: "Given the data shown to each agent (fundamental, technical, news), does the final explanation actually reference the most important signals?"- 1 = major signals missing / made up
- 5 = covers key signals correctly
-
internal_consistency_1_5: "Does the explainer's conclusion (e.g. why they rated SELL) follow logically from the signals it described?"- 1 = contradicts itself
- 5 = strongly consistent
Yes/No/NA Checklist:
mentions_fundamental: If fundamental data exists, did the explanation talk about it? (yes/no/na)mentions_technical: If technical data exists, did the explanation talk about it? (yes/no/na)mentions_news: If news exists, did the explanation talk about it? (yes/no/na)calls_out_missing_data: If a modality is missing, does the explanation acknowledge that instead of hallucinating numbers? (yes/no/na)
Save the CSV with a new name (e.g., explainer_human_study_samples_rated.csv).
Run the aggregation script to compute summary metrics:
python tests/eval_explainer_aggregate.py --input_path evaluation/explainer_human_study_samples_rated.csvWhat this computes:
-
Mean Scores:
- Mean Plausibility (average of
plausibility_1_5) - Mean Signal Coverage (average of
signal_coverage_1_5) - Mean Internal Consistency (average of
internal_consistency_1_5)
- Mean Plausibility (average of
-
Modality Alignment Percentages:
- % of samples where fundamentals were available AND
mentions_fundamental == "yes" - % of samples where technicals were available AND
mentions_technical == "yes" - % of samples where news was available AND
mentions_news == "yes" - % of samples where any modality was missing AND
calls_out_missing_data == "yes"
- % of samples where fundamentals were available AND
-
Data Quality:
- Average data completeness score of the sampled set
Output:
The script prints a summary to the console and optionally writes a markdown report (default: same directory as input CSV, with .md extension).
Example Output:
Explainer Human-Study Evaluation (N = 20 samples)
Mean Scores:
Mean Plausibility: 4.1 / 5
Mean Signal Coverage: 3.8 / 5
Mean Internal Consistency: 4.3 / 5
Modality Alignment:
Mentions fundamentals when available: 85%
Mentions technicals when available: 90%
Mentions news when available: 75%
Explicitly calls out missing data: 92%
Average data completeness score (on sampled set): 2.7 / 3.0
# 1. Generate high-quality samples
python tests/eval_explainer_sample.py --n_samples 20 --output_path evaluation/explainer_human_study_samples.csv
# 2. Teammates fill in ratings in the CSV
# 3. Aggregate results
python tests/eval_explainer_aggregate.py --input_path evaluation/explainer_human_study_samples_rated.csvThe backtest evaluation answers a simple question: "If we had actually traded on the Recommender's rating on each date, how would we have performed compared to human analysts and a baseline?"
The backtest system:
- Runs the Recommender on historical IBES recommendations
- Records for each (ticker, date):
- The model's rating (StrongBuy/Buy/Hold/UnderPerform/Sell)
- The human analyst's rating from IBES
- The future 1-month and 3-month returns of the stock
- Simulates trading strategies based on:
- Recommender rating
- Human rating
- A buy-and-hold baseline (always long)
- Computes summary metrics:
- Directional accuracy (did rating get the sign of future return right?)
- Average trade return
- Cumulative strategy return vs baseline
The backtest uses a simple, fully specified trading rule:
- StrongBuy / Buy → go long (+1)
- Sell / UnderPerform → go short (-1)
- Hold → no position (0)
For each rating, the system computes:
- 1-month forward return (approximately 21 trading days after rec_date)
- 3-month forward return (approximately 63 trading days after rec_date)
P&L per trade:
- If signal ∈ {+1, -1}, P&L = signal × future_return
- If signal = 0 (Hold), P&L = 0 (flat position)
Run the backtest script to generate trading data:
python tests/eval_recommender_backtest.py \
--max_samples 300 \
--output_path evaluation/recommender_backtest_trades.csvWhat this does:
- Samples historical IBES recommendations (randomly or sequentially)
- For each recommendation:
- Runs the full Recommender pipeline
- Extracts the model's final rating
- Gets the human analyst's rating from IBES
- Computes future 1M and 3M returns using adjusted prices from the FUND dataset
- Maps ratings to trading signals
- Computes P&L for model, human, and baseline strategies
- Exports a CSV with all trade data
Key Options:
--max_samples: Number of recommendations to process (default: 300)--output_path: Where to save the CSV (default:evaluation/recommender_backtest_trades.csv)--news_window_days: News window for Recommender (default: 30)--random_seed: Random seed for sampling (default: 42)
Output CSV columns:
rec_index,ticker,rec_datehuman_raw_rating,model_raw_ratinghuman_signal,model_signal,baseline_signalreturn_1m,return_3mmodel_pnl_1m,human_pnl_1m,baseline_pnl_1mmodel_pnl_3m,human_pnl_3m,baseline_pnl_3mmodel_dir_correct_1m,human_dir_correct_1mmodel_dir_correct_3m,human_dir_correct_3m
Run the aggregation script to compute summary metrics:
python tests/eval_recommender_backtest_aggregate.py \
--input_path evaluation/recommender_backtest_trades.csvWhat this computes:
- Directional Accuracy (for non-Hold trades):
- % of trades where signal direction matched return direction
- Computed separately for model, human, and baseline
- Average P&L per trade:
- Mean P&L for model, human, and baseline strategies
- Computed for both 1M and 3M horizons
- Cumulative P&L:
- Sum of all P&L for each strategy
- Shows total strategy performance
- Model vs Human comparison:
- Count of cases where model was correct and human was wrong
- Count of cases where human was correct and model was wrong
Output:
The script prints a summary to the console and optionally writes a markdown report (default: same directory as input CSV, with .md extension).
Example Output:
RECOMMENDER BACKTEST: Aggregating Results
1-MONTH HORIZON
Directional Accuracy (non-Hold trades only):
Model: 62.0% (N=250)
Human: 58.0% (N=280)
Baseline: 52.0% (N=300)
Average P&L per trade:
Model: +1.80%
Human: +1.20%
Baseline: +0.90%
Cumulative P&L:
Model: +450.00%
Human: +336.00%
Baseline: +270.00%
# 1. Generate backtest dataset
python tests/eval_recommender_backtest.py --max_samples 300 --output_path evaluation/recommender_backtest_trades.csv
# 2. Aggregate results
python tests/eval_recommender_backtest_aggregate.py --input_path evaluation/recommender_backtest_trades.csvNote: The backtest uses adjusted prices from the FUND dataset and handles missing future prices gracefully (skips trades where future prices are unavailable).
Security:
- Never commit
.envfile (already in.gitignore) - Never commit API keys in code or documentation
- Review
.gitignorebefore committing large files
Data Files:
- Dataset files are in
.gitignore(too large for repo) - Datasets are cached after first load (10-20 second startup time)
Code Style:
- Python: Follow PEP 8 guidelines
- TypeScript: ESLint configuration in frontend
- Use type hints in Python code
- Use TypeScript interfaces for data structures
Performance Considerations:
- Each LLM call takes 5-15 seconds
- 4 agents per analysis = ~60-90 seconds total
- Consider batch processing for multiple analyses
- Datasets are loaded once at startup, then cached in memory
Why CrewAI?
- Provides agent orchestration framework
- Handles task sequencing and memory management
- Integrates well with various LLM providers
Why Google Gemini?
- Free tier for development
- Good performance for analytical tasks
- Reliable API availability
Why FastAPI?
- Modern async Python framework
- Automatic API documentation
- Excellent performance for I/O-bound tasks
Why React + Vite?
- Fast development experience
- Modern build tooling
- Great TypeScript support
For issues, questions, or contributions:
- Review this README and troubleshooting section
- Check the technical documentation in
Docs/ - Search existing issues (if using GitHub)
- Create a new issue with detailed error information