TerraMind — Data Analysis & ML Pipeline

Longitudinal Agronomy & Soil Memory | Cognee Hackathon 2026

"Digital Twin" memory graph for agricultural land — Using Cognee's graph RAG to connect multi-year timelines of weather, chemical inputs, and soil data to explain crop yield anomalies.

---

🌾 Problem Statement

Agriculture operates on multi-year cycles. A crop failure today isn't just about today's weather — it's a cascading result of the fertilizer used in 2024, the cover crop planted in 2025, and the specific seed variant bought in 2026. No AI can currently remember or connect a 3-year timeline of physical actions to a current soil anomaly.

💡 Our Solution

TerraMind uses Cognee's hybrid graph-vector memory layer to build a temporal knowledge graph for agricultural plots. When a farmer asks "Why did Field B's yield drop by 20%?", Cognee executes multi-hop graph traversal to connect hidden relationships across years.

---

📊 Datasets

File	Records	Description
crop_yield.csv	~19,690	Crop yields by state, year, season with fertilizer/pesticide data
state_soil_data.csv	31	Soil nutrients (N, P, K, pH) by Indian state
state_weather_data_1997_2020.csv	720	Temperature, rainfall, humidity by state & year

Synthetic Datasets (Generated)

File	Description
field_notes.json	500+ synthetic farmer field notes across 5 farms
chemical_logs.csv	Chemical application records with causal patterns
weather_events.json	Extreme weather event records
seed_data.json	Farm/plot definitions with demo scenarios

---

🧠 Pipeline Architecture

┌─────────────────────────────────────────────────────────────┐
│                    DATA SOURCES                              │
│  crop_yield.csv │ weather.csv │ soil.csv │ field notes       │
└────────┬────────┴──────┬──────┴────┬─────┴──────┬───────────┘
         │               │           │            │
         ▼               ▼           ▼            ▼
┌─────────────────────────────────────────────────────────────┐
│              DATA ANALYSIS (EDA)                             │
│  Trends │ Correlations │ Anomaly Detection │ Feature Import. │
└────────┬────────────────────────────────────────┬───────────┘
         │                                        │
         ▼                                        ▼
┌───────────────────────┐    ┌────────────────────────────────┐
│     ML MODELS         │    │   COGNEE KNOWLEDGE GRAPH       │
│  • Yield Predictor    │    │  • Temporal edges              │
│  • Anomaly Detector   │    │  • Causal chains               │
│  • Pattern Miner      │    │  • Multi-hop traversal         │
└───────────────────────┘    └────────────────────────────────┘
         │                                        │
         ▼                                        ▼
┌─────────────────────────────────────────────────────────────┐
│              QUERY ENGINE                                    │
│  "Why did Field B's yield drop by 20%?"                      │
│   → Graph traversal → Evidence trail → Confidence label      │
└─────────────────────────────────────────────────────────────┘

---

🚀 Quick Start

1. Install Dependencies

pip install -r requirements_ml.txt

2. Run Data Analysis

python data_analysis/eda_crop_yield.py
python data_analysis/eda_weather_soil.py
python data_analysis/merged_analysis.py

3. Train ML Models

python ml_models/yield_predictor.py
python ml_models/anomaly_detector.py
python ml_models/temporal_pattern_miner.py

4. Generate Synthetic Data

python data_generation/generate_field_notes.py
python data_generation/generate_chemical_logs.py
python data_generation/generate_weather_events.py

5. Build Knowledge Graph & Run Queries

python cognee_pipeline/ingest_data.py --mode offline
python cognee_pipeline/query_examples.py

6. Run Backend Web API (FastAPI)

Configure your .env file inside the backend directory (see backend/.env.example as a template), then start the dev server:

cd backend
source venv/bin/activate
python -m uvicorn app.main:app --reload --port 8000

7. Run Frontend Dashboard (Next.js)

Install node modules and start the development server:

cd frontend
npm install
npm run dev

Open http://localhost:3000 in your browser.

---

📈 Key Findings

ML Model Performance

• Yield Prediction: R² > 0.85 using Random Forest + Gradient Boosting ensemble
• Anomaly Detection: Identifies yield drops with causal attribution
• Temporal Patterns: Discovers 1-3 year lag effects of chemical inputs on yields

Hidden Relationships Discovered

1. Pesticide + Drought → Yield Crash: High-dose Chlorpyrifos in 2024 + 2025 drought = 30% yield drop
2. Ammonium Sulphate Overuse → pH Decline: Repeated nitrogen fertilizer without lime = progressive soil acidification
3. Fertilizer Reduction + Heatwave: 60% fertilizer cut in 2025 + 48°C heatwave in 2026 = 40% cotton yield crash

---

📁 Project Structure

├── crop_yield.csv                    # Raw crop yield dataset
├── state_soil_data.csv               # State-level soil data
├── state_weather_data_1997_2020.csv  # Weather data 1997-2020
├── requirements_ml.txt               # Python dependencies
├── README.md                         # This file
│
├── data_analysis/                    # Exploratory Data Analysis
│   ├── eda_crop_yield.py             # Crop yield analysis (7 charts)
│   ├── eda_weather_soil.py           # Weather & soil analysis (5 charts)
│   └── merged_analysis.py           # Multi-factor causal analysis
│
├── ml_models/                        # Machine Learning Models
│   ├── yield_predictor.py            # RF + GBM yield prediction
│   ├── anomaly_detector.py           # Yield anomaly detection
│   └── temporal_pattern_miner.py     # Multi-year pattern discovery
│
├── data_generation/                  # Synthetic Data Generation
│   ├── generate_field_notes.py       # Farmer field notes
│   ├── generate_chemical_logs.py     # Chemical application records
│   ├── generate_weather_events.py    # Extreme weather events
│   └── generated/                    # Generated data files
│
├── cognee_pipeline/                  # Cognee Integration
│   ├── ingest_data.py                # Data ingestion pipeline
│   └── query_examples.py            # Demo query showcase
│
└── outputs/                          # Generated Outputs
    ├── charts/                       # Visualization charts (19 PNGs)
    ├── models/                       # Trained ML models (.joblib)
    ├── reports/                      # Analysis reports (.md, .json)
    └── cognee_graph/                 # Knowledge graph JSON files

---

🏆 Why This Project Wins

1. AgriTech + Graph RAG: Massively underrepresented at AI hackathons
2. Temporal Reasoning: Demonstrates Cognee's multi-hop traversal across years
3. Real + Synthetic Data: Comprehensive data pipeline from raw CSVs to knowledge graph
4. Production-Ready: Full ML pipeline with models, evaluation, and demo queries
5. Compound Causal Chains: Discovers non-obvious multi-factor relationships

---

🛠️ Tech Stack

• Data: pandas, numpy
• ML: scikit-learn, XGBoost
• Visualization: matplotlib, seaborn
• Knowledge Graph: Cognee (graph RAG + vector search)
• Backend: FastAPI, PostgreSQL, Neo4j, Qdrant, Redis
• Frontend: Next.js

---

Built for the Cognee Hackathon 2026 by Team TerraMind

Interactive graph and sample records

The knowledge graph supports animated layout, mouse/touch node dragging, background panning, anchored zoom, fit-to-view, keyboard selection and movement, search, entity type and date filters, confirmed-link filtering, and JSON export. Reduced-motion preferences are respected. Live graphs refresh every 10 seconds and after document status changes, and discard responses from previously selected fields.

The local demo includes 18 additional synthetic CSV documents (72 data rows) covering soil, crops, inputs, weather, irrigation, and yields across all three fields. Open Ingest & Review → Sample document library to download the originals. Signed-in users can import a pack of six into their selected field; ordinary upload also accepts multiple files. Imports use the normal authenticated upload pipeline, and repeated identical files are deduplicated within that field. Synthetic examples are labeled in every CSV row and are not farming recommendations.

Saved documents appear as Document nodes with ingestion status and HAS_DOCUMENT links even before extraction finishes. These links describe stored records, not causal claims. Agronomic evidence continues to come from the configured graph service. If that service is temporarily unavailable, the UI explicitly reports that only saved records are shown. Uploads still require working database, storage, worker, and model configuration; no production credentials or real farm data are included with the sample pack.

Run python scripts/generate_sample_documents.py to reproduce the documents. Regression checks: cd frontend && npm test; python -m unittest discover -s checks. CI also runs the production build, authenticated API tests, and Chromium interaction checks using mocked API responses (no real documents or model calls in tests).