TESTING.md

Testing Guide

This document describes the testing setup and strategy for SIMON.

Test Framework

SIMON uses Vitest as its testing framework, which provides:

• Fast test execution
• Built-in TypeScript support
• Coverage reporting
• Watch mode for development

Running Tests

All tests run inside Docker containers. See DOCKER_DEVELOPMENT.md for Docker setup.

Run All Tests

# Run all tests in Docker container
docker-compose -f docker-compose.dev.yml exec simon-dev npm test

# Or use npm script
npm run docker:dev:exec npm test

Run Tests for Specific Package

# Run tests for core package
docker-compose -f docker-compose.dev.yml exec simon-dev npm test --workspace=@simon-ai/core

# Run tests for CLI package
docker-compose -f docker-compose.dev.yml exec simon-dev npm test --workspace=@simon-ai/cli

Watch Mode

# Run tests in watch mode (auto-rerun on file changes)
docker-compose -f docker-compose.dev.yml exec simon-dev npm run test:watch --workspace=@simon-ai/core

Coverage Reports

# Generate coverage report
docker-compose -f docker-compose.dev.yml exec simon-dev npm run test:coverage --workspace=@simon-ai/core

# Coverage reports are generated in:
# - packages/core/coverage/ (HTML report)
# - packages/core/coverage/coverage-final.json (JSON report)
# 
# Access reports from host machine (volume mounted)

Test Structure

Tests are organized alongside source files with the .test.ts suffix:

packages/core/src/
├── config.ts
├── config.test.ts
├── storage/
│   ├── database.ts
│   ├── database.test.ts
│   ├── storage.ts
│   └── storage.test.ts
└── test/
    ├── setup.ts          # Test setup and teardown
    ├── helpers.ts        # Test utilities
    └── server-helper.ts  # Server test utilities

Test Categories

Unit Tests

Unit tests focus on individual components in isolation:

• Configuration Management (config.test.ts)

  • Loading configuration from environment variables
  • Default value handling
  • Validation

• Database Manager (database.test.ts)

  • Database connection
  • Schema initialization
  • Migration handling

• Storage Service (storage.test.ts)

  • CRUD operations for snapshots, files, warnings, artifacts
  • Query operations with pagination

• Ingestion Engine (ingestion.test.ts)

  • Repository scanning
  • File processing
  • Language detection
  • Exclusion patterns

Integration Tests

Integration tests verify components working together:

• API Endpoints (server.test.ts)
  • HTTP request/response handling
  • Error handling
  • Data flow through the system

Test Utilities

Test Helpers

Located in packages/core/src/test/helpers.ts:

• createTestConfig() - Create test configuration
• createTestDatabaseManager() - Create test database
• createTestStorageService() - Create test storage service
• createTestIngestionEngine() - Create test ingestion engine
• createTestFile() - Create test files

Test Setup

Located in packages/core/src/test/setup.ts:

• Creates temporary storage directories
• Cleans up after tests
• Configures test environment

Writing Tests

Example Test Structure

import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { MyComponent } from './my-component';

describe('MyComponent', () => {
  let component: MyComponent;

  beforeEach(() => {
    // Setup before each test
    component = new MyComponent();
  });

  afterEach(() => {
    // Cleanup after each test
  });

  describe('methodName', () => {
    it('should do something', () => {
      const result = component.methodName();
      expect(result).toBe(expected);
    });
  });
});

Best Practices

1. Isolation: Each test should be independent
2. Cleanup: Always clean up resources in afterEach
3. Descriptive Names: Use clear test descriptions
4. Arrange-Act-Assert: Structure tests clearly
5. Mock External Dependencies: Use mocks for external services

Test Coverage

Current coverage targets:

• Statements: > 80%
• Branches: > 75%
• Functions: > 80%
• Lines: > 80%

View coverage reports:

# Generate coverage
docker-compose -f docker-compose.dev.yml exec simon-dev npm run test:coverage --workspace=@simon-ai/core

# View HTML report (from host machine)
open packages/core/coverage/index.html

Continuous Integration

Tests should be run:

• Before committing (pre-commit hook recommended)
• In CI/CD pipeline
• Before releasing

Debugging Tests

Run Specific Test

# Run tests matching a pattern
docker-compose -f docker-compose.dev.yml exec simon-dev npm test --workspace=@simon-ai/core -- -t "should ingest repository"

# Run tests in a specific file
docker-compose -f docker-compose.dev.yml exec simon-dev npm test --workspace=@simon-ai/core -- config.test.ts

Debug Mode

# Access container shell for debugging
docker-compose -f docker-compose.dev.yml exec simon-dev sh

# Then inside container:
npm test --workspace=@simon-ai/core -- --inspect-brk

Future Improvements

• Add E2E tests for CLI
• Add E2E tests for UI
• Add performance/load tests
• Add mutation testing
• Add visual regression tests for UI