TransientTracker

This work is dedicated to the public domain under CC0 1.0 Universal.

English | 日本語

TransientTracker is a Proof of Concept (PoC) for detecting and tracking moving-object candidates in a time series of astronomical images taken of the same field of view.

Purpose of this project

This is a PoC created to learn xtensor.

https://github.com/xtensor-stack/xtensor

This repository is a PoC and has not been tested on real astronomical photographs.

It currently consists of the following two commands.

• transient_tracker_generate Generates a synthetic dataset as a substitute for real astronomical images.
• transient_tracker_analyze Analyzes a generated dataset and extracts tracks.

Overview

The analyzer processes data roughly in the following flow.

1. Loads the sequence of frame images.
2. Estimates the inter-frame positional shift and registers (aligns) the frames.
3. Builds a reference image and residual images.
4. Extracts candidate points from the residual.
5. Links candidate points across frames to form tracks.
6. Outputs track CSV, summary JSON, and visualization images.

The current scenario is the single Scenario S-01.

• Scenario S-01: a PoC scenario that places 20 moving objects randomly from a seed.

The initial position, velocity, and flux of each moving object are generated deterministically from --seed. There is no upper limit on the number of detections in the analyzer.

Analysis modes

The analyzer automatically switches between two modes depending on whether ground truth is available.

• Observation mode: when labels.csv is absent. Outputs summary.json, tracks.csv, etc., and does not perform ground-truth matching (evaluation.json).
• Evaluation mode: when labels.csv is present. In addition to the observation-mode outputs, it generates evaluation.json.

The output data are summarized below (see How to read the outputs for details).

• summary.json: Overview of the whole analysis. Aggregates key metrics such as the number of detected tracks and high-confidence accepted tracks.
• tracks.csv: List of detected tracks. Records per-track score, fit error, autonomous accept/reject result, and so on.
• evaluation.json: Ground-truth matching results (evaluation mode only). Records the correspondence between detected tracks and ground-truth objects.

Requirements

• OS: macOS (developed and verified with Apple Clang)
• Compiler: Apple Clang (C++20 capable)
• CMake 3.16 or later
• OpenCV
• xtensor

Compilers other than those on macOS (e.g., g++) have not been verified.

Example installation with Homebrew:

brew install cmake opencv xtensor

Build

Run the following at the project root.

cd /Users/mitsuruk/devs/@GitHub/@Project/TransientTracker
cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build
ctest --test-dir build --output-on-failure

How to run

1. Generate a synthetic dataset

Example of generating S-01.

./build/transient_tracker_generate \
  --scenario S-01 \
  --output ./output/S-01 \
  --seed 20260608 \
  --write-all-preview

Main arguments:

• --scenario <name>
• --output <dir>
• --seed <value>
• --width <value>
• --height <value>
• --num-frames <value>
• --num-stars <value>
• --write-all-preview

2. Analyze the dataset

./build/transient_tracker_analyze \
  --dataset ./output/S-01 \
  --output ./analysis/S-01 \
  --threshold-scale 3.5 \
  --tolerance-radius 1 \
  --area-min 1 \
  --area-max 64 \
  --max-velocity 3.0 \
  --min-track-length 3 \
  --fit-error-threshold 1.5 \
  --min-continuity-ratio 0.6 \
  --duplicate-merge-distance 1.5 \
  --truth-match-distance 2.0 \
  --write-debug-images

Main arguments:

• --dataset <dir>
• --output <dir>
• --tolerance-radius <value>
• --threshold-scale <value>
• --area-min <value>
• --area-max <value>
• --max-velocity <value>
• --min-track-length <value>
• --fit-error-threshold <value> (upper bound of the line-fit error for high-confidence acceptance)
• --min-continuity-ratio <value> (lower bound of the frame continuity ratio for high-confidence acceptance)
• --duplicate-merge-distance <value> (distance threshold for merging duplicate tracks)
• --truth-match-distance <value> (evaluation mode only)
• --write-debug-images
• --disable-registration

Directory layout

TransientTracker/
├── apps/
├── build/
├── design-docs/
├── include/
├── output/
├── src/
└── tests/

Example of a generated dataset:

output/S-01/
├── frames/
├── labels.csv
├── metadata.json
└── preview/

Example of analysis output:

analysis/S-01/
├── detections.csv
├── evaluation.json   (evaluation mode only)
├── summary.json
├── tracks.csv
├── results/
└── debug/

How to read the outputs

Generator outputs

metadata.json

Basic information about the whole dataset. Check the following first.

• scenario_name
• num_frames
• num_objects
• seed

labels.csv

The ground truth of the synthetic data. Mainly check the following.

• frame_index
• object_id
• object_x
• object_y
• object_visible

preview/

Preview images for visual inspection. Used to confirm the synthesized moving-object positions.

Analyzer outputs

summary.json

A summary of the whole analysis. Mainly check the following.

• num_detected_tracks (number of detected tracks)
• num_high_confidence_tracks (number of accepted high-confidence tracks; the primary metric in observation mode)
• has_truth (true in evaluation mode)
• num_matched_objects (valid in evaluation mode only)
• status

tracks.csv

A list of detected tracks. Mainly check the following.

• track_id
• frame_index
• track_score
• fit_error
• is_high_confidence (result of autonomous acceptance)
• matched_object_id (valid in evaluation mode only)

evaluation.json

Ground-truth matching results, generated only in evaluation mode (with labels.csv). Mainly check the following.

• num_visible_objects
• num_matched_tracks
• num_matched_objects
• matched_objects

results/trajectory_result.png

A summary image with detected tracks overlaid on the first star field. A good image to look at first.

results/result_XXXX.png

Result images with tracks overlaid on each frame. Lets you check the tracking status per frame.

debug/

Intermediate products. The following images let you isolate each processing step.

• reference/ the reference image
• aligned/ the registered (aligned) frames
• residual/ the residual images
• mask/ the candidate-extraction masks
• overlay/ the candidate overlay images

Visualizing the analysis pipeline (S-01 example)

Using the images in analysis/S-01/ (generated with --write-debug-images), two images are shown for each processing step. This is an example with seed 20260608, 1024x1024, 16 frames. The appearance changes with the seed and resolution.

1. Reference image debug/reference.png

The "static background" reference, built from the per-frame median with a max-envelope of a tolerance radius applied. Only stationary stars whose positions do not change across frames remain; moving objects are removed (by nature of this step, only one image).

2. Aligned frames debug/aligned/

Each frame after cancelling the translational shift estimated by phase correlation, so the stationary-star positions are aligned. After registration, the stars overlap on the same pixels.

3. Residual images debug/residual/

The difference of each aligned frame minus the reference image. Stationary stars cancel out and almost disappear, leaving only the moving objects—absent from the reference—as bright points (stationary-star positions remain as faint dark dots due to small subtraction residuals).

4. Threshold mask debug/mask/

A binary mask in which pixels exceeding the threshold (threshold_scale times the noise sigma estimated from the residual via MAD, Median Absolute Deviation) are set to white. It shows the positions of candidate pixels.

5. Candidate overlay debug/overlay/

Connected components extracted from the mask as candidate points, overlaid as circles on the aligned frame. Lets you confirm where moving-object candidates were detected in each frame.

6. Per-frame results results/result_XXXX.png

Tracks formed by linking candidates across frames, drawn on each frame. In the first frame (0000) each object starts as a point; by the last frame (0015) the track extends as a line. Labels read "obj (truth ID) / track (track ID)".

Final: results/trajectory_result.png

A summary image overlaying all 20 accepted object tracks—with start and end markers—on the first star field. The overall analysis flow is as follows.

1. Load frames and normalize by the median.
2. Estimate translational shift by phase correlation → register (aligned).
3. Build the reference image with the median + max-envelope (reference).
4. Aligned frame − reference = residual (residual).
5. Estimate noise via MAD → threshold mask (mask).
6. Extract candidates from the mask (overlay).
7. Link candidates across frames → form tracks.
8. Accept high-confidence tracks by track length, line-fit error, and frame continuity ratio, then merge duplicate tracks.
9. Draw the accepted tracks → trajectory_result.png.

All 20 moving objects are detected and drawn as individual tracks.

Files to check first

When you analyze Scenario S-01, looking at the following first makes it easier to grasp the overall picture.

• output/S-01/metadata.json
• analysis/S-01/summary.json
• analysis/S-01/evaluation.json (evaluation mode only)
• analysis/S-01/results/trajectory_result.png

Notes

• The generator is a program for creating PoC datasets with ground truth.
• The analyzer supports ground-truth-independent autonomous track detection (observation mode). When ground truth is available, it also generates evaluation.json in evaluation mode.
• There is no upper limit on the number of detected moving objects. In Scenario S-01, 20 objects are placed randomly from a seed, and all 20 are confirmed to be recognized at the default size (1024x1024, 16 frames).