dopt_study — D-Optimality vs Energy/Entropy Active Learning Comparison

Comparison of two cluster-selection strategies in ChIMES active learning (AL) for the reactive water system (132 atoms: 88 H + 44 O, 1000 K).

Methods compared

Label	Tarball	Selection criterion
energy_entropy	TestCluster_original.tar.gz	Energy/entropy maximization (original)
dopt	TestCluster_2.tar.gz	D-optimality (maximize det of A^T A)

Both runs used 4 AL cycles (ALC-0 through ALC-3) on the same MD trajectory pool.

Directory layout

dopt_study/
├── data/
│   ├── energy_entropy/   # TestCluster_original (untarred)
│   └── dopt/             # TestCluster_2 (untarred, FAILED/ excluded)
├── analysis/
│   ├── 01_extract_metrics.py       # scrape dlars.log → training_metrics.csv
│   ├── 02_selection_comparison.py  # frame overlap/diversity
│   ├── 03_model_convergence.py     # RMSE + #params plots
│   ├── 04_dopt_gamma_analysis.py   # gamma distribution per cycle
│   ├── 05_md_stability.py          # T/P/E from md_statistics.out
│   ├── 06_energy_distribution.py   # energy histograms of selected frames
│   └── run_all_analysis.sh         # driver: runs all scripts in order
├── results/
│   ├── figures/   # all PDF plots
│   └── tables/    # all CSV summaries
└── README.md

Running the analysis

cd dopt_study
bash analysis/run_all_analysis.sh python3

All outputs land in results/figures/ and results/tables/.

Key findings (first pass)

1. Training RMSE

Method	ALC-0	ALC-1	ALC-2	ALC-3
Energy/Entropy	6.31	~6973	~7178	~6610
D-Optimality	6.35	6.58	11.1	11.0

D-optimality maintains dramatically lower training RMSE across cycles. The energy/entropy method shows a ~1000× RMSE increase after ALC-0, suggesting the newly selected frames are in a very different (high-energy or high-uncertainty) regime that the model struggles to fit.

2. MD stability (critical result)

Method	ALC-1 T_mean	ALC-3 T_mean	ALC-1 E_drift	ALC-3 E_drift
Energy/Entropy	5277 K	3262 K	−40 kcal/mol/atom/fs	−13 kcal/mol/atom/fs
D-Optimality	1003 K	1000 K	−0.016 kcal/mol/atom/fs	+0.001 kcal/mol/atom/fs

D-opt models run stable MD at the target 1000 K. Energy/entropy models show runaway temperature and massive energy drift — physically unstable. This is the most striking quantitative difference between the methods.

3. Cluster selection counts

Method	ALC-0	ALC-1	ALC-2	ALC-3
Energy/Entropy	100	100	100	100
D-Optimality	21	187	1	0

D-opt ALC-2 and ALC-3 selected very few frames. The driver log shows maxvolpy not found error in the final cycle — the D-opt selection (maxvol algorithm) fell back / failed for the last two cycles. This needs investigation: install maxvolpy and re-run ALC-2+.

4. D-opt gamma distributions

Mean candidate gamma: 0.59 → 0.83 → 0.47 → 0.40 across cycles. ALC-1 has the highest mean gamma (most informative candidate pool). The drop in ALC-2/3 is consistent with the maxvolpy selection failure. See results/figures/dopt_gamma_*.pdf.

Notes / caveats

• energy_entropy does not produce all.energies_normed for dopt cycles (script 06 only plots energy distributions for the energy/entropy run).
• The FAILED/ directory in TestCluster_2 was excluded from the main analysis. It contains earlier failed attempts at ALC-3.
• The maxvolpy issue means ALC-2 and ALC-3 dopt results should be treated cautiously until re-run with maxvolpy installed.

Dependencies

Standard scientific Python: numpy, scipy, matplotlib, csv, glob