Skip to content

Eventdisplay CTAO Pipeline — Issue Report #198

Description

@GernotMaier

Eventdisplay CTAO Pipeline — Issue Report

Repositories reviewed: Eventdisplay, Eventdisplay_AnalysisScripts_CTA, Eventdisplay_AnalysisFiles_CTA, Eventdisplay-ML, logFiles

Scope: code correctness, algorithm consistency, and parameter propagation for the CTA IRF Monte Carlo production chain. Large auxiliary data products are not part of this checkout; physics-validity items are therefore assessed from code and log evidence rather than numerical output.


Pipeline overview

Step Mode key Entry script Executable
1 EVNDISP CTA.EVNDISP.sub_convert_and_analyse_MC_VDST_ArrayJob.sh evndisp
2 DISPBDT CTA.DISPTRAINING.sub_analyse.sh trainTMVAforGammaHadronSeparation
3 MAKETABLES CTA.MSCW_ENERGY.sub_make_tables.sh mscw_energy -filltables=1
4 ANATABLES CTA.MSCW_ENERGY.sub_analyse_MC.sh mscw_energy + smoothLookupTables
5 XGBSTEREOTRAIN / XGBSTEREOANA CTA.XGBSTEREO.sub_train/analyse.sh eventdisplay-ml-train/apply-xgb-stereo
6 PREPARETMVA CTA.prepareTMVA.sub_train.sh WriteTrainingEvents
7 TRAIN / TRAIN_RECO_* CTA.TMVA.sub_train.sh trainTMVAforGammaHadronSeparation
8 ANGRES / QC / CUTS CTA.EFFAREA.sub_analyse_list.sh makeEffectiveArea
9 PHYS CTA.WPPhysWriter.sub.sh writeCTAWPPhysSensitivityFiles

Dispatcher: Eventdisplay_AnalysisScripts_CTA/CTA.runAnalysis.sh, invoked from CTA.mainRunScriptsReduced.sh.


Severity definitions

Label Meaning
Critical Logic inversion or memory error that corrupts results unconditionally
High Bug that silently produces wrong physics output, crashes, or causes silent job loss under realistic inputs
Medium Fragile code or parameter mismatch that can produce wrong results in specific but plausible conditions
Low Metadata / diagnostic error with no direct physics impact

1. Environment and orchestration

ENV-1 · High — Hard-coded DESY AFS path makes the pipeline non-portable

  • No an issue and now changes required (we run these scripts only at DESY)

setSoftwarePaths.sh hard-codes a single ROOT installation path and derives all other paths from the current shell directory:

export ROOTSYS=/afs/ifh.de/group/cta/cta/software/root/root_v6.30.02.Linux-almalinux9.3-x86_64-gcc11.4/
TDIR=$(pwd)
export EVNDISPSCRIPTS="$TDIR"
export WORKDIR="${CTA_USER_WORK_DIR%/}/analysis/AnalysisData/${DSET}"

Source: Eventdisplay_AnalysisScripts_CTA/setSoftwarePaths.sh:20-27, :36-45.

Impact: running the same dataset on any other site or from a different checkout directory silently selects different software or fails late in the job.

ENV-2 · High — Dataset-to-array mapping uses cascading ARRAY=() reassignments, leaving dead configurations

CTA.runAnalysis.sh maps dataset names to subarray lists via repeated unconditional assignment inside the same if branch. For example, seven consecutive ARRAY=(...) lines appear for prod5 South:

ARRAY=( "subArray.prod5.South-BL.list" )
ARRAY=( "subArray.prod5.South-Alpha-2LSTs42SSTs.list" )
...
ARRAY=( "subArray.prod5.South-Betab.list" )

Source: Eventdisplay_AnalysisScripts_CTA/CTA.runAnalysis.sh:255-267.

Impact: only the last assignment survives. Earlier options are unreachable dead code and mislead anyone reconstructing which array list was used.

ENV-3 · Medium — Versioning encoded in script literals

ANADATE, EFFDATE, PHYSDATE, and related IRF/TMVA version tags are string literals in CTA.runAnalysis.sh:

Source: Eventdisplay_AnalysisScripts_CTA/CTA.runAnalysis.sh:66-76, :226-233, :293-298.

Impact: reproducibility requires preserving the exact script revision; changing a single date literal redirects the whole chain to a different table/TMVA/IRF namespace without any structural safeguard.


2. Stage: EVNDISP

EVN-1 · High — continue outside a loop terminates the EVNDISP branch abnormally

The EVNDISP dispatch block ends with:

cd ../
done
continue

Source: Eventdisplay_AnalysisScripts_CTA/CTA.runAnalysis.sh:416-427.

Impact: continue outside a loop is a POSIX shell error. The EVNDISP mode does not exit cleanly before worker jobs are submitted.

EVN-2 · High — Input discovery is fragile when TMPDIR contains zero or multiple simtel files

The worker reconstructs simtel inputs with:

SIMFIL=`ls $TMPDIR/*.simtel.${EXTE}`

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.EVNDISP.qsub_convert_and_analyse_MC_VDST_ArrayJob.sh:129-162.

Impact: if the glob matches zero or multiple files, the converter call becomes ambiguous. Batches of files are copied into a shared $TMPDIR before conversion, making multiple matches plausible.

EVN-3 · Medium — Extensive unquoted path handling makes file operations shell-fragile

Representative examples:

  • cp -v -f $F $TMPDIR"/"CTA.EVNDISP.qsub_convert_and_analyse_MC_VDST_ArrayJob.sh:77-95
  • cp $F $DDIR/CTA.MSCW_ENERGY.qsub_analyse_MC.sh:74-77
  • sed -n "$l,$k p" $TFILE > $IFILCTA.MSCW_ENERGY.qsub_analyse_MC.sh:45

Impact: paths containing spaces or shell metacharacters break silently. Current DESY-style paths avoid this in practice but the code is brittle.

EVN-4 · Medium — Missing low-gain calibration multipliers observed in production logs

  • Not an issue. Low-gain calibration is not relevant in this context.

From logFiles/evndisp.log:646-652:

VEventLoop::analyzeEvent warning: No low gain multipliers available for sumwindow 1 (3 samples); trace integration method 2

Impact: reconstruction proceeded with incomplete calibration; trace integration method 2 uses a missing conversion coefficient for low-gain channels.

EVN-5 · Medium — Low-gain average-T0 assignment always overwrites the high-gain field

  • Not an issue. Low-gain calibration is not relevant in this context.

VCalibrationData::setAverageTZero() is missing an else branch:

void VCalibrationData::setAverageTZero(double iAverageTzero, bool iLowGain)
{
    if(iLowGain)
    {
        fAverageTZero_lowgain = iAverageTzero;
    }
    fAverageTZero_highgain = iAverageTzero;   // executes unconditionally
}

Source: Eventdisplay/src/VCalibrationData.cpp:380-388.

Impact: every low-gain T0 calibration call corrupts fAverageTZero_highgain, invalidating the high-gain timing constants for any subsequent event.

EVN-6 · Medium — Low-gain average-T0 calculation uses the high-gain charge extraction path

  • Not an issue. Low-gain calibration is not relevant in this context.

VCalibrator::calculateAverageTZero(bool iLowGain) passes iLowGainOnly=false to calcSums unconditionally:

calcSums(fRunPar->fCalibrationSumFirst,
         fRunPar->fCalibrationSumFirst + fRunPar->fCalibrationSumWindowAverageTime,
         false, false, 2);   // second false = iLowGainOnly

Source: Eventdisplay/src/VCalibrator.cpp:868.

Impact: low-gain T0 constants are derived from charges integrated in high-gain mode; the resulting timing calibration is wrong for low-gain channels.


3. Stage: Lookup tables and stereo reconstruction

TAB-1 · High — Table-filling and analysis stages use different stereo-angle cuts

  • Note an issue (as long as table filling uses a smaller angle than the analysis.

MAKETABLES fixes the stereo-angle cut at 10 degrees:

MOPT="$MOPT -redo_stereo_reconstruction ... -minangle_stereo_reconstruction=10"

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.MSCW_ENERGY.qsub_make_tables.sh:87-90.

ANATABLES uses 15 degrees for 2-telescope arrays and 5 degrees otherwise:

if [[ $NTEL == "2" ]]; then
    MOPT="$MOPT -minangle_stereo_reconstruction=15."
else
    MOPT="$MOPT -minangle_stereo_reconstruction=5."
fi

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.MSCW_ENERGY.qsub_analyse_MC.sh:133-138.

Impact: the lookup tables are filled under a different stereo phase space than the analysis stage that consumes them. This biases mean-scaled parameters and energy reconstruction, especially for low-multiplicity configurations where the 10° vs 5° difference is largest.

TAB-2 · High — CTA.MSCW_ENERGY.sub_make_tables.sh uses an undefined variable MEANDIST in generated filenames

FNAM="$SHELLDIR/EMSCW.table-$TAFIL-W$MEANDIST-${ARRAY}${AZ}"

MEANDIST is never set anywhere in the script.

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.MSCW_ENERGY.sub_make_tables.sh:99-105.

Impact: job and script names are unstable (expand to an empty-token form). Name collisions are possible if different invocations happen to produce the same effective filename prefix.

TAB-3 · Medium — CTA.MSCW_ENERGY.sub_analyse_MC.sh uses an undefined variable NC in output names

TFIL=$PART$NC"."$SUBAR"_ID${RECID}_${MCAZ}-"$DSET

NC is never assigned in the script.

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.MSCW_ENERGY.sub_analyse_MC.sh:121-123.

Impact: the naming convention is broken by a stale variable from an earlier script revision. The empty expansion risks latent name collisions if NC becomes set in the environment.

TAB-4 · Medium — DISP analysis and training cuts are synchronized only by a comment

# IMPORTANT: this must be the same or lower value as in dispBDT training
if [[ $RECID == "1" ]]; then
    MOPT="$MOPT -maxloss=0.1 -minfui=0."

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.MSCW_ENERGY.qsub_analyse_MC.sh:139-148.

Impact: a training/analysis cut mismatch is not detected automatically and can alter DISP validity or reconstruction quality silently.

TAB-5 · Medium — VTableLookupRunParameter contains contradictory CLI validation

  • For -fill, the error text documents 2=read lookup tables, but the code only accepts 0 or 1. Source: Eventdisplay/src/VTableLookupRunParameter.cpp:168-183.
  • For -tmva_nimages_max_stereo_reconstruction, the code rejects only values > 40000 while the error message states the maximum is 4. Source: Eventdisplay/src/VTableLookupRunParameter.cpp:205-214.

Impact: misleading or stale CLI validation on the mscw_energy path makes misconfiguration easy to miss.


4. Stage: DISP training

DISP-1 · High — CTA.DISPTRAINING.sub_analyse.sh argument parsing is broken in two ways

Argument-position error: both TMVAQC and QSUBOPT are gated on $8 rather than $7 and $8 respectively:

if [ -n $8 ]
then
   TMVAQC="$7"    # should be guarded by [ -n "$7" ]
fi
if [ -n $8 ]
then
   QSUBOPT="$8"
fi

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.DISPTRAINING.sub_analyse.sh:57-64.

Unquoted guards always true: all three positional guards ($6, $8, ${9}) are unquoted. In bash, [ -n $UNSET ] reduces to [ -n ], which is always true because -n is itself non-empty. The guards never detect an absent argument.

Source: lines 52, 57, 61, 91.

Consequence of empty TMVAQC: line 72 runs QCA=\cat $TMVAQC`. When TMVAQCis the empty string,catreads from stdin and hangs, or returns empty; the inner submission loopfor QC in ${QCA}` then iterates zero times and no DISP jobs are submitted.

Impact: whenever the quality-cut file argument is absent, the DISP training stage silently submits no jobs. Combined with the unquoted guards, this failure is invisible without explicit log inspection.

DISP-2 · High — DISP training is submitted for telescope types absent from the selected array (confirmed by logs)

Production logs show repeated training failures:

total number of telescopes: 30 (selected 0)
..nothing to do. Exiting.

Source: logFiles/BDTDisp-10408618.training.log:46-48, BDTDisp-201109916.training.log:46-48, BDTDisp-205008707.training.log:46-48.

Successful training occurred only for types 10608418 and 138704810, which are the only types present in the prod6 North subarray (confirmed by logFiles/makeTable.log:60-95 and logFiles/mscwTable.log:181-218).

Impact: telescope-type enumeration in the DISP dispatch loop is inconsistent with the array composition. Training jobs are wasted for every absent type; the pipeline provides no error signal for this condition.

DISP-3 · Medium — Failed telescope-type training is detected only by log-text grep

The qsub worker decides whether training was meaningful by searching for the literal string Number of telescope types: 0 in the log:

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.DISPTRAINING.qsub_analyse.sh:79-88.

Impact: failure detection depends on exact log wording. Any upstream message change silently defeats this safeguard.


5. Stage: TMVA preparation and gamma/hadron training

TMVA-1 · High — CTA.prepareTMVA.sub_train.sh mis-parses optional arguments

The documented interface is [qsub options] [direction] [job_dir], but the implementation assigns:

MCAZ=${5:-$MCAZ}
QSUBOPT=${5:-$QSUBOPT}
LDIR=${6:-$LDIR}

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.prepareTMVA.sub_train.sh:59-68.

Argument $5 is assigned to both MCAZ (azimuth direction) and QSUBOPT (batch options). The documented direction slot ($6) is used as the log directory.

Impact: with an empty QSUBOPT, the azimuth direction can slide into the batch-options slot and vice versa, silently selecting the wrong input path pattern (gamma_cone.*ID$RECID$MCAZ*.mscw.root) and corrupting submission options.

TMVA-2 · High — Gamma/hadron BDT energy bins 0 and 1 overlap completely (bin 0 ⊂ bin 1)

The energy bin arrays in CTA.TMVA.sub_train.sh are:

EMIN=( -1.90 -1.90 -1.45 -1.20 -0.95 -0.50 -0.10  0.45  0.90 )
EMAX=( -1.40 -1.30 -1.15 -0.80 -0.25  0.25  0.75  1.50  2.50 )

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.TMVA.sub_train.sh:84-85.

Bin 0 spans log₁₀(E/TeV) = −1.90 to −1.40 (12.6–39.8 GeV). Bin 1 spans −1.90 to −1.30 (12.6–50.1 GeV). Bin 0 is a strict subset of bin 1; both share the same lower energy edge and train on largely identical event populations.

At application time VTMVAEvaluator::getDataBin() selects the nearest spectral-weighted mean energy bin. Because bins 0 and 1 share the same lower edge, their spectral-weighted means are nearly identical, and assignment of low-energy events to one or the other is effectively arbitrary.

Source: Eventdisplay/src/VTMVAEvaluator.cpp:1120-1155.

Impact: two BDTs are trained on near-identical event sets and applied with near-identical selection criteria. No exclusive low-energy classifier exists for the 12.6–40 GeV range. If the intended configuration was two adjacent bins (e.g., −1.90 to −1.40 and −1.40 to −1.30), the lower edge of bin 1 should be −1.40, not −1.90.

TMVA-3 · High — TMVA training aborts entirely if the first expected training file is missing

CTA.TMVA.sub_train.sh exits immediately when MVA${MCAZ}-${RECID}-${OFFMEA[0]}.training.root is absent, with no per-file retry or diagnostic:

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.TMVA.sub_train.sh:123-129.

Impact: one missing training product blocks all remaining offset/energy bins for the affected array and run mode.

TMVA-4 · Medium — Unquoted positional-parameter checks are always true in CTA.WPPhysWriter.sub.sh

if [ -n $9 ]; then
if [ -n ${10} ]; then
if [ -n ${11} ]; then

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.WPPhysWriter.sub.sh:36-52.

Impact: these guards always evaluate true in bash regardless of whether the arguments are set. Optional late arguments are therefore always consumed.

TMVA-5 · Medium — Diagnostic artifacts deleted unconditionally after training

CTA.TMVA.qsub_train.sh deletes all generated .C files and the entire complete_BDTroot directory after training:

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.TMVA.qsub_train.sh:25-31.

Impact: post-mortem analysis of suspicious classifier behavior is not possible without re-running training.


6. Stage: XGB stereo reconstruction

XGB-1 · High — XGB apply wrapper can load a different model than training produced

Training writes models under a prefix that includes the requested minimum-image count:

PREFIX="${ODIR}/dispdir_bdt_mintel${MINTEL}"

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.XGBSTEREO.qsub_train.sh:46.

The apply wrapper silently caps MINTEL at 3 before constructing the same prefix:

[ "$MINTEL" -ge 3 ] && MINTEL=3
PREFIX="${XGBDIR}/dispdir_bdt_mintel${MINTEL}"

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.XGBSTEREO.qsub_analyse.sh:42-46.

Impact: for any training run with NIMAGESMIN > 3, the apply step loads dispdir_bdt_mintel3_ebin*.joblib.gz even though training produced dispdir_bdt_mintel4 (or higher). This is a silent model-path mismatch.

XGB-2 · High — XGB classification is not wired into the orchestration layer

The script repository contains sub/qsub wrappers for XGB stereo regression training and application. No corresponding wrappers for XGB classification (train-xgb-classify, apply-xgb-classify) exist in Eventdisplay_AnalysisScripts_CTA.

Impact: gamma/hadron separation in the shell pipeline relies entirely on the TMVA path. The XGB classification capabilities of Eventdisplay-ML are unavailable through the current orchestration.

XGB-3 · High — configure_train() treats energy_bins_log10_tev as a dict, but loaders treat it as a list of per-bin dicts

model_configs["pre_cuts"] = pre_cuts_classification(
    e_min=np.power(10.0, model_parameters.get("energy_bins_log10_tev", []).get("E_min")),
    e_max=np.power(10.0, model_parameters.get("energy_bins_log10_tev", []).get("E_max")),
)

Source: Eventdisplay-ML/src/eventdisplay_ml/config.py:162-165.

The loader and updater logic in utils.py:109-125 and models.py:189-202 treat the same field as a list of per-bin dicts.

Impact: configuration shape mismatch causes a runtime AttributeError (list has no .get()) or silently applies incorrect energy pre-cuts before classification training.

XGB-4 · Medium — Training wrapper bypasses the external hyperparameter-config hook

The XGB CLI supports JSON override of hyperparameters via --hyperparameter_config. The CTA training wrapper never passes this option:

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.XGBSTEREO.qsub_train.sh:56-63.

Impact: production XGB training cannot be controlled through Eventdisplay_AnalysisFiles_CTA/ParameterFiles or the package's own JSON mechanism without editing the wrapper. Hyperparameters are locked to Python-package defaults.

XGB-5 · Medium — CTA.XGBSTEREO.sub_train.sh advertises a direction argument it never reads

The usage string documents [direction (e.g. _180deg)] as argument $6, but the implementation only reads $5 as QSUBOPT and $6 as LDIR; no azimuth/direction variable is used anywhere:

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.XGBSTEREO.sub_train.sh:16-22, :48-56.

Impact: callers believe they are training a direction-specific model while all input paths are direction-independent. Azimuth-mixing in XGB stereo models is invisible.

XGB-6 · Low — DL2.production.runparameter is a legacy TMVA example

The shipped file references TMVA products rather than the current XGB production path:

* TMVAPARAMETER noepoch BDT ...

Source: Eventdisplay_AnalysisFiles_CTA/ParameterFiles/DL2.production.runparameter:4-7.

Impact: the file is not an authoritative reference for the current ML production configuration and can mislead users reconstructing the analysis.


7. Stage: IRF calculation and PHYS writing

IRF-1 · High — PHYS passes a filename with a trailing space

./CTA.WPPhysWriter.sub.sh \
        "$NFILARRAY "\
        ${EFFFULLDIR}/BDT."$OOTIME"-${EFFVERSION}.$EFFDATE \

Source: Eventdisplay_AnalysisScripts_CTA/CTA.runAnalysis.sh:754-756.

Impact: the subarray-list argument has a spurious trailing space. Depending on the tool's path handling, this can make the wrapper fail to locate the intended file.

IRF-2 · High — Run-parameter key THETA2MINENEERGY appears misspelled

echo "THETA2MINENEERGY 1." >> "$PARA"

Source: Eventdisplay_AnalysisScripts_CTA/CTA.runAnalysis.sh:590-597.

The correct key is likely THETA2MINENERGY. VInstrumentResponseFunctionRunParameter silently ignores unknown keys (see LIB-4 below).

Impact: the θ² minimum-energy threshold may be silently dropped, allowing events below the intended energy threshold to enter the IRF calculation.

IRF-3 · Medium — Effective-area success is inferred from file size rather than content

minimumsize=300
DS=$(du -k $OFIX.root | cut -f 1)
if [[ ${DS} -le $minimumsize ]]; then
    touch $OLOG.SMALLFILE
    mv -v $OFIX.root ${ODIR}/
    continue
fi

Source: Eventdisplay_AnalysisScripts_CTA/analysis/CTA.EFFAREA.qsub_analyse_list.sh:581-607.

Impact: a semantically broken but large IRF file passes validation; a valid but compact file fails. This is weak validation for a stage that produces final IRF products.

IRF-4 · Medium — Summary-tree builder is hard-wired to one archived directory convention

prepareSummaryTrees.sh unconditionally overrides PHYSDIR immediately after setting it:

PHYSDIR="$DATADIR/Phys-${DDAT}/"
PHYSDIR="$DATADIR/archive.202105/Phys-${DDAT}-${S}/"

Source: Eventdisplay_AnalysisScripts_CTA/analysis/prepareSummaryTrees.sh:31-34.

Impact: the nominal output path is never used; only the hardwired archive path is accessible.


8. C++ library: signal extraction (VTraceHandler)

SIG-1 · High — Sliding-window raw mode returns the wrong quantity

In calculateTraceSum_slidingWindow(..., fRaw=true), the function accumulates raw tail samples into charge but returns FADC[1] instead:

for( unsigned int i = 0; i < ( unsigned int )iIntegrationWindow; i++ )
{
    charge += ( float )fpTrace.at( n - 1 - i );
}
return FADC[1];

Source: Eventdisplay/src/VTraceHandler.cpp:643-654.

Impact: trace integration method 2 in raw mode returns a single FADC sample rather than the integrated charge. Signal estimates are physically wrong for this combination.

SIG-2 · High — Sliding-window loop can access one sample past the trace end

The search range upper bound is:

if( ( n - window ) <= SearchEnd )
    SearchEnd = n - window + 1;

The update step is then:

xmax = xmax - FADC[i] + FADC[i + window];

Source: Eventdisplay/src/VTraceHandler.cpp:611-615, :663-680.

At the final iteration, i + window == n, which is one past the valid FADC[0..n-1] range.

Impact: undefined memory access at trace edges. This can perturb integrated charge and timing values and may crash on short traces.

SIG-3 · Medium — Low-gain saturation threshold computed from uninitialized tmax

getTraceMax() derives the saturation count threshold before resetting tmax:

unsigned int nMax = ( unsigned int )( fDynamicRange * tmax );
n255 = 0;
tmax = -10000.;

Source: Eventdisplay/src/VTraceHandler.cpp:391-395.

Impact: nMax is derived from the previous call's tmax, not the current trace maximum. Low-gain saturation diagnostics (n255) are unreliable.

SIG-4 · Low — Muon-ring correction divides by zero at the first grid point

The muon-ring image-size correction loop computes xi_tmp = (float)i / (float)numSteps and immediately divides by it before any guard:

for(int i = 0; i < numSteps; i++) {
    float xi_tmp = (float)i / (float)numSteps;   // zero when i == 0
    float kTest_tmp = 2.0 * ngExi[i] * kRatio / xi_tmp;  // div-by-zero

Source: Eventdisplay/src/VImageParameterCalculation.cpp:744-745.

Impact: muon calibration produces inf/NaN at the first grid point and an unstable correction factor. Muon ring analysis is not part of the standard CTA MC-only IRF production, but the defect exists in the shared code and would affect any CTA run that enables muon calibration.


9. C++ library: stereo reconstruction

STEREO-1 · Critical — VStereoReconstruction::removeDataSet has inverted logic and always reports failure

bool VStereoReconstruction::removeDataSet( unsigned int iDataSet )
{
    if( fData.size() < iDataSet )    // removes out-of-range indices, not valid ones
    {
        fData.erase( fData.begin() + iDataSet );
    }
    return false;    // always signals failure
}

Source: Eventdisplay/src/VStereoReconstruction.cpp:289-295.

Impact: valid dataset indices are never removed; out-of-range indices can be erased (undefined behaviour); the function unconditionally reports failure to the caller. Any code path relying on this method to manage reconstruction datasets is broken.

STEREO-2 · High — VOnOff mutates the OFF histogram during on-off subtraction

hTemp->Add( hon, hoff, 1., -1.*i_norm_alpha );
hoff->Scale( i_norm_alpha );

Source: Eventdisplay/src/VOnOff.cpp:149-153.

Impact: hoff is scaled in-place after being used in the subtraction. Callers expecting the original OFF histogram to remain unchanged reuse a scaled copy, producing wrong background normalization.

STEREO-3 · High — Method-4 stereo pair-angle rejection evaluates the wrong telescope slopes

In the method-4 all-pair weighted intersection loop, the current pair slopes are loaded into mm[0]/mm[1] (m[ii]/m[jj]), but the angle-rejection cut uses m[0]/m[1] — the slopes of the first two telescopes, not the current pair:

mm[0] = m[ii];
mm[1] = m[jj];
rcs_perpendicular_fit(xx, yy, ww, mm, 2, &xs, &ys, &stds);

float i_diff = fabs(atan(m[0]) - atan(m[1]));   // should use mm[0], mm[1]
if(i_diff < fAxesAngles_min / TMath::RadToDeg() ...) continue;

Source: Eventdisplay/src/VArrayAnalyzer.cpp:1339-1354.

Impact: for events with more than two images (routine in CTA multi-telescope arrays), the parallel-image cut is applied to the first telescope pair for every pair in the loop, not to the actual current pair. Valid pairs whose slopes are nearly parallel to telescopes 0 and 1 are accepted; pairs with good angular separation but small m[0]m[1] angle are discarded. The weighted direction estimate is biased across the full multi-telescope sample.

STEREO-4 · Medium — Emission-height pair weights undefined for small image sizes

The pairwise emission-height weight formula 1./((1./log10(size[i])) + (1./log10(size[j]))) is not guarded against size <= 1:

iEmissionHeightWeightTemp = 1. / ((1. / log10(size[i])) + (1. / log10(size[j])));

Source: Eventdisplay/src/VEmissionHeightCalculator.cpp:61.

Impact: an image with size == 1 makes log10(1) = 0, producing a division by zero; size < 1 produces a negative weight. Both yield NaN/inf or ill-defined emission-height estimates for low-intensity images that pass the size threshold.

STEREO-5 · Medium — VEmissionHeightCalculator accumulates telescope positions across reinitialisations

Both setTelescopePositions() overloads only push_back() and never clear() the position vectors:

void VEmissionHeightCalculator::setTelescopePositions(vector<float> x, ...)
{
    fNTel = x.size();
    for(unsigned int i = 0; i < fNTel; i++)
    {
        fTelX.push_back(x[i]);  // no clear() before the loop
        ...
    }
}

Source: Eventdisplay/src/VEmissionHeightCalculator.cpp:100-128.

Impact: reusing the same calculator instance (e.g., across array configurations or runs) doubles the telescope-position list, making all pairwise baseline distances and emission-height geometry wrong from the second call onward.


10. C++ library: TMVA training

ML-1 · Critical — intersect_mc_error ignores the Y-component due to a typo

intersect_mc_error = sqrt( (Xoff_intersect-MCxoff)*(Xoff_intersect-MCxoff)
                         + (Yoff_intersect-Yoff_intersect)*(Yoff_intersect-Yoff_intersect) );

Source: Eventdisplay/src/trainTMVAforGammaHadronSeparation.cpp:344-346.

The Y term is (Yoff_intersect - Yoff_intersect)², which is always zero. The intersection angular error is computed from the X offset only.

Impact: the angular-reconstruction method selector (TrainAngularReconstructionMethod) trains labels that compare DISP against a systematically wrong intersection error whenever the Y component is non-negligible. This biases method-selection in the active angular training stage.

ML-2 · High — Zenith-bin range check is off by one

if( iRun->fZenithCutData.size() < iZenithBin || iRun->fOutputFile[0].size() < iZenithBin )

Source: Eventdisplay/src/trainTMVAforGammaHadronSeparation.cpp:438-441.

The condition uses < instead of <=. Index iZenithBin == size() passes the guard even though it is out of range.

Impact: the subsequent array access at [iZenithBin] is out-of-bounds and invokes undefined behaviour.

ML-3 · High — get_largest_weight_disp_entries() sorts ascending, selecting the smallest-weight entries

The function name implies selecting the largest-weight DISP images, but the comparator sorts ascending:

sort(indexed_disp_woff.begin(), indexed_disp_woff.end(),
     [](const pair<Float_t, unsigned int>& a, const pair<Float_t, unsigned int>& b) {
         return a.first < b.first;
     });

Source: Eventdisplay/src/trainTMVAforGammaHadronSeparation.cpp:96-101.

Impact: the per-event telescope-image subset selected for BDT features is the lowest-weight group, contradicting the intended selection. This corrupts TMVA feature construction for multi-telescope events.

ML-4 · High — Per-telescope feature remapping writes into its own source arrays

When DispNImages < NImages, the code remaps arrays in place:

for( unsigned int i = 0; i < matching_indexes.size(); i++ )
{
    size[i] = size[matching_indexes[i]];
    width[i] = width[matching_indexes[i]];
    length[i] = length[matching_indexes[i]];
    ...
}

Source: Eventdisplay/src/trainTMVAforGammaHadronSeparation.cpp:103-117.

Impact: early writes overwrite values still needed by later iterations. Per-telescope BDT feature vectors are corrupted for events with missing DISP entries.

ML-5 · Medium — Reduced training tree stores a reconstruction-error feature under a misleading branch name

The helper get_largest_weight_disp_entries() fills an energy-error array named d_erec_mc:

d_erec_mc[i] = (ES[original_index] - ErecS) / ES[original_index];

Source: Eventdisplay/src/trainTMVAforGammaHadronSeparation.cpp:141-143.

But the reduced tree declares and writes this quantity as d_erec_es:

Float_t d_erec_es[d_n];
iDataTree_reduced->Branch( "d_erec_es", d_erec_es, htemp );
get_largest_weight_disp_entries(..., d_erec_es, d_R)

Source: Eventdisplay/src/trainTMVAforGammaHadronSeparation.cpp:248, :285-286, :352.

Impact: the branch name does not match the quantity stored. Any TMVA configuration referencing d_erec_es or d_erec_mc by name may apply the variable to incorrect physics context.

ML-6 · Medium — TrainAngularReconstructionMethod splits both classes from the same signal tree

Both the "prefer DISP" and "prefer intersection" training classes are extracted from iSignalTree_reduced:

TTree* sigTree = iSignalTree_reduced->CopyTree(signalCut);
TTree* bkgTree = iSignalTree_reduced->CopyTree(backgrCut);

Source: Eventdisplay/src/trainTMVAforGammaHadronSeparation.cpp:596-602.

Impact: this is the correct design for an angular-method selector (all events are gammas; the label encodes which reconstructor is more accurate). However, combined with the broken intersect_mc_error calculation (ML-1), the labels are wrong. The two issues compound.


11. C++ library: IRF and effective area

IRF-LIB-1 · High — makeEffectiveArea documents the MC header as optional, but the library hard-exits if it is missing

// read MC header (might not be there, no problem; ...)
VMonteCarloRunHeader* iMonteCarloHeader = fRunPara->readMCRunHeader();

Source: Eventdisplay/src/makeEffectiveArea.cpp:200-201.

Inside readMCRunHeader():

if( !iMC )
{
    cout << "VInstrumentResponseFunctionRunParameter::readMCRunHeader: no MC run header found ...";
    exit( EXIT_FAILURE );
}

Source: Eventdisplay/src/VInstrumentResponseFunctionRunParameter.cpp:570-577.

Impact: the executable-level comment and the library behaviour disagree. Runs the caller describes as tolerable still terminate with a hard exit.

IRF-LIB-2 · High — VInstrumentResponseFunctionReader disables true-energy-axis handling unconditionally

bXaxisIsEtrue = false;
if( bXaxisIsEtrue ) { ... } else { ... }

Source: Eventdisplay/src/VInstrumentResponseFunctionReader.cpp:902-910.

Impact: the code path for migration matrices with true energy on the x-axis is unreachable. Any caller expecting Etrue-axis handling silently receives reconstructed-energy handling.

IRF-LIB-3 · High — CTA IRF reader compensates for inconsistent histogram naming and units in the files it reads

// name and axis units are not consistent in the CTA files!!!

The reader then falls back through multiple alternative histogram names (EffectiveArea, EffectiveAreaEtrue, harea_gamma, ERes, EnResol_RMS, Ebias…).

Source: Eventdisplay/src/VInstrumentResponseFunctionReader.cpp:185-265.

Impact: the reader infers histogram semantics from inconsistent naming conventions. IRF reuse across production versions depends on fallback heuristics that can silently pick the wrong histogram.

IRF-LIB-4 · High — VInstrumentResponseFunctionRunParameter silently ignores unknown runparameter keys

The parser handles a known list of * KEY ... directives with no final else branch:

if( temp == "ENERGYSPECTRUMINDEX" ) { ... }
else if( temp == "MONTECARLOENERGYRANGE" ) { ... }
...
else if( temp == "WOBBLEISOTROPIC" ) { ... }
// unknown keys fall through silently

Source: Eventdisplay/src/VInstrumentResponseFunctionRunParameter.cpp:109-462.

Impact: misspelled or stale script-generated keys (e.g., THETA2MINENEERGY, see IRF-2) are silently ignored. Configuration drift between scripts and C++ parser goes undetected.

IRF-LIB-5 · Medium — IRF binning defaults are duplicated by contract between two classes

VEffectiveAreaCalculator documents that changing default energy/offset axes requires synchronized changes in VEffectiveAreaCalculatorMCHistograms:

Source: Eventdisplay/src/VEffectiveAreaCalculator.cpp:53-60.

Impact: this is a maintainability hazard; mismatched changes alter IRF shapes without compile-time protection.

IRF-LIB-6 · Medium — TMVA XML parsing depends on exact string layout

VTMVAEvaluator::getTrainingVariables parses XML weight files using substring offsets around NVar=", Expression=", and Label=:

Source: Eventdisplay/src/VTMVAEvaluator.cpp:95-151.

Impact: TMVA format changes or slightly different XML serialization silently break variable extraction.

IRF-LIB-7 · Medium — TMVA weight grid silently passes initialisation with missing interior bins

VTMVAEvaluator::initializeWeightFiles() emits a warning and continues when interior energy or zenith bins are missing, allowing an incomplete grid to be used in production:

// allow that first files are missing (this happens when there are no training events in the first energy bins)
if(!bGoodRun) {
    if(i == iMinMissingBin || j == jMinMissingBin) {
        cout << "warning: TMVA root file not found ..." << endl;
        continue;   // incomplete grid proceeds
    }

Source: Eventdisplay/src/VTMVAEvaluator.cpp:275-320.

Impact: missing interior bins are filled by the nearest-neighbour fallback in getDataBin(); events in those bins are evaluated with an incorrect classifier. There is no hard failure that would flag the incomplete IRF set.

IRF-LIB-8 · Medium — VInstrumentResponseFunction mutates the shared MC azimuth in place

MC azimuth normalisation is applied by modifying the shared event object directly:

if(fData->MCaz > 180.) {
    fData->MCaz -= 360.;    // in-place mutation of shared state
}

Source: Eventdisplay/src/VInstrumentResponseFunction.cpp:258-260.

Impact: this mutation occurs inside the azimuth-bin loop. After the first iteration the MCaz value is already normalised; subsequent iterations in any outer loop (e.g., spectral index) operate on the already-modified value. If the normalisation is applied more than once, MCaz can drift below −180°, causing all azimuth-bin membership tests to fail.

IRF-LIB-9 · Medium — makeEffectiveArea IRF resolution duplication is order-dependent

Resolution products (e.g., angular resolution, core resolution) are filled by looking up the source IRF by vector index via getDuplicationID():

else if(f_IRF[i]->getDuplicationID() < f_IRF.size()
        && f_IRF[f_IRF[i]->getDuplicationID()])
{
    f_IRF[i]->fillResolutionGraphs(f_IRF[f_IRF[i]->getDuplicationID()]->getIRFData());
}

Source: Eventdisplay/src/makeEffectiveArea.cpp:297-300.

Impact: this only works correctly if the source IRF has been initialised earlier in f_IRF. Any refactor changing the IRF vector ordering or adding new IRF types silently breaks resolution product generation.

IRF-LIB-10 · Medium — copyMCHistograms() leaves the first ROOT input file open

The helper function opens each input file to accumulate MC histograms. For the first file (z == 0) it borrows the histogram pointer directly and never closes the file:

TFile* ifInput = new TFile(chEl->GetTitle());
if(z == 0) {
    iMC_his = (VEffectiveAreaCalculatorMCHistograms*)ifInput->Get("MChistos");
    // ifInput is never closed
} else {
    iMC_his->add(...);
    ifInput->Close();   // only subsequent files are closed
}

Source: Eventdisplay/src/makeEffectiveArea.cpp:567-579.

Impact: the first input file handle is leaked for the lifetime of the returned histogram object; in large IRF productions with many input files this contributes to file-descriptor exhaustion.


12. C++ library: writeCTAWPPhysSensitivityTree

PHYS-LIB-1 · High — Hard-coded analysis root path

string iDataDirectory = "/lustre/fs22/group/cta/users/maierg/analysis/AnalysisData/" + iSite;

Source: Eventdisplay/src/writeCTAWPPhysSensitivityTree.cpp:248-250.

Impact: the physics summary tree writer is non-portable; it fails on any system where this path does not exist.

PHYS-LIB-2 · High — SC-MST telescope entries are never stored

The bounds check makes the write path unreachable under normal conditions:

if( fTelescopeData.size() > 5 )
{
    fTelescopeData[4].push_back( new VTelescopeData() );
}

Source: Eventdisplay/src/writeCTAWPPhysSensitivityTree.cpp:415-424.

Impact: SC-MST geometry metadata is silently dropped from the summary tree.

PHYS-LIB-3 · High — Energy bin centres read from ROOT underflow bin

for( int i = 0; i < h->GetNbinsX(); i++ )
{
    fEnergy_logTeV[i] = h->GetBinCenter( i );    // ROOT bin 0 = underflow
}

Source: Eventdisplay/src/writeCTAWPPhysSensitivityTree.cpp:515-518.

Impact: the exported energy array starts with the underflow bin and drops the last real bin, shifting all exported energy coordinates by one bin.


13. C++ library: misc

MISC-1 · High — printRunParameter hides file-open failures and returns success on missing metadata

  • If the input file cannot be opened, the program exits with code 0 (success). Source: Eventdisplay/src/printRunParameter.cpp:563-568.
  • readObservingDirection() returns true when neither MC_runheader nor runparameterV2 is present. Source: Eventdisplay/src/printRunParameter.cpp:162-208.

Impact: shell scripts using printRunParameter for file naming or control flow continue with empty or misleading metadata instead of failing.

MISC-2 · Low — VEventLoop::printRunInfos() reports wrong telescope types for subset arrays

if( i < getDetectorGeometry()->getTelType().size() )
    cout << " (type " << getDetectorGeometry()->getTelType()[i] << ")";

Source: Eventdisplay/src/VEventLoop.cpp:134-142.

Impact: when fTelToAnalyze is a subset or reordered telescope list, the reported types do not match the selected IDs. Diagnostic output is misleading.

MISC-3 · Low — logFile falls back to a default log on an unrecognized name

Source: Eventdisplay/src/logFile.cpp:81-107.

Impact: a misspelled log name succeeds silently, printing to the wrong log. Automated provenance extraction can record incorrect metadata.


14. FITS / DL2 / DL3 export

FITS-1 · High — DL2 FITS converter omits reconstructed event-direction coordinates

The ROOT DL2EventTree contains reconstructed camera offsets (xoff, yoff) and pointing, but generate_DL2_file.py exports only:
MC_AZ, MC_ALT, AZ, ALT, MC_ENERGY, ENERGY, MULTIP, PNT_AZ, PNT_ALT, GH_MVA.

Source: Converters/DL2/generate_DL2_file.py:126-137; Eventdisplay/src/VEffectiveAreaCalculator.cpp:408-413, :3592-3599.

Impact: the FITS DL2 output contains no reconstructed sky position and no camera offsets from which it could be rebuilt. Downstream physics analysis cannot recover event directions from this file.

FITS-2 · High — DL3 background FITS header declares the wrong field-of-view coordinate system

The background HDU is filled with synthetic DETX/DETY detector-offset coordinates, but the FITS header states:

write_fits_keyword( "FOVALIGN", "RADEC", "FOV alignment" );

Source: Converters/DL3-IRFs/src/VDL3IRFs.cpp:187-193, :610-699.

Impact: the file claims celestial RA/Dec alignment while the data are detector-offset coordinates. Downstream IRF consumers that honour FOVALIGN will misinterpret background geometry.

FITS-3 · High — DL3 PSF 3-Gauss normalization is dimensionally inconsistent

The sigma is derived in degrees but the normalization uses:

scale.push_back( 1./ (2.*TMath::Pi()*data.back() ) );

with SCALE stored in sr⁻¹ and SIGMA_* in deg.

Source: Converters/DL3-IRFs/src/VDL3IRFs.cpp:465-496, :533-546.

Impact: the normalization misses at least one power of sigma and does not convert between degrees and steradians. The written PSF cannot be a correctly normalised 2D angular PDF.

FITS-4 · Medium — DL2 event-type export is broken for multi-file input

The CLI accepts multiple ROOT files (nargs=-1) but in event_type mode loads exactly one sidecar file:

event_types = np.loadtxt(filenames[0].replace(".eff-0.root", ".txt"), dtype=float)
data_mask[data_mask] = event_types == event_type

Source: Converters/DL2/generate_DL2_file.py:33, :59-63, :111-119.

Impact: the event-type mask applies only to events from the first file while data_mask spans all files, producing logically inconsistent output.

FITS-5 · Medium — Legacy VFITS has a stack-buffer overrun for non-square sky maps

createImageFitsFile() allocates row pointers sized by X bins but initialises them over Y bins:

double* array[hSkyMap->GetNbinsX()];
for( int ii = 1; ii < naxes[1]; ii++ )
    array[ii] = array[ii - 1] + naxes[0];

Source: Eventdisplay/src/VFITS.cpp:883-904.

Impact: when Y bins > X bins, the initialisation loop writes past the stack-allocated pointer array — a memory-safety bug.

FITS-6 · Medium — Legacy VFITS table writers read the ROOT underflow bin and miss the last real bin

Multiple writers iterate i = 0 to < GetNbinsX() and call ROOT bin accessors with i (not i+1):

runValues.push_back( h->GetBinCenter( i ) );
runValues.push_back( h->GetBinContent( i ) );

Source: Eventdisplay/src/VFITS.cpp:540-565, :618-624.

Impact: FITS tables start with the underflow value and drop the highest real bin, silently shifting histogram exports.

FITS-7 · Low — Legacy VFITS graph writer uses a self-acknowledged wrong Δx formula

runValues.push_back( x - x1 ); // Note: this is wrong and should be fixed (delta_x)

Source: Eventdisplay/src/VFITS.cpp:667-670.

Impact: the first Delta_* entry is wrong by construction; all later entries represent point-to-point spacing rather than a real bin width.

FITS-8 · Medium — DL2 and DL3 converters hard-code stale production identifiers

  • generate_DL2_file.py writes CREATOR = "Eventdisplay prod5" regardless of production. Source: line 175-183.
  • convertSensitivityFilesToFITS.cpp writes CTA (MC prod5, v0.1.1) in all primary headers. Source: line 74-76.

Impact: provenance metadata in exported FITS files is incorrect for prod6 and later productions.


15. Analysis parameter files

PAR-1 · High — prod6 full-moon calibration IPR files are missing for most zenith/site combinations

The EVNDISP worker constructs:

PEDFIL="$CTA_EVNDISP_AUX_DIR/Calibration/prod6/prod6-${obs}-full-ze${ZE}-IPR.root"

Source: CTA.EVNDISP.sub_convert_and_analyse_MC_VDST_ArrayJob.sh:112-126.

The directory contains only prod6-north-full-ze40deg-IPR.root and prod6-south-full-ze40deg-IPR.root. Files for 20°, 52°, and 60° zenith are absent for both sites. Site-generic files (prod6-full-ze20deg-IPR.root, etc.) exist but do not match the active naming pattern.

Impact: prod6 full-moon EVNDISP jobs at any zenith angle other than 40° fail when resolving the calibration file.

PAR-2 · Medium — EFFECTIVEAREA.runparameter references a non-existent cut file and an obsolete path convention

* CUTFILE ANASUM.GammaHadron.dat
(if no path is given, file is searched in $VERITAS_EVNDISP_ANA_DIR/ParameterFiles/)

Source: Eventdisplay_AnalysisFiles_CTA/ParameterFiles/EFFECTIVEAREA.runparameter:45-46.

The current CTA wrappers use split .gamma.dat/.CRbck.dat templates, not ANASUM.GammaHadron.dat, and use $CTA_EVNDISP_ANA_DIR, not $VERITAS_EVNDISP_ANA_DIR.

Impact: running makeEffectiveArea directly from this file starts from an outdated environment convention and a non-existent cut file.

PAR-3 · Low — README under-documents active analysis-file inputs

Eventdisplay_AnalysisFiles_CTA/README.md does not mention calibration IPR files or TMVA.BDTDispQualityCuts.runparameter, both of which are required by the active pipeline.

Source: Eventdisplay_AnalysisFiles_CTA/README.md:11-31.


16. Log file evidence

LOG-1 · High — DISP training submitted for telescope types absent from the array (log-confirmed)

total number of telescopes: 30 (selected 0)
..nothing to do. Exiting.

Observed in logFiles/BDTDisp-10408618.training.log:46-48, BDTDisp-201109916.training.log:46-48, BDTDisp-205008707.training.log:46-48.

Successful training logs exist only for types 10608418 and 138704810 — the only types present in the prod6 North subarray, as confirmed by logFiles/makeTable.log:60-95 and logFiles/mscwTable.log:181-218.

Impact: the pipeline submits DISP training jobs for every telescope type regardless of array composition. All jobs for absent types are wasted and generate no usable models.

LOG-2 · Medium — Production log set spans multiple incompatible production tags

The bundled logs for the same prod6 North array use different tags:

Stage Tag
Lookup tables g20260214
TMVA training g20250822
Effective areas g20260325
PHYS/IRF g20260129

Source: logFiles/makeTable.log:16, logFiles/tmva.log:5, logFiles/effArea_angres.log:7, logFiles/irf.log:21.

Impact: these logs do not represent a single coherent end-to-end production run. Cross-stage comparisons within this log set must account for different analysis namespaces.


Summary table

ID Severity Component Short description
STEREO-1 Critical VStereoReconstruction removeDataSet inverted logic; always returns false
ML-1 Critical TMVA training intersect_mc_error Y-component zeroed by typo
ENV-1 High Orchestration Hard-coded DESY AFS ROOT path
ENV-2 High Orchestration Dead array configurations from cascading reassignment
EVN-1 High EVNDISP continue outside loop terminates EVNDISP branch abnormally
EVN-2 High EVNDISP Fragile simtel input discovery via unguarded glob
TAB-1 High Lookup tables Stereo-angle cut differs between table filling (10°) and analysis (5°/15°)
TAB-2 High Lookup tables Undefined MEANDIST in table job filenames
DISP-1 High DISP training Argument parsing broken: wrong positional guards, unquoted tests always true, silent job loss
DISP-2 High DISP training Training submitted for telescope types absent from array
TMVA-1 High TMVA training prepareTMVA conflates direction and batch-options in positional argument $5
TMVA-2 High TMVA training BDT energy bins 0 and 1 overlap completely; no exclusive low-energy classifier
TMVA-3 High TMVA training Training aborts on first missing file with no per-bin retry
XGB-1 High XGB stereo Apply wrapper caps MINTEL at 3; loads wrong model when training used MINTEL > 3
XGB-2 High XGB stereo XGB classification not wired into orchestration layer
XGB-3 High XGB / Python configure_train() treats energy_bins_log10_tev as dict; loader treats it as list
IRF-1 High PHYS dispatch Trailing space in subarray filename argument
IRF-2 High IRF runparameter Key THETA2MINENEERGY likely misspelled; silently ignored by parser
SIG-1 High VTraceHandler Sliding-window raw mode returns FADC[1] instead of integrated charge
SIG-2 High VTraceHandler Sliding-window loop accesses one sample past trace end
IRF-LIB-1 High makeEffectiveArea MC header documented as optional but library hard-exits if missing
IRF-LIB-2 High IRF reader True-energy axis handling disabled unconditionally
IRF-LIB-3 High IRF reader Histogram name/unit inconsistency compensated by heuristic fallbacks
IRF-LIB-4 High IRF runparameter Unknown runparameter keys silently ignored
PHYS-LIB-1 High Physics tree Hard-coded LUSTRE path in writeCTAWPPhysSensitivityTree
PHYS-LIB-2 High Physics tree SC-MST telescope entries never stored (unreachable branch)
PHYS-LIB-3 High Physics tree Energy centres read from ROOT underflow bin
ML-2 High TMVA training Zenith-bin range check off by one; out-of-bounds access
ML-3 High TMVA training get_largest_weight_disp_entries sorts ascending, selects smallest-weight images
ML-4 High TMVA training Per-telescope feature remapping overwrites source arrays in place
FITS-1 High DL2 converter Reconstructed event direction absent from FITS output
FITS-2 High DL3 converter Background FITS header declares wrong FOV coordinate system
FITS-3 High DL3 converter PSF 3-Gauss normalization dimensionally inconsistent
PAR-1 High Analysis files prod6 full-moon IPR calibration files missing for most zenith/site combinations
LOG-1 High Log evidence DISP training jobs submitted for telescope types absent from selected array
STEREO-2 High VOnOff OFF histogram mutated during on-off subtraction
STEREO-3 High VArrayAnalyzer Method-4 pair-angle rejection tests slopes of first telescope pair, not current pair
MISC-1 High printRunParameter File-open failure exits with code 0; missing metadata returns true
TAB-3 Medium Lookup tables Undefined NC in analysis output filenames
TAB-4 Medium Lookup tables DISP training/analysis cut synchronisation enforced only by comment
TAB-5 Medium mscw_energy CLI Contradictory -fill and -tmva_nimages_max validation messages
DISP-3 Medium DISP training Training failure detected only by log-text grep
TMVA-4 Medium PHYS wrapper Unquoted positional guards always true in CTA.WPPhysWriter.sub.sh
TMVA-5 Medium TMVA training Diagnostic training artifacts deleted unconditionally
XGB-4 Medium XGB stereo Training wrapper bypasses external hyperparameter-config hook
XGB-5 Medium XGB stereo Direction argument documented but never read
IRF-3 Medium IRF validation Effective-area success inferred from file size
IRF-4 Medium IRF writing Summary-tree builder overrides PHYSDIR immediately after setting it
SIG-3 Medium VTraceHandler Low-gain saturation threshold computed from previous call's tmax
IRF-LIB-5 Medium IRF binning Default energy/offset axes duplicated by contract between two classes
IRF-LIB-6 Medium TMVA evaluator XML variable extraction depends on exact string layout
IRF-LIB-7 Medium TMVA evaluator Incomplete TMVA weight grids pass initialisation with warnings only
IRF-LIB-8 Medium IRF filling Shared MCaz mutated in-place inside the azimuth-bin loop
IRF-LIB-9 Medium makeEffectiveArea IRF resolution duplication is order-dependent by vector index
IRF-LIB-10 Medium makeEffectiveArea First ROOT input file handle never closed in copyMCHistograms()
ML-5 Medium TMVA training Reconstruction-error feature stored under misleading branch name
ML-6 Medium TMVA training Angular-method training splits both classes from signal tree (correct design, but compounds ML-1)
STEREO-4 Medium VEmissionHeightCalculator Pair weights divide by zero for size ≤ 1
STEREO-5 Medium VEmissionHeightCalculator Telescope positions accumulate without clear across reinitialisations
FITS-4 Medium DL2 converter Event-type export broken for multi-file input
FITS-5 Medium VFITS Stack-buffer overrun for non-square sky maps
FITS-6 Medium VFITS Table writers read ROOT underflow bin and drop last real bin
FITS-8 Medium Converters Hard-coded prod5 creator string in DL2 and DL3 FITS headers
PAR-2 Medium Analysis files EFFECTIVEAREA.runparameter references non-existent cut file and obsolete path
ENV-3 Medium Orchestration Production version tags encoded as script literals
EVN-3 Medium EVNDISP Extensive unquoted paths in file operations
EVN-4 Medium EVNDISP Low-gain calibration multipliers absent for trace integration method 2
EVN-5 Medium EVNDISP / calibration Low-gain T0 assignment always overwrites the high-gain field
EVN-6 Medium EVNDISP / calibration Low-gain T0 calculation uses high-gain charge extraction path
LOG-2 Medium Log evidence Log set covers multiple incompatible production tags
SIG-4 Low VImageParameterCalculation Muon-ring correction divides by zero at first grid point
XGB-6 Low Analysis files DL2.production.runparameter is legacy TMVA example
FITS-7 Low VFITS Graph writer Δx formula acknowledged wrong in source comment
PAR-3 Low Analysis files README omits required calibration and quality-cut files
MISC-2 Low VEventLoop Wrong telescope type reported for subset arrays in diagnostic output
MISC-3 Low logFile Misspelled log name silently returns a different log object

Open items requiring confirmation

  • Signal-efficiency threshold per multiplicity: CUTS applies 80% max signal efficiency for MST >= 4 and 95% otherwise (CTA.runAnalysis.sh:722-747). The physics motivation for this split should be verified.
  • Exposure-dependent re-running: CUTS and PHYS are the only stages executed for non-50h observation times (CTA.runAnalysis.sh:548-553). If upstream lookup tables or BDT models should also vary with exposure, this is a scope inconsistency.
  • DL2FILLING mode compatibility: FULLTREES is set for DL2plus or prod6, DL2 otherwise (CTA.runAnalysis.sh:600-608). Compatibility with downstream DL2 consumers for both modes should be confirmed.
  • Testing coverage gap: the scripts repository performs presence/count checks on DISP, TMVA, and effective-area outputs (testProduction/README.md:1-40), but no end-to-end parameter-consistency validation exists across the MAKETABLESANATABLESTRAINCUTS chain.

Cross-check against VERITAS pipeline review

The VERITAS (VTS) pipeline uses the same Eventdisplay C++ codebase. Issues flagged in vts_eventdisplay_pipeline_review.md were cross-checked against the CTA codebase. The following VTS issues were verified as also present in CTA and have been added to this report:

VTS ID CTA ID Notes
A7 STEREO-3 Confirmed at VArrayAnalyzer.cpp:1350
A10 STEREO-4 Confirmed at VEmissionHeightCalculator.cpp:61
A11 STEREO-5 Confirmed in both setTelescopePositions() overloads
A4 EVN-5 Confirmed at VCalibrationData.cpp:380-388
A5 EVN-6 Confirmed at VCalibrator.cpp:868
A9 SIG-4 Confirmed at VImageParameterCalculation.cpp:744-745
B4 IRF-LIB-7 Confirmed in VTMVAEvaluator::initializeWeightFiles()
B6 IRF-LIB-8 Confirmed at VInstrumentResponseFunction.cpp:258-260
B7 IRF-LIB-9 Confirmed at makeEffectiveArea.cpp:297-300
B8 IRF-LIB-10 Confirmed at makeEffectiveArea.cpp:567-579

The following VTS issues were verified as not applicable to CTA:

VTS ID Reason
A1 CTA VTraceHandler.cpp has an explicit maxpos-- guard; access is bounded
A2 Already documented as CTA SIG-1
A3 VFitTraceHandler.cpp is not present in the CTA codebase
A6 CTA VArrayAnalyzer.cpp:1056 correctly tests both ximp and yimp
A12 CTA VInstrumentResponseFunctionData.cpp:485-486 was already fixed (comment: "was xcore both times")
A13 Already documented as CTA ML-2
A14 CTA trainTMVAforGammaHadronSeparation.cpp:510 correctly uses AnalysisType=Regression
B1/B2 CTA integrates XGB via E_ReconstructionType::XGBSTEREO enum, not the VTS getXGBTree() pattern
B3 Already documented as CTA IRF-LIB-6
B9 The analysisType == 5 dead branch is not present in the CTA anasum.cpp
B10 VTS-specific (real-data epoch reevaluation); CTA pipeline is MC-only
C1–C22 VTS-specific shell orchestration (VERITAS DB, laser calibration, epoch handling)
D1–D11 VTS-specific IRF production scripts
E6 Already documented as CTA XGB-3
E13 CTA TMVA parameter files use AdaBoost/Bagging; the NegWeightTreatment+Grad mismatch does not apply
F1–F12 VTS-specific auxiliary files and epoch metadata
G1–G7 VTS-specific release-test infrastructure
H1–H7 VTS-specific data-analysis performance (real-data evndisp/anasum throughput)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions