Skip to content

Conversation

@GernotMaier
Copy link
Contributor

Improve file management for light emission simulations:

  • use consistently the updated runner_services to have a consistent directory management.
  • use runner_services for improved output and log file naming consistent with rest of simtools.
  • submission scripts and stdout/stderr are now in /sub (as in the other simulation production tools)
  • several changes to have function names and calls mirrored from simulate_prod and simulate_flasher / illuminator (simplifies maintenance)

e.g. the file tree for flasher simulations looks now like:

simtools-output
├── light_emission
│   └── flasher_pulse_shape_MSTN-04_MSFx-NectarCam.dat
├── model
│   └── 7.0.0
│       ├── atm_profile_model_1.dat -> /workdir/external/simtools/simtools-output/model/7.0.0/atmprof_ecmwf_north_winter_fixed.dat
│       ├── atm_trans_2156_1_3_2_0_0_0.1_0.1.dat
│       ├── atmprof_ecmwf_north_winter_fixed.dat
│       ├── atmprof1.dat -> /workdir/external/simtools/simtools-output/model/7.0.0/atmprof_ecmwf_north_winter_fixed.dat
│       ├── Benn_LaPalma_sky_converted.lis
│       ├── camera_CTA-MST-NectarCam_20191120_majority-3nn.dat
│       ├── CTA-North-MSTN-04_simulate_flasher.cfg
│       ├── mirror_CTA-100_1.20-86-0.04.dat
│       ├── NectarCAM_lightguide_efficiency_POP_131019.dat
│       ├── nsb_spectrum_dark.ecsv
│       ├── Pulse_template_nectarCam_17042020-noshift.dat
│       ├── qe_R12992-100-05c.dat
│       ├── ray-tracing-North-MST-NectarCam-D-d10.0-za20.0_validate_optics.ecsv
│       ├── ref_MST-North-MLT_2022_06_28.dat
│       ├── spe_afterpulse_pdf_NectarCam_18122019.dat
│       └── transmission_lst_window_No7-10_ave.dat
├── sim_telarray
│   ├── full_simulation_run000010_North_7.0.0_simulate_flasher.hdata.zst
│   └── full_simulation_run000010_North_7.0.0_simulate_flasher.simtel.zst
└── sub
    ├── full_simulation_run000010_North_7.0.0_simulate_flasher.err
    ├── full_simulation_run000010_North_7.0.0_simulate_flasher.out
    └── full_simulation_run000010_North_7.0.0_simulate_flasher.sh

while before it looked like:

simtools-output
├── ff-1m.ctsim.hdata
├── ff-1m.simtel.zst
├── logfile.log
├── model
│   └── 7.0.0
│       ├── atm_profile_model_1.dat -> /workdir/external/simtools-dev/simtools-output/model/7.0.0/atmprof_ecmwf_north_winter_fixed.dat
│       ├── atm_trans_2156_1_3_2_0_0_0.1_0.1.dat
│       ├── atmprof_ecmwf_north_winter_fixed.dat
│       ├── atmprof1.dat -> /workdir/external/simtools-dev/simtools-output/model/7.0.0/atmprof_ecmwf_north_winter_fixed.dat
│       ├── Benn_LaPalma_sky_converted.lis
│       ├── camera_CTA-MST-NectarCam_20191120_majority-3nn.dat
│       ├── CTA-North-MSTN-04_simulate_flasher.cfg
│       ├── mirror_CTA-100_1.20-86-0.04.dat
│       ├── NectarCAM_lightguide_efficiency_POP_131019.dat
│       ├── nsb_spectrum_dark.ecsv
│       ├── Pulse_template_nectarCam_17042020-noshift.dat
│       ├── qe_R12992-100-05c.dat
│       ├── ray-tracing-North-MST-NectarCam-D-d10.0-za20.0_validate_optics.ecsv
│       ├── ref_MST-North-MLT_2022_06_28.dat
│       ├── spe_afterpulse_pdf_NectarCam_18122019.dat
│       └── transmission_lst_window_No7-10_ave.dat
├── pulse_shapes
│   └── flasher_pulse_shape_MSTN-04_MSFx-NectarCam.dat
├── scripts
│   └── ff-1m-light_emission.sh
└── sim_telarray

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves file management for light emission simulations by consistently using runner_services for directory and file naming, aligning with conventions used in other simulation production tools.

Changes:

  • Refactored SimulatorLightEmission to use runner_services for consistent file and directory management
  • Generalized RunnerServices to support both CorsikaConfig and dict configurations
  • Removed output_prefix parameter and updated file naming to match other simulation tools
  • Moved submission scripts and logs to <output dir>/sub subdirectory

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/simtools/simtel/simulator_light_emission.py Major refactoring to use runner_services, removed prefix logic, renamed methods
src/simtools/runners/runner_services.py Added support for dict configs and iact_output file type
src/simtools/runners/simtel_runner.py Renamed corsika_config parameter to config
src/simtools/runners/corsika_runner.py Refactored to use runner_service.get_file_name()
src/simtools/runners/corsika_simtel_runner.py Updated RunnerServices initialization with named parameters
src/simtools/simtel/simulator_array.py Updated parent class initialization with named parameters
src/simtools/applications/simulate_flasher.py Removed output_prefix parameter, added verify_simulations call
src/simtools/applications/simulate_illuminator.py Removed output_prefix parameter, added run_mode to config
tests/unit_tests/test_simulator.py Added zenith_angle to mock configuration
tests/unit_tests/simtel/test_simulator_light_emission.py Updated tests for renamed methods and refactored implementation
tests/unit_tests/runners/test_simtel_runner.py Updated test to use config parameter name
tests/unit_tests/runners/test_runner_services.py Updated tests with named parameters, added new test case
tests/integration_tests/config/*.yml Updated expected file paths to match new directory structure

GernotMaier and others added 5 commits January 13, 2026 13:12
@ctao-sonarqube
Copy link

@GernotMaier GernotMaier marked this pull request as ready for review January 13, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants