DCSim simulator release v0.3

Production ready version of a distributed computing simulator (DCSim) based on WRENCHv2.2 and SimGridv3.34.
Allows to simulate freely selectable workflows of batch and streaming jobs on arbitrary computing platforms processing freely selectable (amounts of) data.
Thanks to performance updates in WRENCH and SimGrid the memory and computational requirements on executing resources for the simulator could be improved.
This version is used in the studies presented in https://indico.jlab.org/event/459/contributions/11490/.

What's Changed

• Extension the support for platforms consisting of arbitrary numbers of GRID storages and workers by @HerrHorizontal in #24
• Enable fluid model in file read by @HerrHorizontal in #47
• Additional features for the workload generation by @ArturAkh in #49 by @HerrHorizontal in #51
• Extend the sampling of job characteristics by @HerrHorizontal in #54
• Config and plot changes for exemplary scenarios by @HerrHorizontal in #56
• Enable simulation of scheduling and executing multi-core jobs by @HerrHorizontal in #57
• Enable dataset configuration by @rsreds in #53
• Rework simulator configuration to be completely steering-file based
• Update plotting and helper scripts
• Extended examples
• Minor bug fixes

New Contributors

• @rsreds made their first contribution in #52

Full Changelog: v0.2...v0.3

SG-Batch-simulator pre-release v0.2

Completed implementation of the SG-machines-batch benchmark simulator based on WRENCH-2.0. It uses a new job concept of active jobs available in WRENCH-2.0 to enable the simulation of workflows consisting of jobs, which stream their input files, while simultaneously processing the information or jobs, which copy their input-files previous to computation.

The plotting and analysis tools have been updated to support the new job concept.

Thanks to performance improvements in WRENCH-2.0 the scaling of the simulation has also improved significantly (see assets below), when compared to previous releases (approx. factor 10 for memory, factor 0.8 for simulation time).

What's Changed

• New job concept of active jobs performing actions by @HerrHorizontal and @henricasanova in #15
• Support of jobs streaming input files by @HerrHorizontal
• Improvements and adjustments in plotting and analysis tools by @HerrHorizontal and @ArturAkh

Full Changelog: v0.1-hacky...v0.2

DistCachSim SGBatch-simulator pre-release v0.1

First edition of the SG-machines-batch benchmark simulator. This is based on a private hacky version of WRENCH-1.10. Due to a probable bug in the workflow builder (see issue #7) only the simulation of copy-jobs gives meaningful results. For the simulation of streaming-jobs, newer versions of this simulator based on WRENCH-2.0 must be considered.

However, with this release the basic foundations for the simulation of copy-jobs and the analysis of the simulation outputs are given (see attached figures).

Also, it is possible to perform a simple study on the runtime and memory consumption of the simulation with increasing numbers of jobs (see attached figure).

What's Changed

• XRootD-ish blockwise data streaming by @HerrHorizontal in #2. However this is bugged and should not be used, see issue #7
• Test/sgbatch henri small fix by @henricasanova in #3
• Refactor simulation monitoring dump/output and memory usage improvements by @HerrHorizontal in #4
• CLI with the help of boost::program_options by @HerrHorizontal in #9
• Plotting tools by @HerrHorizontal in #13
• Checkout script for easy installation by @HerrHorizontal in #14
• SG-machines-batch benchmark simulator by @HerrHorizontal in #1

New Contributors

• @HerrHorizontal made their first contribution in #2
• @henricasanova made their first contribution in #3
• @ArturAkh

Full Changelog: https://github.com/HerrHorizontal/DistCacheSim/commits/v0.1