Examples of model checks and comparisons

[hazhirr]

Hi,
I have been trying to use the QA, comparison, and checking functionalities but have found the process a bit complex, not knowing what files exactly need to be edited (csv vs. yaml for example) and with what formatting and naming conventions. Are there examples that anybody can share to help one get started with these functionalities? More broadly, additional examples that use the full functionalities of sdeverywhere, including in building more complex graphs, would be extremely helpful.
thanks a lot!
Hazhir

[chrispcampbell]

Hi @hazhirr, thanks for the questions. I've been wanting to improve the documenation and rationale for the model-check tool in SDEverywhere for quite some time, and your questions were a good prompt for continuing that work. Rather than provide long answers here, I expanded the documentation in the Testing and Comparing Your Model page in the wiki. I hope that page will help answer some/most of your questions. Let me know if there are things you'd like clarified.

One thing I'll note here is that the long-term goal of model-check was for the tests to be configurable in an easy-to-use UI, but unfortunately the UI remains only partially developed. The yaml files weren't intended to be the primary way of defining tests; they were supposed to only be the file format for encoding the tests developed in the UI. The yaml file format is relatively human-readable compared to others, but it has many shortcomings (for example, indentation matters), and though we have a well-defined schema for the syntax of check and comparison tests, it's not as easy as it should be to define tests manually in these yaml files.

Until we revive the effort on the easy-to-use UI, I recommend using VS Code to edit these files, and make sure you have the YAML extension installed. The yaml files that are provided in the Quick Start templates are set up with a reference to the schema at the top (which the YAML extension uses) so that you will get some syntax highlighting (and red squiggles to indicate when the syntax is incorrect). It's not great, but better than nothing.

Re: additional examples, I'd recommend checking out this recent discussion thread and the issue for the new Svelte template, which includes slightly more advanced graph configuration.

Due to increased life and (climate) work demands, I haven't been able to devote as much time to SDEverywhere development as I have in the past (or as much as I would like in general). As such I'm always open to others taking a more active role in the project.

Ways you and others can help:

• Please continue to ask questions and provide feedback on things that aren't clear in the documentation.
• Feel free to submit feature/improvement requests if you encounter use cases that aren't possible to express currently (an example would be "add support for model check predicates that look for oscillations").
• If you continue to iterate on your own project (e.g., SimpleCity) and are comfortable developing it in the open, it would be great if you could create a GitHub repo that contains both the mdl and the project structure (including any checks and comparisons you define). Not only will this help you track changes to your model, it'll give you a foundation for (eventually) setting up continuous integration (via GitHub Actions) to run those checks/comparisons automatically with each change, and it would serve as a good real-world example for others in the community to learn from.
• I imagine you are in almost constant contact with bright and energetic students 😄 It would be great if one or more of those students could get involved with SDEverywhere development. I have a long list of things I would like to see improved in and added to SDEverywhere, and these things would be more attainable with support and involvement from the broader SD community, especially those in academia.

[hazhirr]

The documentation you provided is great, and answers a lot of my questions. Thank you Chris. I will try to get myself familiarized with github so I can share more of the modeling work I do on the city dynamics; and will keep in mind if a good student with interest in enhancing sdeverywhere comes into my radar.