1.0.1

What's New

Bug Fixes

• Convert dtypes during joining using lydata's cast_dtypes().

1.0.0

What's New

Compared to the last "stable" version 0.7.3, everything is basically a full rewrite, has been entirely removed, or is an entirely new feature.

Most importantly, we now rely heavily on pydantic for the CLI and the validation of all command inputs. This allows us to merge and validate inputs from several sources, e.g. multiple YAML files and the CLI args. A schema defining the structure of these is available under schemas/ly.json. This means that e.g. in VS Code, one can reuse this schema for any *.ly.yaml files and get docs and autocompletion.

Below are the changes relative to the last pre-release (1.0.0rc3):

Bug Fixes

• Change info to core for mid-level lydata columns
• Use bug-fixed lydata .ly.enhance() method.

Documentation

• Update documentation for integrate and evidence commands.
• Improve data collect description.

Features

• Add integrate command for thermodynamic integration. Thanks @noemibuehrer!
• Add command spawning websever for interactive data collection.

Miscellaneous Tasks

• Add missing links to changelog.
• Add CITATION.cff.

Testing

• Update tests for new lydata.

Build

• Add uvicorn, fastapi to deps.
• Require at least lydata 0.4.0.

Change

• Make compatible with new lyDATA version.
• Centralize inverse temperature schedule generation.
• Store selected log-level globally.
• Disable properties in collector.

1.0.0rc3

What's New

Documentation

• Fix join command's example call.

Features

• Add data fetch command. Fixes #75.

Change

• Access data only via lydata for compatibility.
  We have changed the lydata 2nd level headers slightly for the patient and tumor info (see lycosystem/lydata#21 for more info).
  Since the lydata package was already updated to be compatible with that change, we simply need to route every access the lyscripts make to the data through lydata package and hence be compatible too.
• Make CLI work with new lydata format.
  This is also related to lycosystem/lydata#21

1.0.0rc2

What's New

Bug Fixes

• Divide by midext prevalence. Fixes #72.
  This fixes a bug we reintroduced where we didn't compute observed and
  model prevalence in an analogous and comparable way.

Documentation

• Fix build badge in README.
• Fix outdated links to rmnldwg.

Miscellaneous Tasks

• Update pre-commit & ruff rules.

Testing

• Fix the dataset used for testing prevalences.

Build

• Switch to src layout. Fixes #74.

Ci

• Use tests action with coverage.

1.0.0rc1

What's New

The new package version (v1) is getting smaller and smaller bug fixes, indicating that we may be able to release 1.0.0 soon. Below are the most recent changes:

Bug Fixes

• Use 'fork' start method under MacOS.
• Specify config file encoding for other OSes.
• Cast scenario pattern to bool if possible. Fixes #70.
  Since we allow defining a pattern with keywords like "involved" or 1
  instead of only True, we also need to make sure that is correctly cast
  to its boolean value for lydata's C objects.

Documentation

• Add warning for Windows & MacOS regarding multiprocess(ing).

Miscellaneous Tasks

• Add link to changelog.

Testing

• Ensure observed prevalence is correct. Related #70.

Build

• Exclude buggy pydantic-settings. pydantic/pydantic-settings#605 is present in
  version 2.9.0 and 2.9.1 of pydantic-settings. So, these versions must be
  excluded.

Ci

• Update release scripts to use OIDC. This is more secure.

Remove

• Unnecessary custom help formatter.

1.0.0.a7

What's New

Bug Fixes

• Make config -> model -> config round trip test pass.
• Remove thin_by factor in wrong place.
• Pass involvement & diagnosis correctly to risks and prevalences.

Features

• Create Modality config from model.
• Create Graph config from model.
• Create Model config from model.

Note that the three features above come with certain limitations. It is not possible
to export all aspects of a model to a configuration. Especially the distributions
over diagnosis times cannot be converted to a DistributionConfig.

Miscellaneous Tasks

• Fix changelog version link.
• Update ly schema.

Testing

• Round trip config -> model -> config.

1.0.0.a6

What's New

Mostly fixes and tests regarding the correct reproducibility of (midline) models created from a v0 config.

Bug Fixes

• Better handle midline model.
  This means disabling the evolution over midline extension. Also, since the new
  version of lymph-model, the midext_prob parameter is not epected to be the
  first one anymore when passed to set_params().
• Pass only ipsilateral diagnosis to unilateral model.
• Pass diagnose & involvement correctly to models as dict.

Testing

• Ensure unilateral model receives correct diagnosis.
• Test that diagnosis is used correctly in posteriors.

Build

• Bump lydata & lymph-model dependency.

1.0.0.a5

What's New

After some more "real-world" testing of the new CLIs and APIs while working on LyProX, some improvements where made and some bugs were fixed.

Bug Fixes

• Provide rich console to compute progress bars.
• Correctly build deprecated models.

Features

• Enable sampling only named parameter subset.

Testing

• Check model construction & named params.
• Update integration test config YAML files.
• Add external model symbol check.
• Adding dists works correctly.
  Previously, it could happen that in a Bilateral or Midline model the
  individual submodel's distributions where not synced.

Build

• Bump lymph-model.

Change

• Require every YAML file to have version.
• Better version-related error and docs.
• Make in-/output names more consistent.

Remove

• Drop CLI required argument version.

1.0.0.a4

What's New

In this pre-release, all CLIs were refactored or rewritten to make heavy use of pydantic models and settings. This allows for both cleaner and pydantic's amazing validation out of the box.

It also allows us to provide configuration YAML files to all commands that may even be split and merged and reused as we wish.

After some testing of the package in this state, we may be able to publish a stable 1.0 release.

Bug Fixes

• Update data loading to new lydata API.
• Add sampling config back to sample CLI.
• Finish data filter command.
• Correctly log number of excluded pateints in lyproxify.
• Allow extra args in CLI cmds.
• Logging during progress bar.

Documentation

• Deactivate help of removed commands.
• Link only to stable versions.
• Fix intersphinx links.
• Update link to schedule module.
• Configure how pydantic models are displayed.
• Add more info about schema.
• Better explain sampling.
• Add proper info to cli_cmd() methods.

Features

• Add mandatory version field to command settings.
  This will allow to differentiate between old and new configs and create
  the models accordingly.
• Add translation of old to new model configs.
• Add dynamic YAML config source.
• Configure logging nicely.
• Update data enhance command.
• Update data join command.
• Update data filter command.
• Update data split command.
• Capture lydata logging output.
• Update YAML schema for CLIs.
• Update data lyproxify cmd.
• Allow sampling only specified params.
  Via a new CLI arg named param_names one may restrict the parameters
  sampled to a named subset. In combination with the fact that any Python
  model may be loaded, this results in an enormously flexible sampler.
• Update inv temp schedule cmd.
• Allow providing start state to sampling func.

Testing

• Replace subprocess calls with monkeypatch.
  This allows for better debugging during test calls.
• Load generated data correctly.
  The synthetic data for testing already has "early" and "late" as
  T-stages. Thus, the mapping needed to be adapted.

Build

• pydantic-settings >= 2.7 needed.

Change

• Make version field in command settings required.
• Use pydantic for subcommands.
• Use loguru over default logging.
• Rename data field to input.
• Use pydantic for plot utils, too.
• Use rich logging handler.

Refac

• Slightly change CLI inheritances.
• Sort configs alphabetically.
• Make sampling more reusable.

Remove

• Unused utility functions.
• Plotting scripts except histogram/betaposterp helpers.

1.0.0.a3

What's New

This is (again) a fairly substantial rewrite of the scripts. They now use pydantic for configuration management. This allows multiple YAML files to be merged with command line arguments and also implements exceptional data validation. Also, we now use joblib to cache intermediate results. This also allows for effortless caching when the functions are used as a library instead of a CLI.

What's now left to do is tidy up everything, read through the docs and then publish a proper 1.0.0 release.

Bug Fixes

• (plot) Turn off label by passing None.
• (plot) Don't fail on wrong wrong in .draw().
• Polish dataclass configuration.
• Make pydantic work with subparsers.
• (data) Correct argument name of save function.
• (comp) Add distributions & fix dir type for priors.
• (config) Don't thin samples twice.

Documentation

• (configs) Add to sphinx docs.
• Add intersphinx link to lydata.
• (config) Improve construct_model docstring.
• Correct copyright year.
• Clean up refs to deleted modules.
• Fix links to documentation in readme.

Features

• (config) Write new methods to assemble model.
• (plot) Add func split_legends(). Related [#60].
  This allows the user to separate many plot's labels into a number of
  different legends.
• (configs) Add graph config.
• (sample) Better config management.
  The new sample command can merge configurations from multiple YAML files
  as well as the command line. It does a better job at reporting progress
  when the log-level is set to debug and it more consistently stores
  samples and metric logs during burn-in.
• (sample) Add JSON schema for params.
  With this JSON schema one can get auto-completion in most popular
  code editors for the configs of most commands.
• (config) Allow loading external model.
  It is now possible to not only specify a model using a fixed YAML
  schema, but also via a plain Python file that defines a global model
  variable.
• (sample) Add inverse temperature.
  With this addition, it is now in principle possible to do thermodynamic
  integration. But it is not yet fully fleshed out for a nice user
  experience.
• Add tiny script to generate JSON schema.
  The script creates a JSON schema based on all CLI commands' settings.
• (comp) Rewrite posteriors command that uses pydantic and joblib
• (comp) Rewrite risks command that uses pydantic and joblib
• (sample) Add iterations/second column to burnin progress display.
• (sample) Show total iterations to sampling progress display.
• (config) Allow converting diagnosis to involvement.
• (compute) Rewrite prevalence command that uses pydantic and joblib

Miscellaneous Tasks

• Bump pre-commit hooks.
• Add ruff linting rules.
• Switch to ruff, drop pycln & isort.
• Run ruff & clean up codebase.
• Ignore some ruff rules in tests dir.

Styling

• More cleanup to satisfy ruff.
• Improve docstrings and code style a bit.

Testing

• Add some basic testing for config.
• (data) Basic integration test for generate.
• Add sampling step to integration tests.
• (config) Check the external loading feature.
• Extend & unify integration test for priors.

Build

• Remove upper cap in dependencies.
• Bump lymph-model to 1.2.3.
• Remove dev from optional dependencies.
  This is because a lot of dev tools like ruff, pre-commit, ... are
  installed globally (e.g. with pipx) instead of per venv.
• Bump lydata dependency to 0.2.0.

Change

• (plot) Improve beta post & hist. Fixes [60].
  The histograms and beta posteriors are now better implemented, allowing
  a user to extend the draw() function's abilities by adding classes
  similar to Histogram and BetaPosterior.
• (plot) More flexible labels.
• Use pydantic over dacite.
• Switch to pydantic config for sampling (WIP).
• (compute) Use pydantic & joblib over dataclasses and custom caching.
• ⚠ BREAKING (compute) Add priors joblib cache
• ⚠ BREAKING (data) Replace the generate cmd.
  This was just supposed to be a little script to generate data for an
  integration test, but it turns out that it could just be used to update
  the old generate command.
  BREAKING CHANGES: generate command is better configurable
• (config) Merge sample/sampling configs.
• Use lydata's ModalityConfig.
  Since the lydata package is
  evolving quickly, I added it as a dependency and moved the first bit of
  code over there.
• Enable use of lydata to load patient data.
• (comp) HDF5 file storage more versatile.
• (sample) Store history in .tmp file.
  This serves an indication about whether or not a burn-in sampling round
  has been interrupted. The sampler may then continue from where it left off.
• ⚠ BREAKING (compute) Update prevalence computation.
• More useful YAML load/merge logging.
• Improve logging of some utilities.

Remove

• Outdated streamlit app.
• Temporary test file.
• Delete remaining streamlit code.
• ⚠ BREAKING Unused HDF5 cache and scenarios.
  Both these things are superseded by better stuff based on pydantic.