Version 1.0.0

This is a major release which offers a serious update to the interface, documentation, and backend of FISSA.

Please note that some of the changes to the codebase are backward-incompatible changes. For the most part, the only breaking changes which users will need to be concerned with are listed below. We also recommend looking through our updated tutorials and examples.

• The format of the cache used by the new release is different to previous versions. The new version is not capable of restoring previous results saved in the old cache format. If it is run with folder set to a directory containing a cache in the old format, it will ignore the old cache and run from scratch, saving the new cache in the same directory with a different file name and format.
• The shape of experiment.deltaf_raw has been changed to be a numpy.ndarray of 2d numpy arrays shaped (1, time) to match the shape of the other outputs.

Although we have noted some other breaking changes, end users are very unlikely to be affected by these. These other changes will only affect users that have written their own custom datahandler, or who are directly calling backend tools such as the ROI module, or the functions fissa.core.separate_func and fissa.core.extract_func, all of which which were either removed or refactored and renamed in this release.

The most important addition is fissa.run_fissa, a new high-level function-based interface to FISSA. This function does the same operations as fissa.Experiment, but has a more streamlined interface.

Breaking changes

• The names of the cache files have changed from "preparation.npy" and "separated.npy" to "prepared.npz" and "separated.npz", and the structure of the contents was changed. FISSA output caches generated with version 0.7.x will not no longer be loaded when using version 1.0.0. The new version stores analysis parameters and TIFF means along with the raw and decontaminated traces. (#177, #223, #245, #246)
• The shape of experiment.deltaf_raw was changed from a numpy.ndarray shaped (cells, trials), each containing a numpy.ndarray shaped (time, ), to numpy.ndarray shaped (cells, trials), each element of which is shaped (1, time). The new output shape matches that of experiment.raw and experiment.deltaf_result. (#198)
• The way data handlers were defined was reworked to use classes instead. The datahandler and datahandler_framebyframe modules were dropped, and replaced with an extraction module containing both of these data handlers as classes instead. Custom data handlers will need to be rewritten to be a class inheriting from fissa.extraction.DataHandlerAbstract instead of a custom module.
• The ROI module was renamed to polygons. (#219)
• In neuropil.separate, the keyword arguments maxiter and maxtries were renamed to be max_iter and max_tries. This change was made so that the parameter name max_iter is the same as the parameter name used by sklearn.decomposition.NMF. (#230)
• The internal functions fissa.core.extract_func and fissa.core.separate_func were removed. New functions which are comparable but actually have user-friendly interfaces, fissa.core.extract and fissa.core.separate_trials, were added in their place.

Changed

• The outputs experiment.raw, experiment.sep, and experiment.deltaf_raw were changed from a list of lists of 2d numpy arrays, to a 2d numpy array of 2d numpy arrays. Other outputs, such as experiment.result were already a 2d numpy array of 2d numpy arrays. (#164)
• Output arrays (experiment.result, etc.) are now initialized as empty arrays instead of lists of lists of None objects. (#212)
• The multiprocessing back-end now uses joblib instead of the multiprocessing module. (#227)
• Unnecessary warnings about ragged numpy.ndarrays were suppressed. (#243, #247)
• Output properties are now automatically wiped when parameter attributes are changes. (#254)
• The set of extra requirements named "dev" which specified the requirements needed to run the test suite was renamed to "test". This can be installed with pip install fissa[test]. There is still a "dev" set of extras, but these are now development tools and no longer include the requirements needed to run the unit tests. (#185)

Fixed

• The preparation and separation steps are no longer needlessly re-run (#171, #172)
• Mean images are saved with float64 precision regardless of the precision of the source TIFFs file. (#176)
• Various bugs in the Suite2p workflow. (#181, #257)
• Variables set to None are no longer saved as numpy.ndarray. (#199)
• An error is now raised when both lowmemory mode and a manual datahandler are provided. (#206)
• Mismatches between the number of rois/trials/etc. and the array shapes (which can occur when the data in experiment.raw is overwritten) are resolved by determining these attributes dynamically. (#244)
• Use np.array instead of the deprecated np.matrix. (#174)
• Use np.float64 instead of the deprecated np.float. (#213)
• Iterate over elements in shapely.geometry.MultiPolygon by using the geoms attribute instead treating the whole object as an iterable (which is now deprecated). (#272)

Added

• Added fissa.run_fissa, a high-level function-based interface to FISSA. This does the same operations as fissa.Experiment, but in a more streamlined interface. (#169, #237)
• Added a verbosity argument to control how much feedback is given by FISSA when it is running. (#200, #225, #238, #240)
• A new fissa.Experiment.to_matfile method was added. The arrays saved in this matfile have a different format to the previous fissa.Experiment.save_to_matlab method, which is now deprecated. (#249)
• A new data handler extract.DataHandlerTifffileLazy was added, which is able to handle TIFF files of all data types while in low-memory mode. (#156, #179, #187).
• In fissa.Experiment, arguments max_iter, tol, and max_tries were added which pass through to neuropil.separate to control the stopping conditions for the signal separation routine. (#224, #230)
• In fissa.Experiment, add __repr__ and __str__ methods. These changes mean that str(experiment) describes the content of a fissa.Experiment instance in a human readable way, and repr(experiment) in a machine-readable way. (#209, #231)
• Support for arbitrary sklearn.decomposition classes (e.g. FactorAnalysis), not just ICA and NMF. (#188)

Deprecated

• The fissa.Experiment.save_to_matlab method was deprecated. Please use the new fissa.Experiment.to_matfile method instead. The new method has a different output structure by default (which better matches the structure in Python). If you need to continue using the old structure, you can use fissa.Experiment.to_matfile(legacy=True). (#249)

Documentation

• Reworked all the tutorial notebooks to have better flow, and use matplotlib instead of holoviews which is more approachable for new users. (#205, #228, #239, #279)
• The Suite2p example notebook was moved to a separate repository. This change was made because we want to test our other notebooks with the latest versions of their dependencies, but this did not fit well with running Suite2p, which needs a precise combination of dependencies to run.
• Integrated the example notebooks into the documentation generated by Sphinx and shown on readthedocs. (#273)
• Other various notebook improvements. (#248)
• Various documentation improvements. (#153, #162, #166, #167, #175, #182, #183, #184, #193, #194, #204, #207, #210, #214, #218, #232, #233, #236, #253)

Dev changes

• Changed the code style to black. (#215, #258)
• Add pre-commit hooks to enforce code style and catch pyflake errors. (#161, #180, #217, #234, #261)
• Migrate CI test suite to GitHub Actions. (#154, #195)
• Various changes and updates to the test suite. (#170, #191, #197, #201 #202, #211, #221, #222, #226, #235, #255)
• Notebooks are now automatically deployed on github pages. (#178)

Version 0.7.2

Fixed

• Loading ovals and ellipses which are partially offscreen (to the top or left of the image). (#140)

Changed

• Attempting to load any type of ROI which is fully offscreen to the top or left of the image now produces an error. (#140)

Version 0.7.1

Fixed

• Loading oval, ellipse, brush/freehand, freeline, and polyline ImageJ ROIs on Python 3. (#135)

Added

• Support for rotated rectangle and multipoint ROIs on Python 3. (#135)

Version 0.7.0

Security

• Caution: This release knowingly exposes a new security vulnerability. In numpy 1.16, the default behaviour of numpy.load changed to stop loading files saved with pickle compression by default, due to potential security problems. However, the default behaviour of numpy.save is still to save with pickling enabled. In order to preserve our user-experience and backward compatibility with existing fissa cache files, we have changed our behaviour to allow numpy to load from pickled files. (#111)

Changed

• Officially drop support for Python 3.3 and 3.4. Add python_requires to package metadata, specifying Python 2.7 or >=3.5 is required. (#114)
• Allow tuples and other sequences to be image and roi inputs to FISSA, not just lists. (#73)
• Multiprocessing is no longer used when the number of cores is specified as 1. (#74)
• Changed default axis argument to internal function fissa.roitools.shift_2d_array from None to 0. (#54)
• Documentation updates. (#112, #115, #119, #120, #121)

Fixed

• Allow loading from pickled numpy saved files. (#111)
• Problems reading ints correctly from ImageJ rois on Windows; fixed for Python 3 but not Python 2. This problem does not affect Unix, which was already working correctly on both Python 2 and 3. (#90)
• Reject unsupported axis argument to internal function fissa.roitools.shift_2d_array. (#54)
• Don't round number of npil segments down to 0 in fissa.roitools.split_npil when using adaptive_num=True. (#54)
• Handling float num_slices in fissa.roitools.split_npil, for when adaptive_num=True, which was causing problems on Python 3. (#54)

Added

• Test suite additions. (#54, #99)

Version 0.6.4

Release date: 2020-04-07. Full commit changelog.

Fixed

• Fix multiprocessing pool closure on Python 3.8. (#105)

Version 0.6.3

Release date: 2020-04-03. Full commit changelog.

Fixed

• Specify a maximum version for the panel dependency of holoviews on Python <3.6, which allows us to continue supporting Python 3.5, otherwise dependencies fail to install. (#101)
• Save deltaf to MATLAB compatible output. (#70)
• Wipe downstream data stored in the experiment object if upstream data changes, so data that is present is always consistent with each other. (#93)
• Prevent slashes in paths from doubling up if the input path has a trailing slash. (#71)
• Documentation updates. (#91, #88, #97, #89)

Version 0.6.2

Release date: 2020-03-11. Full commit changelog.

Fixed

• Specify a maximum version for tifffile dependency on Python <3.6, which allows us to continue supporting Python 2.7 and 3.5, which otherwise fail to import dependencies correctly.
  (#87)
• Documentation fixes and updates. (#64, #65, #67, #76, #77, #78, #79, #92)

Version 0.6.1

Release date: 2019-03-11. Full commit changelog.

Fixed

• Allow deltaf.findBaselineF0 to run with fewer than 90 samples, by reducing the pad-length if necessary. (#62)
• Basic usage notebook wasn't supplying the correct datahandler_custom argument for the custom datahandler (it was using datahandler instead, which is incorrect; this was silently ignored previously but will now trigger an error). (#62)
• Use ncores_preparation for perparation step, not ncores_separation. (#59)
• Only use ncores_separation for separation step, not all cores. (#59)
• Allow both byte strings and unicode strings to be arguments of functions which require strings. Previously, byte strings were required on Python 2.7 and unicode strings on Python 3. (#60)

Version 0.6.0

Release date: 2019-02-26. Full commit changelog.

Added

• Python 3 compatibility. (#33)
• Documentation generation, with Sphinx, Sphinx-autodoc, and Napoleon. (#38)

Version 0.5.3

Release date: 2019-02-19. Full commit changelog.

Fixed

• Fix f0 detection with low sampling rates. (#27)