Skip to content

feat(dask): remove dask#1365

Draft
paul-nechifor wants to merge 4 commits intodevfrom
paul/feat/remove-dask
Draft

feat(dask): remove dask#1365
paul-nechifor wants to merge 4 commits intodevfrom
paul/feat/remove-dask

Conversation

@paul-nechifor
Copy link
Contributor

@paul-nechifor paul-nechifor commented Feb 25, 2026

Problem

  • Dask is slow to start.
  • Causes issues with shutdown (you get ugly errors when hitting Ctrl+C).
  • Required a lot of ugly patches in core/__init__.py
  • Causes confusion because things are imported once per worker. (E.g.: see Jeff's rerun issue, or my actor confusion)

Closes DIM-609

Solution

  • Removed the dask package entirely.
  • Moved imports out of dimos.core. Now you have to import from the original place.
  • Removed everything in dimos/core/__init__.py.
  • Made dimos shut down cleanly, without any exceptions, and right away.

Breaking Changes

  • UtilizationModule is gone.
  • Blueprint configurations cannot use lambdas anymore because multiprocessing requires params to be pickleable, and lambdas are not because they don't have names. This worked with Dask because Dask serialized the bytecode of the lambda and reconstructed it on the other side.

How to Test

Test any blueprint. Example:

uv run dimos run unitree-go2

Contributor License Agreement

  • I have read and approved the CLA.

@paul-nechifor paul-nechifor marked this pull request as draft February 25, 2026 06:33
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 25, 2026

Too many files changed for review. (133 files found, 100 file limit)

opacity=0.2,
background="#484981",
),
"world/camera_info": _convert_camera_info,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason this change is needed is because multiprocessing can't pickle lambdas. Lambdas don't have names so they can't be pickled because there's nothing to reference them from the other side.

Dask cheats by serializing the bytecode of the lambda.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant