Deploying Immich-ML

[DocRuben]

Hello again.

I hope the holidays has treated you well, and I also hope you have found the energy to continue this project.
As I have said before I'm running the machine learning part on a separate server and I'm now pondering
helping to figure out how to get the ML part running from deb-packages.
My server running ML is latest Debian LTS and I've been thinking about the dependencies to make such a package work. A working install of the CUDA drivers (in my case) is obvious.
Have you made any progress in this matter so far?
Right now I'm running the ML-container directly from the Immich project but I'd rather replace it with at non containerized version and I'm willing to help find a way to make that work.

Hope to hear from you soon.
Best regards, Fredrik

[dionysius]

Hey Fredi

Thank you! All is well here. Hope you enjoyed the holidays as well. I was also a bit busy with that apt side project :)

I haven't yet tried using HW-accelleration yet, but I've read through a lot of their building code and this what I think should/could work. I have no idea how to detect whether accelleration is active. Worst case you should be able to use a tool from your driver to see if usage goes up.

Right now it should also be easier, once we do #5 (low prio) this will require more testing or whether this plan will work at all. Right now the immich-machine-learning package only brings immich python code without dependencies, that's also why it is so small. All python-dependencies are loaded and handled by uv at runtime (service start), so any required compilation happens for the current machine while the service starts and thus hopefully is easier to manage and influence. Check journalctl -u immich-machine-learning for any progress, hints or other output.

In general this means:

• Install packages/drivers for your hardware
• Configure machine-learning env and set the driver type
• Re-/start the service

For your case, according to https://github.com/immich-app/immich/blob/main/machine-learning/Dockerfile, you'd need nvidia-cuda runtime. You'd have to try out if the repository available drivers or official drivers are needed. I would try to make it work with repository-provided packages, but we never know. In Ubuntu I believe it should be the nvidia-cuda-toolkit package. I've used https://hashcat.net in the past to check whether the driver was recognised.

Then technically you'd only need to set DEVICE=cuda in the machine-learning.env, as this variable is passed to uv as well that it should load the correct dependencies.

That should be it and good luck!

[dionysius]

Btw you basically also help me with the second point from the main issue #1. So if you find anything useful, let me know

[dionysius]

note: apt repository is now available. See readme or go directly to https://apt.crunchy.run/immich/