GPU used by a user

[scatenag]

I rephrase this.

It would be nice to have 'per user' GPU monitoring.

Something similar to what we can get with
users_cpu_user_percentage_average

I would like to have
users_gpu_user_percentage_average.

I have seen something similar in gpustat.

Could it be possible to get some useful info?

If I'm not wrong both (this and gpustat) use the same library pynvml...

I asked the same here...

--it was ---

Would it be possible to create an alarm that triggers
when a user uses more than one GPU?

That is: if on a GPU there are running processes of more than a user?

Thanks

[wookayin]

One can add a chart entry for GPU memory utilization per GPU, which consists of many lines (one per user). This should be straightforward to implement.

[scatenag]

For the moment I implemented per user GPU memory utilization modifying nvidia-smi collector netdata/netdata#9372

Implementing by pynvml should be more efficient
(but, for the moment, I am not python-ninja enough to try).

Do you think it would be possible to monitor per user GPU utilization (not memory)?

[coraxx]

Correct me if I'm wrong, but the only per user metric that is extractable, is how much memory is used. I am not able to find a metric for percent GPU usage per user.

Please let me know if I am just blind ^^

[scatenag]

You are right!
I haven't found any way to get percent of GPU usage per user.

With nvidia-smi plugin I use percent of GPU memory usage (as approximation of) for user GPU usage that, for user accounting, is still better than no user info ..