Cluster failed to start: Timed out trying to connect

[shreyaslr]

Describe the issue:

I am using prefect and dask getting below error

RuntimeError: Cluster failed to start: Timed out trying to connect to tcp://dask-root-d.default:8786 after 30 s

even though the environment value is set to 300 seconds

ENV DASK_DISTRIBUTED__COMM__TIMEOUTS__CONNECT=300s

Environment:

dask-kubernetes==2023.7.0
dask==2023.7.0
distributed==2023.7.0
prefect==1.2.4

[jacobtomlinson]

To help you troubleshoot this we are going to need a minimum reproducible example. It would also help if you can share the logs from the Dask scheduler Pod.