0

Running Dask Scheduler on system A and workers on system A and B. NFS volume from system A is shared on the network through NFS with system B, and contains the data files. This folder has a symbolic link to my home directory due to path issues.

When running with workers from system B I get the following error:

OSError: [Errno 5] Can't synchronously read data (file read failed: time = Tue Apr 30 18:12:40 2024
, filename = '/home/steffan/raw_sarez2017_desc/S1_20170511_ALL_F2.grd', file descriptor = 18, errno = 5, error message = 'Input/output error', buf = 0x757e7079f720, total read size = 40136, bytes this sub-read = 40136, bytes actually read = 18446744073709551615, offset = 0)

When only working with system A I get no errors. Is there a reason why Dask is throwing synchronous IO errors when using workers from a different system on the same network?

Improve this question
2
  • Hi there is no reason Dask would throws such error, it probably comes from your NFS share, on a lower lever. NFS can often be a bottleneck or present performance issues. Commented May 1, 2024 at 9:02
  • can you read files from the NFS share outside of Dask? Commented May 12, 2024 at 9:30
Related questions
5
Dask Distributed: Reading .csv from HDFS
2
How to pinpoint cause of Dask worker to worker connection timeout issues?
2
Why does dask worker fails due to MemoryError on "small" size task? [Dask.bag]
Load 5 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.