2

I am testing file read/write performance in both Python and Dart, and I encountered a surprising result: Python is significantly faster than Dart for these operations. Here are the times I recorded:

Python:

  • Write time: 10.28 seconds
  • Read time: 4.88 seconds
  • Total time: 15.16 seconds

Dart:

  • Write time: 79 seconds
  • Read time: 10 seconds
  • Total time: 90 seconds

I expected Dart to perform similarly or even better than Python in this context, so I'm puzzled by these results. I’m running both tests on the same system (Mac) with similar code structures to ensure a fair comparison.

Questions:

  1. Are there any known reasons for this significant performance difference in file handling between Python and Dart?
  2. Could specific libraries, encoding formats, or other system-level factors in Python or Dart affect file I/O speed?
  3. Are there optimization techniques in Dart to improve file read/write performance?

I would appreciate any insights or suggestions on what might be causing this discrepancy and how to potentially optimize Dart's file I/O performance.

Here's the Python code:

def benchmark(cnt=200):
    block_size = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" * (1024 * 1024)
    file_path = "large_benchmark_test.txt"

    start_time = time.time()
    with open(file_path, "w") as file:
        for _ in range(cnt):
            file.write(block_size)
    write_end_time = time.time()

    with open(file_path, "r") as file:
        while file.read(1024):
            pass
    read_end_time = time.time()

    write_time = write_end_time - start_time
    read_time = read_end_time - write_end_time
    total_time = read_end_time - start_time

    print(f"Python - Write: {write_time:.2f} s")
    print(f"Python - Read: {read_time:.2f} s")
    print(f"Python - Total: {total_time:.2f} s")
    os.remove(file_path)

And the Dart code:

void benchmark({int cnt=200}) {
  final blockSize = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' * 1024 * 1024;
  final filePath = 'large_benchmark_test.txt';
  final file = File(filePath);

  final writeStartTime = DateTime.now();
  final sink = file.openSync(mode: FileMode.write);
  for (int i = 0; i < cnt; i++) {
    sink.writeStringSync(blockSize);
  }
  sink.closeSync();
  final writeEndTime = DateTime.now();
  final writeTime = writeEndTime.difference(writeStartTime).inSeconds;
  print("Dart (Synch) - Write: $writeTime s");
  
  final readStartTime = DateTime.now();
  final reader = file.openSync(mode: FileMode.read);
  while (true) {
    final buffer = reader.readSync(1024);
    if (buffer.isEmpty) break;
  }
  reader.closeSync();
  final readEndTime = DateTime.now();

  final readTime = readEndTime.difference(readStartTime).inSeconds;
  final totalTime = readEndTime.difference(writeStartTime).inSeconds;

  print("Dart (Synch) - Read: $readTime s");
  print("Dart (Synch) - Total: $totalTime s");
  file.deleteSync();
}
4
  • 2
    User have cross-posted the question on Reddit with some discussion going on: reddit.com/r/FlutterDev/comments/1glohox/… Commented Nov 7, 2024 at 14:05
  • Would it be possible to re-run the benchmarks using benchmark(cnt: 10)? Are you observing a similar discrepancy? Commented Nov 7, 2024 at 14:17
  • @DanR yes it finished in 0s Commented Nov 7, 2024 at 15:45
  • I see, could you run benchmark(cnt: 10) using the function I posted below? This would measure writing/reading 260MB of data. And perhaps mention the Python score for cnt 10 as well. Commented Nov 8, 2024 at 6:18

1 Answer 1

0

The scores below were generated on an Intel i5-10210u with 16GB, Python 3.10.0, Dart SDK version: 3.5.3 (stable) (Wed Sep 11 16:22:47 2024 +0000) on "windows_x64"

Scores for cnt = 200 (writing/reading 5.07GB):

>>> main.benchmark(200)
Python - Write: 50.13 s
Python - Read: 20.21 s
Python - Total: 70.34 s
Dart (Synch) - Write: 28 s
Dart (Synch) - Read: 17 s
Dart (Synch) - Total: 45 s

To provide more granularity, I also tried to run the Dart benchmark scores using the function:

void benchmark({int cnt = 200}) {
  final blockSize = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' * (1024 * 1024);
  final filePath = 'large_benchmark_test.txt';
  final file = File(filePath);

  final clock = Stopwatch();
  clock.start();
  final sink = file.openSync(mode: FileMode.write);
  for (int i = 0; i < cnt; i++) {
    sink.writeStringSync(blockSize);
  }
  sink.closeSync();
  clock.stop();
  final writeTime = clock.elapsedMicroseconds / 1e6;
  print("Dart (Synch) - Write: $writeTime s");

  clock.reset();
  clock.start();

  final reader = file.openSync(mode: FileMode.read);
  while (true) {
    final buffer = reader.readSync(1024);
    if (buffer.isEmpty) break;
  }
  reader.closeSync();
  clock.stop();

  final readTime = clock.elapsedMicroseconds / 1e6;
  final totalTime = writeTime + readTime;

  print("Dart (Synch) - Read: $readTime s");
  print("Dart (Synch) - Total: $totalTime s");
  file.deleteSync();
}

In the function above, a Stopwatch is used to calculate the scores. This makes it possible to write/read a smaller amount of data and still get a meaningful benchmark result.

Scores for cnt = 200 (writing/reading 5.07GB)

$ dart .\main.dart
Dart (Synch) - Write: 29.183918 s
Dart (Synch) - Read: 17.287486 s
Dart (Synch) - Total: 46.471404 s

Scores for cnt = 20 (writing/reading 520MB)

>>> main.benchmark(20)  
Python - Write: 5.86 s
Python - Read: 2.17 s
Python - Total: 8.02 s
$ dart .\main.dart
Dart (Synch) - Write: 3.447846 s
Dart (Synch) - Read: 2.434338 s
Dart (Synch) - Total: 5.8821840000000005 s

The benchmark scores above (generated on a Windows 11 system) do not reflect a huge difference in performance when writing/reading data using Python and Dart.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.