Kafka KRaft Mode - Access Denied Exception When Deleting Snapshot Files

Question

I'm facing an issue with Kafka running in KRaft mode (without Zookeeper) on a Windows environment, and I keep getting an AccessDeniedException in the logs when Kafka attempts to delete specific snapshot files.

Here’s the detailed error from the logs:

java.nio.file.AccessDeniedException: C:\tmp\kraft-combined-logs\__cluster_metadata-     0\00000000000000943744-0000000001.checkpoint.deleted at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)

Kafka version: Using KRaft mode (without ZooKeeper)
OS: Windows 10
log.retention.bytes=1073741824 (1 GB)
log.segment.bytes=1073741824 (1 GB)
log.retention.hours=24
log.retention.check.interval.ms=300000 (5 minutes)

What I’ve Tried So Far: I tried increasing log.segment.bytes and log.retention.bytes to 2 GB to reduce the frequency of segment rotation and deletion. This didn't resolve the issue, and the AccessDeniedException still occurs during the retention check interval

I manually stopped Kafka, deleted the snapshot files that were causing the error, and restarted Kafka. This temporarily resolves the issue, but it reoccurs after some time when Kafka again tries to delete files automatically I checked using Process Explorer (Windows tool) to see if any other processes were locking the files, but it seems that only Kafka is accessing them.

Kafka Shutdown and Restart: I ensured that Kafka was fully shut down before restarting it to release any file locks, but the issue persists.

OneCricketeer · Accepted Answer · 2024-09-19 12:37:32Z

0

This is a common problem on Windows as Kafka has no automated tests on Windows environments (confluent-platform also doesn't support running Kafka on Windows).

Please use WSL2 or Docker, or a Linux VM to run Kafka locally.

answered Sep 19, 2024 at 12:37

OneCricketeer

193k20 gold badges146 silver badges276 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

nishant chauhan Over a year ago

we are restricted with Windows OS only .......and the this window machine doesnt have any network access . So , to run kafka on target window machine ....i can only get the files first downloaded in USB then configure it . for wsl , ubuntu image is 1GB appx format then i tried to configure kafka on wsl and mapped wsl port with windows port ..then i am planning to export the wsl image and use it remote machine ....is there any way around or easy way to manage it?

OneCricketeer Over a year ago

As mentioned, Kafka isn't tested/documented as running well in Windows. I'm not sure about using WSL2 offline (obviously, you'll need some network connection to keep it and Windows updated and free of CVE). If you must use Windows, you can first try setting log.preallocate=true, as mentioned in the docs

Collectives™ on Stack Overflow

Kafka KRaft Mode - Access Denied Exception When Deleting Snapshot Files

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related