77 questions
0
votes
0
answers
48
views
Kafka KRaft Cluster SASL Inter-Broker Authentication Failure: Unexpected Request During Handshake [closed]
I am setting up a 3-node Apache Kafka cluster using KRaft mode (3.9.0) with SCRAM-SHA-256 for inter-broker communication on the SASL_PLAINTEXT listener. The cluster is unable to form a quorum due to ...
0
votes
0
answers
152
views
Failed to start Kraft controller in K8s
We are manually trying out Zookeeper to Kafka KRaft migration in our K8s environment. For that the initial step is to deploy Kraft controller in migration mode. We used configmaps to provide the ...
- 39
0
votes
0
answers
50
views
Incorrect configuration of Kafka in Kraft mode cluster with KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: SCRAM-SHA-512
Trying to start Kafka in Kraft mode with image: apache/kafka:3.9.0
with next podman compose of config first node:
kafka-1:
hostname: kafka-1
container_name: kafka-1
image: apache/kafka:3....
- 1
0
votes
0
answers
47
views
Kafka doesn't create consumer groups
I'm running Kafka with KRaft in docker-compose with the following configuration:
services:
kafka:
image: apache/kafka:4.0.0
container_name: kafka
restart: always
ports:
- "...
1
vote
1
answer
150
views
Can Single kraft controllers setup handle multiple kafka setups?
Can a Kafka Controller Cluster support multiple Kafka setups like ZooKeeper does (with chroot paths)?"
Existing setup:
In existing setup we are using single zookeeper setup for multiple kafka ...
0
votes
0
answers
148
views
How to verify the state of Kafka Migration from ZooKeeper to KRaft
I’m in the middle of migrating from Zookeeper to KRaft in my Kafka cluster running on Kubernetes. Following the official Zookeeper to KRaft migration guide, I provisioned the KRaft controller quorum, ...
0
votes
0
answers
120
views
Kafka KRaft with SASL_SSL | Error: Unexpected Kafka request of type METADATA during SASL handshake
I already have a functioning Kafka KRaft cluster with mTLS. Now I'm trying to setup a cluster with SASL_SSL but I get the following error on the broker listener:
INFO [SocketServer listenerType=BROKER,...
- 11
2
votes
0
answers
807
views
Issues trying to consume Kafka messages on kubernetes using KRaft
I'm using docker desktop running Kubernetes.
I'm setting up my environment using the next configuration:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: kafka-network
spec:
...
0
votes
1
answer
559
views
Kafka authentication with scram 512 and KRAFT mode doesn't work
Kafka authentication with scram 512 and KRAFT mode doesn't work. My docker compose looks like this.
services:
kafka-1:
image: confluentinc/cp-kafka:7.7.1
ports:
- '9092:9092'
...
- 11
1
vote
1
answer
426
views
Kafka KRaft: 1 out of 3 brokers is always down
I'm trying to create a cluster with KRaft which has 3 brokers and 3 controllers. But each time one broker is going down while the previous downed one is getting up.
This is my docker configs:
version: ...
- 142
1
vote
1
answer
91
views
python. kafka raft connect to controller
I want to connect to kraft controller on port 9093 (non broker). And I can't to connect
code example:
#!/usr/bin/env python3
from confluent_kafka.admin import AdminClient
import json
import os
...
- 63
0
votes
1
answer
231
views
Kafka KRaft Mode - Access Denied Exception When Deleting Snapshot Files
I'm facing an issue with Kafka running in KRaft mode (without Zookeeper) on a Windows environment, and I keep getting an AccessDeniedException in the logs when Kafka attempts to delete specific ...
-1
votes
1
answer
150
views
Kafka-storage.sh is not behaving as expected in podman container
I am trying to start a Kafka(version 3.7.1) in kraft mode from an Podman image. I am able to create image without any issues. However when i am trying to run Kafka from the custom script(run.sh) which ...
- 81
0
votes
1
answer
730
views
Error Starting Multiple Kafka Brokers with KRaft using Docker Compose: Metadata and Checkpoint File Issues
When attempting to deploy multiple Kafka brokers using Docker Compose with KRaft consensus protocol, the setup fails to initialize correctly. Each Kafka broker is configured to use KRaft (...
1
vote
0
answers
539
views
Kafka Kraft cluster logs indicating potential issues
Currently I am running cluster of 3 Kafka machines. Two of those are hosted in same data center and last one is in different. At the end of post I will include node property file and jvm params.
...
- 11
0
votes
0
answers
188
views
Using Kraft with SASL_SSL
i'm Having a lot of issues to make this work, here is my files
server1.properties
############################# Server Basics #############################
process.roles=broker,controller
node.id=1
...
- 5
1
vote
1
answer
387
views
Kafka Kraft - Reassign broker as broker,controller
We have had a small KRaft cluster running successfully for over a year. Current cluster consists of 6 hosts:
3 broker,controller (nodes 1,2,3)
3 broker (nodes 4,5,6)
We want to transition nodes 4 &...
- 11
0
votes
1
answer
2k
views
Kraft cluster configuration ERROR Could not find a 'KafkaServer' or 'controller.KafkaServer' entry in the JAAS configuration
Please forgive the wall of text and bear with me as I am new to Apache Kafka. I chose to adopt Kraft as I see zookeeper is being deprecated even if guides for it are more available. I've read/watched ...
- 63
4
votes
0
answers
2k
views
Kafka Kraft | No Authorizer is configured when using kafka-acls
create a scram user using kafka-configs:
/bin/kafka-configs --bootstrap-server broker:9092 --alter --add-config 'SCRAM-SHA-256=[password="password"]' --entity-type users --entity-name "...
- 43
1
vote
0
answers
627
views
Kafka Cluster doesn't elect new leader after broker goes down
I'm trying to setup a cluster locally. Mostly so I can learn a Cluster's behavior before sending to production. I'm running everything bare metal
For context:
My cluster consists of 3 controllers and ...
- 2,328
1
vote
1
answer
1k
views
kafka kraft cluster is it right to set dedicated controller and dedicated broker on same machine
according to documentation ( from https://developer.confluent.io/learn/kraft/ _
"If you want to try it yourself, follow the Kafka Local Quickstart. This quick start runs in KRaft combined mode, ...
- 1,927
0
votes
1
answer
721
views
ERROR Exiting Kafka due to fatal exception (kafka.Kafka$) java.lang.IllegalArgumentException: No MetadataVersion with metadata version 14
I'm trying to start Kafka-KRaft server, and I'm getting this error:
~/kafka_2.13-3.5.0$ bin/kafka-server-start.sh config/kraft/server.properties
[2023-09-05 01:28:23,226] INFO Registered kafka:type=...
- 1
0
votes
0
answers
114
views
KAFKA 3.5.1 create replica on controller
I have configuration 5 nodes, 4 broker+controller and 1 controller.
Create topic with replication factor 5, and it is created, and describe show that topic partition have 5 replicas.
/opt/kafka/latest/...
0
votes
1
answer
2k
views
Has anyone got a working configuration for Kafka Kraft with SASL_SSL or even just SSL enabled?
I'm trying to get a Kafka cluster with Kraft off the ground. Plan to use it in production, and the Kafka documentation recommends not running a node in the "broker,controller" mode in ...
- 41
0
votes
1
answer
864
views
Kafka get stuck in adding replicas
I have testing cluster with 3 Kafka instances in KRaft mode. Each instance is Broker and Controller. I have inter communication secured with SSL certificate.
After cluster start, each instances know ...
- 63