34

I have a problem communicating with Kafka secured with sasl using console scripts. Kafka is secured with sasl, listener is SASL_PLAINTEXT and mechanism is PLAIN.

What I did: I tried listing some data using one of kafka scripts:

bin/kafka-consumer-groups.sh --bootstrap-server (address) --list

However I get 

WARN Bootstrap broker (address) disconnected (org.apache.kafka.clients.NetworkClient)

and command fails, which is understandable because it's secured with sasl.

So I tried how to add client username/password to that command. First, I tried to run kafka-console-consumer script, I used --command-config to add necessary file. I quickly discovered that I can't add jaas file directly and I needed to use .properties file, so I did.

My properties file(keep in mind that brackets indicate "censored" data, I can't put all real data here):

bootstrap.servers=(address)
zookeeper.connect=127.0.0.1:2181
zookeeper.connection.timeout.ms=6000
sasl.jaas.config=(path)/consumer_jaas.conf
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
group.id=(group)

My jaas file:

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username=(username)
    password=(password);
};

This jaas file works in my standard java applications.

However, when I'm trying to run either kafka-consumer-groups script or kafka-console-consumer, I get this error:

Exception in thread "main" org.apache.kafka.common.KafkaException: java.lang.IllegalArgumentException: Login module not specified in JAAS config
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:94)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:93)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:51)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:84)
at kafka.admin.AdminClient$.create(AdminClient.scala:229)
at kafka.admin.AdminClient$.create(AdminClient.scala:223)
at kafka.admin.AdminClient$.create(AdminClient.scala:221)
at kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.createAdminClient(ConsumerGroupCommand.scala:454)
at kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.<init>(ConsumerGroupCommand.scala:389)
at kafka.admin.ConsumerGroupCommand$.main(ConsumerGroupCommand.scala:65)
at kafka.admin.ConsumerGroupCommand.main(ConsumerGroupCommand.scala)
Caused by: java.lang.IllegalArgumentException: Login module not specified in JAAS config
at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:68)
at org.apache.kafka.common.security.JaasUtils.jaasConfig(JaasUtils.java:59)
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:85)

This jaas file is a direct copy of a file that I'm using in java app that communicates with kafka and it works, however here, using console tools, it just doesn't work. I tried searching for a solution but I can't find anything useful.

Can anyone help me with this?

Improve this question

3 Answers 3

Reset to default
54

There are 2 ways to provide the JAAS configuration to the Kafka clients.

  • Via the client property: sasl.jaas.config. In that case you set it to the actual JAAS configuration entry. For example, your configuration file becomes:

    bootstrap.servers=(address)
zookeeper.connect=127.0.0.1:2181
zookeeper.connection.timeout.ms=6000
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="(username)" password="(password)";
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
group.id=(group)

    As you've already figured out, you can use --command-config to pass a properties file to kafka-consumer-groups.sh.

  • Via the Java property: java.security.auth.login.config. In this case, you set it to the path of your JAAS file. Also if you set it in KAFKA_OPTS, kafka-consumer-groups.sh will pick it up automatically.

    export KAFKA_OPTS="-Djava.security.auth.login.config=(path)/consumer_jaas.conf"
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I put jaas configuration entry directly into .properties file and it worked. Now I can use these scripts without any errors. I don't know why it had problems with external file. Thank you!
When using sasl.jaas.config, you can only set it to the JAAS configuration entry it doesn't work with the path to a file. On the contrary with the Java property, it only works with the path to a file.
I have set sasl.jaas.config properly, but I still get [main] ERROR io.confluent.admin.utils.cli.KafkaReadyCommand - Error while running kafka-ready. and Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set. How do I make it stop looking for a .jaases file and instead just use what I've specified?
0

You can also include the SASL configuration in the route. Here's an example using Kerberos:

additionalProperties.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required refreshKrb5Config=true useKeyTab=true keyTab=&quot;<path to keyTab>&quot; storeKey=true principal=<principal>;

I had to use the &quot; around the keyTab path. Using keyTab=/path/path/file.keyTab didn't work.

Improve this answer

Comments

0 
spring:
  kafka:
    jaas:
      enabled: true
      control-flag: required
      login-module: org.apache.kafka.common.security.plain.PlainLoginModule
      options:
        username: "username"
        password: "{cipher}password"

Take a look inside KafkaAutoConfiguration.class

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.