Kafka - Producers CLI 🏭

Refer back to the theories section for more information on producers.

Description 👀

Produce data for a topic. This is the most common way to write data to Kafka. It reads data from standard input and sends it to the Kafka cluster.

Basic Commands 📝

• produce data to a topic

    kafka-console-producer.sh --bootstrap-server <kafka-broker> --topic <topic-name>
  

• produce data to a topic with properties

    kafka-console-producer.sh --bootstrap-server <kafka-broker> --topic <topic-name> --producer-property <property-name>=<property-value>
  

• produce data to a topic with properties and a key

    kafka-console-producer.sh --bootstrap-server <kafka-broker> --topic <topic-name> --property parse.key=true --property key.separator=<key-value-separator>
  

Examples 🧩

• produce data to a topic

    kafka-console-producer.sh --bootstrap-server localhost:9092 --topic first_topic 
    >Hello World
    >My name is Guerrero
    >I love learning new things 
  

  • Ctrl + C is used to exit the producer

• produce data to a topic with properties

    kafka-console-producer.sh --bootstrap-server localhost:9092 --topic first_topic --producer-property acks=all
    > some message that is acked
    > just for fun
    > fun learning!
  

• produce to a topic that does not exist

    kafka-console-producer.sh --bootstrap-server localhost:9092 --topic new_topic
    > hello world!
  

  • by default, Kafka will create a topic if it does not exist

  • the new topic only has one partition

    kafka-topics.sh --bootstrap-server localhost:9092 --list
    

    kafka-topics.sh --bootstrap-server localhost:9092 --topic new_topic --describe
    

    • output

         🚀  kafka-zero-to-pro ➜ kafka-topics.sh --bootstrap-server localhost:9092 --topic new_topic --describe
        Topic: new_topic        TopicId: gKHMWVz-Q9a51Zs0wezdNg PartitionCount: 1       ReplicationFactor: 1    Configs: segment.bytes=1073741824
        Topic: new_topic        Partition: 0    Leader: 1       Replicas: 1     Isr: 1
      

      • by default, Kafka will create a topic with a single partition and a replication factor of one

        • to edit the default settings, edit config/server.properties or config/kraft/server.properties

            num.partitions=3
          

        • produce to another non existing topic

            kafka-console-producer.sh --bootstrap-server localhost:9092 --topic new_topic_2
            > hello world again!
          

        • this time the topic has 3 partitions

            kafka-topics.sh --bootstrap-server localhost:9092 --topic new_topic_2 --describe
          

            Topic: new_topic_2      TopicId: 5Q3Z2ZzvQXq2Z2Z2Z2Z2Zg PartitionCount: 3       ReplicationFactor: 1    Configs: segment.bytes=1073741824
            Topic: new_topic_2      Partition: 0    Leader: 1       Replicas: 1     Isr: 1
            Topic: new_topic_2      Partition: 1    Leader: 1       Replicas: 1     Isr: 1
            Topic: new_topic_2      Partition: 2    Leader: 1       Replicas: 1     Isr: 1
          

          • even though Kafka provides this functionality, its best to create topics before producing to them!

• produce to a topic with a key

    kafka-console-producer --bootstrap-server localhost:9092 --topic first_topic --property parse.key=true --property key.separator=:
    >example key:example value
    >name:Guerrero
  

↩️