CCDAK Questions and Answers

Preferred leader is a broker that was leader when topic was created. It is preferred because when partitions are first created, the leaders are balanced between brokers. Otherwise, any of the in-sync replicas (ISR) will be elected leader, as long as unclean.leader.election=false (by default)

Consumers do not directly write to the __consumer_offsets topic, they instead interact with a broker that has been elected to manage that topic, which is the Group Coordinator broker

Point-to-point (request-response) style will couple client to the server.

retention.ms can be configured at topic level while creating topic or by altering topic. It shouldn't be set at the broker level (log.retention.ms) as this would impact all the topics in the cluster, not just the one we are interested in

Although any Kafka Streams application is stateless as the state is stored in Kafka, it can take a while and lots of resources to recover the state from Kafka. In order to speed up recovery, it is advised to store the Kafka Streams state on a persistent volume, so that only the missing part of the state needs to be recovered.

Here offset is committed before processing the message. If consumer crashes before processing the message, message will be lost when it comes back up.

we have two tables, so the max number of tasks is 2

auto.offset.reset=none means that the consumer will crash if the offsets it's recovering from have been deleted from Kafka, which is the case here, as 10 < 45

GlobalKTables have their datasets replicated on each Kafka Streams instance and therefore no repartitioning is required

doc represents optional description of message

Replicas are spread across available brokers, and each replica = one broker. RF 3 = 3 brokers

Two brokers can go down, and one replica will still be able to receive and serve data

The correct option is the only one where the two consumers share an equal number of partitions amongst the two topics of three partitions. An interesting article to read ishttps://medium.com/@anyili0928/what-i-have-learned-from-kafka-partition-assignment-strategy-799fdf15d3ab

unclean.leader.election.enable=true allows non ISR replicas to become leader, ensuring availability but losing consistency as data loss will occur

Log compaction is evaluated every time a segment is closed. It will be triggered if enough data is "dirty" (see dirty ratio config)

Kafka Streams is a library for building streaming applications, specifically applications that transform input Kafka topics into output Kafka topics

batch.size controls how many bytes of data to collect before sending messages to the Kafka broker. Set this as high as possible, without exceeding available memory. Enabling compression can also help make more compact batches and increase the throughput of your producer. Linger.ms will have no effect as the batches are already full

Here, you shouldn't commit offsets 11 or 10 as it would indicate that the message #10 has been processed successfully.

(Topic,Partition,Offset) uniquely identifies a message in Kafka