In modern distributed systems, real-time data processing has become a cornerstone of scalable architectures. Apache Kafka, as a leading event streaming platform, plays a pivotal role in enabling high-throughput, fault-tolerant data pipelines. When integrated with the Spring ecosystem via Spring Kafka, developers gain a robust, declarative model for producing and consuming messages with minimal boilerplate.

This guide compiles critical Spring Kafka concepts frequently assessed in technical interviews—particularly at companies like Kuaishou that emphasize scalable, event-driven backends. The questions span foundational knowledge to advanced configurations, covering message handling, error resilience, transaction support, and performance tuning.

1. Core Features of Spring Kafka

---

Q: What key capabilities does Spring Kafka provide?

A: Spring Kafka simplifies Kafka integration in Spring applications through:

• KafkaTemplate for synchronous/asynchronous message production.
• @KafkaListener for declarative message consumption.
• Built-in support for transactions using KafkaTransactionManager.
• Configurable error handlers (e.g., SeekToCurrentErrorHandler) for dead-letter handling and retries.
• Batch listening and custom deserialization strategies.

2. Kafka Architecture Fundamentals

---

Q: Describe Kafka’s core components.

A:

• Broker: A Kafka server node storing and serving messages.
• Topic: A logical channel for categorizing messages.
• Partition: An ordered, immutable sequence within a topic; enables parallelism.
• Producer: Publishes records to topics.
• Consumer: Reads records from topics.
• Consumer Group: A set of consumers sharing topic partitions for load balancing.

3. Producer Configuration

---

Q: How do you configure a Kafka producer in Spring?

A: Use DefaultKafkaProducerFactory with essential properties:

@Bean
public ProducerFactory<String, String> kafkaProducerFactory() {
    Map<String, Object> props = new HashMap<>();
    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
    props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
    props.put(ProducerConfig.ACKS_CONFIG, "all");
    return new DefaultKafkaProducerFactory<>(props);
}

@Bean
public KafkaTemplate<String, String> kafkaTemplate() {
    return new KafkaTemplate<>(kafkaProducerFactory());
}

4. Consumer Setup

---

Q: What are key consumer configuration parameters?

A: Critical settings enclude group.id, auto.offset.reset, and deserializers. Example:

@Bean
public ConsumerFactory<String, String> kafkaConsumerFactory() {
    Map<String, Object> props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    props.put(ConsumerConfig.GROUP_ID_CONFIG, "event-group");
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
    props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
    return new DefaultKafkaConsumerFactory<>(props);
}

@Bean
public ConcurrentKafkaListenerContainerFactory<?, ?> kafkaListenerContainerFactory() {
    ConcurrentKafkaListenerContainerFactory<String, String> factory =
        new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(kafkaConsumerFactory());
    return factory;
}

5. Using @KafkaListener

---

Q: How does @KafkaListener work?

A: It marks a method as a message handler for specified topics:

@Component
public class EventConsumer {

    @KafkaListener(topics = "user-events", groupId = "analytics-group")
    public void handleEvent(String payload) {
        // Process message
    }
}

6. Serialization Strategies

---

Q: How to use JSON serialization in Spring Kafka?

A: Configure JsonSerializer and JsonDeserializer:

@Bean
public ProducerFactory<String, Object> jsonProducerFactory() {
    Map<String, Object> config = new HashMap<>();
    config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    config.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
    config.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
    return new DefaultKafkaProducerFactory<>(config);
}

@Bean
public ConsumerFactory<String, Object> jsonConsumerFactory() {
    JsonDeserializer<Object> deserializer = new JsonDeserializer<>(Object.class, false);
    return new DefaultKafkaConsumerFactory<>(
        Map.of(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"),
        new StringDeserializer(),
        deserializer
    );
}

7. Message Filtering

---

Q: Can you filter messages before processing?

A: Yes, via RecordFilterStrategy:

factory.setRecordFilterStrategy(record -> 
    record.value().contains("ERROR") // skip error logs
);

Filtered records are not passed to the listener.

8. Error Handling Mechanisms

---

Q: How to handle consumption failures?

A: Use SeekToCurrentErrorHandler with backoff retry logic:

factory.setErrorHandler(
    new SeekToCurrentErrorHandler(new FixedBackOff(2000L, 5L))
);

After 5 failed attempts with 2s delays, the record may be sent to a dead-letter topic.

9. Transaction Support

---

Q: How to enable Kafka transactions in Spring?

A: Configure transactional producer and template:

@Bean
public ProducerFactory<String, String> transactionalProducerFactory() {
    Map<String, Object> props = new HashMap<>();
    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    props.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "txn-" + UUID.randomUUID());
    return new DefaultKafkaProducerFactory<>(props);
}

@Bean
public KafkaTransactionManager<String, String> kafkaTransactionManager() {
    return new KafkaTransactionManager<>(transactionalProducerFactory());
}

@Bean
public KafkaTemplate<String, String> transactionalKafkaTemplate() {
    KafkaTemplate<String, String> template = new KafkaTemplate<>(transactionalProducerFactory());
    template.setTransactionIdPrefix("txn-");
    return template;
}

Use @Transactional on service methods to group sends atomically.

10. Batch Consumption

---

Q: How to consume messages in batches?

A: Enable batch mode in the container factory:

@Bean
public ConcurrentKafkaListenerContainerFactory<String, List<String>> batchFactory() {
    ConcurrentKafkaListenerContainerFactory<String, List<String>> factory =
        new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(kafkaConsumerFactory());
    factory.setBatchListener(true);
    return factory;
}

// Listener method
@KafkaListener(topics = "batch-topic")
public void processBatch(List<String> messages) {
    messages.forEach(this::process);
}

11. Listener Container Tuning

---

Q: What are important container configuration options?

A:

• setConcurrency(n): Number of consumer threads per topic.
• getContainerProperties().setPollTimeout(ms): Controls poll duration.
• setAckMode(): Defines acknowledgment behavior (e.g., MANUAL_IMMEDIATE).
• setAutoStartup(false): Delay listener initialization.

12. Performance Best Practices

---

Q: How to optimize Spring Kafka performance?

A: Apply these strategies:

• Increase concurrency to match partition count.
• Tune fetch.min.bytes and fetch.max.wait.ms on the consumer side.
• Use compact serialization (e.g., Avro, Protobuf) over JSON when possible.
• Enable batch sending (linger.ms > 0, batch.size) on producers.
• Avoid blocking operations inside listeners; offload to thread pools if needed.