kafkaProducer

// TODO Support Transactional behavior Kafka producer as a Flow. Will automatically close, and flush when finished streaming. The Flow will close when the KafkaProducer is consumed from the Flow.

This means that the KafkaProducer will not be closed for a synchronous running stream, but when running the Flow is offloaded in a separate Coroutine it's prone to be collected, closed and flushed. In the example below we construct a producer stream that produces 100 indexed messages.

fun <Key, Value> KafkaProducer<Key, Value>.produce(topicName: String, count: Int): Flow<Unit> =
  (0..count).asFlow().map { sendAwait(ProducerRecord(topicName, "message #it")) }

val producerStream = kafkaProducer(Properties(), StringSerializer(), StringSerializer())
  .flatMapConcat { producer -> producer.produce("topic-name", 100) }

Here the KafkaProducer will only get collected (and closed/flushed) when all 100 messages were produced.

DO NOT If instead we'd do something like the following, where we offload in a buffer then the KafkaProducer gets collected into the buffer and thus closed/flushed.

kafkaProducer(Properties(), StringSerializer(), StringSerializer()).buffer(10)