Kafka over gRPC: a Rust Kafka proxy with generated clients for browser and native

Our second launch is kafka-proxy: a Rust proxy that exposes Apache Kafka through a generated gRPC service, grpcdc.kafka.v1.Kafka, so any generated grpc-over-datachannels client — browser or native — can produce to and consume from Kafka without a Kafka client library in that language.

The proxy terminates generated gRPC on one side and speaks the real Kafka protocol on the other, to both Apache Kafka (KRaft) and Redpanda. Producer sessions and consumer-group sessions live on the proxy side: clients open streams, publish records, subscribe, and commit offsets through gRPC calls, while the proxy handles broker connections, group membership, and rebalances. Broker connections reconnect after transport failures, and offset commits retry through generation churn, so a rebalance does not silently lose progress.

Coverage

The proxy is exercised by an extensive generated end-to-end suite — 21 tests per broker — covering produce/consume round trips, consumer groups, offset commits across rebalances, and resuming a group after the proxy itself restarts. The same suite runs against Apache Kafka and Redpanda in CI, in generated Dockerfiles that readers can run themselves.

Run it

Code, guide, and CI live at browser-grpc/kafka-proxy. A prebuilt container image is published as ghcr.io/browser-grpc/kafka-proxy, and the main project is at browser-grpc/grpc-over-datachannels. If you can call gRPC from your language — including from a browser tab — you can now reach Kafka through it.

For the internals — how producer and consumer-group sessions, rebalances, and offset commits are held on the proxy — see Inside kafka-proxy: consumer groups, rebalances, and offset commits over gRPC.