Redpanda acquires Benthos to expand its end-to-end streaming data platform

Redpanda, the well-funded Kafka-compatible streaming data platform that competes with the likes of Confluent, on Thursday announced that it has acquired open-source stream-processing platform Benthos.

The two companies did not disclose the purchase price, but it’s worth noting that Redpanda raised a $150 million Series C round a year ago that now allows it to make these kinds of moves. This marks Redpanda’s second acquisition after acquiring CloudHut, which built a web-based user interface for streaming data platforms, in 2022.

Benthos helps enterprises with common data engineering tasks like setting up pipelines to connect over 200 data sources and services, and transform and filter their data streams in the process to enable real-time applications. One of the company’s most recent launches is Benthos Studio, a low-code interface for building data pipelines and transformations.

Redpanda CEO Alexander Gallego told me that this acquisition now completes Redpanda’s platform. “What Benthos does for us is that it creates the most complete end-to-end streaming platform for data-intensive applications,” he said. “Benthos and Redpanda is the only product at a technical level that can be deployed from edge devices […] all the way to the largest cloud workloads.” That’s because Benthos is delivered as a single binary that can run on a low-powered devices with just a few megabytes of memory and on the most powerful servers. “It is the same software stack end-to-end, regardless of how you choose to use it.”

Gallegos noted that close to a third of Redpanda users were already Benthos users, and he said he has known Benthos founder Ashley Jeffs for about five years now. The fact that Benthos is written in Go, just like much of the Redpanda platform, also helped seal the deal, given that it makes integrating the service quite a bit easier. He also stressed that trying to build a product like Benthos from scratch would be a heavy lift for any company; thanks to its most recent funding round, Redpanda decided to acquire Benthos.

Indeed, Redpanda has already integrated Benthos into its own service and has made it the core technology of its new Redpanda Connect service. Redpanda Connect will feature 220 pre-built connectors to help engineers quickly build their streaming in data transformation pipelines. Redpanda Connect — just like Benthos — is open source and available from the Redpanda GitHub repo (with the exception of the two previously mentioned connectors).

One of Redpanda’s reference customers for the new service is Zafin, which helps banks modernize their data platforms. “At the heart of Zafin IO is a Streaming ETL capability based on Redpanda,” said Zafin CTO Shahir Daya. “With Redpanda’s acquisition of Benthos, Redpanda Connect will enable Zafin IO to ingest data from a broad variety of systems efficiently without any unnecessary hops, process it in real time, and deliver it to the target system, be it the Zafin platform or other modernized system.”

It’s also worth noting that now, more than in recent years, enterprises are trying to consolidate where they spend their money, so being able to offer an end-to-end platform is an advantage in today’s market where point solution often struggle to gain traction.

Existing Benthos users and customers won’t see any immediate changes to their existing workflows, but Gallegos did stress that Redpanda is making the previously open-source Snowflake and Splunk API integrations a paid service (the existing open-source integrations will continue to work, but they won’t get updates from Redpanda going forward). “All the users that are using those services are used to paying for the integration with those services,” Gallegos told me. “So those are the only two that we are launching with as enterprise-only.”

There is an AI piece to all of this, too. Gallegos explained that a lot of Redpanda customers have struggled because AI APIs are expensive. But going forward, Redpanda will offer a service that can buffer events to meter the usage of external APIs. “We’re going to be building a portfolio of connectivity for the most popular AI API platforms in a way that doesn’t allow you to exceed your averages. That kind of logic tends to be very tedious and very specific to every API,” he said.