Redpanda Cloud, Now Generally Available

At Redpanda we’ve been laser focused on delivering the most performant, cost-effective and developer friendly streaming data experience in the industry. We were able to do so by building a completely new architecture for Kafka with the Redpanda platform. We proved that Redpanda is at least 10x faster tail latencies and up to 6x more cost-effective than Apache Kafka. Next, we delivered Redpanda Console to help developers reign in the chaos that can surface in distributed data streaming environments. Recently, we reimagined tiered storage, inventing a cloud-first storage architecture leveraging our shadow indexing technology and significantly reducing cost of storage.

Today, we’re excited to introduce you to the generally available Redpanda Cloud, which brings together all those innovations in a fully managed streaming data cloud service, complete with privacy preserving architecture, built-in developer tools, connectors and SOC 2 certifications necessary to run mission critical Kafka workloads at scale – including 40GB/s aggregate throughput (yes, you read that right!) with confidence.

Industry-leading performance, without compromising data privacy, developer friendliness and TCO

Redpanda Cloud provides a fully managed Kafka cloud service that lets you, in a few clicks, deploy Redpanda clusters within your own cloud accounts or in a dedicated Redpanda cloud environment, on cloud providers of your choice, and the tools and capabilities to help you focus more on streaming data, and less on platform engineering while utilizing your cloud spend more efficiently.

Highlights include:

• Fully managed, zero-operations cloud for Kafka workloads
• Fast, scalable and resource efficient
• Data sovereignty and compliance
• Integrated Redpanda Console
• Fully managed connectors
• Infinite data retention with tiered storage
• Multi-AZ availability and cloud networking
• Security and team management
• Easy upgrades and monitoring

A zero-operations Kafka API-compatible cloud

Redpanda Cloud is a fully managed service operated and managed by Redpanda. The cloud console facilitates self-service cluster provisioning, configuration, and development. Redpanda takes care of resource automation, patching, performance monitoring, and cluster maintenance so you can focus on engineering your Kafka applications and data pipelines. Unlike other Kafka cloud services, Redpanda Cloud comes with data balancing capabilities built in, without requiring third-party tools or infrastructure that further simplifies operations and provides a hand's off approach as you scale your Kafka workloads. Redpanda also monitors key service level indicators that could impact your application and takes preemptive action.

Fast, scalable and resource efficient

Redpanda Cloud gives you the choice of fully managed Kafka API-compatible clusters that live either within your cloud account/VPCs (called Bring Your Own Cloud or BYOC clusters), or within the Redpanda cloud resources (called Dedicated clusters). Based on Redpanda’s high-performance streaming data platform, Redpanda Cloud instances deliver predictable tail latest latencies that are at least 10x faster than Apache Kafka, even as your workloads scale to gigabytes of data per second. Furthermore, Redpanda does so with significantly fewer compute resources, enabling you to realize your throughput goals with smaller clusters, reducing your total cost of ownership (TCO). Also, each Redpanda cluster comes complete with a schema registry and HTTP proxy so that you can write Kafka applications without any additional infrastructure.

Redpanda Cloud is cloud-provider agnostic and lets you choose cloud regions in Amazon AWS and Google GCP so that you can deploy your streaming data close to your applications. Azure support to follow soon.

Data sovereignty & SOC 2 compliance

Redpanda Cloud employs a privacy-preserving architecture that makes it easy for you to build streaming data clusters that comply with data sovereignty laws and compliance regulations. Today’s business environments require stringent controls over where your data resides, who and what process it, and how it is secured and protected. With streaming data environments, enforcing sovereignty can be complex, given the distributed nature of the data sets.

Redpanda Cloud’s BYOC clusters are engineered to preserve your data privacy and help you meet your data sovereignty requirements. With BYOC, the Redpanda Console is integrated directly into your VPC! So even though we provide the capability to inspect messages, that data or the underlying security credentials are never stored in any Redpanda infrastructure. Instead, your browser securely talks with the backend hosted in your VPC during each browsing session, and furnishes the message data without impacting responsiveness!

Redpanda Cloud is the only Kafka-compatible cloud service in the industry that supports this privacy-preserving model spanning both the data and your security credentials. Today, we support Amazon AWS and Google GCP for hosting your clusters.

Not only that, Redpanda Cloud is now SOC 2 Type 1 compliant. We are committed to ensuring the security of your data and our systems. Please contact us for a detailed report from the auditors!

Integrated Redpanda Console

Redpanda Cloud eliminates development bottlenecks by baking the capabilities of the Redpanda Console directly into the cloud experience. One of the primary challenges for developers working with streaming data is the lack of visibility into data streams, the ability to debug and manage your own topics as your applications evolve. Within this integration, developers can now easily avail Redanda Console’s features such as time-travel debugging and push-down filters to debug data streams, as well as manage their own topic lifecycle for the cloud clusters they create, without administrative support.

Fully managed connectors

Redpanda Cloud now includes integrated connectors and fully manages them within the cloud experience so you don’t have to! A primary need for any data pipeline is to connect your streaming data with a variety of sources and sinks. This update includes support for the following connectors:

• Mirror Maker: Migrate data from external Kafka workloads to Redpanda Cloud using active-standby and active-active topologies.
• AWS S3 and Google GCS: Sink data to downstream object storage systems using built-in connectors for S3 and GCS.

Connectors are automatically provisioned as part of Redpanda cluster creation. Based on CPU usage, the connector cluster autoscales up or down as necessary, freeing you from having to do resource management and capacity planning for your connector infrastructure.

Infinite data retention with tiered storage

Redpanda Cloud enables you to retain your data for any time frame that suits your application and regulatory needs. To manage your data more cost-effectively, Redpanda Cloud leverages the Redpanda platform’s tiered storage capabilities to transparently offload data from local disks to object stores such as AWS S3 or Google GCS without disrupting the read latencies of your clusters. Redpanda makes it simple to scale your storage needs from a few hundred megabytes to petabytes while passing the cost savings down to you!

Multi-AZ availability and cloud networking

The latest updates to Redpanda Cloud also bring support for new capabilities essential to cloud-native environments. Specifically, Redpanda Cloud supports single-zone and multi-zone deployment flexibility to balance availability and cost during cloud zone failure. Multi-AZ offers high availability during a zone failure typically needed by production applications. The single-AZ option can be used when multi-zone availability is not a requirement

Redpanda Cloud also supports private networking through VPC peering, which can reduce your networking cost by peering your application VPC with Redpanda Cloud.

Security and user management

Redpanda Cloud provides authentication, authorization and encryption in transit for your cloud users and data. Supported security protocols for cloud endpoints are compatible with those commonly used clients for both Kafka Clients and Http Clients. For example, the Kafka endpoint is secured with SASL+TLS and the Schema Registry and HTTP Proxy endpoints are secured with HTTP Basic Auth where the credentials are the same as the Kafka clients.

Your Redpanda Cloud admins can invite other team members, and assign permissions for cluster access. Cloud users are managed by assigning user roles. These permissions are synchronized with the Redpanda Console UI to control access permissions. For example, some team members can create clusters, but others may only create new topics or view cluster settings.

Self-service upgrades and monitoring

Redpanda Cloud comes with additional goodies to help prevent surprises in your streaming data environment.

First, it lets you upgrade your clusters to the latest version of the Redpanda platform at your own pace. Redpanda Cloud uses rolling upgrades to ensure zero application downtime. You can also run clusters of different versions side-by-side to maintain application continuity. Say goodbye to forced scheduled maintenance windows, or overnight surprises that might break your applications!

Second, you can integrate its built-in Prometheus endpoint to integrate with a third-party monitoring system of your choice. Want to monitor your Redpanda Cloud instances with your Datadog or Grafana investments – go right ahead!

Try Redpanda Cloud today!

Interested in rolling out your own streaming data clusters? We offer a 15-day trial for Redpanda Cloud. The trial lets you build your clusters (both dedicated and BYOC), and test drive all the integrated development capabilities within the cloud. Request yours here or contact us for more information. Happy streaming!