Exclusive: Cloudera builds Hadoop-free future platform

Cloudera is drawing a clear architectural line between its established Hadoop-rooted data platform and its future platform, promising six years of support for existing CDP deployments while directing new workloads towards Anywhere Cloud, a ground-up architecture containing no Hadoop code.

Its latest CDP release, code-named Kona, will remain the stable platform for existing workloads. Anywhere Cloud - whose first release is internally code-named Maui - is intended for new AI, analytics and compute projects.

The two platforms can operate side by side, allowing customers to retain established CDP environments while using their data with newer engines and AI services through Anywhere Cloud.

"Anywhere Cloud has no bytes from Hadoop; it's completely detached from that ecosystem," said Sergio Gago, CTO, Cloudera.

Clean break

Cloudera's existing platform traces its architecture through CDP, Cloudera Manager and the company's Hadoop heritage, although it has been expanded substantially over the years. The company is now separating that mature estate from the technology stack intended for new projects.

Anywhere Cloud is not an incremental version of CDP. It is a separate framework designed to run across public clouds, data centres, sovereign cloud infrastructure and air-gapped environments, while remaining compatible with data held in Cloudera's existing platform.

Current customers can keep established workloads on Kona while deploying new applications elsewhere. Organisations building a new data platform can adopt Anywhere Cloud without first deploying CDP.

"Kona is the 7.3.2 line of code. It is CDP, it is your crown jewels that has six years' long-term support, and this is what's going to be resilient, secure, and we will support you almost forever. Now, the opposite of security and resilience and robustness is crazy innovation and speed and fast time to market. That's why we're saying Anywhere Cloud, aka Maui, can work side by side. It's a new framework. It's a new development. It has nothing from Hadoop, but it's completely compatible with a cloud data platform," added Gago.

The separation allows Cloudera to introduce newer technologies without repeatedly changing the platform running customers' most sensitive workloads. New processing engines and emerging technologies can instead be deployed through Anywhere Cloud without affecting compatibility on Kona.

Gago cited newer Spark releases, ClickHouse, PuppyGraph and future compute capabilities as examples of workloads that could sit on the newer platform.

A customer could, for example, retain a large data lake in CDP in its own data centre while running AI inference or newer analytics engines through Anywhere Cloud elsewhere. Customers that moved their data estates to public cloud could also bring some compute back to local infrastructure.

Legacy support

Kona will receive six years of long-term support, twice Cloudera's typical three-year LTS period. The commitment is intended to give organisations confidence that existing CDP estates will continue to receive maintenance and security updates as development expands around Anywhere Cloud.

Many of those deployments support large operational workloads in banking, oil and gas and other regulated industries, making rapid replacement impractical.

"For example, in Europe, there is a specific regulation. It's called the EU Digital Resilience Act, that is forcing every company to provide five years of support on any software that we deliver. We wanted to go beyond that and say six years," added Gago.

"Just because we're moving into this new direction doesn't mean that the support for CDP falls off. We wanted to make that very strong," said Leo Brunnick, Chief Product Officer, Cloudera.

Security maintenance will also become a greater focus. Cloudera has tripled the size of its internal product security team over the past 12 months and is introducing processes to respond more quickly to vulnerabilities.

Customer requirements have shifted accordingly. Some organisations that previously sought to minimise upgrades are now asking suppliers to provide security patches far more frequently when vulnerabilities emerge.

"We tripled the size of our internal security, our product security team, in this last 12 months, and we're putting in place entirely new, enhanced processes because these banks were telling us a year ago, stop making us upgrade so much. We only want a patch or an upgrade every six months or every year, and now they're telling us you need to be able to give us an upgrade, a patch every week," added Brunnick.

New workloads

Anywhere Cloud is in limited availability, with general availability scheduled for November 30.

Cloudera began putting early software into the hands of selected design partners on August 1. Nine organisations are in the design programme, while another 90 volunteered but were not selected.

Those customers are testing different production configurations ahead of wider availability. Selected participants can run the technology in production during the limited-availability period.

"Today was the launch of Anywhere Cloud. It remains in design partnership," added Brunnick.

Agentic AI is one of the workloads shaping the new platform. Agents can generate substantially more database queries and compute activity than human users carrying out comparable tasks, increasing both infrastructure and governance demands.

Traditional enterprise systems are often sized around human users and predictable query volumes. A single request to an AI system can instead trigger multiple searches, analytical queries and other operations.

The challenge is not only scale. Organisations also need to control which information an agent can access and what it is permitted to return.

"When it comes to agentic behaviour, you have several constraints: the latency, the quality of the data, but the most important one for me is the governance, access controls, and so on on the data itself," added Gago.

Cloudera is carrying governance concepts already used in its data platform into Anywhere Cloud, including identity, lineage, masking and access controls. An AI agent can therefore be governed against enterprise data in a similar way to an analytics query or machine-learning workload, although agents can require more complex permissions.

An HR agent, for example, might require access to a broad payroll dataset to perform its function while being permitted to return only an individual employee's information to that person.

"The same system, the same software, can have access to the whole dataset or chunks of it, depending on who's the person speaking to it. This is a second order, a second derivative on the data, because when you send a query to a database, that's only one degree of distance. I am this user. I have access to these tables. That's it. In this case, the agent needs to have all the information for context, but will give back to you only part of that, and that has to be baked into your orchestration system, your governance system, and the way the harness of the AI works," added Gago.

AI scale

Cloudera also expects agentic systems to change where and how organisations run compute.

Gago said a single AI request can trigger many more queries than a traditional search or dashboard interaction. As businesses deploy more agents, infrastructure designed around predictable human activity could face substantially higher loads.

Anywhere Cloud is therefore designed to operate across multiple infrastructure environments rather than being tied to a single public cloud.

Customers could expand workloads from a data centre into AWS, Azure or Google Cloud when additional capacity is needed, or bring AI workloads onto privately owned infrastructure for cost, sovereignty or privacy reasons.

The architecture also targets organisations that migrated heavily to public cloud and subsequently reduced their ability to operate data-centre infrastructure. Anywhere Cloud is intended to provide a common operating layer without requiring separate specialist teams each time workloads move between environments.

The same principle applies to AI model deployment. Gago expects enterprises to combine smaller specialised models with retrieval-augmented generation and governed corporate data rather than train large models with all internal information.

Sensitive information such as payroll data also cannot simply be embedded permanently in a model when access must vary according to the identity and permissions of each user.

"A company with 100 employees can actually become a 100,000-employee company with agents that are supporting each of us. We call this the AI rearchitecture," added Gago.