Just over a year ago Red Hat announced that the company is changing gears on CentOS, dropping support for the stable release of CentOS that’s so universally popular, while continuing to drive development of CentOS Stream.
At the time of announcing this change, Red Hat appeared to suggest that many workloads could migrate seamlessly from CentOS 7 or 8 to CentOS Stream, and that the change in approach was broadly beneficial, including for the ongoing development of enterprise Linux.
The community backlash was firm. Most CentOS users considered it a drastic step that disrupted established IT plans, particularly given that a promised CentOS 8 support window was radically cut by several years.
Yet, in many ways, CentOS Stream is still CentOS, a binary compatible RHEL clone. But a single difference – the fact that CentOS Stream is updated more frequently at weekly intervals rather than every six months – has important implications for many workloads, if not all workloads.
Red Hat is not entirely wrong when it says CentOS Stream could be a replacement for CentOS 8. That said, for many users, this single change makes CentOS Stream unviable. So, can you switch from CentOS 8 to Stream without fuss? And under which circumstances is it simply not an option? Read on to find out.
What does Red Hat say?
It’s a mixed message, to say the least. In an interview, Hervé Lemaitre, Red Hat’s chief technologist in the EMEA said that it’s easy to migrate from CentOS 8 to CentOS Stream, because doing so involves just two commands. In the same interview, Brian Exelbierd, a RHEL product manager, said that the company’s VP of engineering suggests he could sneak into a data center at night and, in many cases, replace CentOS 7 or 8 with CentOS Stream without anyone noticing.
Those comments makes it sound as if CentOS Stream is perfectly fit for production. As if CentOS 8 and CentOS Stream is interchangeable. In the original announcement, Red Hat also implied that CentOS Stream can work well in the enterprise environment, by pointing to FaceBook as an example, because the social media company is running large workloads on an OS derived from CentOS Stream.
However, a document published by Red Hat includes this quote about CentOS Stream: “…it is not designed for production use. It is intended as a development platform…” The document suggests that users should consider Red Hat Enterprise Linux instead which, of course, unlike CentOS, isn’t free.
What is the truth?
Clearly, the truth is a bit more nuanced when it comes to CentOS Stream. We can safely say that “a stable release OS with a guaranteed maintenance window” is not what CentOS Stream is. By discontinuing CentOS as we knew it over the years, Red Hat caused significant challenges for many users who relied on CentOS Stream, but not for everyone.
In fact, in some ways the wholesale move to CentOS Stream has its advantages. Red Hat can focus limited CentOS resources on a single product that is better suited to ongoing development and community feedback, and which could even be seen as a better solution for a cloud centric world.
However, there’s also the reality of production workloads. Used in simple, isolated, non-critical circumstances without any third-party applications depending on it, CentOS Stream might well be a drop-in replacement for CentOS Linux 8.
But there’s a critical issue: most production workloads depend on a range of software solutions, of which the OS is just one. These software pieces must be certified to work with each other, and certification requires stable snapshots for comparison.
It’s one thing to certify software against Centos Linux 8.x, which will be around for a while, but it’s impractical to certify against CentOS-Stream-8-x86_64-2021XXXX-dvd1.iso which won’t be around in a week’s time. And that goes for software that’s developed in-house too.
Furthermore, certification usually happens against a particular version of Redhat Enterprise Linux – which CentOS used to follow accurately but Stream no longer does. Redhat itself specifically mentions this certification issue in their response to a change in Java’s release cadence that recently was announced by Oracle: “(…) certification of complex applications on a version of Java can often take many months and it has significant resource costs attached to it”. Replace “Java” with “CentOS” and it still holds true.
Use cases that could work just fine with CentOS Stream
Switching to CentOS Stream may work well for you. In this section we’ll take a look at some of the scenarios where you may find that switching to CentOS Stream is as simple as running a command line.
The OS you use IS of little relevance
In some instances the applications that you run on your OS depend very lightly on the underlying OS. In other words, where even major changes to the underlying OS won’t affect the applications that run on top of it.
For example, if you run web-based apps these apps are far more sensitive to changes in the web application platform. Changes in the OS won’t affect these applications very much – unless it affects the platform your apps run on, but if that’s the case you may well find that responses to changes in CentOS Stream are well organized.
Small, non-critical environments that can adapt
CentOS is popular not just in large production environments – it’s commonly used in small environments, sometimes even running as just a single instance. Depending on the workload you could decide that the relatively small risk something might break after a CentOS Stream update is completely manageable, and that you can afford the downtime.
Likewise, if you use CentOS as a desktop OS within a development environment you could also find that CentOS Stream works just fine. On the flipside, if your development work depends on a stable release it could trip you up.
You have a strong team and you build your own OS
In some particularly large-scale or highly specialized environments teams may well take a CentOS release and modify it for production purposes, and then repeat the process when there are sufficient improvements in a next CentOS release. These teams would have in-house expertise that can take on these tasks, and that’s what Redhat leadership pointed to in their Facebook example.
Here, CentOS Stream simply fast-forwards the existing process: you can still opt to “pause” CentOS Stream and derive your OS from it when you choose to do so. Losing stable release version of CentOS won’t matter to you. It is, however, a strategy that a relatively small proportion of CentOS users can afford to adopt. The required skill-set is in high demand and hard to find, thus accruing considerable cost to have in-house.
Use cases where CentOS Stream probably won’t work
We suggested earlier that there was a lot of community unhappiness around the Red Hat decisions, and there’s a reason for that – the simple act of removing a stable 1:1 connection to RHEL releases causes significant problems for many kinds of workloads. Here are some examples.
When OS stability and compatibility with RHEL is paramount
It all comes down to how your workloads are structured, what your applications depend on, and what the implications are if your workload experiences downtime – including how any machines are affected.
Simply put, there is a risk that a CentOS Stream update may break something in your workload due to, say, an updated library. With stable releases, you would always have a testing and implementation period – but not so with streaming updates. If even small changes in CentOS carry the risk of wreaking havoc on your workloads you should definitely think twice before switching to CentOS stream. The fast pace of updates means that testing environments will have a “moving target” to strive for, and the usual testing window will be made obsolete quickly. This will impact other processes in the company like feature rollouts, for example.
Application certification is required
Applications are usually certified to run on specific versions of RHEL, and would run equally well on matching CentOS Linux versions e.g. RHEL 8 v. CentOS 8. Certification is an involved process that requires intensive testing – but it does guarantee that an application will run, and if it doesn’t, guarantees a fix or support.
It used to be one of the biggest benefits of CentOS, as enterprise-grade applications that were certified to run on an expensive enterprise-grade OS (RHEL) would also run on the free-to-use CentOS.
CentOS Stream break this certification because any number of things may change from one release to the next. Vendors just cannot certify their applications on a weekly basis, as simple as that. So users who choose to run applications certified for RHEL on CentOS Stream risk losing support, and breaking compliance.
Cases that require that baseline performance must be an exact match
Another reason why CentOS Stream may not meet requirements are situations where baseline performance has to match previous runs. Think about scenarios such as scientific calculations that will be compared to prior results, for example.
These results can be affected by minute changes in library versions, which will happen more frequently in CentOS Stream. Similarly, high performance computing (HPC) platforms require very stable platforms, a level of stability that will no longer be there with CentOS Stream.
This is not always a bad thing. In fact, recent tests showed CentOS Stream 9 had marked performance improvements. It does, however, completely invalidate any results obtained previously – in fact, results obtained in one week may no longer be valid the week after, given a fast update pace.
A more nuanced view of how CentOS Stream relates to the now discontinued CentOS Linux is slowly filtering through the community. But the reality is that the change affects a large proportion of users for whom CentOS Stream is simply not an option. In fact, two RHEL clones – AlmaLinux and RockyLinux – have been set up with great effort to act as alternatives.
If you’re on CentOS 7, either of these makes for a relatively smooth switch to a 1:1 binary compatible RHEL clone with stable releases which should also carry across to application certification. As for CentOS 8, with EOL now a reality, you’re likely out of time to test a different release – which you should do before switching.
One option is to buy yourself some time with extended support from a third party vendor – such as TuxCare’s EOL support for CentOS 8. Not acting is the one option you don’t have. So think through your CentOS usage patterns and decide which category your workload fits in – and act fast to get extended support if you need it.