Release 7279: [BETA] Debian Buster hypernode-docker and configuration specific images

Images for different configurations

For the Ubuntu Xenial hypernode-docker the container would default to a specific PHP and MySQL version. It was then possible to change the version from inside the (running) container or from a Dockerfile based on the image. For MySQL this was a bit more clunky than for PHP because unlike PHP for which all versions are installed alongside each other, the image only contains one MySQL installation.

To make things more user friendly we have now published multiple images for different combinations of the PHP and MySQL versions. Instead of writing a Dockerfile around the hypernode-docker image to configure the right PHP or MySQL version a different image can now simply be used.

The new default Debian Buster Docker image can pulled at:

This default box currently contains PHP 7.1 and MySQL 5.6:

For specific version combinations you can pull from these urls:

The Xenial image can (for now) still be found at:

Check out the README on github for more info.

If you wish to make other changes like enabling Varnish or adding other functionality you still have to craft something yourself around the base image. An example of this is composed-mutagen-hypernode-elastic project by Ivan Chepurnyi where he uses Docker Compose to connect an ElasticSearch container to a hypernode-docker. Another great example of how people are extending the hypernode-docker image by basing a Dockerfile on it is this blog by Elgentos where they explain how they set up disposable testing environments using Kubernetes.

Docker containers vs a real Hypernode

The previous hypernode-docker image was based on Ubuntu Xenial like our Hypernode production platform. While the Docker is not exactly the same as a ‘real’ Hypernode, it is very similar. The hypernode-docker is a container while on the production platform each Hypernode is a proper virtual machine. Because Dockers are lightweight and an accepted part of the modern web developer’s toolchain it makes sense to use a container image similar to the production environment for testing and development.

Docker has good integration with tools for local development, automated testing in CI/CD like for example using GitLab or Jenkins and is easy to dynamically deploy in a cloud native environment using orchestration systems like Kubernetes on a public cloud like Google Kubernetes Engine so temporary environments for testing and acceptance can be quickly spun up and destroyed.

While we encourage people to use the hypernode-docker for development and testing, there are some key differences between the container image and our production platform. Superficially there are differences between how on your local Docker there is no init system while on a real Hypernode there is systemd, and the fact that in the container the kernel is shared with the host system (which has all kinds of implications). But on a more abstract level Hypernode (the platform) is a PaaS where the gritty complexity of DevOps and infrastructure is abstracted away from the user.

When you use Hypernode to host your Magento, Shopware or Akeneo application you give up a lot of flexibility compared to hosting that application yourself on an unmanaged service. You can’t access the AWS EC2 interface directly to hook up any cloud service like RDS or plug in an ELB. There is no cluster you can kubectl to in order to deploy an application or run a Helm “chart” on.

But in trade for that flexibility you get many features that would not attainable for any entity not devoting all their time to their platform. The Hypernode production platform exists of a large array of off the shelf ready to go (some opt-in) features, sophisticated cloud automation for scaling, ongoing security improvements, monitoring and alerting, auto-healing and recovery systems, backups and more. Perhaps the most important feature is that we get beeped out of bed at night if there’s something wrong and you don’t.

Our job at Hypernode is to spend our time improving the platform, while as a web developer this task would be secondary to the development of the application or shop. The world of modern web development is growing ever more complex with an infinite amount of JavaScript frameworks and web technologies like PWA, but the same is happening in the world of DevOps. By being able to focus solely on the platform we provide standardized functionality that would be difficult and expensive to reproduce in perpetuity by doing it yourself.

The hypernode-docker is a representative (enough) environment for testing changes in a shop to be deployed on a Hypernode on the production platform. We use the image ourselves too for integration tests and local development. Not for building shops, but for the configuration management of our systems.

For this reason we have published this new Debian Buster Docker image already as we are currently in the process of preparing to upgrade our production platform to Debian buster. While we develop our platform to run on top of this new operating system, any potential issues can be spotted in this image already by early adopters. If you run into any problems don’t hesitate to create an issue on github and we’ll take a look.

The Hypernode production platform

In the Magento hosting space it is common to encounter hosters that provide a ‘set and forget’ experience where when you order a new plan it is installed once (often manually) and never touched again. It is not rare that you will be charged for software updates, server maintenance and version upgrades. If there is package update automation it is often minimal (standard unattended upgrades) or configured to download system packages directly from third party repositories which might break, get compromised or disappear over time instead of being centrally maintained and vetted for quality by the hoster.

If left unchecked for too long this means that you risk running a shop on outdated software or that services and software change versions and functionality without knowledge of the hoster and your shop might break. And if there is an urgent security vulnerability the upgrade path might have become time intensive and complicated, while in such scenario you should be able to act swiftly and with confidence.

The Hypernode platform is very different in this regard. Because everything is automated, all Hypernodes are always up to date. If you order a Hypernode or scale your capacity in the middle of the night or on the weekend there is no waiting until Monday morning for a sysadmin to pick up the ticket from a queue, but our cloud automation handles everything immediately. You can even use the hypernode-api or the command-line to keep track of the automated process. If we add a feature it will instantly be available on all nodes on the platform. You get this consistent experience of quality even on our least expensive plan.

Hypernode is rolling release. There is no version 2 or 3, you’re always on the latest version. Our automation periodically updates each node (at least once per week) with the latest version of our software. Configuration changes, software updates and new features are constantly propagated throughout the platform. Our approach to the platform is continuous integration and continuous delivery.

We have an elaborate build pipeline where we integration test everything a customer can do from the point of clicking ‘buy’ in the control panel to delivering an installed application on a new cloud server. For each pull request we merge, we boot around 40 cloud instances to make sure things like scaling to a bigger plan, changing major PHP versions, configuring Varnish and installing the latest version of a CMS still work as expected.

This gives us a high velocity even though Hypernode has grown to be a massive platform. In the year 2019 we did 772 deploys on our production platform, that’s more than 2 improvements to your Hypernode per day. These are things from small optimization tweaks for configuring better defaults, new additions to our WAF, platform stability and performance improvements and allowing more configurable settings, but also major infrastructure overhauls like implementing special inter-provider scaling techniques for faster scaling in certain scenarios.

We have build pipelines for many of the relevant software packages that make up the web stack (for example PHP and Nginx) and our own Debian repository infrastructure for deploying those packages. This allows us to do things like making modifications to PHP in order to optimize active request throughput, keep the minor version of PHP up to date on all nodes and sometimes even patch security vulnerabilities even before the official operating system packages have.

In order to facilitate this constantly evolving platform we have to keep our software stack up to date. The expected features available on a Magento or Shopware hosting environment have grown in complexity. Many web-shops now require services like ElasticSearch for advanced search and RabbitMQ for asynchronous processing. Established software like PHP gets new versions to add functionality to the programming language and improves single core performance, which can make a great difference for your page load time.

As time progresses these newer versions of software will become harder and harder to run on an older operating system. We spend an increasing amount of time adjusting upstream packaging the older an OS gets while the demand for the features of this new software is often valid. The world of software doesn’t stop moving, and there comes a point where packages can’t even be backported to the old system anymore because of deep underlying changes. The same goes the other way around, running old software makes it so that eventually there might come a point where it is not possible to run it on a newer OS (without resulting to time intensive and often fragile ‘creative’ solutions).

Preparations for an OS upgrade of the production platform

Most of the mainstream Linux server operating systems like Debian, Ubuntu and CentOS have a release schedule where every couple of years an LTS (Long Term Support) version is released. The development of the operating system goes on and nothing is stopping you from using an unstable testing version of the upcoming release in the mean time, or cherry pick features from it. Debian and Ubuntu currently support their LTS versions for five years. This means security fixes and select software updates will be ported by their maintainers to these versions during this time.

The fact that an OS is LTS is not the only factor in play in regards to upgrading an operating system. The longer you stay on one version, the harder it will become to upgrade to the next version. It is possible to skip an LTS upgrade and stay within the support period, but upgrading multiple versions makes you have to deal with that much more incompatibilities and changes. Additionally, while the OS is LTS, it does not mean all security issues or software updates are in fact addressed. And if they are, likely not to the same extent or more importantly with the same urgency as the most used version of the OS at that time.

In 2017 we upgraded the operating system underlying all Hypernodes for the first time. We upgraded from Ubuntu Precise to Ubuntu Xenial. This was a rather complicated process because in that migration we skipped one LTS version (Ubuntu Trusty), this was the first time we’d run systemd on the Hypernodes (previously upstart) and at the time we did not have floating IPs. Even though at that point Ubuntu Precise (which was released in 2012) had become pretty old we were still able to provide modern software like PHP 7 by building our own packaging, but this was becoming more arduous to keep up. Now in 2020 we’re at a similar point with Ubuntu Xenial.

Back in 2014 when the first Hypernodes were deployed, Ubuntu was chosen instead of the Debian OS that ran most of our datacenter because in a production platform you want to encounter as few surprises as possible. Back then compute cloud was relatively new and the most common OS on AWS and other similar clouds was Ubuntu and not Debian. Now years later the cloud has matured and the critical mass is there that these issues are less of a consideration. All this time we’ve been building mostly Debian packaging (to run on this Ubuntu OS) because in our experience that ecosystem generally is more transparent, better structured and often the packaging that Ubuntu provides is derivative of Debian anyway.

For this reason we’re not upgrading to Ubuntu Bionic or Ubuntu Focal this time, instead we’re upgrading to Debian Buster. This for us will remove some friction in deploying and maintaining software and from the user perspective the two operating systems are mostly indistinguishable. For us internally this also means our supporting infrastructure and the servers used by customers will run the same operating system for the first time in years, which hopefully will provide some synergy benefits for our code-base as we develop the platform.

Currently we’re still very much in development for the new Debian Buster Hypernode. Ostensibly we’re re-creating what constitutes a Hypernode from scratch. We’re taking a look at ‘what can it currently do’ at a very low level and are re-implementing those features on top of this new OS. We’re keeping all relevant software the same to make the upgrade process as smooth as possible, but many smaller system packages that we don’t explicitly managed will be upgraded to a newer version.

For example, the git version control command line utility will upgrade from version 2.7.4 to 2.20.1. Many small changes like that will benefit the user as newer flags and features will be available, (increasing the chance that those snippets you copy-paste from stack-overflow don’t error out on missing functionality ;)). In the mean time we’re paving the way for future features that currently aren’t possible on Xenial. An example of this is TLSv1.3 which will work out of the box on Buster.

A migration like this also gives us a chance to iron out some pre-existing inconsistencies like making it so that all (Excellence) nodes will have a 20 GB root disk like our Openstack and DigitalOcean plans instead of the 8 GB currently allocated on AWS. More specific changes will be communicated as we progress.

For now all Hypernodes on the production platform will remain on the Ubuntu Xenial OS. But eventually we’ll switch over any newly ordered nodes to use Buster and some time after that we’ll migrate over all existing Xenial nodes. Of course there will be advance notice for when that will happen.

Currently we have arrived at being able to boot a Buster Hypernode and it will on first sight be the exact same thing as the Xenial Hypernode. We’re testing internally for missing or broken features in specific configurations, but we’re at a point where we’d be comfortable with having BETA-testers.

Request a BETA Hypernode on the new OS

If you checked out the hypernode-docker and would like to take this new BETA Debian Buster Hypernode (the real server, not the container) out for a spin on the production platform, send a mail to support@hypernode.com and we’ll see if we can set you up with a free and temporary Hypernode for testing. We’d love to get some feedback and perhaps find any potential issues.

Keep in mind that these test Hypernodes will be thrown away later during this development process, will not be actively monitored or alerted on, and will really only be for testing and poking around. We’ll only give out a couple, so if you’re interested in this type of preview and giving some feedback let us know.