In the coming days of the holiday season, the last thing you want to worry about is your hosting environment. This is why we have taken extra proactive measures to ensure the stability of your Hypernode. We do this with the changes that are listed in this changelog.
These changes have been released alongside our recent performance improvements. Read more about those in the previous changelog.
Randomized dehydrated cronjobs
Turns out that our support documentation is quite popular! So popular in fact, that the cron we suggest for renewing Let’s Encrypt certificates is widely used across our Hypernode platform. This is causing a thundering herd problem where we see an increasing amount of failed requests for Let’s Encrypt SSL certificates. We believe this is caused by several hundreds of Hypernodes who had a cron configured to renew their certificate every night at 1 AM UTC.
In hindsight, the amount of times this cron checks for a possible certificate renew is a bit excessive. That’s why we decided to automatically detect this cron and configure it to be less aggressive with it’s renewal period.
If you have this cron configured, you’ll notice that the renewal check has changed from a daily check to a weekly check. It now happens at a random weekday, somewhere between 0 and 7 AM UTC instead of daily at 1 AM UTC.
Automatic block for aggressive Petalbot
Earlier this year we posted a blog regarding the increasing impact of the Aspiegel/Petalbot. The pattern in requests doesn’t seem to be changing, as there are still millions of requests coming from this bot to our platform on a daily basis.
Previously we applied a ratelimit to this bot to reduce the impact it has on Hypernodes. However, since that time we’ve had complaints about this bot having performance impact on Hypernodes. This is why we have decided to automatically block all requests coming from Petalbot who cross our threshold of malicious requests.
We’ll inform you by mail if we’ve seen malicious activity of the Petalbot on your Hypernode, with the appropriate actions that we’ve taken. If you want to deactive the automatic block that we’ve placed, just comment out the 403 block in /data/web/nginx/server.malicious_aspiegelbot.
Pin composer version to version 1
Composer has released their version 2 of composer last month. At the day of the release, we’ve received many complaints about composer not working as intended. We cannot risk composer not working during the holiday season, so to counteract this we’re enforcing composer version 1. To prevent composer’s self-update to automatically upgrade to version 2, we now run composer self-update --1.
We will have another look at the update process to Composer 2 after the holiday season.
Set PHP-FPM’s pm.max_requests
The parameter PHP-FPM called pm.max_requests specifies how many requests one PHP-FPM worker should handle before being respawned. This parameter defaults to 0, and thus will never respawn any PHP-FPM workers. However, setting this value is useful to prevent a memoryleak in a PHP application. Using this value might provide a solution for these leaks without having a significant impact on the application.
We’ve ran several benchmarks to find the best value for this parameter. Out of the values 0 (default, unlimited), 256, 512, 1024 and 2048, we’ve found the value 1024 to perform the best. We found that the difference in response times between results were minuscule. This gives us confidence that setting this value will not have any performance impact on any running applications.