This release adds the hypernode-auto-logrotate command to our bundle of hypernode- system utilities. This is a command line tool which automatically rotates log files, which helps avoid disk space and backup issues.

Log rotation

A hypernode contains files that applications write their logs to. For example, Magento writes to /data/web/magento2/var/log/debug.log. These log files can get quite large (we’ve seen examples of over 30GB), so for example when doing a backup, you end up transferring obscenely large files over the internet.

You can avoid this problem by rotating the log files. This means that when a log file /data/web/example.log gets too big, you compress it, move it to /data/web/example.log.1.gz, and give the application an empty log file to write to. The hypernode-auto-logrotate command automates this process.

Example usage

For a log file ~/example.log, the tool can be used as follows:

app@levkd4-example-magweb-cmbl:~$ hypernode-auto-logrotate example.log 
Adding logrotate cron job 
Adding logrotate config entry for "/data/web/example.log"

This will cause the log file to be rotated on a daily basis, at around midnight. If the file is over 50MB, the next day the old logs will have been moved to ~/example.log.1.gz.

app@levkd4-example-magweb-cmbl:~$ ls example.log*
example.log  example.log.1.gz

The tool also has a --detect option, with which it will search for log files and rotate them:

app@levkd4-example-magweb-cmbl:~$ hypernode-auto-logrotate --detect
Searching for logfiles...
Logfile found: /data/web/example.log
Logfile found: /data/web/magento2/var/log/system.log
Logfile found: /data/web/magento2/var/log/connector.log
Logfile found: /data/web/magento2/var/log/debug.log
Logfile found: /data/web/magento2-sample-data/dev/tools/exclude.log
Logfile "/data/web/example.log" is already being rotated
Adding logrotate config entry for "/data/web/magento2/var/log/system.log"
Adding logrotate config entry for "/data/web/magento2/var/log/connector.log"
Adding logrotate config entry for "/data/web/magento2/var/log/debug.log"
Adding logrotate config entry for "/data/web/magento2-sample-data/dev/tools/exclude.log"

The tool has a --dry-run option which will print what the tool will do without actually doing it. For more detailed output, run it with the --verbose option.


The hypernode-auto-logrotate command comes with a sensible out-of-the-box configuration. Among other things, this means that it will rotate log files daily, only when they exceed 50MB, and that it will keep four files in rotation (meaning you end up with example.log.1.gz up to example.log.4.gz). These configurations are stored in /data/web/hypernode_logrotate.conf, and can be customized per log by editing this file. For example, the entry added for example.log reads

/data/web/example.log {
    rotate 4
    # do not raise an error if the logfile does not exist
    # file size at which rotation is triggered
    size 50M
    # copy the file and truncate the original, in case
    # another process cannot be told to close the logfile

The logrotate man page contains instructions on how to customize this.