In this release we deploy the possibility for Hypernodes to run RabbitMQ. Now that Magento 2.3 has been released, RabbitMQ can be used on Magento Open Source installations. Previously it was already possible to use RabbitMQ in the Enterprise version of Magento 2, or with the help of custom third party modules. While Hypernode has always been compatible with shops that depend on RabbitMQ for event processing, this release will make it possible to use the service out of the box without having to arrange it externally.
Because RabbitMQ is by definition not as latency sensitive as services like MySQL (because it is implemented to deal with asynchronous actions), previously we recommended to connect an externally hosted RabbitMQ instance (like CloudAMQP) to your shop if you required it. Now that is no longer necessary and you can run the RabbitMQ service on the node itself at no additional cost.
To enable RabbitMQ on your Hypernode you can use the hypernode-api or run the hypernode-systemctl command-line tool on the Hypernode. Note that RabbitMQ is only supported for products Professional M and larger.
# Check if RabbitMQ is currently enabled
app@s2lsnn-vdloo-magweb-aws:~$ ps aux | grep /usr/sbin/rabbitmq-server | grep -v grep
app@s2lsnn-vdloo-magweb-aws:~$ # No output
# Enable RabbitMQ using the commandline-tool
app@s2lsnn-vdloo-magweb-aws:~$ hypernode-systemctl settings rabbitmq_enabled --value True
Operation was successful and is being processed. Please allow a few minutes for the settings to be applied. Run 'livelog' to see the progress.
# Wait for the node update to finish (`hypernode-log` or `livelog`)
app@s2lsnn-vdloo-magweb-aws:~$ hypernode-log
ACTION START END STATE TASKS RUNNING
update_node 2018-12-28T11:11:22Z 2018-12-28T11:11:23Z running 3/4 rabbitmq_update_node_to_update_flow
# When the update has finished you should now see a RabbitMQ process
app@s2lsnn-vdloo-magweb-aws:~$ ps aux | grep /usr/sbin/rabbitmq-server | grep -v grep
rabbitmq 13327 0.0 0.0 4504 700 ? Ss 09:58 0:00 /bin/sh /usr/sbin/rabbitmq-server
If you wish to disable RabbitMQ later (to perhaps save some RAM) you can do so with:
app@s2lsnn-vdloo-magweb-aws:~$ hypernode-systemctl settings rabbitmq_enabled --value False
Operation was successful and is being processed. Please allow a few minutes for the settings to be applied. Run 'livelog' to see the progress.If your product is not Professional M or larger, you will see this message:
app@b3acqq-vdloo-magweb-do:~$ hypernode-systemctl settings rabbitmq_enabled --value True
Looks like something went wrong: b'{"rabbitmq_enabled":["RabbitMQ cannot be enabled for this app. Please upgrade to a plan that supports RabbitMQ if you want to make use of this feature."]}'Once your system is running the rabbitmq-server you can access the web interface on port 15672. But because this port is only bound to localhost on the Hypernode (and not publicly accessible) you have to tunnel through SSH. Later we’ll also make a change to allow the hypernode-vpn to access it. For now, to create an SSH tunnel to forward the remote port to your local host you can run:
ssh app@appname.hypernode.io -L 55672:localhost:15672
And then access the RabbitMQ Management UI on http://localhost:55672 in your browser. The username / password is the default guest / guest but you can configure more granular permissions on the UI.
To use RabbitMQ in your Magento 2.3 Open Source installation you can do something like this in the env.php:
array (
'amqp' =>
array (
'host' => 'localhost',
'port' => '5672',
'user' => 'guest',
'password' => 'guest',
'virtualhost' => '/',
),
)And then run an event-listener like for example:
bin/magento queue:consumers:start async.operations.allIf you want Magento to emit events into this queue from for the Asynchronous Web API for POSTS to /async/V1/products for example, you can do something like this in your test environment to see if it works (note that you should never edit core files in production shops but always write a nice module that overwrites functionality from the outside).
# Change queue_consumer.xml from
~/magento2$ cat ./vendor/magento/module-webapi-async/etc/queue_consumer.xml
<?xml version="1.0"?>
<!--
/**
* Copyright © Magento, Inc. All rights reserved.
* See COPYING.txt for license details.
*/
-->
<config xmlns_xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi_noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/consumer.xsd">
<consumer name="async.operations.all" queue="async.operations.all" connection="amqp"
consumerInstance="MagentoAsynchronousOperationsModelMassConsumer"/>
# To
~/magento2$ cat ./vendor/magento/module-webapi-async/etc/queue_consumer.xml
<?xml version="1.0"?>
<!--
/**
* Copyright © Magento, Inc. All rights reserved.
* See COPYING.txt for license details.
*/
-->
<config xmlns_xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi_noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/consumer.xsd">
<consumer name="async.operations.all" queue="async.operations.all" connection="amqp"
consumerInstance="MagentoAsynchronousOperationsModelMassConsumer"/>
<consumer name="async.V1.products.POST" queue="async.V1.products.POST" connection="amqp"
consumerInstance="MagentoAsynchronousOperationsModelMassConsumer"
handler="MagentoCatalogApiProductRepositoryInterface::save"/>
</config>
# And queue_topology.xml from
$ cat ./vendor/magento/module-webapi-async/etc/queue_topology.xml
<?xml version="1.0"?>
<!--
/**
* Copyright © Magento, Inc. All rights reserved.
* See COPYING.txt for license details.
*/
-->
<config xmlns_xsi="http://www.w3.org/2001/XMLSchema-instance" xsi_noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
<exchange name="magento" type="topic" connection="amqp">
<binding id="async.operations.all" topic="async.#" destinationType="queue" destination="async.operations.all"/>
</exchange>
</config>
# To
~/magento2$ cat ./vendor/magento/module-webapi-async/etc/queue_topology.xml
<?xml version="1.0"?>
<!--
/**
* Copyright © Magento, Inc. All rights reserved.
* See COPYING.txt for license details.
*/
-->
<config xmlns_xsi="http://www.w3.org/2001/XMLSchema-instance" xsi_noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
<exchange name="magento" type="topic" connection="amqp">
<binding id="async.operations.all" topic="async.#" destinationType="queue" destination="async.operations.all"/>
<binding id="async.V1.products.POST" topic="async.V1.products.POST" destinationType="queue" destination="async.V1.products.POST"/>
</exchange>
</config>
If you then add a product using the async web endpoint in the Magento API you should see events passing through the async.operations.all queue. Note that any transient message will NOT be persisted to disk. Data that will persist to disk and that will still be available after scaling your Hypernode or after restoring from backup are queues, exchanges, users and messages tagged as persistent. For more information about RabbitMQ see the official documentation.
Also check out this Hypernode knowledge base article about RabbitMQ.
Other small tweaks in this release:
– We have decreased the upper limit on active PHP requests per IP from 20 to 15.
– We have slightly increased the max amount of allowed connections for MySQL based on vCPU count.
These changes should benefit the stability of larger nodes somewhat.