Mind your AMQPs… or Kafka… or Interconnect! What is in the Red Hat messaging box?

11 min readAug 12, 2020

I was asked recently to explain some of the pieces in Red Hat’s AMQ messaging software. The confusion was from non-integration folks who wanted to know what components they might use to solve a problem and why there is seemingly a large number of options.

<TLDR>
AMQ Broker — multi protocol, modern, and legacy transactional messaging
AMQ Streams — Kafka Broker/Connect/Mirrormaker for high throughput, high scale streaming
AMQ Interconnect — AMQP distributed brokerless message routing, connecting multiple clouds or geos with networked layer 7 routers
</TLDR>

Red Hat AMQ… Java Messaging Service, right?

No… well, not exactly. I’ve seen Red Hat AMQ grow from providing only a JMS Message broker (based on the Apache Active MQ open source project) delivering distributed asynchronous messaging, through to today where Red Hat AMQ is a multi-product, multi-technology, multi use case messaging suite that delivers messaging across pretty much any solution space in modern container and Kubernetes native architectures.

It’s the latest incarnation of the AMQ product that is really interesting as it provides both cloud native and traditional messaging paradigms in various guises, and most interestingly can combine into massively distributed and scaled messaging architectures through interconnected message routing.

In the “box” we have AMQ Broker, AMQ Streams, AMQ Interconnect, and AMQ Online. Let’s take a look at each and see what they are, along with some key technology points and use cases for each. I’m not going to deep dive into the technologies, as each of these are huge topics that have been covered elsewhere. I just want to give a high level view of what they are and what they are for. As I was writing this, a friend of mine, Bilgin Ibryam was also writing a more technical blog on the same subject, take a look here: http://www.ofbizian.com/2020/07/choosing-between-activemq-and-kafka-for.html

WHAT IS AMQ BROKER?

AMQ Broker is the latest release of the more traditional broker, re-architected in the latest version 7.x from the Active MQ Artemis project, with non blocking reactive architecture to support high levels of scalability and message delivery guarantees.

A broker in the context of Red Hat AMQ messaging is a piece of software that acts as an intermediary between message producers and message consumers. A brokers job is to decouple systems, with asynchronous messaging (fire and forget) patterns used to ease scalability issues related to having large volumes of messages, message producers, message consumers, or all three.

AMQ Broker is a real “swiss army knife” of messaging, with a huge array of broker topologies supporting almost any use case you can think of. The AMQ Broker technology is very mature. Customers have poked at all of the edge cases of the broker and the community has responded and corrected these over the years, to such an extent that there are sometimes an overwhelming number of feature flags and configurations to cope! AMQ Broker has a wide list of advanced messaging use cases such as: last value queues, message groups, topic hierarchies, and large message support.

At its heart, is a very smart broker which knows about topologies, clients, message statuses, what has been delivered, and what is yet to be delivered. The smarts are in the broker and the AMQ client APIs. The connecting consumer application needs very little effort to get things like client side failover or broker discovery, which is something that is a difference from AMQ Streams and Kafka, where the consumer needs more coding to handle complexity.

AMQ Broker supports point-to-point messaging through Queues (anycast) and one-to-many messaging through Topics (multicast). The actual physical configuration of queues and topics are abstracted through AMQP addressing.

Messages are sent to an address by a message producer, and the broker handles the internal storage and distribution. Message consumers take messages from an address. The broker only retains the message until a consumer has read the message, at which point the message is gone from a queue (…in general, although there are other configurations). This typical case, and how the brokers manage the messages, is one of the distinctions between Kafka and AMQ Broker message handling.

The AMQ Broker is scaled through multiple brokers providing redundancy, and either shared storage (main and redundant brokers both sharing the same message store, fail over to another broker who then reads the message from the store) or through message replication to another broker waiting to pick up and continue when failed over.

This is only the tip of the iceberg for AMQ Broker, but the main “what is it” points are here.

What Technologies?

If we look at the technology points for AMQ Broker, the diagram below gives a view of some of the main characteristics. The takeaway is that AMQ Broker is highly flexible with many protocols, clients, and topologies supported. If high levels of message guarantee and flexibility are needed, then AMQ Broker is perfect.

What Use Cases

If I take a look at a typical use case for AMQ Broker, a real use case for a UK airline, who had regulatory requirements to disseminate passenger events to multiple legacy and 3rd party systems. At each stage of a passenger journey (booking, APD, Check-in, security gates, boarding), the passenger event needed to be sent to internal and external systems.

The first part included a transactional guarantee that the event had landed within the system of record database, and therefore included the message broker in an XA distributed transaction.

The internal and third party systems were mixed legacy and needed to talk over specific protocols etc.

WHAT IS AMQ STREAMS?

AMQ Streams provides a set of technologies to build very large distributed messaging brokers with massive scalability of clients. The AMQ Streams set of technologies are Apache Kafka based, including supported versions of Apache Kafka components such as: Kafka Broker, Kafka Connect, Kafka Bridge, and Kafka MirrorMaker.

I love the simplicity of how Kafka handles messages, it’s the simplicity that underlies the power and versatility of this messaging technology. Unlike AMQ Broker, Kafka understands very little about the clients that are connected to the Kafka broker and instead hands off a lot of the intelligence back to client applications that are connected to the broker.

So How does this work?

The way the broker handles messages is its key. Kafka is essentially a distributed commit log that stores streams of records into logical categories called topics.

A message producer sends a message to a topic, that message is then appended to the topic and made available for consumers to read.

Message consumers can read a message from any position in a topic. The position to read from is identified by a pointer to a message location in the topic called a message offset. The client maintains the offset, so can read from any point backwards (or forwards :))

Because clients effectively manage how they consume data (the broker only records consumer offset positions when the consumer commits the new offset) this makes consumers very cheap to add, meaning that Kafka can scale to massive amounts of consumers. That’s what I’m talking about, simple makes it powerful!

Scaling and high availability is achieved through the partitioning or sharding of topics and distributing partitions across multiple servers, adding partition replication so that we have partitions available for failover spread across multiple servers.

Now, for me, the more complexity you put into the Kafka broker by replicating, partitioning etc. the more you lose out on the massive scalability. If you are trying to replicate highly available transactional guarantees by adding complexity to your Kafka brokers then you need to think, “Am I using the right messaging technology?”

But… keeping it simple, Kafka makes for a massively scalable streaming platform able to handle huge message loads and massive numbers of clients beyond AMQ Broker.

What technologies?

The power of Kafka is derived from its simplicity. Hence the versatility of topologies, protocols and other configurable options are much fewer than AMQ Broker. But hey, if massively scalable streaming is what you need, then AMQ Streams is the choice.

What use cases?

A simple use case of banking fraud detection, completely fabricated by me :)

The use case has transactions happening by a card holder at various locations: online, at the ATM, or a store transaction. Patterns of usage are a good way for detecting fraudulent use cases, for example a small tester transaction at a suspicious online store followed by a large cash withdrawal in an unusual location. Each of these transactions update a system of record, but we can also use the transaction to perform actions.

In our fictitious example, we use Debezium as a ‘change data capture’ tool to scrape transactions and send the stream of transactions to a Kafka topic. We have partitioned the topic by transaction type, so we have a set of streams for each type. By pushing these streams into our AI tooling, or even simply pushing into a complex event processor, we can infer things about the transactions and create alerts that must be handled. We then use AMQ Broker to push transactional alerts that are guaranteed to reach the end user systems, and make sure our transactions are safe!

AMQ Interconnect

What is it?

Interconnect is different to the other two in our AMQ bag of tools in that it’s not a brokering technology, it’s actually a routing tool working at the application layer to reliably route messages to destinations.

Hmm… it’s a router?

Yes, but a router that uses AMQP as a protocol (the upstream project is the Apache Qpid Dispatch Router), which makes it really interesting in our messaging bag of tools. AMQP as a protocol describes how to establish links between endpoints over a network. Consumers and producers use AMQP clients to connect to the routers and exchange messages.

Now in this simple network we can have brokerless communication between AMQP clients. We can even have request/response messaging so that a message consumer can immediately see that a message is consumed.

“Hang on a minute”, I hear you say. “We have some damn fine protocols that allow us to do that across the network stack!”

Well yes, but by adding brokering and other messaging semantics and capabilities into the mix, we can start to create very interesting use cases. Firstly, because Interconnect uses layer 7 (application) for communicating, our network uses TCP/HTTP(S). This means we can work with network firewalls to allow our message routers into private networks etc.

Other things we can do are: create redundant networking routes that the interconnect can use to route messages to destinations even if some of our routes are out of action, creating high availability routing paths.

We can distribute our routers across… anywhere: geographies, datacenter, edge, etc.

We can add brokers in the middle of our routes, creating networks of store and forward interconnected brokers. Awesome!

We can add mobile addresses, meaning that a client can attach to a router in the network, receive messages at an address one day. The next day the client (a laptop or device maybe) can fly to another country, connect to a different local router and continue receiving messages with no additional configuration.

The more I think about the versatility of using AMQ interconnect, the more I am like, “wow, what a tool, I can do so much with this!”

Take a look at Skupper as an example of someone who has said, ok, I can take the Qpid interconnect and add Kubernetes services onto my addresses, now I can connect two Kuberenetes clusters via Kubernetes services using Interconnect as the networking between them. Wow!

I love this technology, on top of the awesome messaging brokering tools, we can combine into topologies to solve real multi-cloud, multi-geography messaging problems.

What Technologies

What use cases?

A great use case for AMQ Interconnect is creating a messaging network between multiple clouds in a straightforward way. In this example, we connect multiple public and private clouds though the messaging network to replace batch updates in multiple geographies.

We have production data from Indonesia sitting in Alibaba cloud pushing data into locally hosted cloud databases. We also have online sales activity hosted on AWS distributed globally recording geography specific sales data.

In the past we have batched sales and production data at the end of the day to send through to the HQ clusters that are in a private data centre in the UK. Problems existed with the latency of the data compounded by the differences in timezone end of day. A few unsuccessful real time data solutions were tried and failed. A solution was to combine change data capture with a global interconnected network streaming data into a central Kafka cluster.

The solution:

Capture data in real time at source from redo logs with no impact to existing systems
Push that data across the interconnect to a central Kafka cluster with partitions for each source.
Central systems can read the data from the Kafka clusters at any time. The existing batch processes worked as is, reading data from the last batch read offset. New systems could be built to analyse data in real time as it is streamed into the Kafka cluster.

Phew!

I am amazed when I see new use cases and new topologies like Skupper come out of this tech, and there are so many things that I could talk about across all of these technologies. I haven’t even started on this stuff running on Kubernetes and Red Hat OpenShift! another blog another day!

In case you managed to scroll to the bottom, then here’s my summary!