Code Outside of the Box

AMQP: A Developer's Digest

What is AMQP?

An acronym for the “Advanced Message Queuing Protocol”. It is a Standard (with a capital “S”) application layer protocol defining a messaging protocol for message-oriented software. It’s a binary protocol wire-level protocol. It describes the format and structure of data sent across the network. It’s effectively peer-to-peer protocol, simply defining how two peers may exchange messages, and not does define what roles the peers may or may not have.

You can, for example, build a message broker. Or message queue. Or a gateway. Or an event bus. Or an “exchange”. You get the idea. It supports virtually any type of messaging topology.

This article aims to be update-able reference for those getting started with AMQP.

The Protocol

Here is the actual 1.0 specification. It over 100 pages long, but it’s a relatively quick read. The protocol specification itself is relatively simple and broken into several layers.

Type System

AMQP implements it’s own binary encoding for types. It define primitive types that map to many popular programming languages and platforms. This enables a high-degree of interoperability. It also supports composite types.

Transport Layer / Link Protocol

Data is always transmitted across the wire in a frame. AMQP defines frames for various types of control messages (opening connections, starting sessions, attaching links, transferring messages, etc.).

1. Connections are usually opened over a reliable protocol such as TCP.
2. A session is started on a connection. A single connection may multiplex several independent sessions.
3. On a session, one or more uni-directional links are attached. Either peer may attach a link for sending or receiving. The link is attached, logically, to some terminus such as a queue.
4. Messages are transferred over links.

Since a single connection can multiplex multiple sessions and/or links, this means that network traffic can be streamlined. You don’t need multiple connections.

Messaging

The messaging layer covers:

• Message format. Including structure, properties, and headers.
• Delivery state for messages traveling between nodes. Messages need to be settled which ensures the sender and receiver agree on the state of the transfer.
• Sources and targets, and the states for messages stored at a distribution node. It also covers resumption of broken sessions and links.

Transactions

The protocol supports transactions across transfers and distinct links in either direction. A transactions is first declared which establishes a transaction-id. Subsequent work is includes that transaction-id. At the end, the transaction is discharged, which will either commit or rollback the transaction.

Security

You can encrypt the connection with TLS. Additionally, the protocol supports SASL to authenticate peers. Both protocols are be negotiated.

Who/what uses AMQP?

Microsoft’s Azure Service Bus supports AMQP. But not all Azure Service Bus features are supported over AMQP. The client library implements AMQP (it’s not currently open source).

Microsoft does have an open source AMQP client library: https://github.com/Azure/amqpnetlite. I believe that it’s very similar to the implementation in WindowsAzure.ServiceBus.

The Apache Foundation has a number of open source projects: Apache Qpid and Apache ActiveMq.

There are a number of other commercial and open source brokers, for example IBM MQ Light. But I don’t know a lot about them.

And then there’s RabbitMQ. RabbitMQ is interesting because it is actually built on AMQP 0.9.1. That version was a pre-release and is a very different and basically incompatible protocol. There is a plugin to support AMQP 1, but it’s marked as “experimental”!

Annotated Further Reading

• AMQP Version 1.0. The actual specification.
• Clarifying AMQP. Kelly explains the differences between AMQP 1.0 and the prelease 0.9 specification.
• AMQP Isn’t So Scary… If You Know How To Start!. Another digest style article with links and references.