What is a message queue?

A message queue is a form of asynchronous service-to-service communication used in serverless and microservices architectures. Messages are stored on the queue until they are processed and deleted. Each message is processed only once, by a single consumer.

When should you use a message queue?

Message queues are useful for decoupling heavyweight processing, buffering or batching work, and for smoothing spiky workloads. They enable asynchronous communication, allowing services to interact without being tightly coupled.

System Design Interview

Message Queues

Build robust and scalable systems by understanding how message queues facilitate asynchronous communication and decouple your microservices.

Example: Producer-Consumer with RabbitMQ in Python

A basic example of a producer that sends a message to a RabbitMQ queue and a consumer that receives it. This pattern is fundamental to many distributed systems.


                                # Producer.py
                                import pika
                                connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
                                channel = connection.channel()
                                channel.queue_declare(queue='hello')
                                channel.basic_publish(exchange='', routing_key='hello', body='Hello World!')
                                connection.close()
                                
                                # Consumer.py
                                import pika
                                connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
                                channel = connection.channel()
                                channel.queue_declare(queue='hello')
                                def callback(ch, method, properties, body):
                                    print(f" [x] Received {body}")
                                channel.basic_consume(queue='hello', on_message_callback=callback, auto_ack=True)
                                channel.start_consuming()

Our AI Coach can help you compare different message brokers like RabbitMQ and Kafka, and discuss concepts like message durability, delivery guarantees, and dead-letter queues.