Docker Service Discovery and Load Balancing with Haystack

Haystack provides service discovery and automatic HTTP load balancing in a Docker environment. It uses its own DNS to provide service discovery and dynamically updates its embedded load balancer as services are started or stopped.

Haystack solves two problems:

  • Service Discovery – Haystack automatically manages service registration within DNS by monitoring the Docker container lifecycle. Haystack introspects container metadata exposing available services within its own DNS. Haystack automatically manages the registration of services into a common DNS for discovery by other services.
  • Load balancing of a service as it is scaled up or down to meet demand – Haystack registers DNS service names that are load balanced over the available instances as the service is scaled up or down to meet demand. Each service name is registered in Haystack’s own DNS and Haystack manages the load balancing of the containers providing that service, from one to thousands and back again – dynamically as demand rises or falls.

This post walks through a demo of some of capabilities that Haystack provides using a simple HTTP based micro service. The service accepts HTTP GET requests and responds with the hostname of the container and the HTTP path used in the request.

To start a number of services to demonstrate load balancing in Haystack:

for i in {1..5}; do
     docker run \
     --name demo-$(printf %03d $i) \
     -d shortishly/haystack_demo;

Start Haystack – replace with the location of your Docker Engine that is already listening on a tcp port.

docker run \
    -p 8080:80 \
    -e DOCKER_HOST=tcp:// \
    -d \
    --name haystack \

You should now have Haystack and 5 demo micro services running within Docker:

# docker ps --format="{{.Names}}"


Now start a busybox that uses Haystack for DNS resolution as follows:

  docker run \
  --dns=$(docker inspect --format='{{.NetworkSettings.IPAddress}}' haystack) \
  --tty \
  --interactive \
  --rm busybox /bin/sh

In the busybox shell:

wget \
   -q \
   -O /dev/stdout \

Haystack has registered in
its own DNS service, and is load balancing any
request randomly to one of the demo-001, demo-002,
demo-003, demo-003, demo-004 or demo-005

If you make a number of wget requests to the same URL you will
get responses from the different containers at random:

# wget -q -O /dev/stdout
d617596e70da: /load/balancing
# wget -q -O /dev/stdout
96c6c6f27f03: /load/balancing
# wget -q -O /dev/stdout
296b5208edf9: /load/balancing
# wget -q -O /dev/stdout
1166e110e70d: /load/balancing
# wget -q -O /dev/stdout
9909343b937a: /load/balancing

You can verify which services are available in Haystack by curling /api/info:

curl -s http://$(docker inspect --format='{{.NetworkSettings.IPAddress}}' haystack)/api/info|python -m json.tool

Use you bowser to visit the Haystack UI running on your Docker host:


Leave a Reply

Your email address will not be published. Required fields are marked *