Keep your services healthy with circuit-breaker

The circuit-breaker is not that much different from Stop Loss in stock investment.

When your Titanic has leaking holes, the entire ship will sink if you don’t have another protection layer to isolate those leaking holes.

The essence of Circuit-breaker is to isolate unavailable dependent services to avoid cascaded service unavailability.

Typical issues caused by bad dependent services

  1. Exhausting resource because of dependent service late response or no response for a long period.
    • Ever request will open certain resource in a server. The threads will be locked until the request is timeout. In high concurrent scenario, it could exhaust the server’s resource.
    • If microservice applications run on containers, this could lead to the container got killed because run out of memory.
  2. Causing MQ unavailability. If two services are interacted with each other using ActiveMQ. Service A keeps sending message to the ActiveMQ while Service B is unavailable. This could result in the ActiveMQ runs out of storage such as too much message in the DLQ. And finally, the MQ becomes unavailable.

The Circuit-Breaker Design

Other implementations, frameworks, design and approaches will be provided when I have time.

Leave a comment