Our team has been hard at work building out the next iteration of our customer storefront accelerator using Next.js (the current iteration of our storefront accelerator is built using Create React App). Given that the current version of the storefront is still being used and tested, this gave us a good opportunity to exercise various deployment strategies on top of our microservices architecture. My goal was to allow our team to test out both versions simultaneously on the same Kubernetes infrastructure allowing for a canary type rollout.

Canary and A/B testing strategies are common “asks” in the real world and make a lot of sense to safely and incrementally introduce changes without having to do a “big bang” cutover. So, in this series of blog posts - I will walk you through various strategies that can help you implement similar flows in your microservices environment.

Ingress Strategy: Canary Routing Using Nginx Kubernetes Ingress

In this post, I will be describing how to implement canary routing using an Nginx Ingress. In particular, we’ll showcase canary deploying a service containing a new Next.js storefront application running alongside an existing one all while both connect to the same microservices on the backend.

With Broadleaf’s reference deployment architecture, we utilize an Nginx Ingress Proxy to front our Kubernetes cluster. The Nginx proxy is widely used and supports several interesting capabilities including the ability to do canary routing by keying on different criteria: (https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#canary).

The proxy allows you to route requests based on cookie or header and gives you the ability to define weighted requests to slowly increase the load to the canary service over time.

Deploying the Canary Service(s)

Test Example:

Here’s an example manifest that provisions 2 test “http-echo” resources (one production and another canary) that are fronted by an Nginx Ingress. Run `kubectl create -f http-echo-canary-example.yaml` to create the deployments, services, and ingress with weighted canary enabled. This example should route about 30% of requests to the canary service and the rest to the production service.

You can now test to see this in action using the following script:

for i in $(seq 1 10); do curl -kL http://echo.example.com; done

Broadleaf Specific Example:

In our case, the Broadleaf Microservices stack utilizes a “Backend for Frontend” pattern (https://samnewman.io/patterns/architectural/bff/) with a lightweight gateway built on Spring Cloud Gateway to facilitate the coordination of the storefront application with the core backing APIs. Because Spring Cloud Gateway is flexible and allows you to define routes via properties or environment variables I’m able to deploy a copy of the same production gateway pod that just defines a different environment variable pointing to the new canary storefront and left all the default routes to the backing production service APIs the same.

Here is an example Kubernetes manifest for reference:

The diagram below gives a representation of the resources involved with this routing strategy:

In particular, for this example, we wanted to allow users to selectively switch between the old and new applications. A common user experience strategy to achieve this would be to add a button or action to allow users the ability to switch to the new design/application with the ability to revert if necessary. It may make more sense to configure a randomized percentage-based routing rule in other scenarios or configure certain regions or geographies to get the new flow. Being able to leverage canary routing by “cookie” on the Nginx ingress makes supporting our use case pretty trivial.

Creating the “Try Beta Design” Button

Next, I went ahead and created a small button that drops a specific cookie that Nginx can filter and route to the appropriate services. In our case, we named the cookie “canary” and set the value to “always”.

Here is a reference react component that drops the canary cookie and refreshes the window.

Incorporating this component into the existing storefront header like this allows the users to selectively try out the new design. All subsequent requests will direct that user to the new storefront and if the user ever wishes to go back to the old design, they can clear that particular cookie on the new storefront.

That’s it! With a few steps, we’re able to leverage some basic canary deployment mechanisms built into the Ingress capabilities of our microservices cluster. In a later post, we’ll go over other patterns such as using Spring Cloud Gateway to facilitate additional routing and A/B testing strategies.