Managing and scaling different white label development and testing environments

White-labelling (Part 2): how Agoda manages development environments for white labels

In the previous part, I explained Agoda’s white label architecture, what they are, and how we manage the differences between white labels. If you haven’t read the first part, you can read it here — Building and scaling different travel websites with one codebase

To briefly recap: a white label is another travel website that uses Agoda’s code. We maintain the differences between websites by using a combination of White Label Service and Behaviour Management.

In this article, I will share how we address a challenge of multiple environments for multiple white labels: Development, QA, Pre-Live, Live/Production.

Multiple Environments for Multiple White Labels

At Agoda, we have different environments for development stages. We have URLs for each environment, development.agoda.com, testing.agoda.com, pre-live.agoda.com, and agoda.com, respectively. For each stage, there is a URL for every active pull request. For example, pull1234.pre-live.agoda.com is the pre-live URL for pull request #1234.

The challenge is that when we have created a pull request, developers have to check the website for other white labels. There must be a way for developers to access those white-label websites. Also, the system has to be able to present the website in different stages of the development process.

Using domain names for white labels

Our initial solution was to create a new domain for every white label. For example, for white label 42, assuming the name is Adoga, we would register a domain called notyetadoga.com for this white label.

We would then use the following approach for building a domain name:

• a subdomain for each stage of the development process, e.g., development.notyetadoga.com to indicate the development stage
• a sub-sub domain to represent a pull request, e.g., pull1234.pre-live.notyetadoga.com for pull request #1234.

As you can guess, the domain name solution works fine for the first few white labels. We eventually ran into multiple scalability problems, for example:

• The costs of creating and renewing the domain names and certificates
• Cross-Origin Resource Sharing (CORS) issues when forcing API endpoints on other domain names, etc.

In the end, we decided to move on from this high-cost and unscalable approach.

Using a header for white labels

Essentially, to differentiate white labels, we need to find a way to tell our server which white label ID we are trying to access. We then came up with the header solution.

Instead of using domain names, we can just send a custom header to our server-side to select what white label we want to get. For example, if we want to get white label 42, we can send the header x-agoda-whitelabel-id = 42 along with the requests using a browser extension, for example, ModHeader, etc.

When the reverse proxy receives the x-agoda-whitelabel-id header, it passes on that header downstream to the web server and other services. Those services use the header to get the configuration from White Label Service and compute the results.

This is a simple and scalable solution but does not work as expected on mobile devices. Furthermore, we cannot detect brand leakage (an accident that a white label shows images, links, or strings from another white label) with this approach because every white label still points to agoda.com. We can never know which URL/link points incorrectly.

In a typical development process, developers use mobile view in the browser’s dev tools to simulate viewing the website on mobile devices which work fine 99% of the time. However, in some cases, we found a bug that can only be reproduced on real mobile devices. Forcing the header on mobile devices is highly inconvenient and challenging. Also, it is difficult for our partners to look at the website since they might not have the technical background to know about headers and how to force them, which is a bad experience.

Even though the white label header is a much better approach than creating individual domain names, it is not very practical to do testing on mobile devices and troublesome for our partners and our automation test systems to test. We, again, had to find a new solution.

Back to domain names, but this time, it’s better!

We have observed a few problems with the previous solutions that we have faced:

• Overheads in preparing for a new white label
• No consolidated system for end-to-end (E2E) tests for white-labels
• Difficulties for the partners to access the website for user acceptance test (UAT)

The white-label team has designed a solution that combined both the header and domain name solutions with those problems in mind. We created another layer of a reverse proxy that works as follows:

• We created a unified white label domain name, agoda-whitelabel.com
• We added another layer, white-label proxy, to resolve the new white label URL scheme
• The white label reverse proxy tokenises the URL and maps back to the corresponding pull request, white-label, and environment
• The white label proxy then passes the mapped information to the reverse proxy to prepare information for the downstream systems

The URL scheme works as follows: for each environment, we use a subdomain to specify the white label, e.g., wl999.agoda-whitelabel.com for white label 999. We use the sub-sub domain name to target a pull request and environment, e.g., pull1234-pl.wl999.agoda-whitelabel.com for pull request #1234 on the pre-live environment for white label 999.

Initially, we considered having only one level of subdomain name, e.g. pull1234-pl-wl999.agoda-whitelabel.com to simplify the SSL certification management, but we realised the complexity and headache of cookie management that would follow. So, we decided to add one more level of subdomain name.

The current solution allows the system to be more flexible and extensible. We could add more environments easily. For example, uat.wl999.agoda-whitelabel.com can be used for user acceptance tests. This is much simpler for our partners and us to test the website.

Final Thoughts

To look back, we have learned a lot along the way. We combined the knowledge of two previous solutions, domain names and headers, to create the best solution so far. Of course, in software development, this is not the perfect and final solution. We are improving the domain name systems to support other environments, such as origins and different testing environments.

Next time you book a hotel or flight, you might be booking on Agoda’s partner websites unknowingly. Look out! It might be one of Agoda’s white labels if it is full of features!

Thanks to Vlad Batushkov, Kevin Kenny, Brinda Krishnan, Patthaphol Kulthawaiporn for their help improving this series.