testing-node-apps-with-docker-compose (and some Soul)

Contents of this post

  • Purpose: the reason for this blog post.
  • Scenario: what this example of using docker-compose can be useful for.
  • Prerequisites: basic setup to be able to run the code contained in this post.
  • Code example: an actual step-by-step guide on how you can setup your test environment to run with docker-compose.
  • Improvements: a couple of ideas on how to expand this technique.


As I am sure the audience of this post knows to some extent, Docker is a technology that has grown to become popular over the last few years, allowing developers to deploy pieces of software by packaging them into standardized containers, in a number of various ecosystems (Apache Mesos, Amazon Web Services and many more).

So we can use Docker for our deployment needs, awesome. But let’s pay attention to a key word I used above. Docker grants isolation. And what do we like to perform on our application in isolation? Yeah, you guessed right – testing!

Specifically, with this post, I aim to dig deeper into how to use docker-compose (a specific Docker-based tool that enables creation of multi-container Docker applications) to build and run a Node.js application connected to MongoDB, to test their interaction and the interaction of the app with the external world, all inside containers running on your machine. All isolated and testable thanks to the usage of containers that we can spin up, hit with tests, and clean up with little effort.

Interested? Let’s go!


In this scenario we will use Docker and one of its functionalities, docker-compose, to build a container and spin up our app. Then we build another container with a copy of the database where we can freely create and manipulate data, and finally we perform all the integration testing we want against those self-contained entities, which we can clean up after the tests ran. Total isolation and, very importantly, no need to pollute our development or pre-production environment with superfluous test data.

Let’s imagine an app that we can build and test, for example a directory of soul music artists.

[Read More]


At OpenTable, we have an engineering culture that empowers us to research, experiment and learn.

In an effort to foster innovation and to try new ideas, Chris Cartlidge, Nick Balestra, Tom Martin and myself started to work on a side project nicknamed big-mars. Our project is a mobile-first, responsive web application that uses Falcor by Netflix.

Falcor, a JavaScript library for efficient data fetching, is an implementation of the Backend for Frontend (BFF) or the API Gateway pattern.

One powerful concept that Falcor has is its query model, in which you access your data as if it was a single JSON model in memory. You can navigate your data structure the way you would navigate a JSON object; either with a “dot notation” (e.g. restaurant.address.city) or with an “array notation” (e.g. restaurant[‘address’][‘city’]).

Really easy and intuitive.

However, while experimenting and learning how to properly use its query model, we noticed that it was quite hard to “visualise” the expected result from the API calls without using, for instance, our beloved in-browser Developer Tools console.


The Falcor project released many additional packages in order to facilitate developers who are willing to use this technology (e.g. falcor-express) but in this ecosystem we noticed a lack of tools with a GUI on top.

We also noticed that the GraphQL, another project that shares with Falcor the same core concepts, has tools with visual interfaces (e.g. express-graphql and GraphiQL).

So as a spin-off of our big-mars project, Nick and I decided to build a tool with a nice and intuitive GUI responsible for exercising the Falcor endpoint in order to help us to validate our queries, and we named this tool falcor-postman.

[Read More]

OpenComponents - microservices in the front-end world

Many engineers work every day on opentable.com from our offices located in Europe, America, and Asia, pushing changes to production multiple times a day. Usually, this is very hard to achieve, in fact it took years for us to get to this point. I described in a previous article how we dismantled our monolith in favour of a Microsites architecture. Since the publication of that blog post we have been working on something I believe to be quite unique, called OpenComponents.

Another front-end framework?

OpenComponents is a system to facilitate code sharing, reduce dependencies, and easily approach new features and experiments from the back-end to the front-end. To achieve this, it is based on the concept of using services as interfaces - enabling pages to render partial content that is located, executed and deployed independently.

OpenComponents is not another SPA JS framework; it is a set of conventions, patterns and tools to develop and quickly deploy fragments of front-end. In this perspective, it plays nicely with any existing architecture and framework in terms of front-end and back-end. Its purpose is to serve as delivery mechanism for a more modularised end-result in the front-end.

OC is been in production for more than a year at OpenTable and it is fully open-sourced.


OpenComponents involves two parts:

  • The consumers are web pages that need fragments of HTML for rendering partial contents. Sometimes they need some content during server-side rendering, somethings when executing code in the browser.
  • The components are small units of isomorphic code mainly consisting of HTML, Javascript and CSS. They can optionally contain some logic, allowing a server-side Node.js closure to compose a model that is used to render the view. When rendered they are pieces of HTML, ready to be injected in any web page.

The framework consists of three parts:

  • The cli allows developers to create, develop, test, and publish components.
  • The library is where the components are stored after the publishing. When components depend on static resources (such as images, CSS files, etc.) these are stored, during packaging and publishing, in a publicly-exposed part of the library that serves as a CDN.
  • The registry is a REST API that is used to consume components. It is the entity that handles the traffic between the library and the consumers.
[Read More]

Testing React Components

At OpenTable it’s becoming an increasingly popular trend to use React.
One of the reasons for this is the ability for it to server-side render whilst still
giving us the client side flexibility that we all crave!

We all know to have stable, reliable software you need to have well written tests. Facebook knows this and
provides the handy Test Utilities library to make
our lives easier.

Cool — I hear you all say! But what is the best approach to testing React components?

Well unfortunately this is something that is not very well documented and if not approached in
the correct way can lead to brittle tests.

Therefore I have written this blog post to discuss the different approaches we have available to us.

All code used in this post is avaliable on my GitHub.

The Basics

To make our lives a lot easier when writing test it’s best to use a couple of basic tools. Below is
the absolute minimum required to start testing React components.

  • Mocha - This is a testing framework that runs in the browser or Node.JS (others are available).
  • ReactTestUtils - This is the basic testing framework that Facebook provides to go testing with React.

The Scenario

[Read More]


Puppet is an important tool to us at OpenTable; we couldn’t operate as efficiently without it but Puppet is more than a tool or a vendor, it is a community of people trying to help
each other operate increasing complex and sophisticated infrastructures.

The Puppet community and the open source efforts that drive that community have always been important to us which is why we want to take a step further in our efforts and introduce
you to the “Puppet-community” project.

What is Puppet-community

Puppet-community is a GitHub organisation of like-minded individuals from across the wider Puppet ecosystem and from a diverse set of companies. Its principle aims are to allow the community to synchronise its efforts and to provide a GitHub organisation and Puppet Forge namespace not affiliated with any company.

Its wider aims are to provide a place for module and tool authors to share their code and the burden of maintaining it.

I would like to say that this was our idea, as it’s an excellent one, but actually all credit goes to its founders: Igor Galić, Daniele Sluijters and Spencer Krum

Why communities matter

So why all the fuss about this? Why does it even matter where your code lives?

Well these are the some questions that I asked myself when I first heard about this project at PuppetConf 2014. The answer is that is really does matter and it’s a pattern that is
developing elsewhere (see: packer-community, terraform-community-modules,
cloudfoundry-community) to deal with the problems you’ll face with a large amount of open source code.

Stepping back slightly, if you look at open source then there are three types: product-based (think open-core), corporate/individual sponsored, and community-driven.

[Read More]


Introduction to DNS

Before joining OpenTable I was looking for a software engineer job and I’ve done my fair share of interviews. A question that has popped out a lot, and when I say a lot I mean always, is:

Could you tell me what happens when I type an URL in a web browser on my computer and press enter?

Of course the possible answers could range from “MMMHHH, wellll, I’m not sure where to start…” to a whole book on computer networks.

After a number of attempts to answer briefly and correctly, I’ve concluded that mentioning DNS can make a reasonable start.

Let’s think about it. When we type the address of the resource we want to browse, we use the alphabet, right? With letters and names easily readable and retainable by a human being.

But a machine needs an IP address to recognize another machine connected to a network. An IP address is numerical, for example Less readable, it seems.

And here is where DNS comes to play. DNS stands for Domain Name System, and that represents exactly what it is: a system that translates domain names (e.g www.opentable.co.uk), into IP addresses. I think of it as a phone book. It is queried with a domain name and, after a lookup, returns an IP.

How does the magic happen? Let’s look into it.


[Read More]

Hapi.js and SIGTERM

When we first stood up our hapi.js APIs, we wrote init scripts to start/stop them. Stopping the server, was simply a case of sending SIGKILL (causing the app to immediately exit).

Whilst this is fine for most cases, if we want our apps to be good Linux citizens, then they should terminate gracefully. Hapi.js has the handy server.stop(...) command (see docs here) which will terminate the server gracefully. It will cause the server to respond to new connections with a 503 (server unavailable), and wait for existing connections to terminate (up to some specified timeout), before stopping the server and allowing the node.js process to exit. Perfect.

This makes our graceful shutdown code really simple:

process.on('SIGTERM', function(){
server.stop({ timeout: 5 * 1000}, function(){

When we see a SIGTERM, call server.stop(), then once the server has stopped, call process.exit(0). Easy peasy.

Throw a spanner in the works

Whilst server.stop() is really useful, it has the problem that it immediately prevents the server from responding to new requests. In our case, that isn’t particularly desirable. We use service-discovery, which means that the graceful termination of our app should run like this:

  • Unannounce from Service-Discovery
  • server.stop(...)
  • process.exit(0)

Ideally we want the unannounce to happen before the server starts rejecting connections, in order to reduce the likelihood that clients will hit a server that is shutting down.

Plugins to the rescue!

[Read More]

Dismantling the monolith - Microsites at Opentable

A couple of years ago we started to break-up the code-base behind our consumer site opentable.com, to smaller units of code, in order to improve our productivity. New teams were created with the goal of splitting up the logic that was powering the back-end and then bring to life new small services. Then, we started working on what we call Microsites.


A microsite is a very small set of web-pages, or even a single one, that takes care of handling a very specific part of the system’s domain logic. Examples are the Search Results page or the Restaurant’s Profile page. Every microsite is an independently deployable unit of code, so it is easier to test, to deploy, and in consequence more resilient. Microsites are then all connected by a front-door service that handles the routing.

Not a free ride

When we deployed some microsites to production we immediately discovered a lot of pros:

  • Bi-weekly deployments of the monolith became hundreds of deployments every week.
  • Not anymore a shared codebase for hundreds of engineers. Pull requests accepted, merged, and often deployed on the same day.
  • Teams experimenting and reiterating faster: product was happy.
  • Diversity on tech stacks: teams were finally able to pick their own favourite web-stack, as soon as they were capable of deploying their code and taking care of it in terms of reliability and performance.
  • Robustness: when something was wrong with a microsite, everything else was fine.

On the other hand, we soon realised that we introduced new problems on the system:

  • Duplication: teams started duplicating a lot of code, specifically front-end components such as the header, the footer, etc.
  • Coordination: when we needed to change something on the header, for example, we were expecting to see the change live in different time frames, resulting in inconsistencies.
  • Performance: every microsite was hosting its own duplicated css, javascript libraries, and static resources; resulting as a big disadvantage for the end-user in terms of performance.

SRS - aka Site Resources Service

To solve some of these problems we created a REST api to serve html snippets, that soon we started to call components. Main characteristics of the system are:

[Read More]

A Beginner's guide to REST services

Why this post?

As a junior, I always find it easier to just sit and write code than actually stop to think about the theoretical basis that lie under the applications I work on. REST is one of those terms I heard a lot about, so I decided to try to sum up what it means and how it affects the choices we make everyday as software engineers.

Introduction to REST

REST stands for Representational State Transfer, and it can be defined as an architectural style used to build Web Services that are lightweight, maintainable, and scalable. A service that is designed by REST principles can be called a RESTful service.

It has been described first in 2000 by Roy Fielding, in a dissertation called “Architectural Styles and the Design of Network-based Software Architectures”. The basic idea was to describe the interactions between the components of a distributed system, putting constraints on them and emphasizing the importance of an uniform interface, that is abstracted from the single components.

REST is often applied to the design and development of web services, which is the scenario I’ll try to address in this post.

The purpose of a web service can be summed up as follows: it exposes resources to a client so that it can have access to them (examples of typical resources include pictures, video files, web pages and business data).

Common features of a service that is built in a REST style are:

  • Representations
  • Messages
  • URIs
  • Uniform Interface
  • Statelessness
  • Links between resources
  • Caching

Representations - what are they?

[Read More]

On Strongly Typed Logging

Logging is a crucial element of monitoring highly available systems. It allows not only to find out about errors but also quickly identify their cause. Logs are often used to generate metrics that help business and engineering make informative decisions on future development directions.

At OpenTable we have a central logging infrastructure, that means all logs are stored in the same shared database (ElasticSearch for us). And everybody can access any logs they want without having very specialized knowledge (thanks Kibana!).

ElasticSearch, though living in a NoSQL world, is not actually a schema-free database. Sure, you do not need to provide schema to it but instead ES will infer schema for you from documents you send to it. This is very similar to type inference you can find in many programming languages. You do not need to specify type of field, but if you later on try to assign inappropriate value to it you will get an exception.

This trait of our database goes all the way to the root of our logging system design. Let me explain why I say that we have ‘strongly typed logs’.

In The Beginning There Was String

Before centralization we just logged a single message along with its importance. In code it looked something like:


which resulted in logline on disk having timestamp, severity and message.

{2014-10-10T07:33:04Z [ERROR] Kaboom!}

That worked pretty well. As time passed we often started making log messages more generic to hold relevant data:

logger.INFO(string.Format(“Received {0} from {1}. Status: {2}. Took {3}”, httpMethod, sourceIp, statusCode, durationms));

When we decided to centralize logs we moved the same logs from local disk to a central database. Suddenly things that used to live on single server in a file called ‘application.log’ become part of one huge lump of data. Instead of easing access to logs they were really hard to filter, without even speaking about aggregation, or any simple form of operations to find the source of the problem. ElasticSearch is really good at free text searching, but frankly speaking FTS is never as precise as a good filter.

Then There Was Dictionary Of Strings

[Read More]