The DNS ABC
Before joining OpenTable I was looking for a software engineer job and I've done my fair share of interviews. A question that has popped out a lot, and when I say a lot I mean always, is:
On Strongly Typed Logging
Logging is a crucial element of monitoring highly available systems. It allows not only to find out about errors but also quickly identify their cause. Logs are often used to generate metrics that help business and engineering make informative decisions on future development directions.
Interacting with ElasticSearch using Hubot
At OpenTable, we use a few ElasticSearch clusters. Our aim was to be able to interact with our ElasticSearch clusters via HipChat so that we could troubleshoot easily and without having to log into our VPN. We already use Hubot as part of our systems workflow, so it made sense to be able to interact with ElasticSearch with it.
Testing Puppet with Beaker pt.3 - Testing Roles
In the first two parts of this blog series we have focusing on testing puppet modules with beaker. As an open source contributor there is always a large test matrix so this makes absolute sense. But what about the other large use-case for beaker - what about our day-to-day internal code base? Not all of this is modules, in fact a large portion of it is other puppet code - roles, profiles, facts, hiera data etc. All of this needs testing as well.
Testing Puppet with Beaker pt.2 - The Windows story
In part one we discussed our first steps into the world of acceptance testing our Puppet manifests. By using Beaker we able to test managing local users on our Linux boxes. This was a positive experience for us. It allowed us to get to grips with the basics of configuring Beaker to run tests and configuring our node sets to run those tests against. In this post, we will be discussing how we went about getting Beaker working with Windows.
Continuous Delivery: Automating Deployment Visibility
In our continued effort to drive towards a service oriented architecture each of our teams are continuously improving their deployment processes. Recently our team has focussed on automating as much as possible, putting as much into chat as we can and improving our logging/metrics.
Managing Windows Features with Puppet
Back in June 2013, I wrote about Windows Feature Management with PowerShell. We have since released a Puppet module that will do this for us. We originally wrote PowerShell:
Managing Windows Web Applications with Puppet
As part of our move towards a configuration management tool, we really wanted to start automating as much of our infrastructure as possible. This included our application configuration stack. IIS management is pretty easy with PowerShell. It would look something like this
Upgrading Puppet with Puppet
As part of one of our recent ForgeFriday efforts we released a new module puppetversion with the purpose of managing the installation and upgrade of Puppet in a platform agnostic way.
Testing Puppet with Beaker
One afternoon I got asked to write a new Puppet module to manage local users on our Linux boxes. Not a contrived example but a real-world need as we begin to move our infrastructure from Windows to Linux. Managing users is one of those tasks that is at the core of the Puppet ecosystem and I thought this would be pretty easy as I had done this sort of thing many times before. What added to the complexity was that we needed to support Ubuntu, Centos and FreeBSD machines that we had in our stack and we wanted to make it something that was open source and on the Forge - so lots of testing was required.
ForgeFriday - our commitment to the Puppet Forge
Those of you that read this blog on a regular basis will be aware that here at OpenTable we take our commitment to Open Source very seriously. As part of the infrastructure team at OpenTable we are begining a process of being open-first. That means that everything we write will be open-sourced unless it contains a significant amount of internal information but we will architect to limit this.
Benchmarking APIs - why it’s important, and how
Since I joined OpenTable I’ve been experimenting with performance monitoring, specifically on web services. One of the projects my team is responsible for is a REST API that provides UI elements for HTML5 applications, shaped as HTML snippets and static resources. Our consumers are websites deployed in multiple parts of the world, so our service needs to be fast and reliable.
The adoption of Configuration Management
In years gone by, we were a traditional IT company. We had teams of developers and operations. They rarely mixed. Around nine months ago we started to really try and get these teams working together. We introduced a configuration management tool, Puppet, into our ecosystem.
Quick look at RethinkDB
Someone in the office mentioned RethinkDb and I was impressed by the rhetoric on the site, so I decided to spend a couple of hours spiking one of our existing nodejs apps with RethinkDb. The app currently uses MongoDb so inevitably I'm comparing the two.
Using Vagrant to work with ElasticSearch on your local machine
Recently, I have started to work a lot more with Vagrant as a tool for creating a standard development environment across my team. This essentially means that regardless what the developers' machine is set up or running as, they can still reproduce the same environment as their colleagues just by entering a command.
Managing Windows Certificates with PowerShell
Managing certificates on Windows is really painful. There is no easy way to do it. The general way to install a certificate to a Windows Server 2008 machine is as follows:
Windows Feature Management with PowerShell
In late 2012, our development team started to move towards our systems being much more automated. Long gone are the days of developers creating runbooks in Word and giving them to our operations team to use to set up our production servers.