I gave Ghost a good shot. And I do love it as a blogging platform, the markdown and writing experience is beautiful. But I’ve switched this site back to WordPress because I want it to be more than just a blog. I’ve been inspired by Derek Sivers’s site and want to make this a platform for many different videos, reviews, and projects I’m working on. If you notice anything broken please let me know.

I’ve also installed the Jetpack Markdown plugin because writing in markdown is so much nicer and cleaner than the default wordpress editing experience (or maybe I’ve just become used to it, coding and submitting PR’s on Github for the last 2 years).

I did like that Ghost was written in NodeJS, which I love far more than PHP, but in the end I did very little actual core hacking so that really didn’t matter.

You can use the following command to get the start time of a docker container in a unix timestamp (seconds since 1970).

docker inspect --format='{{.State.StartedAt}}' <CONTAINERID> | xargs date +%s -d

Just replace <CONTAINERID> with the actual container id.

Or you can use this to get the start time of all your containers:

docker ps -q | xargs docker inspect --format='{{.State.StartedAt}}' | xargs -n1 date +%s -d

At Cloud9 we use Datadog for all our server monitoring and Pagerduty for alerts when things break. To do this we use the standard Datadog + Pagerduty integration and make Pagerduty automatically trigger for critical incidents by adding @pagerduty into the “Say what’s happening” field in the Datadog monitor.

Unfortunately datadog triggers the monitor both when it starts and when it has recovered. Because we had @pagerduty in the “Say what’s happening” area this meant we got a pagerduty call both times.

You can fix this by wrapping the @pagerduty trigger with {{#is_alert}}{{/is_alert}}. So your monitor should look something like:

Docker is having trouble creating containers. Please investigate @slack-datadog @slack-warnings {{#is_alert}}@pagerduty{{/is_alert}}

You can also use {{#is_warning}}@pagerduty{{/is_warning}} for warnings (where the monitor has gone over the warning threshold but not the alert threshold).

Then you can go back to bed safe in the knowledge your server isn’t going to wake you up to tell you “Everything is good, nothing is broken”.

If you want to get the output of docker ps in JSON format the easiest way is to run the remote docker API and use that, you can read about how to do that here

However if you can’t run the remote API because of security issues or for other reasons there is another way! With curl version 7.40 (If this isn’t available on your distro click here) and newer you can get data from the local unix socket, and docker always runs the remote api on docker.sock.

Here’s what you have to run:

curl --unix-socket /var/run/docker.sock http://localhost/containers/json

Tada! Nice JSON Formatted docker ps output. I recommend using the jq library if you want an easy way to parse it in bash.

Today I had the fun task of taking Cloud9’s build bot and making it more secure. Primarily because it’s now exposed to the outside world and we don’t want random strangers having the ability to ship or revert our code.

Our bot responds to slash commands on Slack, so we can type /ship [appname] at any time in any channel in slack and the latest tested code will be pushed to production. It also recieves notifications from Jenkins when jobs have started, succeeded or failed.

Securing Slack

The first step was ensuring all Slack commands were actually coming from Slack. Whenever you create a new slash command Slack tells you it will send a specific token with all api calls, and you should use this to verify the call is from Slack.

Now there are multiple routes we wanted our bot to talk to and multiple slash commands to reach them, each with their own tokens. But we don’t want to add if (token == ‘xyz’) to every single route. Firstly because it’s messy, and secondly because then whenever a new developer joins the project they have to remember to do that or they’ll compromise security. So how do we do it? By creating a /slack route that verifies every token for us.

var express = require("express");
var config = require("config);

function verifyToken(req, res, next) {
if (!req.body.token || config.get("slack.tokens").indexOf(req.body.token) === -1) {
return next(new Error("Invalid slack token" + req.body.token));
}
next();
}

var slackRouter = new express.Router();
slackRouter.use(verifyToken);

slackRouter.post("/highfive", highFive.handleRequest.bind(highFive));
slackRouter.post("/ship", ship.startShipping.bind(ship));

app.use("/slack", slackRouter);

We have an array of possible slack tokens stored in our config file and whenever we add or remove commands we can simply add the token to that one list.

Now our routes are /slack/ship and /slack/highfive and whenever anyone sends data to them it will always validate that they have a valid slack token. No more manual verification in each route or having new developers forget to add security to their route, it’s all automatic.

Securing Jenkins

Our bot also listens to build hooks from Jenkins so that it can post to our Slack channel letting us know about the stats of various jobs.

We can secure Jenkins in the same way, but because it doesn’t pass any custom data in these job notifications we’ll secure it based on the requester IP address.

var requestIp = require("request-ip");

function verifyIPIsJenkins(req, res, next) {
var reqIp = requestIp.getClientIp(req);
if (!reqIp || config.get("jenkins.ips").indexOf(reqIp) === -1) {
return next(new Error("Jenkins push request came from " + reqIp + " which isn't a known address"));
}

return next();
};

var jenkinsRouter = new express.Router();
jenkinsRouter.use(verifyIPIsJenkins);

jenkinsRouter.post("/success", jenkins.buildSuccess.bind(jenkins));
jenkinsRouter.post("/failed", jenkins.buildFailed.bind(jenkins));

app.use("/jenkins", jenkinsRouter);

Now just like above we have 2 routes at /jenkins/success and /jenkins/failed and whenever anyone tries to access them it automatically verifies they are our CI server. If they are not the request will fail.

The reason I enjoy using these routes is they keep the code neat and also ensure that when someone comes to work on this project in the future they can easily add another route and won’t accidently allow hackers in the back door. Keeping things simple and automatic so any developer can pick up this code and run with it is my style of programming.

So you want to upgrade a library but don’t know what’s changed? Easy enough to do in git. Most projects tag their releases with something like v1.2.3. If you want to see all the commits between two of these releases simply do:

git log --no-merges v1.2.3 --not v1.2.2 

This shows all commits from v1.2.3 that aren’t in v1.2.2.

Or if you want a short summary grouped by author you can do:

git shortlog --no-merges v1.2.3 --not v1.2.2 

For the longest time I’d had the frustration of working on one branch, needing to create another branch for a bug fix, then creating a PR with this bug fix only to discover all the commits from the original branch are now in the PR.

Today I discovered you can actually specify the base branch in the git checkout command. Simply do this:

git checkout -b newbranch master

It will create and switch to the new branch basing it off master. I’ve created a new script called ‘gnb’ (for ‘git new branch’) to do this all the time for me now. You can see it here

Also if you’ve already submitted the PR and now want to remove those additional commits you can easily do this by rebasing your new branch off master.

git rebase --onto master originalbranch newbranch
git push origin +newbranch # Force push as we're re-writing history here

This will rebase all the commits you did in newbranch onto master skipping commits that are in originalbranch.

While testing some shell scripts I needed a simple way to tie up a port on the server to cause the shell scripts to error out.

I did some googling and couldn’t find any way to do this in pure bash but if you have NodeJS installed on your server it’s as simple as this:

node -e 'require("http").createServer(function(){}).listen(PORT);'

Simply replace PORT with the port number you wish to use. If anyone knows a way to do this in pure bash please let me know.