Using Express 4 routes to secure your web app

Today I had the fun task of taking Cloud9’s build bot and making it more secure. Primarily because it’s now exposed to the outside world and we don’t want random strangers having the ability to ship or revert our code.

Our bot responds to slash commands on Slack, so we can type /ship [appname] at any time in any channel in slack and the latest tested code will be pushed to production. It also recieves notifications from Jenkins when jobs have started, succeeded or failed.

Securing Slack

The first step was ensuring all Slack commands were actually coming from Slack. Whenever you create a new slash command Slack tells you it will send a specific token with all api calls, and you should use this to verify the call is from Slack.

Now there are multiple routes we wanted our bot to talk to and multiple slash commands to reach them, each with their own tokens. But we don’t want to add if (token == ‘xyz’) to every single route. Firstly because it’s messy, and secondly because then whenever a new developer joins the project they have to remember to do that or they’ll compromise security. So how do we do it? By creating a /slack route that verifies every token for us.

var express = require("express");
var config = require("config);

function verifyToken(req, res, next) {
if (!req.body.token || config.get("slack.tokens").indexOf(req.body.token) === -1) {
return next(new Error("Invalid slack token" + req.body.token));

var slackRouter = new express.Router();
slackRouter.use(verifyToken);"/highfive", highFive.handleRequest.bind(highFive));"/ship", ship.startShipping.bind(ship));

app.use("/slack", slackRouter);

We have an array of possible slack tokens stored in our config file and whenever we add or remove commands we can simply add the token to that one list.

Now our routes are /slack/ship and /slack/highfive and whenever anyone sends data to them it will always validate that they have a valid slack token. No more manual verification in each route or having new developers forget to add security to their route, it’s all automatic.

Securing Jenkins

Our bot also listens to build hooks from Jenkins so that it can post to our Slack channel letting us know about the stats of various jobs.

We can secure Jenkins in the same way, but because it doesn’t pass any custom data in these job notifications we’ll secure it based on the requester IP address.

var requestIp = require("request-ip");

function verifyIPIsJenkins(req, res, next) {
var reqIp = requestIp.getClientIp(req);
if (!reqIp || config.get("jenkins.ips").indexOf(reqIp) === -1) {
return next(new Error("Jenkins push request came from " + reqIp + " which isn't a known address"));

return next();

var jenkinsRouter = new express.Router();
jenkinsRouter.use(verifyIPIsJenkins);"/success", jenkins.buildSuccess.bind(jenkins));"/failed", jenkins.buildFailed.bind(jenkins));

app.use("/jenkins", jenkinsRouter);

Now just like above we have 2 routes at /jenkins/success and /jenkins/failed and whenever anyone tries to access them it automatically verifies they are our CI server. If they are not the request will fail.

The reason I enjoy using these routes is they keep the code neat and also ensure that when someone comes to work on this project in the future they can easily add another route and won’t accidently allow hackers in the back door. Keeping things simple and automatic so any developer can pick up this code and run with it is my style of programming.


Above the fold is stupid

A short post for those in Internet Marketing who still believe the “Put everything above the fold because people don’t usually scroll down” myth. Unfortunately whoever originally said this didn’t quite think the data through and it has been repeated ad-nausium by Internet Marketers ever since.

Here’s why you shouldn’t worry about putting your stuff above the fold and it may even be counterproductive to jam it all up there:

  1. Most people aren’t interested in what you’re selling. Especially if your traffic isn’t pre-sold or coming from relevant locations. Not all traffic is equal and I really wish people would stop a/b testing like it is.
  2. 60% of people don’t scroll down because they don’t like what you have to sell, not because they don’t know how to scroll.
  3. Putting your signup form above the fold is counterproductive. Just because people don’t scroll dosn’t mean they’re going to be like “Oh wow a signup form, I better add my details”.
  4. If you had better content or marketing in place of that signup form you’ll probably (test this!) get better results as you hook people in and get them to register later.

The core thing you should be putting above the fold is content that sucks your visitors in, something that within 5 seconds makes them think “Hey this is a problem I have and they may have the solution”.

Here are things that should not be above the fold:

  1. Videos that start slowly (boring)
  2. Signup forms (seriously the only person who would sign up without reading is your Mum, or heavily presold visitors, in which case it doesn’t matter where your signup form is).
  3. Pictures that mean nothing to fresh visitors (your logo, stock photography, banners etc).
  4. Links (If you want your visitors to do something don’t overload them with options).

Now go build a landing page people actually enjoy and remember: test everything. If you think I’m wrong run a test then come back here and tell me how wrong I am. Being told I’m wrong is fantastic, because it helps me constantly grow and be a better market and human being.


Git show all commits between two versions

So you want to upgrade a library but don’t know what’s changed? Easy enough to do in git. Most projects tag their releases with something like v1.2.3. If you want to see all the commits between two of these releases simply do:

git log --no-merges v1.2.3 --not v1.2.2 

This shows all commits from v1.2.3 that aren’t in v1.2.2.

Or if you want a short summary grouped by author you can do:

git shortlog --no-merges v1.2.3 --not v1.2.2 

Git create new branches basing them off master

For the longest time I’d had the frustration of working on one branch, needing to create another branch for a bug fix, then creating a PR with this bug fix only to discover all the commits from the original branch are now in the PR.

Today I discovered you can actually specify the base branch in the git checkout command. Simply do this:

git checkout -b newbranch master

It will create and switch to the new branch basing it off master. I’ve created a new script called ‘gnb’ (for ‘git new branch’) to do this all the time for me now. You can see it here

Also if you’ve already submitted the PR and now want to remove those additional commits you can easily do this by rebasing your new branch off master.

git rebase --onto master originalbranch newbranch
git push origin +newbranch # Force push as we're re-writing history here

This will rebase all the commits you did in newbranch onto master skipping commits that are in originalbranch.


Simple script to use / run a fake process on a port

While testing some shell scripts I needed a simple way to tie up a port on the server to cause the shell scripts to error out.

I did some googling and couldn’t find any way to do this in pure bash but if you have NodeJS installed on your server it’s as simple as this:

node -e 'require("http").createServer(function(){}).listen(PORT);'

Simply replace PORT with the port number you wish to use. If anyone knows a way to do this in pure bash please let me know.


HAProxy – How to run http and https on the same port

Want to have your app run on one just the one port but work in both http and https mode? It’s easily done. You’ll first have to have a normal frontend for ports 80 and 443 similar to the following:

frontend unsecured *:80
timeout client 1d
maxconn 20000

default_backend default

frontend secured
maxconn 20000
bind ssl crt /etc/haproxy/proxycert.cert

default_backend default

You probably already have this setup if you’re running HAProxy, no need to change it if you do.

Now to make another port (9000 in this example) work with both http and https just do the following:

frontend newport
maxconn 20000


mode tcp
option tcplog

tcp-request inspect-delay 100ms
tcp-request content accept if HTTP
tcp-request content accept if { req.ssl_hello_type 1 }

use_backend forward_http if HTTP
default_backend forward_https

backend forward_http
mode tcp
server serverhttp

backend forward_https
mode tcp
server serverhttps

It simply takes 100ms (this could be lowered but I didn’t want things to accidentally break) to detect what mode the connection is in. If it’s in HTTP it forwards the request to itself on port 80 and if not it forwards to itself on port 443.


How to uninstall / delete OSX mail app

Unfortunately Apple love forcing their software on users and providing no way to uninstall. If you’re sick of apple mail popping up every time you click an email link or pressing cmd + shift + i (as I often do trying to get into the web dev tools in chrome) do the following in your terminal to remove it:

sudo -i 
mkdir /dump
mv /Applications/ /dump
mv /usr/bin/mail /dump
chmod 000 /dump

You could also rm -rf the files but I like to keep them around just in case OSX breaks somehow. If you don’t run the chmod command OSX will actually detect the file has moved and ask you to set it up again.


Docker / lxc set memory limit invalid argument

When trying to resize a docker or LXC container by changing the value in /sys/fs/cgroup/memory/docker//memory.limit_in_bytes if you come across the error

write error: Invalid argument

When going above a certain amount (for me it was 1024MB).

It’s most likely due to your memory limit being higher than your memory + swap limit. You need to edit the file memory.memsw.limit_in_bytes and increase it’s value.


How to test memory limits of a docker container

I’ve been playing around with docker containers recently including setting memory limits. But Google had wasn’t much help. So posting a simple method I discovered here. You do need php installed in your container.

truncate -s 1G /tmp/1G
php -r 'file("/tmp/1G");

All it does is create a 1 gigabyte file in the /tmp directory then attempts to read it in using PHP. PHP being PHP tries to load it all at once and if your memory limits are working correctly should give a Fatal error.


Teamcity stop double building the same project

It is annoying when Teamcity builds the same project twice at the same time. Especially when some of the steps are to decomission old servers and deploy new ones and the two builds end up trampling all over each other.

The option to disable this is tricky to find. First go to your project build configuration. Then click on general settings on the left hand side. Then click the ‘show advanced options’ just above the save button in the main column. When you click this it now has an option to “Limit the number of simultaneously running builds (0 — unlimited)”. Enter 1 in this box. Hit save and you’ll no longer have 2 builds of the same project at the same time.

This is with teamcity version 8.1.3. It may be in a different spot in future versions.