in Platform Updates

Speeding Up LiveChat API

Grzegorz Wyszyński in Programming, on March 24, 2014

I’ve recently stumbled upon a [post by Jason Fried]( where he discusses the efforts Basecamp took to speed up their API. When I logged in to my Basecamp account and started poking around, it felt very responsive and much quicker than before.

Having some issues of our own with LiveChat API speed, I was pretty impressed. I decided to dig a bit deeper.

Keeping the SSL connection up

After looking under the hood of Basecamp, mainly at the way their connections are handled, I noticed that all requests are very fast – around 300 ms from Europe and 180 ms from the US. Actually, the only request that took a bit longer was the initial SSL handshake, which took 400ms. Since it requires two roundtrips, the time of a the normal request needs to be doubled.

When sending request in our API, we used to add the SSL information with each request, which significantly increased their duration. The average request took around 1,300 ms in Europe and 400 ms in United States since our data centers are located in Dallas, TX.

I hit up our administrator to ask if we could use keep-alive on the SSL part of our request to make them last for the duration of the session instead of sending the same information with each request. After a bit of testing and setting up [HAProxy]( on a localhost, it turned out it should work for us.

Reducing request times

By default, HAProxy uses `keep-alive` to sustain connections between client and backend. To get the addresses of our clients, instead of the HAProxy address, we needed to add two things to our HAProxy config:
  • mode http - we had to use this option to be able to use http-specific rules
  • forwardfor - this option adds a forwarder to the header

    You only needed to add the x-forwarded-for option to the first request that goes through the keep-alive socket as HAProxy leaves all the following keep-alive traffic untouched.

LiveChat API speed - Load balancer

To make the IP addresses known across all requests, you have two options:
  1. Use option httpclose to turn off keep-alive in HAProxy - When you use this option, HAProxy will add connection:close header to all requests and force the backend and client to disconnect. It also makes it possible to use option forwardfor. This is the worst option as every client request requires a new DNS Lookup, new SSL Handshake and a new socket, making the whole request much longer than necessary.
  2. Keep alive connections between client and balancer. Close connections between balancer and backend - This is the fastest way to make keep-alive work. It’s more resource-demanding as it creates and closes sockets to backend more often. There is only a small delay because balancer needs to perform a DNS Lookup and create a new socket when connecting to your backend. But if we assume that it is in a local network, the delays should be very low. It also doesn’t require any development in your backend.

    We went with the second option, which is very easy to set up. We just had to add the http-server-close option to our HAProxy config defaults.

    Here is full working example:

	mode http
	timeout connect 5000ms
	timeout client 50000ms
	timeout server 50000ms

	option forwardfor
	option http-server-close

frontend http-farm
	bind localhost:8088
	default_backend backend1

backend backend1
	balance roundrobin
	server b1 localhost:8000 check inter 1000
	server b2 localhost:8001 check inter 1000
We prepared a quick HAProxy config change and gave it a go on our test server. The results were pretty neat:

LiveChat API speed - Keep-alive improvement

We’ve managed to cut our request times by over 1,100 ms for each request, which is a significant boost in performance. After the initial SSL handshake, it takes only around 150 ms in the US and 250 ms in Europe to finish a typical request like getting the list of agents.

See other Platform Updates

Installable Web Applications

The majority of today’s apps is web applications (i.e.

Read more

How to Fix Common UI Mistakes With a Few Lines of Code

“The devil is in the details,” people tend to say.

Read more
See all updates