TheGremlin
New Member
- Messages
- 21
- Reaction score
- 0
- Points
- 1
SOLVING THE DREADED VARNISH 503 ERROR
“Error 503 Service Unavailable” / “Guru Meditation” is probably the most common error people get with Varnish. This post will go through a few causes of (and solutions to) this Varnish 503 error.
BEFORE WE START
One of the most confusing (but nonetheless very cool) things about Varnish is that it does not store any logs on disk. All log entries go directly into memory. This allows for better performance and efficient disk usage.
So to view the logs, you’ll have to fire the following command: varnishlog
Once the command is running, load up a page from your website on your browser and look for any errors. This literally means looking for the word “error”.
Pretty sweet, huh! Now let’s get on to our first solution.
SOLUTION 1 – GENERAL CHECK
Most of the times, Varnish 503 means Varnish cannot connect with your backend (Apache, Nginx, etc…). So you should start from there.
The first thing is to make sure your backend is correctly configured. That means making sure your Apache/Nginx/whatever is running properly before installing Varnish.
If you’ve installed Varnish on a properly running instance, then you might just check that installing Varnish hasn’t done anything funny to your Apache configurations. Here’s what you can do:
Open your Varnish configuration file. Mine was in /etc/varnish/user.vcl
backend default {
.host = "127.0.0.1";
.port = "8888";
.connect_timeout = 1s;
}
We can see that Varnish is trying to connect to port 8888 on the same machine. But is our backend responding correctly there?
Let’s try this (make sure you run this on your server, this port will probably be blocked to the outside world):
wget http://www.your-site.com:8888
This will create an HTML file with the output in your current directory. Open the file and see if it matches your website’s homepage. If it does not, then you need to configure your server. This is beyond the scope of this post, I’ll advise you to check the Linode Library for resources.
SOLUTION 2 – INCREASE TIMEOUTS
A frequent cause of Varnish 503 errors is timeouts between Varnish and your backend. This is quite easy to solve.
Open your Varnish configuration file (mine was in /etc/varnish/user.vcl) and add these lines:
backend default { .host = "127.0.0.1"; .port = "8888"; .connect_timeout = 1s; # Wait a maximum of 1s for backend connection (Apache, Nginx, etc...) .first_byte_timeout = 5s; # Wait a maximum of 5s for the first byte to come from your backend .between_bytes_timeout = 2s; # Wait a maximum of 2s between each bytes sent }
The settings are pretty straightforward. The most important settings here is thefirst_byte_timeout. So go ahead and play with those settings until you find the sweet spot for your server.
SOLUTION 3 – TURN KEEPALIVE OFF
If you’ve increased the timeouts and you’re still getting Varnish 503 errors, then the problem might be elsewhere.
Do this test; increase your timeouts to a considerable amount, like 10 seconds. Then, start browsing your website until you get a 503 error. If the error comes up before those 10 seconds, then the error is definitely elsewhere. I’ve seen cases where the error was coming up right after the click, as if the page was cached locally.
The solution to this issue is quite simple. It occurs often on Apache servers and it’s all due to the KeepAlive directive.
Open your Apache configuration file. Mine was in /etc/apache2/apache2.conf
#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive Off
By turning the KeepAlive directive off, we are forcing Apache to drop idle connections. Turning KeepAlive on makes sense when your Apache instance serves files directly to the end users. Basically the end user will open one connection and receive multiple files from it.
But with Varnish and Apache on the same server, I’d advise you to turn KeepAlive Off.
WRAPPING UP
So there you go with my suggestions. Go ahead, try them and let me know if you have any other solutions!
“Error 503 Service Unavailable” / “Guru Meditation” is probably the most common error people get with Varnish. This post will go through a few causes of (and solutions to) this Varnish 503 error.
BEFORE WE START
One of the most confusing (but nonetheless very cool) things about Varnish is that it does not store any logs on disk. All log entries go directly into memory. This allows for better performance and efficient disk usage.
So to view the logs, you’ll have to fire the following command: varnishlog
Once the command is running, load up a page from your website on your browser and look for any errors. This literally means looking for the word “error”.
Pretty sweet, huh! Now let’s get on to our first solution.
SOLUTION 1 – GENERAL CHECK
Most of the times, Varnish 503 means Varnish cannot connect with your backend (Apache, Nginx, etc…). So you should start from there.
The first thing is to make sure your backend is correctly configured. That means making sure your Apache/Nginx/whatever is running properly before installing Varnish.
If you’ve installed Varnish on a properly running instance, then you might just check that installing Varnish hasn’t done anything funny to your Apache configurations. Here’s what you can do:
Open your Varnish configuration file. Mine was in /etc/varnish/user.vcl
backend default {
.host = "127.0.0.1";
.port = "8888";
.connect_timeout = 1s;
}
We can see that Varnish is trying to connect to port 8888 on the same machine. But is our backend responding correctly there?
Let’s try this (make sure you run this on your server, this port will probably be blocked to the outside world):
wget http://www.your-site.com:8888
This will create an HTML file with the output in your current directory. Open the file and see if it matches your website’s homepage. If it does not, then you need to configure your server. This is beyond the scope of this post, I’ll advise you to check the Linode Library for resources.
SOLUTION 2 – INCREASE TIMEOUTS
A frequent cause of Varnish 503 errors is timeouts between Varnish and your backend. This is quite easy to solve.
Open your Varnish configuration file (mine was in /etc/varnish/user.vcl) and add these lines:
backend default { .host = "127.0.0.1"; .port = "8888"; .connect_timeout = 1s; # Wait a maximum of 1s for backend connection (Apache, Nginx, etc...) .first_byte_timeout = 5s; # Wait a maximum of 5s for the first byte to come from your backend .between_bytes_timeout = 2s; # Wait a maximum of 2s between each bytes sent }
The settings are pretty straightforward. The most important settings here is thefirst_byte_timeout. So go ahead and play with those settings until you find the sweet spot for your server.
SOLUTION 3 – TURN KEEPALIVE OFF
If you’ve increased the timeouts and you’re still getting Varnish 503 errors, then the problem might be elsewhere.
Do this test; increase your timeouts to a considerable amount, like 10 seconds. Then, start browsing your website until you get a 503 error. If the error comes up before those 10 seconds, then the error is definitely elsewhere. I’ve seen cases where the error was coming up right after the click, as if the page was cached locally.
The solution to this issue is quite simple. It occurs often on Apache servers and it’s all due to the KeepAlive directive.
Open your Apache configuration file. Mine was in /etc/apache2/apache2.conf
#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive Off
By turning the KeepAlive directive off, we are forcing Apache to drop idle connections. Turning KeepAlive on makes sense when your Apache instance serves files directly to the end users. Basically the end user will open one connection and receive multiple files from it.
But with Varnish and Apache on the same server, I’d advise you to turn KeepAlive Off.
WRAPPING UP
So there you go with my suggestions. Go ahead, try them and let me know if you have any other solutions!
