{"id":3097,"date":"2013-06-06T11:59:20","date_gmt":"2013-06-06T11:59:20","guid":{"rendered":"http:\/\/blog.shineservers.com\/?p=239"},"modified":"2013-06-06T11:59:20","modified_gmt":"2013-06-06T11:59:20","slug":"how-to-optimize-heavy-traffic-wordpress-blog","status":"publish","type":"post","link":"https:\/\/www.shineservers.com\/2013\/06\/06\/how-to-optimize-heavy-traffic-wordpress-blog\/","title":{"rendered":"How To Optimize Heavy Traffic WordPress Blog"},"content":{"rendered":"<p>I\u2019m running\u00a0<a title=\"Varnish Community | Varnish makes websites fly!\" href=\"http:\/\/varnish-cache.org\/\">Varnish<\/a>\u00a0as a front-end to\u00a0<a title=\"nginx news\" href=\"http:\/\/nginx.net\/\">Nginx<\/a>\u00a0which is running<a title=\"WordPress \u203a Blog Tool and Publishing Platform\" href=\"http:\/\/wordpress.org\/\">WordPress<\/a>\u00a0loaded with the\u00a0<a title=\"WordPress \u203a W3 Total Cache \u00ab WordPress Plugins\" href=\"http:\/\/wordpress.org\/extend\/plugins\/w3-total-cache\/\">W3-Total-Cache<\/a>\u00a0plugin. The W3-Total-Cache plugin is configured to use both\u00a0<code><a title=\"memcached - a distributed memory object caching system\" href=\"http:\/\/memcached.org\/\">memcached<\/a><\/code>\u00a0as well as\u00a0<a title=\"Amazon Simple Storage Service (Amazon S3)\" href=\"http:\/\/aws.amazon.com\/s3\/\">Amazon S3<\/a>\u00a0as its CDN. All of this sits on Ubuntu Linux .<\/p>\n<p><span style=\"font-size: 1.5em;\">Nginx<\/span><\/p>\n<p>The first thing I did was dump\u00a0<a title=\"Welcome to The Apache Software Foundation!\" href=\"http:\/\/www.apache.org\/\">Apache<\/a>. I love Apache, don\u2019t get me wrong, but I prefer to go with simple and fast if I have the option, and that\u2019s what Nginx offers.<\/p>\n<p>Again, I\u2019m using Ubuntu, so the installations here are pretty clean.<\/p>\n<div id=\"commandentry\">\n<p># install\u00a0<code>nginx<\/code><\/p>\n<p>aptitude install\u00a0nginx<\/p>\n<\/div>\n<p>You\u2019re also going to need to install\u00a0<code>php-fpm<\/code>\u00a0to cache PHP within Nginx.<\/p>\n<div id=\"commandentry\">\n<p># install\u00a0<code>php-fpm<\/code><\/p>\n<p>aptitude install\u00a0php-fpm<\/p>\n<\/div>\n<p>Configuration is handled by\u00a0<code>\/etc\/nginx\/nginx.conf<\/code>, where you\u2019ll want to just do a few things:<\/p>\n<p>&nbsp;<\/p>\n<pre># Miscellaneous Options\n   sendfile        on;\n   tcp_nopush     off;\n   keepalive_timeout  30;\n   tcp_nodelay        on;\n   multi_accept     on;\n   gzip  on;\n   gzip_proxied any;\n   gzip_comp_level 2;\n   gzip_disable \"MSIE [1-6].(?!.*SV1)\";\ngzip_types text\/plain text\/css application\/x-javascript text\/xml\n application\/xml application\/xml+rss text\/javascript;<\/pre>\n<p>&nbsp;<\/p>\n<p>And then here\u2019s the site config, under<code>\/etc\/nginx\/sites-enabled\/default<\/code>:<\/p>\n<p>[ A couple of options have been omitted, but they should be self-explanatory, such as server name and the port you&#8217;re listening on. ]<\/p>\n<p>&nbsp;<\/p>\n<pre>## Default location\n    location \/ {\n        root   \/var\/www\/;\n        index  index index.php;\n        try_files $uri\/ $uri \/index.php?q=$uri&amp;$args;\n        port_in_redirect off;\n    }\n    location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|xml)$ {\n      access_log        off;\n      expires           30d;\n      root \/var\/www\/;\n    }\n    location ~ .php$ {\n        fastcgi_split_path_info ^(.+.php)(.*)$;\n        fastcgi_pass   backend;\n        fastcgi_index  index.php;\n        fastcgi_param  SCRIPT_FILENAME  \/var\/www\/$fastcgi_script_name;\n        include fastcgi_params;\n        fastcgi_param  QUERY_STRING     $query_string;\n        fastcgi_param  REQUEST_METHOD   $request_method;\n        fastcgi_param  CONTENT_TYPE     $content_type;\n        fastcgi_param  CONTENT_LENGTH   $content_length;\n        fastcgi_intercept_errors        on;\n        fastcgi_ignore_client_abort     off;\n        fastcgi_connect_timeout 60;\n        fastcgi_send_timeout 180;\n        fastcgi_read_timeout 180;\n        fastcgi_buffer_size 128k;\n        fastcgi_buffers 4 256k;\n        fastcgi_busy_buffers_size 256k;\n        fastcgi_temp_file_write_size 256k;\n    }\n    location ~ \/.ht {\n        deny  all;\n    }\n    location ~ \/.git {\n        deny  all;\n    }\n        location ~ \/.svn {\n            deny  all;\n    }\nupstream backend {\n            server 127.0.0.1:9000;\n    }<\/pre>\n<p>&nbsp;<\/p>\n<p>[ I&#8217;ve collected all these options from multiple sources&#8211;mostly from the official docs when possible&#8211;and have tweaked them through experimentation. Hopefully this will save you lots of time getting up and running with a decent config. ]<\/p>\n<p><span style=\"font-size: 1.5em;\">Varnish<\/span><\/p>\n<p>So, Varnish is a wicked fast reverse proxy for serving up content. The idea is that if someone just requested something from your backend (Nginx), and it hasn\u2019t been long, Varnish can serve it much faster than a full web server like Apache or even Nginx.<\/p>\n<p>Varnish is a simple install as well when using Ubuntu.<\/p>\n<div id=\"commandentry\">\n<p># install\u00a0<code>varnish<\/code><\/p>\n<p>aptitude install\u00a0varnish<\/p>\n<\/div>\n<p>&nbsp;<\/p>\n<pre> backend default {\n     .host = \"localhost\";\n     .port = \"8080\";\n}\nacl purge {\n        \"localhost\";\n}\nsub vcl_recv {\n        if (req.request == \"PURGE\") {\n                if (!client.ip ~ purge) {\n                        error 405 \"Not allowed.\";\n                }\n                return(lookup);\n        }\nif (req.url ~ \"^\/$\") {\n               unset req.http.cookie;\n            }\n}\nsub vcl_hit {\n        if (req.request == \"PURGE\") {\n                set obj.ttl = 0s;\n                error 200 \"Purged.\";\n        }\n}\nsub vcl_miss {\n        if (req.request == \"PURGE\") {\n                error 404 \"Not in cache.\";\n        }\nif (!(req.url ~ \"wp-(login|admin)\")) {\n                        unset req.http.cookie;\n                }\n    if (req.url ~ \"^\/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|\nzip|lzma|bz2|tgz|tbz|html|htm)(\\?.<em>|)$\") {\n       unset req.http.cookie;\n       set req.url = regsub(req.url, \"\\?.<\/em>$\", \"\");\n    }\n    if (req.url ~ \"^\/$\") {\n       unset req.http.cookie;\n    }\n}\nsub vcl_fetch {\n        if (req.url ~ \"^\/$\") {\n                unset beresp.http.set-cookie;\n        }\nif (!(req.url ~ \"wp-(login|admin)\")) {\n                        unset beresp.http.set-cookie;\n}\n}<\/pre>\n<p>&nbsp;<\/p>\n<p>So a key thing to realize about Varnish is that you need to get rid of cookies to see the benefit from it. If Varnish sees cookies flying back and forth, it\u2019s going to assume there\u2019s some sensitive functionality at play, and it\u2019s not going to interfere. So part of this is saying that if you don\u2019t see wp-login or wp-admin in the URL, strip the cookies.<\/p>\n<p>The key to the main config here at the top is that Varnish sits on port 80 as your public presentation of your website to the world, and your \u201creal\u201d web server (Nginx in our case) sits behind it on another port, e.g. 8080.<\/p>\n<p>Ok, so if we spin both Nginx and Varnish up at this point we\u2019ll have a website, and it\u2019ll be decently fast. What we\u2019ve done so far is:<\/p>\n<ul>\n<li>In our Nginx config we applied gzip compression, enabled keepalive, and allowed it to handle multiple requests at once (plus a few other settings)<\/li>\n<li>In Varnish, we\u2019ve stripped cookies from most WordPress requests (non-admin\/login), as well as made it so that when we create or update a WordPress post it\u2019ll refresh the Varnish cache appropriately so we don\u2019t have a stale site while waiting for expiration. We\u2019ve also removed the cookies from the front page within Varnish.<\/li>\n<\/ul>\n<p>Now on to W3-Total-Cache.<\/p>\n<p><span style=\"font-size: 1.5em;\">W3-Total-Cache<\/span><\/p>\n<p>In the mode of clean, tight configs with as little clutter as possible, I try to run as few WordPress plugins as possible. One of them is W3-Total-Cache. It\u2019s simply phenomenal at speeding up either Apache or Nginx.<\/p>\n<p>It gets its speed gains by combining a few techniques: Browser Caching, APC for PHP caching (reducing database lookups), and the use of a CDN. I use all three of those.<\/p>\n<p>First we\u2019ll install\u00a0<code>memcached<\/code>\u00a0on Ubuntu:<\/p>\n<div id=\"commandentry\">\n<p># install\u00a0<code>apc<\/code>\u00a0in Ubuntu<\/p>\n<p>aptitude install\u00a0apc<\/p>\n<\/div>\n<p>Then install W3-Total-Cache within WordPress and perform the following steps:<\/p>\n<ol>\n<li>Enable page caching (memcached)<\/li>\n<li>Enable the object cache (memcached)<\/li>\n<li>Enable the browser cache<\/li>\n<li>Do NOT enable the database cache (it\u2019ll slow everything down)<\/li>\n<li>If your kung-fu is strong, enable the CDN functionality (you should be serving as much of your site from a CDN anyway\u2013regardless of this plugin. I use Amazon S3)<\/li>\n<\/ol>\n<p>[ I am actually not using W3 Total Cache now, as I&#8217;ve transfered its functionality to my nginx config and removed the plugin. Remember the principle of simplicity. ]<\/p>\n<p><span style=\"font-size: 1.5em;\">Benchmarks<\/span><\/p>\n<p>Ok, so we now have Varnish acting as the front-end cache to Nginx, with tons of optimization happening at all layers. So the question is: How much did we improve things?<\/p>\n<p>I use a myriad of tools to test web performance, but the two I\u2019ll discuss here are\u00a0<a title=\"ab - Apache HTTP server benchmarking tool - Apache HTTP Server\" href=\"http:\/\/httpd.apache.org\/docs\/2.0\/programs\/ab.html\">Apache Bench<\/a>\u00a0and\u00a0<a title=\"Which loads faster?\" href=\"http:\/\/whichloadsfaster.com\/\">Which Loads Faster<\/a>.<\/p>\n<p>[ Don&#8217;t forget to restart everything before continuing. Here&#8217;s a script I use ]<\/p>\n<div id=\"commandentry\">\n<p># bounce all the web-related services<\/p>\n<p>alias\u00a0whup=\u201dservice mysql restart; \/etc\/init.d\/nginx restart; \/etc\/init.d\/php5-fpm restart<\/p>\n<\/div>\n<h3>Apache Bench\u00a0<code>(ab)<\/code><\/h3>\n<p><code>ab<\/code>\u00a0is a tool for testing the performance of web servers. It sends mad requests to sites in the form of many concurrent connections. You can install\u00a0<code>ab<\/code>\u00a0on Ubuntu really easily by installing the Apache Utilities:<\/p>\n<div id=\"commandentry\">\n<p>aptitude\u00a0install apache-utils<\/p>\n<\/div>\n<p>You can then run\u00a0<code>ab<\/code>\u00a0against your site like so:<\/p>\n<div id=\"commandentry\">\n<p>ab\u00a0-kc 10 -n 1000\u00a0http:\/\/test.shineservers.in\/<\/p>\n<\/div>\n<p><code>ab<\/code>\u00a0allows you to define a port as well, which gives you the option of testing Varnish + Nginx vs. Nginx directly.<\/p>\n<div id=\"commandentry\">\n<p>ab\u00a0-kc 10 -n 1000\u00a0http:\/\/test.shineservers.in:8080\/<\/p>\n<\/div>\n<p>The second example here is a test of Nginx without Varnish. Keep in mind that you\u2019ll have to lower your firewall to be able to connect in to a port other than 80. You\u00a0<em>do<\/em>\u00a0run\u00a0<code>iptables<\/code>\u00a0on your web server, right?<\/p>\n<p>Anyway, here is what I get when hitting a default Apache-based WordPress install (still on Ubuntu and Linode, though) using 10 concurrent connections for 1000 hits (with keepalive):<\/p>\n<p>&nbsp;<\/p>\n<pre>Server Software:        Apache\/2.2.16\nServer Hostname:        somesite.com\nServer Port:            80\nDocument Path:          \/?p=5\nDocument Length:        8577 bytes\nConcurrency Level:      10\nTime taken for tests:   57.812 seconds\nComplete requests:      1000\nFailed requests:        0\nWrite errors:           0\nKeep-Alive requests:    0\nTotal transferred:      8833000 bytes\nHTML transferred:       8577000 bytes\nRequests per second:    17.30 [#\/sec] (mean)\nTime per request:       578.119 [ms] (mean)\nTime per request:       57.812 [ms] (mean, across all concurrent requests)\nTransfer rate:          149.21 [Kbytes\/sec] received\nConnection Times (ms)\n              min  mean[+\/-sd] median   max\nConnect:       59   60   0.9     60      88\nProcessing:   297  517  91.1    521    1055\nWaiting:      236  456  90.9    461     995\nTotal:        357  576  91.2    581    1114\nPercentage of the requests served within a certain time (ms)\n  50%    581\n  66%    616\n  75%    637\n  80%    651\n  90%    689\n  95%    722\n  98%    753\n  99%    777\n 100%   1114 (longest request)<\/pre>\n<p>&nbsp;<\/p>\n<p>This isn\u2019t horrible, with 95% of requests coming in around 755ms, but that\u2019s a default site with virtually no content in it, e.g. no images, plugins, ads, etc.<\/p>\n<p>Now compare that with my pure Nginx performance (without Varnish)\u00a0<em>with<\/em>all those things slowing me down:<\/p>\n<p>&nbsp;<\/p>\n<pre>Server Software:        nginx\/0.7.65\nServer Hostname:        test.shineservers.in\nServer Port:            81\nDocument Path:          \/\nDocument Length:        0 bytes\nConcurrency Level:      10\nTime taken for tests:   17.816 seconds\nComplete requests:      1000\nFailed requests:        0\nWrite errors:           0\nNon-2xx responses:      1000\nKeep-Alive requests:    0\nTotal transferred:      279000 bytes\nHTML transferred:       0 bytes\nRequests per second:    56.13 [#\/sec] (mean)\nTime per request:       178.162 [ms] (mean)\nTime per request:       17.816 [ms] (mean, across all concurrent requests)\nTransfer rate:          15.29 [Kbytes\/sec] received\nConnection Times (ms)\n              min  mean[+\/-sd] median   max\nConnect:       59   60   0.5     60      75\nProcessing:    97  109  12.3    104     172\nWaiting:       97  109  12.3    104     172\nTotal:        157  168  12.3    163     232\nPercentage of the requests served within a certain time (ms)\n  50%    163\n  66%    166\n  75%    170\n  80%    173\n  90%    184\n  95%    196\n  98%    210\n  99%    218\n 100%    232 (longest request)<\/pre>\n<p>&nbsp;<\/p>\n<p>That\u2019s stupid faster.<\/p>\n<p>Now let\u2019s try with Varnish added in (hitting port 80 instead of 8080):<\/p>\n<p>&nbsp;<\/p>\n<pre>Server Software:        nginx\/0.7.65\nServer Hostname:        test.shineservers.info\nServer Port:            80\nDocument Path:          \/\nDocument Length:        39435 bytes\nConcurrency Level:      10\nTime taken for tests:   7.530 seconds\nComplete requests:      1000\nFailed requests:        992\n   (Connect: 0, Receive: 0, Length: 992, Exceptions: 0)\nWrite errors:           0\nKeep-Alive requests:    1000\nTotal transferred:      39806014 bytes\nHTML transferred:       39434008 bytes\nRequests per second:    132.80 [#\/sec] (mean)\nTime per request:       75.301 [ms] (mean)\nTime per request:       7.530 [ms] (mean, across all concurrent requests)\nTransfer rate:          5162.33 [Kbytes\/sec] received\nConnection Times (ms)\n              min  mean[+\/-sd] median   max\nConnect:        0    1   5.9      0      60\nProcessing:    60   74  23.2     61     332\nWaiting:       60   70  12.7     61     151\nTotal:         60   75  28.0     61     391\nPercentage of the requests served within a certain time (ms)\n  50%     61\n  66%     80\n  75%     88\n  80%     89\n  90%     89\n  95%     90\n  98%    120\n  99%    298\n 100%    391 (longest request)<\/pre>\n<p>&nbsp;<\/p>\n<p>Duh-am. That\u2019s 95% of the requests finishing in 90ms or less! Happiness.<\/p>\n<h3>Which Site Loads Faster<\/h3>\n<p>Just for giggles I like to use a site called\u00a0<a title=\"Which loads faster?\" href=\"http:\/\/whichloadsfaster.com\/\">whichloadsfaster.com<\/a>, which graphically and side-by-side tests two sites against each other in terms of load speed. Here\u2019s what I get if I compare my Varnish+Nginx against pure Nginx over 100 pulls (notice the port number in the fields at the top of the screenshot):<\/p>\n<p>So, almost a 2.5x improvement in speed using Varnish + Nginx vs. just Nginx for loading my WordPress front page. And for hitting my non-database PHP content I am at around 25% faster (tests of\u00a0<code>\/study<\/code>\u00a0not shown for brevity).<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I\u2019m running\u00a0Varnish\u00a0as a front-end to\u00a0Nginx\u00a0which is runningWordPress\u00a0loaded with the\u00a0W3-Total-Cache\u00a0plugin. The W3-Total-Cache plugin is configured to use both\u00a0memcached\u00a0as well as\u00a0Amazon S3\u00a0as its CDN. All of this sits on Ubuntu Linux . Nginx The first thing I did was dump\u00a0Apache. I love Apache, don\u2019t get me wrong, but I prefer to go with simple and fast if [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[60],"tags":[126],"class_list":["post-3097","post","type-post","status-publish","format-standard","hentry","category-linux","tag-how-to-optimize-heavy-traffic-wordpress-blog"],"acf":[],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.shineservers.com\/wp-json\/wp\/v2\/posts\/3097","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shineservers.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shineservers.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shineservers.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shineservers.com\/wp-json\/wp\/v2\/comments?post=3097"}],"version-history":[{"count":0,"href":"https:\/\/www.shineservers.com\/wp-json\/wp\/v2\/posts\/3097\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.shineservers.com\/wp-json\/wp\/v2\/media?parent=3097"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shineservers.com\/wp-json\/wp\/v2\/categories?post=3097"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shineservers.com\/wp-json\/wp\/v2\/tags?post=3097"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}