• Controlling Varnish ESI inside your application

    05-07-2010Author: basdenooijer

    For me one of the best features of Varnish is ESI (Edge Side Includes).  It allows you to combine elements with different lifetimes into a single page. This way you don’t need to regenerate a complete page as soon as a news listing somewhere on the page changes. And you can still cache pages while displaying user-specific information somewhere on the page.
    This lowers the load on your application even further and it can also help in simplifying your application, by focussing purely on the content of the page and loading all other elements via ESI.

    There is a small drawback though, parsing for ESI comes at a cost. Although Varnish performs well even with ESI enabled, the effect is noticable. And binary files should be excluded from ESI parsing all together. The usual solution is to enable ESI only for specific requests in your VCL file. A typical example:

    sub vcl_fetch {
        if (req.url == "/news") {
            esi;  /* Do ESI processing */
        }
    }
    

    With this solution you might end up adding all URLs that need to be parsed into your VCL file, complicating application development. Or you can decide to parse all text/html content, so you don’t need to frequently alter your VCL. Both solutions work, but are not optimal. The main issue is that your application should decide when to apply ESI to the content, not Varnish. The knowledge to base this decision on is in your application domain. When developing a page in your application you know whether you want to use ESI, and you should be able to do so easily, right inside the application.

    There is a solution for this issue, and it’s actually quite easy! Since Varnish has full access to the backend (application) response, we can send an ESI ‘marker’ in the form of a special header. In your VCL file you can enable ESI based on this header, and then remove the marker header:

    sub vcl_fetch {
        if (beresp.http.esi-enabled == "1") {
            esi;
            unset beresp.http.esi-enabled;
        }
    }
    

    You can implement this in your application by simply sending a header. A basic example in PHP:

    header('esi-enabled: 1');
    

    When using the MVC model you could let the view object handle this, for even better abstraction. An example for PHP, using the Zend Framework where this can be implemented with a view helper:

    <?php
    
    /**
     * Copyright (c) 2010, Bas de Nooijer
     * All rights reserved.
     *
     * Redistribution and use in source and binary forms, with or without
     * modification, are permitted provided that the following conditions are met:
     *
     *   * Redistributions of source code must retain the above copyright notice,
     *     this list of conditions and the following disclaimer.
     *
     *   * Redistributions in binary form must reproduce the above copyright
     *     notice, this list of conditions and the following disclaimer in the
     *     documentation and/or other materials provided with the distribution.
     *
     *   * Neither the name of Raspberry nor the names of its contributors may be
     *     used to endorse or promote products derived from this software without
     *     specific prior written permission.
    
     * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS &quot;AS IS&quot;
     * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
     * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
     * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
     * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
     * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
     * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
     * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
     * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
     * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
     * POSSIBILITY OF SUCH DAMAGE.
     *
     * @author      Bas de Nooijer
     * @copyright   2010, raspberry.nl
     * @license     http://www.opensource.org/licenses/bsd-license.php
     */
    
    /** Zend_Controller_Front */
    require_once 'Zend/Controller/Front.php';
    
    /** Zend_View_Helper_Abstract.php */
    require_once 'Zend/View/Helper/Abstract.php';
    
    /**
     * Helper for Varnish Edge Side Includes
     */
    class Raspberry_View_Helper_Esi extends Zend_View_Helper_Abstract
    {
    
    	/**
         * Default ESI header name
         *
         * @var string
         */
        protected static $_varnishHeaderName = 'esi-enabled';
    
        /**
         * Default ESI header value
         *
         * @var string
         */
        protected static $_varnishHeaderValue = '1';
    
        /**
         * Has the Varnish header been sent?
         *
         * @var boolean
         */
        protected static $_varnishHeaderSent = false;
    
        /**
         * Sets the ESI header settings (to match your Varnish VCL)
         *
         * @param string $name
         * @param string $value
         */
        public static function setHeader($name, $value)
        {
            self::$_varnishHeaderName = $name;
            self::$_varnishHeaderValue = $value;
        }
    
        /**
         * Create an ESI tag for a given SRC.
         *
         * @param  string $src
         * @return string
         */
        public function esi($src)
        {
        	// If the ESI headers have not been sent yet do it now
        	if(!self::$_varnishHeaderSent)
        	{
                $response = Zend_Controller_Front::getInstance()->getResponse();
                $response->setHeader(self::$_varnishHeaderName, self::$_varnishHeaderValue);
         	    self::$_varnishHeaderSent = true;
        	}
    
            return '<esi:include src="' . $src . '"/>';
        }
    
    }
    

    After adding this view helper to your view helper path you can use it like this in your view scripts:

    ... content ....
    <?php
    echo $this->esi('/user/status');
    ?>
    ... content ....
    

    The view helper will automatically set the ESI header for Varnish as soon as you use it, and as an added bonus it generates the ESI tag for you.

    , , ,
  • 3 comments on “Controlling Varnish ESI inside your application