Andrew Welch
Published , updated · 5 min read · RSS Feed
Please consider 🎗 sponsoring me 🎗 to keep writing articles like this.
ServiceWorkers and Offline Browsing
ServiceWorkers provide a way to provide offline browsing and instant loading (amongst other things).
Once upon a time, there was a thing called the ApplicationCache that was supposed to allow for offline browsing and instant loading of resources by managing a local cache. It ended up being a good idea, but a lackluster implementation.
The idea is that we should provide some way to let the user browse our site even if they are offline or on a spotty Internet connection, and also to have a way to identify critical resources that should be cached locally for instant loading.
While it may seem somewhat fringe in our increasingly connected world to provide for offline browsing, the reality is that mobile devices account for the majority of traffic to many websites, a trend that’s only likely to grow in the future. A mobile device may even be a person’s only gateway to the Internet, and mobile devices are used in all sorts of less than ideal situations & locations.
And while web browsers will cache pages that you’ve already visited — for a limited period of time — they don’t cache pages you haven’t visited. The user might be using an interactive application, and we really don’t want to interrupt that experience with a 404 error just because they ventured to an area of spotty or no cellular coverage.
Imagine someone is using your online store, and they’ve added a bunch of items to their cart, and tries to check out. Do you want to lose that sale just because they have poor reception?
So what we need is a way to cache visited pages/resources, and also pre-cache pages/resources they haven’t visited, but we think they’ll be likely to. It’s even possible to do fun things like let them fill out a multi-page form offline, and sync that up with our server once they have an Internet connection again.
ServiceWorkers lets us do all of this and more. What’s presented here is just a small subset of what you can do with ServiceWorkers, and is based on the Filament Group’s Modernizing our Progressive Enhancement Delivery article (which in turn is based on several other works).
Link Sent into the Coal Mine
Patrick Harrington of MildlyGeeky.com is a friend & colleague of mine, but I swear, he treats me like a canary in a coal mine. He pointed out the Filament Group article to me (which I’d already perused), knowing full well that by doing so, it’d cause me to “go first” and try to get ServiceWorkers going.
He knows me well enough that he can just drop a hint of something interesting, causing me to dive in head-first — and if the canary lives, then he’ll venture down, too.
I’m happy to report that the canary lived!
Standing on the shoulders of giants (read: using their code), I was able to get ServiceWorkers up and working pretty quickly on this very website. It’s important to note that ServiceWorkers are fairly new, and currently are only supported by Chrome, FireFox, Opera, and Android browsers. However, Microsoft is working on support, and even Apple may be getting on board.
The nice thing about ServiceWorkers is that they degrade gracefully; if the client web browser doesn’t support them, no biggy. They just don’t reap the benefits, and your website continues to work normally.
The Lighthouse Chrome extension can be used to test your website to validate your offline browsing implementation, amongst other things.
Lighthouse testing tool results for nystudio107.com
Link So Exactly What is a ServiceWorker?
ServiceWorkers are written in JavaScript, and you register them with the browser when your webpage loads. They don’t run in the context of your webpage, but rather they can be thought of as a script that runs in another (hidden) browser tab, and act as a go-between for your webpage and the Internet.
This is pretty powerful stuff, as you can intercept or modify network requests, so https is required to utilize them, to mitigate against “man in the middle” attack vectors. They can be used for many other things in addition to offline browsing & instant loading, including push notifications, background sync, and so on.
The cool thing about ServiceWorkers (and what makes them far more flexible than the now-deprecated ApplicationCache) is that they don’t define a caching system, they just provide an API that can be used to create one. This makes them incredibly flexible and powerful.
When your webpage loads for the first time, all of the resources load normally, and the ServiceWorker is registered as being used for all of the requests that fall under the scope that you specify. Here’s what the code looks like to do that:
if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('/sw.js', {
scope: "/"
}).then(function(registration) {
// Registration was successful
console.log('ServiceWorker registration successful with scope: ', registration.scope);
// Trim the caches on load
navigator.serviceWorker.controller && navigator.serviceWorker.controller.postMessage({
command: "trimCaches"
});
}).catch(function(err) {
// registration failed :(
console.log('ServiceWorker registration failed: ', err);
});
});
}In the example above, sw.js is our ServiceWorker script. If the browser supports ServiceWorkers, after the webpage has loaded, we attempt to register it with the browser. If it succeeds we send a message to the console, and then post a message to our service worker to send it the trimCaches command.
In this example, we don’t need to explicitly declare the scope our service worker should handle requests from, because by default it uses the same scope as the location of our script, which is at the server root. This means that it can handle all requests from the website.
However if our script was located in, say, /js/sw.js that would give it an implicit scope of /js/ so it would only handle requests from the /js/ directory (and everything below it). So I decided to explicitly declare it, for clarity’s sake.
Now that the service worker has been registered, any further requests to our domain that fall under the scope we specified will be handled by it. Our webpage asks for, say, site.combined.min.css via a <link> tag, and what happens is our ServiceWorker gets a chance to field the request, and maybe do something different with it.
In this context, think of a ServiceWorker as traffic cop that sits between your webpage, and the Internet, deciding what comes and goes.
Using the Filament Group code (which is based on Lyza Danger Gardner & Jeremy Keith’s code) as a basis, we specify the pages we want to be pre-cached:
const offlinePages = [
'/blog/index',
'/offline',
'/'
];And we specify the static resources we want pre-cached:
const staticAssets = [
'{{ baseUrl }}js/lazysizes.min.js',
'{{ baseUrl }}js/prism.min.js',
'{{ baseUrl }}js/system.min.js',
'{{ baseUrl }}js/vue.min.js',
'{{ baseUrl }}js/vue-resource.min.js',
'{{ baseUrl }}css/site.combined.min.css',
'{{ baseUrl }}fonts/esfera-webfont.woff2',
'{{ baseUrl }}fonts/brandon-regular-webfont.woff2',
'{{ baseUrl }}fonts/brandon-bold-webfont.woff2',
'{{ baseUrl }}fonts/brandon-regular-italic-webfont.woff2',
'{{ baseUrl }}fonts/fontello.woff2',
'{{ baseUrl }}fonts/OperatorMonoSSm-Book.woff2',
'{{ baseUrl }}fonts/OperatorMonoSSm-BookItalic.woff2',
'/api/blog.json',
];Note that the implementation used here is just one of many possible implementations; your ServiceWorker listens to events, and can implement a caching system (or something else, for that matter) however it wants.
The offline page just specifies a fallback page to use if they are offline, but they haven’t visited any pages, so nothing is in the cache.
These declarations cause our ServiceWorker to request all of the offlinePages & staticAssets when our page loads, even if it didn’t request them explicitly. These requests happen asynchronously, after our page is loaded, so they don’t impact the page load.
This ensures that all of the content we we want available when our site is offline will be cached locally. This also ensures that anything we think people will want to load soon will be instantly loaded from our cache. Here’s what it looks like:
ServiceWorker page load waterfall
Everything inside the orange box was loaded by our ServiceWorker, after our page was fully loaded. Subsequent requests go through the ServiceWorker, so if the person goes offline, all of these pages and resources will be available. And pages we think they are likely to visit will also be instantly loaded from the cache, whether they are online or offline.
In addition to ensuring that the pages & assets we specify are cached, our ServiceWorker also ensures that any pages the user visits are cached, too. We can take our site offline with the Chrome developer tools to simulate what happens:
Notice that the assets are (from ServiceWorker) and the Time is very quick. We can now browse any of the pages we’ve already visited — or that we explicitly cached — even when offline, rather than getting the dreaded dinosaur:
Offlineosaur
We can even specify an “offline” image to swap in (instead of an ugly broken image link) for the case where a page is cached, but the image on it is not (perhaps it was lazily loaded). As webpages become more and more webapp-ish, it’s natural that we’d want to be able to emulate the offline usability of native apps, too.
Link Route Map Plugin
I’ve created a plugin for Craft CMS called Route Map that makes working with ServiceWorkers even easier.
Let’s say for this website we want to pre-cache the latest 9 blog posts, so that if someone loads any page on the website, and then goes offline or loses their cellular connection, they can still read our articles.
Using the Route Map plugin, we can just do this:
{% set blogEntries = craft.routeMap.getSectionUrls('blog', {'limit': 9}) %}
(function(){
'use strict';
// Constants for the cache size
const maxPagesCache = 30;
const maxStaticAssetsCache = 100;
// borrowing heavily from adactio's sw patterns here... Thanks JK!
const version = 'nys1.1::';
const staticCacheName = version + 'static';
const pagesCacheName = version + 'pages';
const imagesCacheName = version + 'images';
const offlinePages = [
'/blog/index',
{% for blogEntry in blogEntries %}
'{{ blogEntry }}',
{% endfor %}
'/offline',
'/'
];The {% set blogEntries = craft.routeMap.getSectionUrls('blog', {'limit': 9}) %} fetches the URLs to the last 9 blog entries in a performant, cached manner, and then we add them to our offline pages via:
{% for blogEntry in blogEntries %}
'{{ blogEntry }}',
{% endfor %}Well, so what? You could do that already just using craft.entries.section('blog').limit(9) right? Well, we can also use Route Map to extract the Assets uses on those pages, whether in Assets fields or embedded in Matrix/Neo blocks.
That way we could also pre-cache any images needed for those pages with craft.routeMap.getUrlAssetUrls, too:
const staticAssets = [
{% for blogEntry in blogEntries %}
{%- set assetUrls = craft.routeMap.getUrlAssetUrls(blogEntry) -%}
{%- for assetUrl in assetUrls -%}
'{{ assetUrl }}',
{% endfor -%}
{% endfor %}
'{{ baseUrl }}js/lazysizes.min{{staticAssetsVersion}}.js',
'{{ baseUrl }}js/prism.min{{staticAssetsVersion}}.js',
'{{ baseUrl }}js/vue.min{{staticAssetsVersion}}.js',
'{{ baseUrl }}js/vue-resource.min{{staticAssetsVersion}}.js',
'{{ baseUrl }}css/site.combined.min{{staticAssetsVersion}}.css',
'{{ baseUrl }}fonts/esfera-webfont.woff2',
'{{ baseUrl }}fonts/brandon-regular-webfont.woff2',
'{{ baseUrl }}fonts/brandon-bold-webfont.woff2',
'{{ baseUrl }}fonts/brandon-regular-italic-webfont.woff2',
'{{ baseUrl }}fonts/fontello.woff2',
'{{ baseUrl }}fonts/OperatorMonoSSm-Book.woff2',
'{{ baseUrl }}fonts/OperatorMonoSSm-BookItalic.woff2',
'/api/blog.json',
];This make for some powerful offline browsing capabilities.
This example uses Twig to obtain the Entry URLs via the Route Map plugin, but you could just as easily grab them via XHR.
Link Wrapping Up
This is some seriously cool stuff. And it barely scratches the surface of what ServiceWorkers can do. I can’t wait to see what you do with them.
Want to know more about ServiceWorkers? The Service workers explained and Beyond Offline: Other Interesting Use Cases with Service Workers articles are good places to start.
A really good place to get started with ServiceWorkers is also Workbox from Google. It’s sort of a ServiceWorker construction set, with cool things like caching, pre-caching, background sync, and offline analytics. Pick the features you want, build, and go!
Tangent: Astute readers will notice that we’re using Twig syntax in our sw.js script. I did this because I use a Content Delivery Network (CDN) for production, but not for development, and I wanted to make the ServiceWorker work in all environments. So I set up a route in Nginx:
# Pass our ServiceWorker through Craft so it can work as a template
location ^~ /sw.js {
try_files $uri $uri/ /index.php?$query_string;
}This causes requests for /sw.js to go through Craft, and placed the sw.js script in the templates directory, so Craft will parse it as Twig.