Andrew Welch · Insights · #frontend #devops #offline

Published , updated · 5 min read ·


For more tools, technologies, and techniques, check out the devMode.fm podcast!

ServiceWorkers and Offline Browsing

Ser­vice­Work­ers pro­vide a way to pro­vide offline brows­ing and instant load­ing (amongst oth­er things).

Service Dog

Once upon a time, there was a thing called the Appli­ca­tion­Cache that was sup­posed to allow for offline brows­ing and instant load­ing of resources by man­ag­ing a local cache. It end­ed up being a good idea, but a lack­lus­ter implementation.

The idea is that we should provide some way to let the user browse our site even if they are offline or on a spotty Internet connection, and also to have a way to identify critical resources that should be cached locally for instant loading.

While it may seem some­what fringe in our increas­ing­ly con­nect­ed world to pro­vide for offline brows­ing, the real­i­ty is that mobile devices account for the major­i­ty of traf­fic to many web­sites, a trend that’s only like­ly to grow in the future. A mobile device may even be a person’s only gate­way to the Inter­net, and mobile devices are used in all sorts of less than ide­al sit­u­a­tions & locations.

And while web browsers will cache pages that you’ve already vis­it­ed — for a lim­it­ed peri­od of time — they don’t cache pages you haven’t vis­it­ed. The user might be using an inter­ac­tive appli­ca­tion, and we real­ly don’t want to inter­rupt that expe­ri­ence with a 404 error just because they ven­tured to an area of spot­ty or no cel­lu­lar coverage.

Imagine someone is using your online store, and they’ve added a bunch of items to their cart, and tries to check out. Do you want to lose that sale just because they have poor reception?

So what we need is a way to cache vis­it­ed pages/​resources, and also pre-cache pages/​resources they haven’t vis­it­ed, but we think they’ll be like­ly to. It’s even pos­si­ble to do fun things like let them fill out a mul­ti-page form offline, and sync that up with our serv­er once they have an Inter­net con­nec­tion again.

Ser­vice­Work­ers lets us do all of this and more. What’s pre­sent­ed here is just a small sub­set of what you can do with Ser­vice­Work­ers, and is based on the Fil­a­ment Group’s Mod­ern­iz­ing our Pro­gres­sive Enhance­ment Deliv­ery arti­cle (which in turn is based on sev­er­al oth­er works).

Link Sent into the Coal Mine

Patrick Har­ring­ton of Mild​lyGeeky​.com is a friend & col­league of mine, but I swear, he treats me like a canary in a coal mine. He point­ed out the Fil­a­ment Group arti­cle to me (which I’d already perused), know­ing full well that by doing so, it’d cause me to go first” and try to get Ser­vice­Work­ers going.

Canary Coalmine

He knows me well enough that he can just drop a hint of some­thing inter­est­ing, caus­ing me to dive in head-first — and if the canary lives, then he’ll ven­ture down, too.

I’m happy to report that the canary lived!

Stand­ing on the shoul­ders of giants (read: using their code), I was able to get Ser­vice­Work­ers up and work­ing pret­ty quick­ly on this very web­site. It’s impor­tant to note that Ser­vice­Work­ers are fair­ly new, and cur­rent­ly are only sup­port­ed by Chrome, Fire­Fox, Opera, and Android browsers. How­ev­er, Microsoft is work­ing on sup­port, and even Apple may be get­ting on board.

The nice thing about Ser­vice­Work­ers is that they degrade grace­ful­ly; if the client web brows­er doesn’t sup­port them, no big­gy. They just don’t reap the ben­e­fits, and your web­site con­tin­ues to work normally.

The Light­house Chrome exten­sion can be used to test your web­site to val­i­date your offline brows­ing imple­men­ta­tion, amongst oth­er things.

Lighthouse Report

Light­house test­ing tool results for nys​tu​dio107​.com

Link So Exactly What is a ServiceWorker?

Ser­vice­Work­ers are writ­ten in JavaScript, and you reg­is­ter them with the brows­er when your web­page loads. They don’t run in the con­text of your web­page, but rather they can be thought of as a script that runs in anoth­er (hid­den) brows­er tab, and act as a go-between for your web­page and the Internet.

This is pret­ty pow­er­ful stuff, as you can inter­cept or mod­i­fy net­work requests, so https is required to uti­lize them, to mit­i­gate against man in the mid­dle” attack vec­tors. They can be used for many oth­er things in addi­tion to offline brows­ing & instant load­ing, includ­ing push noti­fi­ca­tions, back­ground sync, and so on.

The cool thing about Ser­vice­Work­ers (and what makes them far more flex­i­ble than the now-dep­re­cat­ed Appli­ca­tion­Cache) is that they don’t define a caching sys­tem, they just pro­vide an API that can be used to cre­ate one. This makes them incred­i­bly flex­i­ble and powerful.

When your web­page loads for the first time, all of the resources load nor­mal­ly, and the Ser­vice­Work­er is reg­is­tered as being used for all of the requests that fall under the scope that you spec­i­fy. Here’s what the code looks like to do that:

if ('serviceWorker' in navigator) {
    window.addEventListener('load', function() {
        navigator.serviceWorker.register('/sw.js', {
            scope: "/"
        }).then(function(registration) {
            // Registration was successful
            console.log('ServiceWorker registration successful with scope: ', registration.scope);
            // Trim the caches on load
            navigator.serviceWorker.controller && navigator.serviceWorker.controller.postMessage({
                command: "trimCaches"
            });
       }).catch(function(err) {
            // registration failed :(
            console.log('ServiceWorker registration failed: ', err);
        });
    });
}

In the exam­ple above, sw.js is our Ser­vice­Work­er script. If the brows­er sup­ports Ser­vice­Work­ers, after the web­page has loaded, we attempt to reg­is­ter it with the brows­er. If it suc­ceeds we send a mes­sage to the con­sole, and then post a mes­sage to our ser­vice work­er to send it the trimCaches command.

In this exam­ple, we don’t need to explic­it­ly declare the scope our ser­vice work­er should han­dle requests from, because by default it uses the same scope as the loca­tion of our script, which is at the serv­er root. This means that it can han­dle all requests from the website.

How­ev­er if our script was locat­ed in, say, /js/sw.js that would give it an implic­it scope of /js/ so it would only han­dle requests from the /js/ direc­to­ry (and every­thing below it). So I decid­ed to explic­it­ly declare it, for clarity’s sake.

Now that the ser­vice work­er has been reg­is­tered, any fur­ther requests to our domain that fall under the scope we spec­i­fied will be han­dled by it. Our web­page asks for, say, site.combined.min.css via a <link> tag, and what hap­pens is our Ser­vice­Work­er gets a chance to field the request, and maybe do some­thing dif­fer­ent with it.

In this context, think of a ServiceWorker as traffic cop that sits between your webpage, and the Internet, deciding what comes and goes.

Using the Fil­a­ment Group code (which is based on Lyza Dan­ger Gard­ner & Jere­my Keith’s code) as a basis, we spec­i­fy the pages we want to be pre-cached:

const offlinePages = [
  '/blog/index',
  '/offline',
  '/'
];

And we spec­i­fy the sta­t­ic resources we want pre-cached:

const staticAssets = [
  '{{ baseUrl }}js/lazysizes.min.js',
  '{{ baseUrl }}js/prism.min.js',
  '{{ baseUrl }}js/system.min.js',
  '{{ baseUrl }}js/vue.min.js',
  '{{ baseUrl }}js/vue-resource.min.js',

  '{{ baseUrl }}css/site.combined.min.css',

  '{{ baseUrl }}fonts/esfera-webfont.woff2',
  '{{ baseUrl }}fonts/brandon-regular-webfont.woff2',
  '{{ baseUrl }}fonts/brandon-bold-webfont.woff2',
  '{{ baseUrl }}fonts/brandon-regular-italic-webfont.woff2',
  '{{ baseUrl }}fonts/fontello.woff2',
  '{{ baseUrl }}fonts/OperatorMonoSSm-Book.woff2',
  '{{ baseUrl }}fonts/OperatorMonoSSm-BookItalic.woff2',

  '/api/blog.json',
];

Note that the imple­men­ta­tion used here is just one of many pos­si­ble imple­men­ta­tions; your Ser­vice­Work­er lis­tens to events, and can imple­ment a caching sys­tem (or some­thing else, for that mat­ter) how­ev­er it wants.

The offline page just spec­i­fies a fall­back page to use if they are offline, but they haven’t vis­it­ed any pages, so noth­ing is in the cache.

These dec­la­ra­tions cause our Ser­vice­Work­er to request all of the offlinePages & staticAssets when our page loads, even if it didn’t request them explic­it­ly. These requests hap­pen asyn­chro­nous­ly, after our page is loaded, so they don’t impact the page load.

This ensures that all of the con­tent we we want avail­able when our site is offline will be cached local­ly. This also ensures that any­thing we think peo­ple will want to load soon will be instant­ly loaded from our cache. Here’s what it looks like:

Service Worker Page Waterfall

Ser­vice­Work­er page load waterfall

Every­thing inside the orange box was loaded by our Ser­vice­Work­er, after our page was ful­ly loaded. Sub­se­quent requests go through the Ser­vice­Work­er, so if the per­son goes offline, all of these pages and resources will be avail­able. And pages we think they are like­ly to vis­it will also be instant­ly loaded from the cache, whether they are online or offline.

In addi­tion to ensur­ing that the pages & assets we spec­i­fy are cached, our Ser­vice­Work­er also ensures that any pages the user vis­its are cached, too. We can take our site offline with the Chrome devel­op­er tools to sim­u­late what happens:

Service Worker Offline

Notice that the assets are (from Ser­vice­Work­er) and the Time is very quick. We can now browse any of the pages we’ve already vis­it­ed — or that we explic­it­ly cached — even when offline, rather than get­ting the dread­ed dinosaur:

No Internet Connection

Offli­neosaur

We can even spec­i­fy an offline” image to swap in (instead of an ugly bro­ken image link) for the case where a page is cached, but the image on it is not (per­haps it was lazi­ly loaded). As web­pages become more and more webapp-ish, it’s nat­ur­al that we’d want to be able to emu­late the offline usabil­i­ty of native apps, too.

Link Route Map Plugin

I’ve cre­at­ed a plu­g­in for Craft CMS called Route Map that makes work­ing with Ser­vice­Work­ers even easier.

Let’s say for this web­site we want to pre-cache the lat­est 9 blog posts, so that if some­one loads any page on the web­site, and then goes offline or los­es their cel­lu­lar con­nec­tion, they can still read our articles.

Using the Route Map plu­g­in, we can just do this:

{% set blogEntries = craft.routeMap.getSectionUrls('blog', {'limit': 9}) %}

(function(){
'use strict';

// Constants for the cache size
const maxPagesCache = 30;
const maxStaticAssetsCache = 100;

// borrowing heavily from adactio's sw patterns here... Thanks JK!

const version = 'nys1.1::';
const staticCacheName = version + 'static';
const pagesCacheName = version + 'pages';
const imagesCacheName = version + 'images';
const offlinePages = [
  '/blog/index',
{% for blogEntry in blogEntries %}
    '{{ blogEntry }}',
{% endfor %}
  '/offline',
  '/'
];

The {% set blogEntries = craft.routeMap.getSectionUrls('blog', {'limit': 9}) %} fetch­es the URLs to the last 9 blog entries in a per­for­mant, cached man­ner, and then we add them to our offline pages via:

{% for blogEntry in blogEntries %}
    '{{ blogEntry }}',
{% endfor %}

Well, so what? You could do that already just using craft.entries.section('blog').limit(9) right? Well, we can also use Route Map to extract the Assets uses on those pages, whether in Assets fields or embed­ded in Matrix/​Neo blocks.

That way we could also pre-cache any images need­ed for those pages with craft.routeMap.getUrlAssetUrls, too:

const staticAssets = [
{% for blogEntry in blogEntries %}
        {%- set assetUrls = craft.routeMap.getUrlAssetUrls(blogEntry) -%}
        {%- for assetUrl in assetUrls -%}
            '{{ assetUrl }}',
        {% endfor -%}
{% endfor %}
  '{{ baseUrl }}js/lazysizes.min{{staticAssetsVersion}}.js',
  '{{ baseUrl }}js/prism.min{{staticAssetsVersion}}.js',
  '{{ baseUrl }}js/vue.min{{staticAssetsVersion}}.js',
  '{{ baseUrl }}js/vue-resource.min{{staticAssetsVersion}}.js',

  '{{ baseUrl }}css/site.combined.min{{staticAssetsVersion}}.css',

  '{{ baseUrl }}fonts/esfera-webfont.woff2',
  '{{ baseUrl }}fonts/brandon-regular-webfont.woff2',
  '{{ baseUrl }}fonts/brandon-bold-webfont.woff2',
  '{{ baseUrl }}fonts/brandon-regular-italic-webfont.woff2',
  '{{ baseUrl }}fonts/fontello.woff2',
  '{{ baseUrl }}fonts/OperatorMonoSSm-Book.woff2',
  '{{ baseUrl }}fonts/OperatorMonoSSm-BookItalic.woff2',

  '/api/blog.json',
];

This make for some pow­er­ful offline brows­ing capabilities.

This exam­ple uses Twig to obtain the Entry URLs via the Route Map plu­g­in, but you could just as eas­i­ly grab them via XHR.

Link Wrapping Up

This is some seri­ous­ly cool stuff. And it bare­ly scratch­es the sur­face of what Ser­vice­Work­ers can do. I can’t wait to see what you do with them.

Want to know more about Ser­vice­Work­ers? The Ser­vice work­ers explained and Beyond Offline: Oth­er Inter­est­ing Use Cas­es with Ser­vice Work­ers arti­cles are good places to start.

A real­ly good place to get start­ed with Ser­vice­Work­ers is also Work­box from Google. It’s sort of a Ser­vice­Work­er con­struc­tion set, with cool things like caching, pre-caching, back­ground sync, and offline ana­lyt­ics. Pick the fea­tures you want, build, and go!

Tan­gent: Astute read­ers will notice that we’re using Twig syn­tax in our sw.js script. I did this because I use a Con­tent Deliv­ery Net­work (CDN) for pro­duc­tion, but not for devel­op­ment, and I want­ed to make the Ser­vice­Work­er work in all envi­ron­ments. So I set up a route in Nginx:

# Pass our ServiceWorker through Craft so it can work as a template
    location ^~ /sw.js {
        try_files $uri $uri/ /index.php?$query_string;
    }

This caus­es requests for /sw.js to go through Craft, and placed the sw.js script in the templates direc­to­ry, so Craft will parse it as Twig.