Offline first with progressive web apps: caching strategies [part 2/3]

The internet is constantly evolving. We saw this in 2004 when ajax became really popular and made websites more dynamic. The next revolution was the mobile-first approach when we stopped thinking that the user always had a 24-inch screen but instead the website should be accessible on mobile as well. And now we see a new revolution happening called progressive web apps (PWA). And this is where our journey continues.

This is part 2 of a 3 part blog series. In the previous blog post we started our journey by explaining what a PWA is and mentioned there advantages. We as well looked how service workers fit into PWA. After this we created a service worker and changed the default look when the user has no network to a custom based one. This part builds on top of what we did in part 1.

The target audience for this post are developers (or something like that) or in the second paragraph, we could explain that this part also is trying to help developers to get acquainted with the technology.

Now the journey continues, and we will cover a lot in this part. We will start by improving our basic offline screen and moving the html inside the service worker to its own file. To do this we will see basic caching with the new CacheStorage API and a new service worker life-cycle hook.

After we improved our offline screen we will look into more advanced and complex caching strategies and how to use them to improve the load time of your website when the user comes back. Of course, we will also implement these caching strategies in our application.

Improving our offline screen

Currently, we have a very basic offline screen with the name of our app and a message for the user with a dark grey background as shown in the picture below in picture 01.

This is very basic and not so nice for users to land on it. To make it a bit sleeker we could add an image and centre the text on the screen with CSS as shown above in picture 02. The problem here is that its difficult to include this all as plain text in our service-worker.js. It would look pretty messy in the code, so we will go for another approach. Instead of having all the HTML in the JavaScript code, we will put it in its own file. That way we can develop it just like any other web page.

If we use this approach we encounter another problem. The files are on the server but we want our users to view this page when they can not connect to our server. To solve this we need to leverage the CacheStorage API and use a life-cycle hook from the service workers. Before we get into our solution I’ll explain both.

CacheStorage API

CacheStorage is a new type of caching layer that is completely under your control. It can be used by service workers or the window object. The CacheStorage has only 5 methods and all of them return a Promise:

• open(cacheName): Opens the cache with the given name (Creates a new cache when it can not find one). Promise resolves in a Cache object.
• has(cacheName): Checks if it has a cache with the given name. Promise resolves to a Boolean.
• delete(cacheName): Deletes the cache with a given name. Promise resolves to a Boolean.
• keys(): Returns a promise that will resolve with an array containing strings to all of the names of saved Cache objects.
• match(): Checks if a given request is a key in any of the Cache objects. Promise resolves to that matched Cache object.

As you can see the CacheStorage works with Cache objects. So what is this object? The Cache objects provide a way to cache requests so we can fetch them without needing to get them from the server again. The Cache object has 7 methods, here are the most important ones:

• add(request): Takes a URL, retrieves the response and adds it to the cache. Returns a Promise that resolves in a void.
• delete(request, options): Deletes the cache for a current URL. Returns a Promise that resolves in a Boolean.
• match(request, options): Returns a Promise that resolves to the response associated with the first matching request.

For more info about the CacheStorage API you can check the following links

• CacheStorage: https://developer.mozilla.org/en-US/docs/Web/API/CacheStorage
• Cache Object: https://developer.mozilla.org/en-US/docs/Web/API/Cache

Service workers and its life-cycle hooks

A service worker goes through a few states from being installed to being destroyed. Below we explain briefly when we enter each state and why we can use it.

• Installing: The service worker enters this state when it’s registered by using navigator.serviceWorker.register. If the installation of the service worker is successful it will go to the Installed state. But is there is an error it will go the Redundant state.
• Installed/waiting: We will enter this state when the service worker is installed. And the service worker will wait there until the previous service worker is in the Redundant state
• Activating: This is a pass-through state unless you use the event.waitUntil() it will go immediately to the next state. We will use this state in the advanced caching strategies part.
• Activated: When we reach this state the service worker is ready and will take control of the page. It will start listening to events such as fetch
• Redundant: The service worker comes in this state through 2 ways. The first one is when there is an error with the registering or installing of the service worker. The second way is when every browser tab with the website open is closed. Only then will the next and updated service worker become activated when we open up the website again.

The coding part

Let’s put all this theory in action and develop our improved offline screen. Be sure to have all the code from the previous part or checkout tag: PT2_0-starting-point. And delete everything in the service-worker.js.

We will start by creating a new file called offline.html and you can add the following code to that and check the result at http://localhost:8080/offline.html if you are running a local web server as explained in part 1.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Video Voter</title>

    <link href="css/bootstrap.min.css" rel="stylesheet" />
    <link href="css/cover.css" rel="stylesheet" />
</head>
<body class="text-center full-screen-background-image">
<div class="cover-container d-flex w-100 h-100 p-3 mx-auto flex-column">
    <header class="masthead mb-auto"></header>

    <main role="main" class="inner cover">
        <h1 class="cover-heading">Youtube video voter.</h1>
        <p class="lead">There seems to be a problem with your connection.</p>
    </main>

    <footer class="mastfoot mt-auto">
        <div class="inner">
            <p>Developed by Wout Schoovaerts.</p>
        </div>
    </footer>
</div>
</body>
</html>

As you can see from the picture 04 below we need to load in the HTML, 2 CSS and one image from the server. Which we can not do when we are offline.

Our next step is to cache these 4 files the first time when we visit the web site, so we can display this page when we do not have any internet. We will use the install state of the service worker. We can do this with adding the following code to the service-worker.js:

const CACHE_NAME = 'vid-voter'
const URLS_TO_CACHE = [
    "/offline.html",
    "/css/bootstrap.min.css",
    "/css/cover.css",
    "/image/cover.jpg"
];

self.addEventListener("install", function(event) {
    event.waitUntil(
        caches.open(CACHE_NAME).then(function(cache) {
            return cache.addAll(URLS_TO_CACHE);
        })
    );
});

This code snippet does a few things let’s go over them:

• We define in the beginning our name for the cache and define an array of URL’s we want to cache.
• We add a new listener that listens when we come in the install state of the service worker.
• We have an event.waitUntil so the service worker waits until the promise given as its agrument is resolved to go to the next state.
• We open the cache with caches.open and when this is successful we cache all the URLs we have to cache with cache.addAll.

This takes care of caching the files we need, but how will we show this when the user had no internet? We will do it the same way as in part 1. We use the fetch method and catch when it fails. We can add the following to the service-worker.js:

self.addEventListener("fetch", function(event) {
    event.respondWith(
        fetch(event.request).catch(function() {
            return caches.match(event.request).then(function(response) {
                if (response) {
                    return response;
                } else if (event.request.headers.get("accept").includes("text/html")) {
                    return caches.match("/offline.html");
                }
            });
        })
    );
});

So what does this code do?

• We start by listening to fetch events.
• We try to fetch all requests over the network but when it fails we enter the catch part.
• The catch part will first check if the request is already in the cache, if so we return it.
• If the request is not in the cache and we have the header Accept: text/html. We will start loading offline.html and its dependencies.

If you save you changes in service-worker.js and refresh the index page, you should see the same thing. But if you go to Developer tools → Application → Cache Storage → vid-voter you will see all your entries from the URLS_TO_CACHE array. If you now toggle off the network: developer tools → Network → Tick the offline checkbox. You should see the improved offline screen. You can find all the code to this point with the tag: PT2_1-improved-offline-screen

Going offline

It is nice we have an improved offline screen, but wouldn’t it be better if we just showed our normal website as we would do when we are online? To do this we need to cache everything we need. Because our site does not require a lot of images we can do this. But then what about the second page where we fill in a form and send that data to the server? We will address this in part 3.

In this next part we will see some more advanced caching strategies and start applying those so we can view our site when we do not have any internet.

Some common caching strategies

• Cache only: This approach assumes that the resource you want was already cached in the service workers installation hook. This pattern can be useful for resources that do not change a lot. For example our background image on the homepage. If you want to update the file that is caches you will have to give it a new name and cache that file (e.g. bg-img_v2.png).
• Cache, falling back to network: Here we first check if the cache can serve this resource. If the resource is not cached if will try and fetch it through the network.
• Network, falling back to cache: We first check if we can fetch the resource through the network. If the request fails we check if the cache can give us this resources. This pattern is useful for data that changes frequently and the user needs an up to date representation of the data, but you still want the user to have the possibility to view the data offline.
• Cache, Then network: With this strategy you can display the resource immediately. And after the cache served we start fetching the newest data and once this comes back we replace the cached data with the new up to date data.

There are more caching strategies out there I recommend checking **The Offline Cookbook **for more patterns and details. With these four patterns we can make our small video voter app fully offline accessible. We can do this by mix and matching these patterns together for different use cases.

What will we use?

So the next step is to identify in our application where to use caching and which strategy to use. This all depends on how much the data changes, and how important it is that the data is up to date.

Let’s list all the different strategies we will use.

• Cache only: All third party css, js (bootstrap, fonts, jQuery) and the images (cover.jpg, did-logo.png).
• Network, falling back to cache: Our own css, js (app.js, cover.css) and the HTML. As well for the voter page fetching the videos with the votes.

For our voter page we will only use network, falling back to cache.

Coding time!

Let’s start implementing all these strategies. We will start with our homepage and make it completely offline accessible. First thing we need to do is to start clean so you can remove all the code from service-worker.js and add the following:

const CACHE_NAME = 'vid-voter-v3'
const URLS_CACHE_ONLY = [
    "/css/bootstrap.min.css",
    "/js/bootstrap.min.js",
    "/js/jquery-3.3.1.min.js",
    "/image/cover.jpg",
    "/image/did-logo.png",

    // font-awesome
    "/css/all.min.css",
    "/webfonts/fa-brands-400.eot",
    "/webfonts/fa-brands-400.svg",
    "/webfonts/fa-brands-400.ttf",
    "/webfonts/fa-brands-400.woff",
    "/webfonts/fa-brands-400.woff2",
    "/webfonts/fa-regular-400.eot",
    "/webfonts/fa-regular-400.svg",
    "/webfonts/fa-regular-400.ttf",
    "/webfonts/fa-regular-400.woff",
    "/webfonts/fa-regular-400.woff2",
    "/webfonts/fa-solid-900.eot",
    "/webfonts/fa-solid-900.svg",
    "/webfonts/fa-solid-900.ttf",
    "/webfonts/fa-solid-900.woff",
    "/webfonts/fa-solid-900.woff2",
];

const URLS_OVER_NETWORK_WITH_CACHE_FALLBACK = [
    "/index.html",
    "/voter.html",
    "/css/app.css",
    "/css/cover.css",
    "/js/app.js",
    "/js/voter/voter.js",
    "http://localhost:3000/videos"
];

self.addEventListener("install", function(event) {
    event.waitUntil(
        caches.open(CACHE_NAME).then(function(cache) {
            return cache.addAll(
             URLS_CACHE_ONLY.concat(URLS_OVER_NETWORK_WITH_CACHE_FALLBACK)
            );
        }).catch((err) => {
            console.error(err);
            return new Promise((resolve, reject) => {
                reject('ERROR: ' + err);
            });
        })
    );
});

self.addEventListener("fetch", function (event) {
    const requestURL = new URL(event.request.url);

    if (requestURL.pathname === '/') {
      event.respondWith(getByNetworkFallingBackByCache("/index.html"));
    } else if (URLS_OVER_NETWORK_WITH_CACHE_FALLBACK.includes(requestURL.href) ||
        URLS_OVER_NETWORK_WITH_CACHE_FALLBACK.includes(requestURL.pathname)) {
        event.respondWith(getByNetworkFallingBackByCache(event.request));
    } else if (URLS_CACHE_ONLY.includes(requestURL.href) || 
       URLS_CACHE_ONLY.includes(requestURL.pathname)) {
        event.respondWith(getByCacheOnly(event.request));
    }
});

self.addEventListener("activate", function (event) {
    event.waitUntil(
        caches.keys().then(function (cacheNames) {
            return Promise.all(
                cacheNames.map(function (cacheName) {
                    if (CACHE_NAME !== cacheName && cacheName.startsWith("vid-voter")) {
                        return caches.delete(cacheName);
                    }
                })
            );
        })
    );
});

/**
 * 1. We fetch the request over the network
 * 2. If successful we add the new response to the cache
 * 3. If failed we return the result from the cache
 *
 * @param request
 * @param showAlert
 * @returns Promise
 */
const getByNetworkFallingBackByCache = (request, showAlert = false) => {
    return caches.open(CACHE_NAME).then((cache) => {
        return fetch(request).then((networkResponse) => {
            cache.put(request, networkResponse.clone());
            return networkResponse;
        }).catch(() => {
            if (showAlert) {
                alert('You are in offline mode. The data may be outdated.')
            }

            return caches.match(request);
        });
    });
};

/**
 * Get from cache
 *
 * @param request
 * @returns Promise
 */
const getByCacheOnly = (request) => {
    return caches.open(CACHE_NAME).then((cache) => {
        return cache.match(request).then((response) => {
            return response;
        });
    });
};

Whats happening in the code?

The first thing we do is define our cache name, when we want to cache everything again (for example: after a new release of you website). We change the name to vid-voter-v4. This will make sure that the serviceworker will cache everything again, and the end users have the latest code and functionalities.

The second thing we do is define which urls need to be cached. I like to create arrays with the name of the caching strategy I’m using for this batch of urls. That’s why I used URLS_CACHE_ONLY and URLS_OVER_NETWORK_WITH_CACHE_FALLBACK*,* this makes it clear to anyone who reads the code, what happens with what url.

The event listener install is still the same, we only added a bit of error handling.

In the event listener of fetch we added a special case for index.html so we can serve this file over cache when the network is failing, so this page will become offline available. We use the getByNetworkFallingBackByCache which is explained next.

The getByNetworkFallingBackByCache method, will try to fetch the content over the network. If it succeeds, we update the cache with the newest content. If we are offline or the server, we will check if this content is inside the cache. When this is the case, we return the content from the cache and show a warning that the data may be outdated.

Our last method is getByCacheOnly which just retrieves content from the cache.

You can find all the code under the tag: PT2_2-homepage

What have we done?

After we implemented this code, visit the homepage with a network connection. After that you can toggle offline mode in the chrome developer tools and we see the same page. So this means our homepage is offline accessible.

In the last part we will use indexedDB and background-sync to make our form functioning when we are offline.