Andrew Welch

Andrew Welch · Insights · #frontend #gulp #package.json

Making the web better one site at a time, with a focus on performance, usability & SEO

2016.12.23 · 5 min read ·

A Better package.json for the Frontend

Frontend workflow tools like gulp are almost mandatory for webdev these days; here’s what I consider to be a better package.json setup

Package

If you’re doing webdev, odds are good you’re using a workflow tool like Gulp to automate your build process. Whether you’re using Gulp, Grunt, Slushwebpack, or npm scripts, it doesn’t really matter. Use whatever tool works for you.

If you’re not using a workflow automation tool yet, you should be. It’s worth investing the upfront time to get it working, for the time it’ll save you on every project going forward.

I’m currently using Gulp on most projects, so that’s what I’ll be presenting here, but the methodologies described work for any of the frontend workflow tools.

The package.json file is the packaging format for node.js applications, and Gulp is a node.js application. But we can also leverage the package.json file to encompass our entire website.

In the past, people were using Bower as a frontend package manager for the third party Javscript/CSS used on their websites. The advantage of using a package manager is threefold:

  1. You have a centralized manifest of all of the third party packages your website uses
  2. You can easily update to more recent versions of the third party packages your website uses
  3. You are always obtaining the packages from their canonical source

In the bad old days before Bower, people manually downloaded their frontend packages, which made updating them a pain. So most of the time, it never happened.

I’ve used Bower in the past, but since we’re already using npm as a package manager for our node.js packages used by Gulp, why not use it for the frontend packages as well?

Using more than one package manager for our website seems silly. I’ve never run into a case where a package is in Bower, but is not in NPM

Tangent: While I talk about npm throughout the rest of this article, I’m actually using yarn these days to manage my npm packages. It uses the exact same npmjs.com package registry that npm uses, it just does it better and faster. And it uses a yarn.lock file to ensure that the package installs are deterministic. I urge you to check it!

Instead of having to do npm install and bower install to get our project bootstrapped, and then remembering to do both npm update and bower update to update our packages, we just do npm install or npm update. Or if we’re using yarn, we can just do yarn! Great!

If you’re not familiar with the package.json format, check out this package.json Interactive Guide. While there are many keys in the package.json that have special meaning, it’s instructive to keep in mind that it’s just JSON. We can add whatever we want to it. So we will! More on that later.

Package All the Things!

For our websites, we’ll use the devDependencies key in our package.json for node.js packages that Gulp uses via our gulpfile.js for automating our frontend workflow. Here are the devDependencies for this very website you’re reading now:

    "devDependencies": {
        "babel-preset-es2015": "^6.16.0",
        "babel-plugin-transform-runtime": "^6.15.0",
        "chalk": "^1.1.3",
        "critical": "0.8.0",
        "fancy-log": "^1.2.0",
        "git-rev-sync": "^1.7.1",
        "gulp": "^3.9.0",
        "gulp-autoprefixer": "^3.1.0",
        "gulp-babel": "^6.1.2",
        "gulp-concat": "^2.6.0",
        "gulp-cssnano": "^2.1.2",
        "gulp-debug": "^2.1.2",
        "gulp-favicons": "^2.2.6",
        "gulp-fontello": "^0.4.6",
        "gulp-header": "^1.8.7",
        "gulp-if": "^2.0.1",
        "gulp-imagemin": "^3.1.1",
        "gulp-load-plugins": "^1.3.0",
        "gulp-newer": "^1.2.0",
        "gulp-plumber": "^1.1.0",
        "gulp-print": "^2.0.1",
        "gulp-rename": "^1.2.2",
        "gulp-replace": "0.5.4",
        "gulp-rev": "^7.1.0",
        "gulp-sass": "^2.1.0",
        "gulp-size": "^2.1.0",
        "gulp-sourcemaps": "^2.2.1",
        "gulp-uglify": "^1.5.4",
        "moment": "^2.14.1"
    },

It’s not important that you know what all of these things are; rather just that you see they are all node.js packages that we use for automating our frontend workflow.

Since we consider the website we’re building “the app”, we’ll use the dependencies key in our package.json for the frontend packages (JavaScript/CSS) that we use in-browser. Here are the dependencies for this very website you’re reading now:

    "dependencies": {
        "fg-loadcss": "^1.2.0",
        "flexboxgrid": "^6.3.1",
        "fontfaceobserver": "^2.0.5",
        "lazysizes": "^2.0.6",
        "normalize.css": "^5.0.0",
        "picturefill": "^3.0.2",
        "prismjs": "^1.5.1",
        "systemjs": "^0.19.40",
        "vue": "^2.1.0",
        "vue-resource": "^1.0.3"
    },

Boom, that’s it! We do an npm install or yarn and all of the third party packages we use to make our website—both in-browser and for our workflow automation—are installed! 

You can find the packages on npmjs.com, and install them as dependencies via npm install -S <package-name>, or install them as devDependencies via npm install -D <package-name>.

If you’re using yarn instead of npm (which you should be!), you can find the packages on npmjs.com, and install them as dependencies via yarn add <package-name>, or install them as devDependencies via yarn add --dev <package-name>.

Or if you prefer, you can just edit your package.json file yourself, and then do npm install or yarn.

If for some crazy reason you run into a situation where an npm package doesn’t exist for something you want to use, or you’re installing something from a private git repo, you can always just do this in your package.json:

        "private-package": "git+ssh://git@github.com/yourhandle/private-package.git#master",

One thing that’s fairly critical to understanding how to use npm is the package version, it’s what appears after the package name in our package.json file. The version is expressed as a semver which is just a way to express the version you want installed, and the threshold for newer version that it can be updated to.

For instance, we might want all bug fix versions of of a particular package, but we don’t want any breaking changes. Using semver, we can rest easy that when we type npm update or yarn, we’ll get only the non-breaking bug fix or patch versions, and our project won’t blow up.

I typically do '^1.0.0' for the packages I install, which says “I want all updates up to the next major version”, 2.0.0 in this case); a more common pattern is to do '~1.0.0' (which will give you only patch updates, in this case up to 1.1.0). A great way to learn semver is to check out the npm semver calculator and play around with it a bit. It’s really not hard to learn, and it’s time well-spent.

Tangent: We should consider anything we’ve gotten from a package manager (whether it is Bower or npm or whatever) as read-only. Use your gulpfile.js to copy or transform things from these third-party packages into your website. Don’t use them directly, or—gasp—modify them.

Cleaning up our gulpfile.js

With all of these npm packages (aka node modules), our gulpfiles tend to end up with a ton of cruft at the top, where we can’t even see our code until we scroll down past a few pages of things like this:

var gulp        = require('gulp'),
    sass        = require('gulp-sass'),
    rename      = require('gulp-rename'),
    cssmin      = require('gulp-minify-css'),
    concat      = require('gulp-concat'),
    uglify      = require('gulp-uglify'),
    jshint      = require('gulp-jshint'),
    scsslint    = require('gulp-scss-lint'),
    cache       = require('gulp-cached'),
    prefix      = require('gulp-autoprefixer'),
    browserSync = require('browser-sync'),
    reload      = browserSync.reload,
    size        = require('gulp-size'),
    imagemin    = require('gulp-imagemin'),
    pngquant    = require('imagemin-pngquant'),
    plumber     = require('gulp-plumber'),
    deploy      = require('gulp-gh-pages'),
    notify      = require('gulp-notify'),
    changed     = require('gulp-changed'),
    favicons    = require('gulp-favicons'),
    fontello    = require('gulp-fontello'),
    print       = require('gulp-print'),
    replace     = require('gulp-replace'),
    csslint     = require('gulp-csslint'),
    newer       = require('gulp-newer'),
    log         = require('fancy-log'),
    chalk       = require('chalk'),
    jsonminify = require('gulp-jsonminify'),
    critical    = require('critical');

This really is fairly pointless, we’re already declaring what we want in our devDependencies.

Instead, let’s clean it up a bit by using the gulp-load-plugins Gulp plugin. What it does is it’ll load in your node.js modules namespaced under the $ variable (hello, jQuery!). Since we know we’re using everything in devDependencies for our workflow automation via Gulp, we can change that raft of require()’s above into just:

// load all plugins in 'devDependencies' into the variable $
const $ = require('gulp-load-plugins')({
        pattern: ['*'],
        scope: ['devDependencies']
    });

Then to use our node.js modules in our gulpfile.js, we just access them via the $ variable (well, really a const in this case, but whatever) like this:

    $.fancyLog("-> Compiling scss");

Ahhh, so neat and tidy! Now let’s keep on a roll cleaning things up!

Separating your data from your code

A term all of the cool kids are using these days is DRY. They don’t mean that your code is boring, they mean that you Don’t Repeat Yourself. Back in the day, when we’d walk 2 miles uphill in the snow to get to our bus stop, we just called this modularity or reusability.

Whatever you call it, it’s a good thing. Essentially, we want to take the bits of code we write, and re-use them as much as possible from project to project. As a step towards this when building websites, there’s what I do in my gulpfile.js:

// package vars
const pkg = require('./package.json');

Now we have access to the entire package.json data structure in our gulpfile.js. So what we can do is we can separate the code in our gulpfile.js from the data that it operates on.

Want to start a new project? Great! All you will have to change is your package.json to reflect what’s different about this project… since your gulpfile.js just operates on the data that’s fed into it via the package.json, this all just works. And you end up with cleaner, more readable code, too.

For instance, we change this:

// scss - build the scss to the build folder, including the required paths, and writing out a sourcemap
gulp.task('scss', () => {
    $.fancyLog("-> Compiling scss:");
    return gulp.src("./src/scss/style.scss")
        .pipe($.plumber({ errorHandler: onError }))
        .pipe($.sourcemaps.init())
        .pipe($.sass({
                includePaths: "./src/scss/"
            })
            .on('error', $.sass.logError))
        .pipe($.cached('sass_compile'))
        .pipe($.autoprefixer())
        .pipe($.sourcemaps.write('./'))
        .pipe($.size({ gzip: true, showFiles: true }))
        .pipe(gulp.dest("./build/css/"));
});

Into this:

// scss - build the scss to the build folder, including the required paths, and writing out a sourcemap
gulp.task('scss', () => {
    $.fancyLog("-> Compiling scss: " + pkg.paths.build.css + pkg.vars.scssName);
    return gulp.src(pkg.paths.src.scss + pkg.vars.scssName)
        .pipe($.plumber({ errorHandler: onError }))
        .pipe($.sourcemaps.init())
        .pipe($.sass({
                includePaths: pkg.paths.scss
            })
            .on('error', $.sass.logError))
        .pipe($.cached('sass_compile'))
        .pipe($.autoprefixer())
        .pipe($.sourcemaps.write('./'))
        .pipe($.size({ gzip: true, showFiles: true }))
        .pipe(gulp.dest(pkg.paths.build.css));
});

Not only are we not hard-coding the file names and paths inline in our gulpfile.js, we’re not even including them in our gulpfile.js at all—they are pulled from our package.json which defines everything that’s unique about this particular project.

Pretty cool, right? While this isn’t the be-all and end-all of how to do things, it’s worked out well for me. The code is more reusable, and it’s cleaner and easier to read.

That’s it folks, hope you found some of this useful!

I’ll include the full package.json used for this website here, to help you get a better idea how things work under the hood:

{
    "name": "nystudio107",
    "version": "1.0.0",
    "description": "Website for nystudio107.com",
    "main": "gulpfile.js",
    "author": "Andrew Welch, nystudio107 <andrew@nystudio107.com>",
    "copyright": "nystudio107",
    "license": "UNLICENSED",
    "private": true,
    "paths": {
        "src": {
            "base": "./src/",
            "css": "./src/css/",
            "fontello": "./src/fontello/",
            "fonts": "./src/fonts/",
            "json": "./src/json/",
            "js": "./src/js/",
            "img": "./src/img/",
            "scss": "./src/scss/"
        },
        "dist": {
            "base": "./public/",
            "css": "./public/css/",
            "js": "./public/js/",
            "fonts": "./public/fonts/",
            "img": "./public/img/"
        },
        "build": {
            "base": "./build/",
            "css": "./build/css/",
            "fontello": "./build/fonts/fontello/",
            "fonts": "./build/fonts/",
            "js": "./build/js/",
            "html": "./build/html/",
            "img": "./build/img/"
        },
        "favicon": {
            "src": "./src/img/favicon_src.png",
            "dest": "./public/img/site/",
            "path": "/img/site/"
        },
        "scss": [
        ],
        "templates": "./craft/templates/"
    },
    "urls": {
        "live": "https://nystudio107.com/",
        "local": "https://nystudio107.dev/",
        "critical": "https://nystudio107.com/"
    },
    "vars": {
        "siteCssName": "site.combined.min.css",
        "scssName": "style.scss",
        "cssName": "style.css"
    },
    "globs": {
        "distCss": [
            "./node_modules/normalize.css/normalize.css",
            "./node_modules/flexboxgrid/dist/flexboxgrid.min.css",
            "./src/css/prism-theme.css",
            "./build/fonts/fontello/css/fontello-codes.css",
            "./build/css/*.css",
            "./src/css/*.css"
        ],
        "img": [
            "./public/img/"
        ],
        "components": [
            "./src/components/**/*.vue"
        ],
        "fonts": [
            "./build/fonts/fontello/font/*.{eot,ttf,woff,woff2}",
            "./src/fonts/*.{eot,ttf,woff,woff2}"
        ],
        "critical": [
            { "url": "", "template": "index" },
            { "url": "blog/stop-using-htaccess-files-no-really", "template": "blog/_entry" },
            { "url": "blog/stop-using-htaccess-files-no-really", "template": "blog/_amp_entry" },
            { "url": "blog", "template": "blog/index" },
            { "url": "blog", "template": "blog/amp_index" },
            { "url": "wordpress", "template": "wordpress" },
            { "url": "404", "template": "404" }
        ],
        "distJs": [
            "./build/js/*.js",
            "./node_modules/lazysizes/lazysizes.min.js",
            "./node_modules/lazysizes/plugins/bgset/ls.bgset.min.js",
            "./node_modules/picturefill/dist/picturefill.min.js",
            "./node_modules/vue/dist/vue.min.js",
            "./node_modules/vue-resource/dist/vue-resource.min.js"
        ],
        "prismJs": [
            "./node_modules/prismjs/prism.js",
            "./node_modules/prismjs/components/prism-markup.js",
            "./node_modules/prismjs/components/prism-apacheconf.js",
            "./node_modules/prismjs/components/prism-css.js",
            "./node_modules/prismjs/components/prism-json.js",
            "./node_modules/prismjs/components/prism-twig.js",
            "./node_modules/prismjs/components/prism-php.js",
            "./node_modules/prismjs/components/prism-bash.js",
            "./node_modules/prismjs/components/prism-javascript.js",
            "./node_modules/prismjs/plugins/line-numbers/prism-line-numbers.min.js"
       ],
        "systemJs": [
            "./node_modules/systemjs/dist/system-polyfills.js",
            "./node_modules/systemjs/dist/system.js",
            "./src/js/system-config.js"
        ],
        "babelJs": [
            "./src/js/*.js"
        ],
        "inlineJs": [
            "./node_modules/fg-loadcss/src/loadCSS.js",
            "./node_modules/fg-loadcss/src/cssrelpreload.js",
            "./node_modules/fontfaceobserver/fontfaceobserver.js",
            "./src/js/asyncload-blog-fonts.js",
            "./src/js/asyncload-site-fonts.js"
        ],
        "siteIcon": "./public/img/site/favicon.*"
    },
    "dependencies": {
        "fg-loadcss": "^1.2.0",
        "flexboxgrid": "^6.3.1",
        "fontfaceobserver": "^2.0.5",
        "lazysizes": "^2.0.6",
        "normalize.css": "^5.0.0",
        "picturefill": "^3.0.2",
        "prismjs": "^1.5.1",
        "systemjs": "^0.19.40",
        "vue": "^2.1.0",
        "vue-resource": "^1.0.3"
    },
    "devDependencies": {
        "babel-preset-es2015": "^6.16.0",
        "babel-plugin-transform-runtime": "^6.15.0",
        "chalk": "^1.1.3",
        "critical": "0.8.0",
        "fancy-log": "^1.2.0",
        "git-rev-sync": "^1.7.1",
        "gulp": "^3.9.0",
        "gulp-autoprefixer": "^3.1.0",
        "gulp-babel": "^6.1.2",
        "gulp-concat": "^2.6.0",
        "gulp-cssnano": "^2.1.2",
        "gulp-debug": "^2.1.2",
        "gulp-favicons": "^2.2.6",
        "gulp-fontello": "^0.4.6",
        "gulp-header": "^1.8.7",
        "gulp-if": "^2.0.1",
        "gulp-imagemin": "^3.1.1",
        "gulp-load-plugins": "^1.3.0",
        "gulp-newer": "^1.2.0",
        "gulp-plumber": "^1.1.0",
        "gulp-print": "^2.0.1",
        "gulp-rename": "^1.2.2",
        "gulp-replace": "0.5.4",
        "gulp-rev": "^7.1.0",
        "gulp-sass": "^2.1.0",
        "gulp-size": "^2.1.0",
        "gulp-sourcemaps": "^2.2.1",
        "gulp-uglify": "^1.5.4",
        "moment": "^2.14.1"
    },
    "scripts": {
        "start": "gulp",
        "build": "gulp build"
    },
    "repository": {
        "type": "git",
        "url": "git@codewhore.com:nystudio107.git"
    },
    "bugs": {
        "email": "andrew@nystudio107.com"
    }
}
Tidal Wave Disaster

Insights · 2017.03.20

Mitigating Disaster via Website Backups

#backups #devops #craftcms

Web Host Server

Insights · 2017.03.09

How Agencies & Freelancers Should Do Web Hosting

#devops #hosting #vps

Critical Css Stopwatch

Insights · 2017.02.28

Implementing Critical CSS on your website

#critical-css #frontend #performance

Autocomplete Search

Insights · 2017.02.22

Autocomplete Search with the Element API & VueJS

#autocomplete #elementapi #vuejs

Structured Data

Insights · 2017.02.11

JSON-LD, Structured Data and Erotica

#json-ld #structured-data #SEO

Craft Cms 3 Beta

Reviews · 2017.02.09

Craft 3 Beta Executive Summary

#craftcms #Craft 3 #Review

No Robots

Insights · 2017.02.02

Preventing Google from Indexing Staging Sites

#frontend #craftcms #SEOmatic

Light Feather

Insights · 2017.01.24

LoadJS as a Lightweight JavaScript Loader

#loadjs #frontend #Javascript

The Matrix

Insights · 2017.01.16

Creating a Content Builder in Craft CMS

#frontend #matrix #craftcms

${ category } · ${ blog.postDate }

${ blog.title }

#${ tag }