Jeremy Wells

Full-stack software developer sharing learnings with .NET, Angular, and anything else worth tinkering with

Show and tell: my Eleventy plugin

Plug

Photo by Markus Winkler on Unsplash

This year, I've become a pretty big fan of a newer-ish static site generator called Eleventy. After playing around a bit, I took on the project converting this site to an Eleventy-generated blog. Eleventy has got a pretty rich API, and I found that three configuration methods it provides - filter, addShortcode, and addPlugin - lend themselves really well to extending the tool's functionality. This post is to document the process for creating a little Eleventy plugin of my own (Eleventy Gist) by iterating through the different levels of abstraction that can be attained through the Eleventy API.

V2-ing my little tech blog

After a busy year, I decided to dust off this blog to see if I could get back to some regular tech writing. One thing that snuck up on me in the intervening year was my Jekyll setup had broken. I'd need to go down some rabbit holes between getting a Ruby environment reconfigured and figuring out what Gem updates needed sorting out. And not being a Ruby developer, it wouldn't have been the most useful learning experience.

Instead, I looked at a few of the newer static site generators in town and happened upon Eleventy. After going through a couple of basic tutorials, I found a lot to like: it's simple but versatile, it's easy to configure, it generates the site quickly, and best of all it's Javascript all the way down, which is much more in my wheelhouse. There were a few tutorials out there that got me around a few of the well known obstacles when it comes to porting Jekyll sites to Eleventy. Generally, I was able to copy over my old posts without having to modify the files too much - and I really don't have enough posts to necessitate much automation anyway.

But one big obstacle in porting the site was a Jekyll plugin called jekyll-gist that I had made use of for including code snippets. It basically takes the contents of a Github Gist file and outputs it inside of a <code></code> block on the page. As far as I could tell, it has no equivalent in the Eleventy-verse. And going into each post, finding each reference to the Jekyll Gist plugin, hunting down the gist, then manually pasting the contents just did not sound like an afternoon well spent. Plus, I'd still like to use Github Gists in my workflow. Here was an opportunity to go a little deeper into Eleventy and build my first Eleventy plugin.

I came up with the following basic requirements for the plugin:

  • I can output the contents of a Github Gist to a static web page.
  • I can add this functionality to an Eleventy-generated site with a plugin.
  • I can configure this plugin to use a Github bearer token.
  • I can use a similar syntax in my templates as the Jekyll Gist plugin.
  • I can make this plugin available via npm.

Iterating with Eleventy's configuration methods.

After a few false starts, one aspect of Eleventy that helped with iterating over the logic I wanted to implement was the different configuration methods available in the tool, which offer different levels of abstraction.

  1. A Filter can be used to transform a value and output it to the generated page.
  2. A Shortcode is like a method that extends the syntax of the template language you're using.
  3. A Plugin is a separate, reusable module that can be imported into an Eleventy config.

So when developing and testing this new functionality, iterating through the different configuration methods could look something like this:

1. Use a filter to validate the content that gets output to the page

I created a dummy post file ($ touch content/posts/gist-testpost.md). I set up the Github API to return a Gist from an old post and copied the content string, then passed that to a variable in the dummy post:

{% set myGist = '// javascript export function doTheThing(myArg) { console.log("doin the thing");}' %}

Then, I sent that to a yet-unimplemented filter:

{{ myGist | gist }}

And as expected, on rebuilding the site, errors and chaos ensue.

Then, to implement and test how the raw file content gets rendered onto the page, I created the filter in the eleventy.config.js:

module.exports = function (eleventyConfig) {
    // ...
    eleventyConfig.addFilter('gist', content => {
        const ext = 'js';
		return '```' + ext + '\n' + content + '\n```';
	});
    // ...
}

And with that, I was able to work out how the content of one gist would look in my site, adjusting the syntax highlighting and styling as needed.

2. Use a shortcode to test the feature as a standalone module

After seeing basically how the output of a gist would render in the site, it was time to develop the eleventy-gist module itself. I created a temporary plugin directory to store the files ($ mkdir -d plugins/gist) and some files ($ touch plugins/gist/{gist.js,gist.test.js,requestWrapper.js}), and I was able to use tests with Jest to work out the gist function.

One extra file that was necessary was requestWrapper.js that isn't much more than an HTTP request body using the native Node.js https module:

const https = require('https');

/**
 * Wraps Node.js https request for easier testing
 * @param {https.RequestOptions} options
 * @returns {any}
 */
module.exports = async function request(options) {
    return new Promise((resolve, reject) => {
        const req = https.request(options, res => {
            if (res.statusCode < 200 || res.statusCode >= 300) {
                return reject(new Error('API error: statusCode = ' + res.statusCode));
            }
            let body = [];
            res.on('data', d => body.push(d));
            res.on('end', () => {
                body = JSON.parse(Buffer.concat(body).toString());
                resolve(body);
            })
        });
        req.on('error', e => reject(e.message));
        req.end();
    });
}

This helps remove real requests to the Github API from the unit tests, and we can mock it in Jest like this:

jest.mock('./requestWrapper');
const request = require('./requestWrapper');

const createAPIResponse = (fileName, contentText) =>
    ((copyFileName, copyContentText) => {
        let fileName = copyFileName;
        let contentText = copyContentText;
        return {
            getResponse() {
                return {
                    files: { [fileName]: { content: contentText } }
                };
            },
            addFileToResponse(fileName, contentText) {
                const response = this.getResponse();
                response.files[fileName] = { content: contentText };
                return response;
            }
        }
    })(fileName, contentText);

const createOpts = () => {
    return { authToken: '12345', userAgent: 'dave grohl' };
};

Then using the mock in a test looks something like this:

describe('gist() : ', () => {
    test('if called w/o authToken or userAgent does not call github API', async () => {
        const testObj = createAPIResponse('01.sh', ' echo hello > myfile.txt ').getResponse();
        request.mockResolvedValue(testObj);
        const opts = null;
        const result = await gist('', '', opts);
        expect(request).not.toHaveBeenCalled();
        expect(result).toBe('');
    });
});

From there, I can test and develop the gist() function:

async function gist(gistId, fileName, opts) {
   // ...
    try {
        verifyOptions(opts);
        const result = await run(gistId, fileName, opts);
        return result;
    }
    catch (e) {
        // ...
        console.log(errorMessage);
        // ...
        return '';
    }
}

module.exports = { gist };

You can see the final repo for the full implementation.

Finally, using an Eleventy shortcode, I can import and test the gist() function for real:

const gist = require('./plugins/gist/gist/')

const config = {
    // ...(I'll talk about the config a little later)...
};
module.exports = function (eleventyConfig) {
    // ...
    eleventyConfig.addShortcode('gist', async function (gistId, fileName) {
        return gist(gistId, fileName, config);
    });
    // ...
}

Cool. So through a combination of Test-Driven Development, then using a shortcode to see how the gist() function worked in the real site, I was able to find and work out a few issues:

  • I needed a configuration object to store things like a bearer token for the Github API, as well as a user name to pass to the User-Agent HTTP header.
  • When I ran the live development server that comes with Eleventy, on every save, gist() would call the Github API for every single use of the {% gist %} shortcode. I noticed this dramatically increased the build times. I needed a way to cache the results the first time the contents loaded, especially if the site was in development mode.
  • The Github API of course sends back errors as APIs will do. I needed a way to communicate that in the console.
  • Also, if the user adds the ID or a file name of a gist that doesn't exist, when in development, it would be helpful to see it really clearly on the page. I added a debug option to the configuration that will output a big red message on the page so it's at least a little easier to spot. Otherwise, in production errors will return an empty string to hide the issue from the public.

So, when hooked up to dotenv for reading environmental variables, a sample eleventy.config.js will look like this:

require('dotenv').config();
const gist = require('./plugins/gist/gist/')
const config = {
    authToken: process.env.github_access_token,
		userAgent: process.env.github_user_agent,
		debug: process.env.NODE_ENV === 'development',
		useCache: process.env.NODE_ENV === 'development'
};

module.exports = function (eleventyConfig) {
    // ...
    eleventyConfig.addShortcode('gist', async function (gistId, fileName) {
        return gist(gistId, fileName, config);
	  });
    // ...
}

With a .env file set up something like this (don't forget to list it in your .gitignore):

NODE_ENV="development"
github_access_token="YOUR SECRET TOKEN"
github_user_agent="@YOUR USER NAME"

3. Use an Eleventy plugin to test the feature as an NPM package

The last step was to move my gist files to their own directory and set them up as an NPM package. Then, in the Eleventy config, I could call the function with addPlugin instead of addShortcode. Once this worked as expected, Eleventy would be all set to install and use the gist() function as a third-party package.

Before doing any of that, let's point out what an Eleventy plugin does differently. When our plugin is fully converted, adding it to the Eleventy config will look like this:

const config = {
    // ...same config...
};
module.exports = function (eleventyConfig) {
    // ...
    eleventyConfig.addPlugin(gist, config);
}

Basically, any of the logic from the body of the addShortcode callback method will be abstracted away to the plugin. Plugins in Eleventy work by taking the context of the configuration and doing all the work in the plugin rather than in the consuming application.

Back in the plugin project, I needed to export the gist() function within an anonymous function that takes the Eleventy configuration context as it's first argument, then whatever else is needed from the Eleventy config. I added an index.js file to the eleventy-gist project that looks like this:

const { gist } = require('./gist');

module.exports = async function(config, opts = {}) {
	config.addShortcode('gist', async function (gistId, fileName) {
        return gist(gistId, fileName, opts);
  });
}

We can see here that the function receives the Eleventy config object (here it's generically called config), then everything in the addShortcode from above is moved into here.

Now back in the eleventy.config.js, I replaced the addShortcode method with addPlugin:

const config = {
    // ...same config...
};
module.exports = function (eleventyConfig) {
    // ...
    eleventyConfig.addPlugin(gist, config);
}

Now that the gist function is a plugin, it can just be added and configured in eleventy.config.js, and all the other details are abstracted away. Once this change was working, it was time to move the new files to their own NPM project.

The following steps were the standard way of creating an NPM package, so I won't go into too much detail.

  1. Move the files from the plugins directory in the Eleventy project, to their own project (/eleventy-gist in my case).
  2. Initialize the project ($ npm init) and add any required npm modules (I only needed Jest: npm install --save-dev jest).
  3. Back in the eleventy project, add eleventy-gist to the package.json and reference the local path (for some final tests before publishing):
{
  "...": "...",  
  "devDependencies": {
    "eleventy-gist": "../eleventy-gist",
    "...": "..."
  }
}
  1. Adjust the import in the eleventy.config.js accordingly, then give it a test. However it works here should be exactly how it will work when imported via $ npm install ....
  2. If everything is good, create your npm project, or repo, or whatever you need to distribute your project, then npm publish.
  3. Update the package.json in the Eleventy project to reference the package remotely and everything should be ready.

And through the process of switching out different Eleventy configuration methods - filter for initial hard-coding and working out the expected output; addShortcode for working out the actual function through TDD; and addPlugin to convert the function to a separate project.

If Eleventy Gist might be something you could use, give it a go and post and issue if I can make it better.