Use A Whitelist for Your IgnoreJune 11th 2018
Over and over again I've seen open source projects, clients and coworkers publishing modules or pushing images with unnecessary files, tests and sensitive information. This can result from a misconfigured project or inattentive maintainer. An easy way to prevent many of these simple mistakes is to take advantage of project "ignore" files as whitelists.
The two types of projects I'll be focusing on are for Node and Docker.
Node modules are (typically) packaged and uploaded with
npm. By default, npm will ignore files
.gitignore or files from a short builtin list:
.*.swp ._* .DS_Store .git .hg .npmrc .lock-wscript .svn .wafpickle-* config.gypi CVS npm-debug.log node_modules
However, these defaults can be overridden by using an
.npmignore uses the same pattern rules as
.gitignore and by default is a
Docker images are typically built from the contents of a directory using the
command will follow the directives in the local
Dockerfile to build an
docker build will also upload the contents of the local
directory to your Docker build server and any blanket
COPY . /opt/somedir/) will copy these files into the image
We can limit what is copied to the build server and images by specifying a
This file uses the golang
filepath.Match rules instead of
the glob rules in
.npmignore, a slightly different syntax,
but for most purposes it will be the same.
.dockerignore is a blacklist.
What's wrong with blacklists?
For the use case of managing what's included in a published package or image, it's a matter of using the right tool for the job. The logic seems backwards when you have to specify explicitly what cannot be added to the module or image. It's impossible to anticipate what your peers will add to the project.
In the Node ecosystem it's common to use a
.env file to describe your
application environment. Typically this file will include things like
authentication tokens for testing locally. It's easy for an inattentive
maintainer to say, copy their existing
doing some local development. When it comes time to deliver their module or
image they can easily bundle this file in with a
npm publish or
build. This strawman user probably had some rules sprinkled throughout their
ignore files for
.env*, but they still managed to sneak a file
Blacklist to Whitelist
.gitignore will typically be formatted as a
To transform into a whitelist simply follow this format:
# Disallow all files * # Allow specific file with a ! !specific_file # Works with directories in Docker !my_directory/ # Works with directories in Npm !my_directory/**/*
Using your ignore as a whitelist isn't a panacea. You or or peers can easily fall into similar traps as the blacklist by whitelisting entire directory trees. Another issue is the friction of adding new files to the whitelist itself to make sure your project builds properly.
I think that these issues are minor to the ones typically caused by a blacklist ignore.
Why not just do all your builds / deploys from a controlled environment like a CI/CD server?
You should. Unfortunately it's not always possible. A whitelist ignore is more for the uncontrolled build environments.
June 13 2018
January 11 2019
Fixed npm directory list in whitelist with a glob.
By Colin Kennedy