Lockfiles for apps, not packages (still)

12 Sep 2024 • 7 min read • JavaScript

TL;DR: My updated take is Lockfiles for Node.js apps, not for other projects.

When you run npm install, after you add or change a dependency in package.json, npm finds and selects the latest compatible version, downloads it, and replaces your package-lock.json file to describe what it found.

The npm install command does not consider lockfiles from upstream packages you depend on. This is not a bug. It’s by design. The npm publish command explicitly omits lockfiles from any package.^[1]

This and other factors led Sindre Sorhus (@sindresorhus), author of some of the most well-known and popular packages on npm, to adopt this policy in 2017:

Lockfiles for apps, but not for packages.

This was in response to npm enabling package-lock.json in the npm 5.0 release.

Lockfiles are useful

Over the past decade, I found lockfiles to really shine and be “worth it” when:

You maintain a Node.js-based application that you deploy as a finished product. Or;
You maintain a command-line application that developers should install globally on their workstation, via npm install -g.

When developing a Node.js-based service, you can commit a package-lock.json file alongside it. Combine this with a production deployment that runs npm ci (instead of npm install), and you can safely deploy changes (especially rollbacks, time-sensitive reverts after a faulty deployment) to your service without untimely updates to dependencies piggybacking as part of your deployment. There are other and better ways to accomplish this, but lockfiles are a decent start. In this case, I’d also run npm shrinkwrap, which renames the lockfile to npm-shrinkwrap.json. That clearly communicates that this lockfile is tied to your application’s deployment. But, any lockfile will do for this use case.

When installing a package globally, e.g. npm install -g fresnel, npm can consider an upstream lockfile. Such upstream package must supply a shrinkwrap for npm to consider it. And, npm can only utlize it when installing the package standalone, i.e. globally. When developing an end-user application that you expect developers to install via npm install ‑g, by all means use a lockfile. Any lockfile that isn’t “shrinkwrapped”, won’t be published by npm as part of your package, and thus cannot benefit installations.

Global dependencies

Back in the early 2010s, it was common to find projects that couldn’t locally pass linters and tests, because it assumed a different version of JSHint or ESLint than I installed, for another project I contribute to. These kinds of problems tormented many frontend developers, when they first dabbled in CLI and server-side scripting. They would have their project rely on globally installed tools and, invariably, on a specific (yet undocumented) version.

Over the past decade, the Node.js ecosystem has slowly learned its lesson. Packages now generally take care of their own dev tooling. In package.json, each package declares the relevant dev dependencies. We use "scripts" entries to execute commands like eslint, qunit, or grunt. This is especially convenient given that the commands of any dependency can be used directly in "scripts". You need not specify the path to node_modules or call npx here.^[2]

Benefits and costs

Most repositories containing a package.json file are either:

packages published to npm, for use as dependency in another project, or
projects that use Node.js tooling during development only — such as PHP, Ruby, Python, or C++ projects that may use tools like ESLint and QUnit for frontend testing. This includes Composer packages, WordPress plugins, and MediaWiki extensions.

Note that neither of these fall under the categories outlined earlier (Node.js services, and Node.js global tools), and thus have no use for a lockfile. However, as maintainer, it costs you in busywork, support tickets, and sunkcosts you further into justifying other equally-fruitless busywork.

Open faucet splashing water from a fountain. — In Dutch we have the idiom “dweilen met de kraan open“, to mop while the tap is running. This perfectly captures the idea of a boondoggle and busy work more generally. (Image via Wikimedia Commons)

Security updates

Okay, so you’re working on a project or package where you ostensibly don’t need a package-lock.json file. Can this impact security?

For packages, we’ve already established that the lockfile can’t benefit your users. Hence, it does not delay or provide any protection from problematic updates. When they install your package, npm selects the latest compatible version of your dependencies. To pin a dependency, you have to pin it in package.json. This is best paired with a general reduction in risk by reducing your dependencies. Either way, a lockfile cannot help you.

Okay, what about you? Does it help you as maintainer?

For maintainers and contributors to your project, the first install downloads dependencies over the network, either way. Subsequent installs resolve versions against the online registry, then utilize the local npm cache, either way. Lockfiles accomplish nothing but a constant stream of patches (and conflicts) to said lockfile, to keep it identical to… how npm install leaves it. Also, notice what just happened. Yes, when you have a lockfile and run npm install, it changes. That’s because npm isn’t required to follow it. You could locally run npm ci, which does. However, assuming you semi-automatically update the lockfile regularly, what’s the difference? Have you ever not merged a patch that updates a lockfile to match npm install? Any issue captured by that would still be experienced by people using npm install, which is most people.

Pinning dependencies

Perhaps you have scars from a badly behaved dependency that broke compatibility in a semver-minor release. I know I do. It’s rare, but it happens. Lockfiles are an ineffective approach to pinning dependencies, though, as they aren’t applied in most cases, and get overwritten the very next time anyone runs npm install.

A more effective solution, even if you do utilize a lockfile, is to pin dependencies in package.json first.

I like to use the “overrides” key, to further separate these from my own dependencies.

npm audit

npm audit is great, mostly, and works regardless of whether you commit a lockfile.

Dependency update notifications

Perhaps you use GitHub Dependabot, or Wikimedia LibUp. Whether for security, or for other reasons, it’s useful to learn about available software updates, right? Yes! And, the great thing is, these work even better on package.json — without lockfile.

GitHub scans for CVEs in indirect dependencies. It scans package.json too, and knows about affected packages and their downstream dependants. By not checking in your lockfile, it will inform you if, and only if, a change to package.json is needed. In most cases, a CVE or other bug is fixed in a patch release, and your package.json (or the one of the intermediary package) has a caret or tilde version that expands automatically to the newer version. By definition, if the Dependabot only changed package-lock.json, then it didn’t need to be done^[3]. Whether you change the lockfile or not, anyone installing your project was already getting the update. The lockfile is ignored by npm-install, and isn’t part of your package. The lockfile merely describes what npm install last did.

Suppose your project uses eslint and @typescript-eslint/parser, which has an indirect dependency on micromatch. Then, a CVE emerges. The intermediary package uses a tilde or caret version, and the patch release is compatible and in-range. With a lockfile, you’d get notified and “have to” merge a patch to update your lockfile. Without a lockfile, this is a non-event as npm install was already installing said update. Okay, that one was easy.

Suppose the intermediary dependency pinned micromatch to an exact version (or maybe the fix was outside its semver range). To get this update, you’ll need to upgrade @typescript-eslint/parser. And you can, because GitHub Dependabot scans your package.json, notifying you of package versions you rely on that have insecure dependencies. By removing the lockfile, it now only notifies you when your own dependencies are affected and/or when you have to use a newer version of your dependencies to obtain the update.

Adding a lockfile in this scenario only serves to invite noise and churn over already-solved issues. In the event of malicious activity and compromised packages, the company behind npmjs.com (Microsoft/GitHub) deletes those releases from the registry. This isn’t what npm audit or lockfiles are for.

We all care about security. I care about security. But, be wary of performative security, which can cost you valuable code review time, CI resources, and support tickets (from users who mistakenly think you must update your lockfile to help them, when actually they need to update their own).

Except when deploying Node.js apps, a lockfile brings you nothing but lost oppertunities.