Joshua's Cheatsheets - Building an NPM Package - Notes, Tips, and Troubleshooting
Light
help

Note: You might also find my general notes on NPM and Node helpful.

Click to expand resource section

Picking a Module Pattern for your NPM Package

For understanding all the differences between different JS module patterns, I recommend checking out my separate cheatsheet on the topic.

For writing the source code of your package, if you are using a bundler / transpiler, you have a pretty endless amount of options (see below). If you are not bundling / transpiling your code, you should probably stick to hand-coded CommonJS or UMD for maximum compatibility across the most versions of Node.

For distributing your code, the general consensus is that your main entry point (ala main) should point to either CommonJS, or UMD (especially if trying to support cross-environment / isomorphic). This is for maximum compatibility; although ESM is now enabled without a flag in NodeJS, it is still labeled experimental in 2020.

More and more developers are moving to ESM / ES2015 / ES6 module style, which has a lot of benefits. If you want to take advantage of ESM, you can write your source code in it, and optionally ship ESM via NPM alongside a legacy CJS bundle.

Module Pattern - Source Code vs Distribution

The reason why I made the emphasis on distribution vs source code, is because if you are using a transpiler / bundler, then it can do the automatic work of taking source code that is written in a newer, but less compatible module pattern (such as ESM), and transpile it to a more widely accepted pattern (such as UMD). This is a huge advantage, as something like UMD is messy to write by hand and track in source code, but can be auto-generated by your build tool.

In fact, you can even bundle and distribute multiple versions of your code with different module patterns, all in the same package, to support users who want / need legacy support, as well as those that want the newest module pattern. That's right; you can distribute UMD and ESM in the same library, with the same package.json file.

You just need to use the module field to point to the ESM entry, while continuing to use the main field to point to the legacy UMD or CommonJS entry:

{
	"name": "my-cross-env-lib",
	"version": "0.0.1",
	"main": "dist/output.umd.js",
	"module": "dist/output.esm.js"
}

Rich Harris has a wonderful writeup about this, as part of the Rollup docs.

Another (new) relevant method is with NodeJS Conditional Exports. This section of the Node docs covers this strategy for delivering side-by-side ESM and CJS entries, as well as general tips on dealing with dual package delivery.

Module Pattern Continued - ESM in Source Code

As previously mentioned, at least in the year 2020, using CommonJS as the module pattern in your source code offers maximum compatibility with the most versions of NodeJS, and the least headache. However, if you are really itching to write your source code with ESM, you can do so, but ideally should deliver a pre-transpiled CJS version as the final version to support all users (alongside the ESM code for those that can use it).

Note: Even if you are already using a bundler, you might have to add another tool / step to transform the ESM (and or ES6 stuff); often you need to separately add transpiling as a step in-between source code and bundling. Or, your bundler might have an option to specify the use of a tool like babel in the input options / config directly.

A very common pattern is to keep all your ESM code in a subdirectory, such as /src. Then, as your build step, you transpile to /lib. So, to summarize, the common folder structure is:

  • /src

    • Contains your original code, with ESM (and / or other less compatible stuff)
    • Tracked in VC, optionally omitted in NPM upload
  • /lib

    • Contains the transpiled version of /src
    • Usually generated by babel, and via your build command
    • Probably don't want to track in VC, but should include in distribution
  • /dist

    • OPTIONAL: Contains minified / uglified / compressed version of /lib
    • If building a cross-environment (Node + Browser) package, this would probably contain the browser-specific final code (see my cross-env package guide for details).
    • Like /lib, usually don't want to track in VC, but should be included in distribution
    • The reason why this is usually generated by optimizing /lib and not /src, is because many compressors (such as uglifyjs) are not compatible with the newest JS features, so their input has to be transpiled anyways; feeding it /lib is saving an unnecessary duplicate transpiling

      • If your /src folder is already compatible with most modern versions of Node (or to where you feel comfortable) and compatible with your minifier / optimizer, then you could just minify directly from /src to /dist

You can choose to deliver both CJS alongside your original ESM code, or only the final CJS version. If you opt to only include the final CJS version, using the folder structure from above, you would simply exclude the /src directory from the NPM upload, and make absolutely sure the entry point is inside the /dist folder.

If you are delivering both at the same time, you would point package.main to the commonjs entry (e.g. dist/index.js) and package.module to the ESM entry point (e.g. src/index.js).

A benefit to delivering ESM is that it works with tree-shaking, and allows ESM consumers to avoid messy CJS interop syntax.

Transpiling

If you want to use bleeding-edge JS features, or other code that might be natively unsupported by most versions of Node but are supported by a transpiler, you can do so. The normal pattern for doing this is to keep all your code in a subdirectory (e.g. /src) and transpile to another directory (e.g. /dist). You then exclude /dist from version control, but include it in the NPM upload and set your entry point inside it.

My instructions here are very similar to my section on using ES Modules in a package's source code, so check that out if you are looking for a more in-depth explanation.

Distribution Paths and Module Resolution - The Root Directory Issue

Disclaimer: I'll be honest, this part of package development continues to confuse me and always seems convoluted and overly-complicated 😒. It took me considerable effort to gather the resources for this section, which should not have been the case.

In general, it seems like Node and NPM still have a big problem with nested source file resolution, even when package.main and/or package.exports is explicitly set to point to a nested target. I'm loosely referring to this as the "root directory" issue, but as you'll see in my "background" section, there have been multiple proposals to address this which have fallen under different names. This can actually be a really problematic and frustrating issue, and it makes me angry 😡 that so many guides seem to gloss over this, or leave the issue to bundling without discussing how it gets solved...

For example, assuming this structure for my-lib:

  • package.json
  • /src

    • /index.js
    • /validators.js
    • /constants (dir) - /index.js
  • /dist (auto-generated)

    • /index.d.ts
    • /index.js
    • /validators.js
    • /validators.d.ts
    • /constants (dir)

      • /index.js
      • /index.d.ts

If you have package.main point to dist/index.js, you would expect that in the consuming project, you could write validators = require('my-lib/validators)... but that is not the case. That will throw an error, unless you change it to require('my-lib/dist/validators') even though main is pointing inside dist already!

Background on Root Directory Issues with NodeJS

The answer to why this is the case is complicated, and mostly has to do with module resolution, and how the default in node is to resolve to the root of the package inside node_modules, where package.json is located. If you are curious, here are some relevant links that help explain this:

Here is the creator, (and founder + former CEO) of NPM, writing in 2013 about why this functionality was not added, and why he felt it should not be.

There have also been multiple discussions on either changing the default behavior, or adding a field to package.json to specify adding / modifying the default root directory.

Approaches for Working Around NodeJS's Root Directory Issue

Here is probably what you are looking for; how have other developers approached this issue? How can you workaround it? Read below:

Different Approaches

  • Map the subdirectory(s) via module.exports (only for versions of node supporting ESM or --experimental-modules flag)

    • This is basically the newest solution, and is gaining in adoption. However, this really only solves the issue in scenarios where both the library author and library consumer are using ES6 / ES Modules.
    • Example implementations:

  • Import and re-export everything from main entry (e.g. index)

  • Keep everything in project root

    • Pros: Keeps everything simple, requires less build tooling
    • Cons: Can make root of project messy, source files mixed with config files, etc.
    • It looks like this is how sindresorhus builds most of his (non-TS) libraries, for example, slugify
  • Copy everything to subdirectory, or send build output there (e.g. /dist), copy package.json to subdirectory, ensure proper paths in package.json, and run npm publish from subdirectory

  • Add files to the root that simple re-export from the subdirectory

    • This is similar to the "import and re-export everything from index.js" solution
    • If you want to expose my-lib/subdir/alpha.js, you would create a file in the root, my-lib/alpha.js, which simply re-exports: module.exports = require('./subdir/alpha.js)
    • Examples:

  • Keep package.json directly in subdirectory instead of root (or a different package.json file than root), and publish from there

    • Very similar to the "copy everything to subdirectory and publish there" approach, but the difference is that you keep package.json in the subdirectory to start with (as well as any other "root" level files)
    • In general, I think this is one of the worst approaches; it not conventional, is confusing to those browsing NPM vs those browsing actual source code, and seems error-prone
    • If you are leaving a package.json file in the root directory, make sure you have "private": true, so you don't accidentally publish the entire thing
    • Examples:

Relevant resources:

Creating Cross-Environment NPM Packages

I've put together a separate page for some summary information on how to craft and release a cross-environment (aka isomorphic, Node + Browser) package - you can find it here.

Exposing a CLI

You can specify that a specific file will receive CLI commands (instead of index.js), by using the bin field in your package.json.

If you want the command trigger (e.g. what someone types in their CLI to hit your package) to just be your package name, then bin can just be the path to the file:

{
	"bin": "cli.js"
}

But if you want to expose multiple commands, make bin an object with the commands as fields:

{
	"bin": {
		"myapp-foo": "cli-foo.js",
		"myapp-bar": "cli-bar.js"
	}
}

Make sure that all files exposed through bin start with #!/usr/bin/env node, otherwise they are not executed with the node executable!

Warning: If you are using npm link for local development, you might need to re-run npm link every time after modifying bin in order for the changes to be picked up globally. At least, I had to in my case.

See docs for details.

CLI Building Helpers

Hand-building a CLI can be labor intensive and error-prone, since you end up dealing with a lot of variable inputs, string parsing, and random ordered arguments. There are some reputable libraries to help with building a CLI:

Returning an Exit Code / Boolean to the CLI

You might have noticed that a lot of build and dev-ops stuff uses multiple commands chained together, often with logical operators and exit codes. For example:

package.json:

{
	"scripts": {
		"run": "npm run build && npm run serve",
		"build": "___",
		"serve": "___"
	}
}

With the above, if build throws an error (and/or returns anything other than 0 as the exit code), the npm run serve part does not run.

To return an exit code from your NodeJS script, simply use process.exit(errorCode). Just remember that 0 means success, and any other number represents an error!

Exporting Types

Important resources:

Local Package Testing and Loading

To try out your NPM package, you can link it to the project directory where you want to import the local version for trial; below are some options for how to accomplish this, as well as some tips:

  • BEST: yalc CLI!!! (🙏)

  • yarn link or npm link

    • Pros: Fast, since it uses symbolic links to point to source package location
    • Cons: Inconsistencies in how it works, stale data, issues with module resolution
  • install to package.json with local relative path (file: syntax)

    • Under dependencies, this might look something like "my-lib": "file:../../my-lib"
    • This works with both yarn add and npm install
  • Using custom webpack resolvers (resolve config)

Publishing

You can use npm publish --dry-run to get a preview of what files would be published. Or npm pack to generate a local tarball (.tgz).

If you want to exclude files from being distributed as part of your package, you have a few options:

  • Use .gitignore file

    • Downside: this also prevents files from showing up on Github / tracked in Git
    • Not a good solution for preventing large files that you still want tracked in Git
  • Use .npmignore file

    • NOT RECOMMENDED, for several reasons:

      • Conflicts with .gitignore file - NPM will not even look at that file if .npmignore exists.
      • This is dangerous because if you exclude a password file in your .gitignore but forget to duplicate the rule to your .npmignore file, it will be included in your package!
  • Use package.json -> "files": []

    • This is the best option
    • You can move all the files you want to include into a separate folder, and then only include that folder. For example:

      {
      	"files": [
      		"src/"
      	]
      }

Warning: Make sure you don't have any entries pointing at files/directories that are excluded from publishing! E.g., if you are only publishing ./dist, then make sure you don't accidentally have an entry like bin: "./src/cli.js"

Misc Tips

  • Be aware of built-in hooks, like install, as well as all pre / post hooks

  • Know about "scoped packages"

    • You can publish names with slashes in them, like @joshuatz/right-pad
    • These by default are scoped to private
    • use npm publish --access=public to publish public
    • Thanks to this post for the tip
  • Don't forget about npm link for local testing

    • In package folder npm link
    • In other project you want to test the package with npm link {packageName}
  • If you are making both a CLI and a regular module, don't forget to have both fields:

    • main (points to index.js, or whatever your main module file is)
    • bin (points to the cli.js, or whatever your CLI file(s) are)
  • Just because you see a field in someone elses package.json file, doesn't mean it is natively supported by the spec; it might be something special to be consumed by their build system

  • You can find a hosted JSON Schema file for package.json here
  • The package shx is great for cross-OS commands directly in package.scripts

    • e.g. "clean": "shx rm -rf dist && shx mkdir dist"

Troubleshooting

  • Error: cannot find module '___' when trying to import a nested source file from your published / built package

    • Are you trying to import or require a file with a slashed path that does not exactly match the published package? Node's module resolution maps the root folder to the package's root in node_modules, regardless of where package.main points to.
    • See my section on "The Root Directory Issue"
  • Error: File __.d.ts is not a module when trying to export / consume a declaration file that you generated (TypeScript)

    • Make sure you didn't combine module: 'commonjs' with outFile; these are incompatible (use outDir instead)

      • There is a proposal open for the bundling of declaration files - #4433
      • Relevant S/O: #58440177
Markdown Source Last Updated:
Thu Jul 30 2020 04:48:20 GMT+0000 (Coordinated Universal Time)
Markdown Source Created:
Fri Jul 24 2020 01:14:53 GMT+0000 (Coordinated Universal Time)
© 2020 Joshua Tzucker, Built with Gatsby