Multi-Package Repos with Lerna
Lerna is a multi-package repo tool similar to Yarn Workspaces. Many projects choose to use Lerna as the UI for interacting with their multi-package repo while Yarn Workspaces is used under the hood to handle linking packages together.
We've previously covered Yarn Workspaces for multi-package repo management on it's own.
If we take a look at our previous Yarn Workspaces setup, we
can see that our packages got hoisted and linked into a root
node_modules. The -> represents a
symlink which
means that when we try to access package-a through the
node_modules directory, we get re-directed to
../packages/package-a, where all of our files actually
live.
➜ tree ..├── node_modules│ ├── package-a -> ../packages/package-a│ └── package-b -> ../packages/package-b├── package.json├── packages│ ├── package-a│ │ ├── index.js│ │ ├── package.json│ │ └── yarn-error.log│ └── package-b│ ├── index.js│ └── package.json└── yarn.lock
Lets install lerna into our root package to prepare to
transfer from Yarn Workspaces to exclusively Lerna. (note:
we need -W because we've already defined workspaces and
Yarn tries to warn us that we're installing into the root.
yarn add lerna -W
Then we can initialize lerna's config.
➜ yarn lerna inityarn run v1.12.3$ /workspaces/node_modules/.bin/lerna initlerna notice cli v3.10.6lerna info Initializing Git repositorylerna info Updating package.jsonlerna info Creating lerna.jsonlerna info Creating packages directorylerna success Initialized Lerna files✨ Done in 0.86s.
With no arguments, the lerna.json config file that gets
spit out defaults to pointing at packages/*, just like our
Yarn Workspaces but in a dedicated file.
{"packages": ["packages/*"],"version": "0.0.0"}
If we run yarn lerna bootstrap now... nothing interesting
will happen because we haven't added a dependency between
our packages. Add a dependency on package-b in
package-a. We haven't published package-b so we'll use
the "any version" version *.
{"name": "package-a","version": "0.0.1","main": "index.js","author": "Chris Biscardi <chris@christopherbiscardi.com> (@chrisbiscardi)","license": "MIT","scripts": {"test": "echo package-a"},"dependencies": {"package-b": "*"}}
Now when we bootstrap our multi-package repo, we'll see the
packages linked in a slightly different way than our past
Yarn Workspaces attempt. The packages are linked into each
other rather than hoisted to the root node_modules by
default. This is one critical difference in the default
behavior of Yarn Workspaces vs Lerna.
.├── lerna.json├── package.json├── packages│ ├── package-a│ │ ├── index.js│ │ ├── node_modules│ │ │ └── package-b -> ../../package-b│ │ └── package.json│ └── package-b│ ├── index.js│ └── package.json└── yarn.lock
Unfortunately, for very large repos, doing this per-package
installation and linking is really slow. Luckily, we can
speed everything up by using Yarn Workspaces as the
underlying package dependency resolution approach. We'll add
two new keys to lerna.json: npmClient and
useWorkspaces. We can also remove the packages key
because Lerna will now look at our package.json
workspaces key instead.
{"version": "0.0.0","npmClient": "yarn","useWorkspaces": true}
Now running yarn lerna bootstrap will use Yarn Workspaces
and we'll end up with a very similar layout to our original
Yarn Workspaces based approach.
.├── lerna.json├── node_modules│ ├── package-a -> ../packages/package-a│ └── package-b -> ../packages/package-b├── package.json├── packages│ ├── package-a│ │ ├── index.js│ │ └── package.json│ └── package-b│ ├── index.js│ └── package.json└── yarn.lock
So that's great, but why would we go through all this effort to use Lerna if we're just going to end up back where we were with Yarn Workspaces on their own? The answer lies in the way Yarn deals with versioning and handles the publishing of packages.
Lets say, for example, that you wanted to publish a new
version of all of your changed packages on every merge to
master, but also didn't want that release to be the mainline
stable release. You can do this with Lerna by specifying a
canary release and using a special NPM tag (here we choose
ci, but this could be alpha or whatever you want).
lerna publish -y --canary --preid ci --npm-tag=ci
This command gets us versions that look like 0.3.5-ci.263
published to a ci tag on NPM. We can run this on every
merge to master, removing a dependency on maintainers to do
a release immediately after merging code. Any user of your
packages can now pull down the relevant ci version and
test it before it moves on to a stable release.
Lerna includes a set of features build on the concept of
figuring out what's changed since the last release such as
lerna changed. If we run it on our project with no
releases we see:
➜ yarn lerna changedyarn run v1.12.3$ /workspaces/node_modules/.bin/lerna changedlerna notice cli v3.10.6lerna info Looking for changed packages since initial commit.package-apackage-blerna success found 2 packages ready to publish✨ Done in 0.93s.
The full list of commands is found below, along with what they do.
lerna add <pkg> [globs..] Add a single dependency to matched packageslerna bootstrap Link local packages together and install remaining package dependencieslerna changed List local packages that have changed since the last tagged release [aliases: updated]lerna clean Remove the node_modules directory from all packageslerna create <name> [loc] Create a new lerna-managed packagelerna diff [pkgName] Diff all packages or a single package since the last releaselerna exec [cmd] [args..] Execute an arbitrary command in each packagelerna import <dir> Import a package into the monorepo with commit historylerna init Create a new Lerna repo or upgrade an existing repo to the current version of Lerna.lerna link Symlink together all packages that are dependencies of each otherlerna list List local packages [aliases: ls, la, ll]lerna publish [bump] Publish packages in the current project.lerna run <script> Run an npm script in each package that contains that scriptlerna version [bump] Bump version of packages changed since the last release.
Each command also has a set of filters or other options that
we can use to specifically target subsets of our packages.
lerna run for example, allows us to filter by scope
(package name) or filter so we can operate on all packages
that have changed since a specific ref.
➜ yarn lerna run --helpyarn run v1.12.3$ /Users/biscarch/tmp/workspaces/node_modules/.bin/lerna run --helplerna run <script>Run an npm script in each package that contains that scriptPositionals:script The npm script to run. Pass flags to send to the npm client after -- [string] [required]Command Options:--npm-client Executable used to run scripts (npm, yarn, pnpm, ...). [string] [default: npm]--stream Stream output with lines prefixed by package. [boolean]--parallel Run script with unlimited concurrency, streaming prefixed output. [boolean]--no-bail Continue running script despite non-zero exit in a given package. [boolean]--no-prefix Do not prefix streaming output. [boolean]Filter Options:--scope Include only packages with names matching the given glob. [string]--ignore Exclude packages with names matching the given glob. [string]--no-private Exclude packages with { "private": true } in their package.json. [boolean]--since Only include packages that have been updated since the specified [ref].If no ref is passed, it defaults to the most-recent tag. [string]--include-filtered-dependents Include all transitive dependents when running a commandregardless of --scope, --ignore, or --since. [boolean]--include-filtered-dependencies Include all transitive dependencies when running a commandregardless of --scope, --ignore, or --since. [boolean]Global Options:--loglevel What level of logs to report. [string] [default: info]--concurrency How many processes to use when lerna parallelizes tasks. [number] [default: 4]--reject-cycles Fail if a cycle is detected among dependencies. [boolean]--no-progress Disable progress bars. (Always off in CI) [boolean]--no-sort Do not sort packages topologically (dependencies before dependents). [boolean]--max-buffer Set max-buffer (in bytes) for subcommand execution [number]-h, --help Show help [boolean]-v, --version Show version number [boolean]Examples:lerna run build -- --silent # `npm run build --silent` in all packages with a build script✨ Done in 0.31s.
In this way, Lerna becomes a powerful front end on top of Yarn Workspaces that can be used to publish packages when they change or operate on subsets of packages locally when developing.
With the Yarn Workspaces based setup... we can still use the Yarn commands if we're used to them or find them easier.
➜ yarn workspace package-a testyarn workspace v1.12.3yarn run v1.12.3$ echo package-apackage-a✨ Done in 0.07s.✨ Done in 0.49s.