Thursday 23 July 2015

Should I put 'project.lock.json' in source control?

TL;DR: No

Why not?

If you depend on specific versions of dependencies, then the lock file contains the same information as project.json, but in an easier way for dnx to consume. As long as the specific versions you depend directly on are the same, dnu will always restore the same versions.

If you have floating dependencies, then you expect the dependencies to change frequently, and any package restore can create noise in source control whenever it picks up new versions.

Longer reasoning

A small categorization of dependencies

Consider each of your dependencies as one of the following:
- stable
- fast moving

A stable dependency is one you consider to be fully featured, bug-free, ready to use and depend on. Newtonsoft.Json might be one, the latest stable release of RavenDB would be another, or an internal library that’s been in use for years and sees new versions once or twice a year.
You want to ensure you’re always working with the version you tested on, and it’s ok to explicitly update that dependency when you need functionality from a new version, or a bug is fixed.
You do not expect to have to update this dependency frequently.
You target a specific version of such dependencies, like 1.2.3, or 4.0.0.

A fast moving dependency is one that you want to track the latest developments, either because it’s still under rapid development and you want to consume always the new functionalities and get the latest bugfixes, you want to know when it breaks your code, it’s being developed at the same time as your project, or it’s just another project that you develop in tandem and it is split for logical reasons, deployment lifecicles or reuse in other projects.
If you’re trying out the latest versions of ASP.NET 5, you might consider beta-6 to be a fast moving dependency, and want to track all latest developments instead of having to manually check and update versions.
In this case you target ranged versions, like 1.0.0-beta6-*.

Project.json and project.lock.json

The project.json file includes the dependencies for your project, besides other information like your project version, commands available for dnx. It then gets transformed into a project.lock.json, which contains the whole dependency tree, resolved into specific versions.

Project.json

Your dependencies are defined on the project.json file.

You can have ranged versions:

    "Microsoft.AspNet.Mvc": "6.0.0-beta6-*",

And specific versions:

    "Newtonsoft.Json": "7.0.1"

These dependencies are used by dnu restore to generate or update the project.lock.json file, which contains all the transitive dependencies of your project, resolved to the specific versions to be used.

Project.lock.json

This file is mainly used by dnx to load the required references for your project, instead of having to recalculate all the dependencies each time the application runs.

Locking a project.lock.file

When restoring packages with dnu, you can opt to lock the project.lock.json file with dnu restore --lock. This means that whatever versions dnu picks up are written into the lock file, and future restore executions will use the same versions, if they continue to satisfy the dependencies on project.json.

What happens is that project.lock.json will contain the locked property set to true, instead of the default false.

If the versions from the lock file do not satisfy the dependencies from project.json, because they changed in the meantime, then the lock file gets rebuilt.

Locking a project.lock.json file is only useful if you depend on ranged versions of a package/project, and it does nothing if you only depend on specific versions.

How are versions picked for a specific project.json

Dependency resolution is described here, but for the purpose of this post there are two main points:

  • By default, versions are considered to be “at least X”. So "Newtonsoft.Json": "7.0.1" means that at least you need 7.0.1.
  • It will pick the earliest version that satisfies all dependencies. So if you depend on A 1.0 and B 1.0, and A 1.0 depends on B 2.0, it will pick B 2.0.

The result is that, for a project.json file that only contains specific versions, subsequent restores will always use the same versions. Hence the first point of section Why not?.

For ranged versions, dnu restore searches for the latest version of the dependency, and the rest of the resolution works as usual. This means that yes, a dnu restore done today can pick up later versions of a dnu restore done yesterday, which is point two of Why not?.

Options

Now that we know what are the main moving parts, here are the three ways a version gets resolved for a dependency:

  • It is specified on project.json (no wild cards)
  • It is wildcarded on project.json, but locked on project.lock.json
  • It is wildcarded on project.json, and unlocked on project.lock.json

The first case is used for stable dependencies. Project.lock.json is never changed unless you change project.json.

The last case is used for fast moving dependencies. Whenever there is a new version of such dependencies, Project.lock.json can change even if nothing else changes.

The middle case is a bit of an odd duck when thinking about stable/fast moving,but it is used when you want to track a fast moving dependency, but for now want to work with a version that you know doesn’t break your code. This is useful in large teams, where the team considers a dependency to be fast moving, but some developers want to stabilize it for a while.

Scenarios

Be notified of breaking changes on upstream projects

If you’re working on a project that depends on other projects, you can use a ranged dependency to always pick up the latest version. That way, you can have a build server checking new versions for problems, since it will always pick up new versions.

Locally, you can lock project.lock.json during a sprint or a feature to avoid getting sidetracked by upstream problems. Since the lock file isn’t commited, you can still get warnings from the build server, and handle them before merging back your code.

Branch to test a bugfix for a library

Consider you found a bug on version 1.42.0 of a library called XBuggy. You can create a branch in source control to test workarounds and fixes, and want to pick up new versions of XBuggy to test as they come up.

Instead of having to specifically update on each new version, you can depend on XBuggy 1.42.*. This means that on each restore, you can pick up new versions of the library.

When the bug is fixed, you then set the dependency to the version 1.42.3 that fixed the bug and merge it back to master.

New functionality that requires new versions of another project

If you have two separate projects that need to be developed in tandem for a given functionality, it is useful to use ranged versions while they’re in development.

The ASP.NET 5 scenario

Looking at the github repos for ASP.NET 5, we see that most of the projects use ranged versions to get early warnings of breaking changes. This enables them to quickly move up new versions, with the option of locking versions for short periods.

Side notes

One scenario where adding project.lock.json to source control is useful is if you want to run dnx without a dnu restore first, since dnx needs the lock file to run (at the time of this post). However, I’m not sure if this would ever crop up outside of my imagination.