SPFx - an overview of node_modules, package.json and other node concepts

N.B. this is a companion post to Avoiding dependency issues in SharePoint Framework (SPFx) development

Any SharePoint Framework (SPFx) solution that does something useful will use external JavaScript libraries – whether bigger frameworks such as React or Angular, or smaller, more focused libraries such as LoDash, moment.js or something else. Since SPFx is based on a node.js-based web stack, the best way to integrate such libraries into your solution is usually to add them from npm, the internet-based package manager for JavaScript. Npm is effectively an equivalent of NuGet in the .NET world. Once you’re set up with npm, a developer simply runs npm install jquery [or whatever] and the source code for the package will be added to your application’s files. As is normal with node development, the library is added under the “node_modules” subfolder, and it can now be referenced in your code.

The role of the node_modules folder

So the node_modules folder stores all your dependencies. But it also stores any dependencies of those, and so on and so on. So, it’s a tree structure which can be quite deep. An app which uses jQuery and React for example, could look like this:

In this example, jQuery has a dependency on “cache-swap”, which also has its own dependencies in its node_modules folder. And so on..

Modules which are actually used by your code are distributed with your app, so everything works at run time. The bundling mechanism built into the SharePoint Framework tooling takes care of this using webpack.

Ensuring dependencies are tracked with package.json

If the --save flag is also specified when adding a package with npm install, an entry is written into the package.json file in the application’s directory. This goes into the “dependencies” section of the file, and will look something like this:

"jquery": "^3.1.1"

Here’s an example of full package.json file:

Notice the caret (^) in that version number above. I’ll talk about tilde and caret dependencies later. But the purpose of that –save flag is is to ensure that the dependency is recorded. The node_modules folder itself is typically *not* checked-in to source control (in the same way all NuGet packages would not be), partly because it is large and unwieldy. Instead, each developer has a local copy on their machine, and the shared package.json file plays a critical role in ensuring all devs have the same files. When another developer subsequently obtains the files from source control, running the npm install or npm update commands will ensure all dependencies are restored on his/her machine, based on what’s stored in the package.json file.

As I detail in Avoiding dependency issues in SharePoint Framework development, if the developer forgets to track the dependency (with the --save flag or similar), then other devs are likely to be missing the module on their machines and the code will not work for them – they’ll get a missing module error. However, even if developers do always remember to specify the --save flag, you can run into problems in dev because npm uses caret dependencies by default, and these are a form of “floating version” dependency. I won’t discuss the pitfall further here, since we’re all about the fundamental concepts in this post – but related to all this is the whole topic of semantic versioning (known as semver).

A brief summary of semantic versioning

Semver refers to the different kinds of version numbers which can be used by npm packages. The semver page in the npm docs is recommended reading, but I’d say the core things you need to understand are:

• The 3 part version number format i.e. MAJOR.MINOR.PATCH, and the rules for when each number should be updated – see http://semver.org
• The different forms of version number used in package.json

Version number forms include:

Type	Example	What it does
Caret dependency	^3.1.1	Allows any version on the same major version, i.e. anything below 4.0.0
Tilde dependency	~3.1.1	Allows any version on the same minor version, i.e. anything below 3.2.0

However, my table above is a simplification because npm deals with things differently if there is a zero in the version number. Additionally, npm supports other forms too such as pre-release versions like “1.0.0-alpha.1” and tags. I recommend spending time in the npm docs, but there are LOTS of simplified posts on the internet too if you prefer another format e.g. https://nodesource.com/blog/semver-tilde-and-caret

Setting npm defaults to get consistency across devs

Elsewhere we’ve talked about parameters to pass to npm commands, such as the --save or--save-exact flags used with npm install. However, a useful approach with a dev team can be to set defaults across the team so that each developer does not need to remember a certain flag each time a command is run. This can be done in a couple of ways:

• Setting environment variables on each machine
• Using a .npmrc file – at the project, user or global level

I like the idea of using the .npmrc file at the user level (e.g. C:\users\chris\.npmrc on my machine), and ensuring all team members have this in place. By creating/editing this file, I can specify for example that all dependencies should be saved into package.json with *exact* version numbers, rather than caret or tilde dependencies. I can do this by adding the following to my file:

save = true
save-exact = true

Now, when any dev installs a package, the following differences to the default behaviour will occur:

• The dependency will automatically be added to package.json, even if the dev accidentally forgets the --save flag
• The dependency will be added with an exact version number, not a caret version number

This is useful, as devs won’t accidentally fall into some of the pitfalls that can come about from the npm defaults.

Also check out the save-prefix config flag as an option to override caret dependencies (e.g. to change to tilde dependencies).

Summary

Hopefully this is some useful background information to concepts underneath the SharePoint Framework. It can seem a whole new world in some respects, but at the same time I don’t think you have to be a complete expert in npm, modules, webpack etc. to be productive. A little extra digging around sub-topics like these is recommended though I think. Have fun!