each of these variables contains a simple 1 or 0, where a 1 means “something is changed” and a 0 is “nothing is changed”.
So if I pull node_modules, and nothing is changed I don’t need to run npm install and I can skip that expensive step.
if [ BITRISE_CACHE_NODE_MODULES == 1 ]; then
npm install
fi
Awesome idea, thanks for creating the #feature-requests@redant!
A technical note: I’d personally rather implement a mechanism like the one we did in the Carthage step, where the step itself can detect whether the caches are available and sufficient, and simply skip the “install/bootstrap” command when not needed.
In short, the step would work the way mentioned by @redant:
But instead of checking an environment variable, it should inspect the required files to determine whether an npm install is required or not. E.g. in case of the Carthage step, “cache facts” are stored by the step into a file, including the resolve/lock file’s content or hash, as well as tool versions (e.g. install might be required if the NPM version changes since the cached version of the deps).
For the cached npm install there’s only one downside.
When there’s a postinstall script.
If I choose to not run npm install because a valid node_modules exists, I may want to run another script, like npm run postinstall that usually contains something like bower install or some action that I want to do using one or more packages I’ve previously installed (like gulp for example).
I think that a general cache step can be useful, maybe a “cache:if”
The problem with an environment variable like BITRISE_CACHE_NODE_MODULES is how you would define this variable, or what would define this variable?
The Build Cache can be configured to cache any directory. Maybe we should add one more “special item” to the format, like the cache indicator file (/what/to/cache -> /only/if/this/file/changed)? Something like
In case of the Carthage step it does handle this internally, so you don’t have to configure anything else, but it only skips the command if the cache is available, it does not perform another / alternative command instead.
Maybe the npm step could do the same, with an optional “run this command if cache available:” input? That way you could set install as the default command, and e.g. run postinstall for the “if cache available” input.
I have made a script that will do nearly what cache:if is supposed to:
#!/bin/bash
# Automatic exit on error
set -e
# Run a cached NPM install
# Arguments
# 1: Directory of the package.json
# 2: Cache file
function runCachedNPMInstall {
originalDir=$(pwd)
cd $1
# Create a unique object with the keys of dependencies and devDependencies of the package.json
currentPackageJSONVariables=$(jq -cM '.dependencies + .devDependencies' package.json)
# Read the dependencies from the cache
cachedPackageJSONVariables=''
if [ -f "$2" ]; then
echo "cached npm file exists"
cachedPackageJSONVariables=$(cat $2)
fi
# If any dependency is changed on package.json
if [ "${currentPackageJSONVariables}" != "${cachedPackageJSONVariables}" ]; then
# Install ignoring root and postinstall
npm config set loglevel warn
echo "Installing package.json deps"
npm install --unsafe-perm --ignore-scripts
echo "${currentPackageJSONVariables}" > $2
else
echo "Using package.json cache"
fi
cd ${originalDir}
}
runCachedNPMInstall "${BITRISE_SOURCE_DIR}" "${BITRISE_SOURCE_DIR}/cached.npm.json"
What the script does is simple:
Extract and merge the fields dependencies and devDependencies from the package.json file (this means that if you make a dependency as a devDependency the cache will not be invalidated)
Check with a file that comes from the cache if the “new” deps are equal to the “old” deps
In case are different run npm install in silent mode, and disabling postinstall scripts
Btw this is quite similar to what the Carthage step does, but instead of getting the dependency list it uses the full “resolve” file as the cache indicator source, combined with Swift version.