# Cache flags

**URL:** https://discuss.bitrise.io/t/cache-flags/405
**Category:** Steps
**Tags:** react-native, build-cache, ionic, contrib-this-feature, nodejs
**Created:** [January 23, 2017, 5:22pm UTC](https://discuss.bitrise.io/t/cache-flags/405 "2017-01-23T17:22:18Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![redant](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/redant/32/130_2.png) [@redant](https://discuss.bitrise.io/u/redant)
#### Post date: [January 23, 2017, 5:22pm UTC](https://discuss.bitrise.io/t/cache-flags/405/1 "2017-01-23T17:22:18Z")

</div>

Will be good to have the possibility to have some flags from the cache addons.  
I imagine a workflow like this:

On cache:pull I specify the same configuration of cache:push, where I say which lock files use:

```
node_modules -> package.json
bower_components -> bower.json

```

After the pull I will get some environment variables like:

```
BITRISE_CACHE_NODE_MODULES
BITRISE_CACHE_BOWER_COMPONENTS

```

each of these variables contains a simple 1 or 0, where a 1 means “something is changed” and a 0 is “nothing is changed”.  
So if I pull node\_modules, and nothing is changed I don’t need to run `npm install` and I can skip that expensive step.

```
if [BITRISE_CACHE_NODE_MODULES == 1]; then
  npm install
fi

```

---

<div class="post-metadata">

### Author: ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)
#### Post date: [January 23, 2017, 5:26pm UTC](https://discuss.bitrise.io/t/cache-flags/405/2 "2017-01-23T17:26:44Z")

</div>

Awesome idea, thanks for creating the #feature-requests @redant!

A technical note: I’d personally rather implement a mechanism like the one we did in the **Carthage** step, where the step itself can detect whether the caches are available and sufficient, and simply skip the “install/bootstrap” command when not needed.

In this case I’d add this logic to the **NPM** step ([GitHub - bitrise-steplib/steps-npm: Bitrise step for running npm commands](https://github.com/bitrise-steplib/steps-npm)), in case the command is “install” (similar to the “bootstrap” command in case of the `Carthage` step)

In short, the step would work the way mentioned by @redant:

> [@redant](#):
>
> So if I pull node\_modules, and nothing is changed I don’t need to run npm install and I can skip that expensive step.
> 
> if [BITRISE\_CACHE\_NODE\_MODULES == 1]; then  
> npm install  
> fi

But instead of checking an environment variable, it should inspect the required files to determine whether an `npm install` is required or not. E.g. in case of the Carthage step, “cache facts” are stored by the step into a file, including the resolve/lock file’s content or hash, as well as tool versions (e.g. `install` might be required if the NPM version changes since the cached version of the deps).

---

<div class="post-metadata">

### Author: ![redant](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/redant/32/130_2.png) [@redant](https://discuss.bitrise.io/u/redant)
#### Post date: [January 25, 2017, 4:42pm UTC](https://discuss.bitrise.io/t/cache-flags/405/3 "2017-01-25T16:42:03Z")

</div>

For the cached npm install there’s only one downside.  
When there’s a postinstall script.

If I choose to not run npm install because a valid node\_modules exists, I may want to run another script, like `npm run postinstall` that usually contains something like `bower install` or some action that I want to do using one or more packages I’ve previously installed (like gulp for example).

I think that a general cache step can be useful, maybe a “cache:if”

---

<div class="post-metadata">

### Author: ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)
#### Post date: [January 25, 2017, 4:54pm UTC](https://discuss.bitrise.io/t/cache-flags/405/4 "2017-01-25T16:54:33Z")

</div>

Great idea!

The problem with an environment variable like `BITRISE_CACHE_NODE_MODULES` is how you would define this variable, or what would define this variable?

The Build Cache can be configured to cache any directory. Maybe we should add one more “special item” to the format, like the cache indicator file (`/what/to/cache -> /only/if/this/file/changed`)? Something like

```auto
/what/to/cache -> /only/if/this/file/changed => ENV_TO_SET_IF_CACHED

```

?

In case of the Carthage step it does handle this internally, so you don’t have to configure anything else, but it only skips the command if the cache is available, it does not perform another / alternative command instead.

Maybe the `npm` step could do the same, with an optional “run this command if cache available:” input? That way you could set `install` as the default command, and e.g. `run postinstall` for the “if cache available” input.

WDYT @redant?

---

<div class="post-metadata">

### Author: ![redant](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/redant/32/130_2.png) [@redant](https://discuss.bitrise.io/u/redant)
#### Post date: [January 26, 2017, 5:41pm UTC](https://discuss.bitrise.io/t/cache-flags/405/5 "2017-01-26T17:41:54Z")

</div>

Well the idea was to use BITRISE\_CACHE\_${FOLDER\_NAME}

But with a syntax where you can specify the name of the variable is even better

---

<div class="post-metadata">

### Author: ![redant](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/redant/32/130_2.png) [@redant](https://discuss.bitrise.io/u/redant)
#### Post date: [January 31, 2017, 12:40pm UTC](https://discuss.bitrise.io/t/cache-flags/405/6 "2017-01-31T12:40:32Z")

</div>

I have made a script that will do nearly what `cache:if` is supposed to:

```
#!/bin/bash

# Automatic exit on error
set -e

# Run a cached NPM install
# Arguments
# 1: Directory of the package.json
# 2: Cache file
function runCachedNPMInstall {
    originalDir=$(pwd)
    cd $1

    # Create a unique object with the keys of dependencies and devDependencies of the package.json
    currentPackageJSONVariables=$(jq -cM '.dependencies + .devDependencies' package.json)

    # Read the dependencies from the cache
    cachedPackageJSONVariables=''
    if [-f "$2"]; then
       echo "cached npm file exists"
       cachedPackageJSONVariables=$(cat $2)
    fi

    # If any dependency is changed on package.json
    if ["${currentPackageJSONVariables}" != "${cachedPackageJSONVariables}"]; then
        # Install ignoring root and postinstall
        npm config set loglevel warn
        echo "Installing package.json deps"
        npm install --unsafe-perm --ignore-scripts
        echo "${currentPackageJSONVariables}" > $2
    else
        echo "Using package.json cache"
    fi

    cd ${originalDir}
}

runCachedNPMInstall "${BITRISE_SOURCE_DIR}" "${BITRISE_SOURCE_DIR}/cached.npm.json"

```

What the script does is simple:

1. Extract and merge the fields dependencies and devDependencies from the package.json file (this means that if you make a dependency as a devDependency the cache will not be invalidated)
2. Check with a file that comes from the cache if the “new” deps are equal to the “old” deps
3. In case are different run npm install in silent mode, and disabling postinstall scripts

For use this script on cache:push I put:

```
$BITRISE_SOURCE_DIR/node_modules -> $BITRISE_SOURCE_DIR/package.json
$BITRISE_SOURCE_DIR/cached.npm.json

```

The script requires jq that can be installed using:

```
DEBIAN_FRONTEND=noninteractive apt-get install -qq -y jq > /dev/null

```

I got a speed-up of 2 minutes (originally npm install required ~150 seconds):

```
+---+---------------------------------------------------------------+----------+
| ✓ | cache-pull@0.9.2 | 26 sec |
+---+---------------------------------------------------------------+----------+
| ✓ | Check cache and install deps | 1.79 sec |
+---+---------------------------------------------------------------+----------+
| ✓ | cache-push@0.9.4 | 3.19 sec |
+---+---------------------------------------------------------------+----------+

```

WDYT @viktorbenei

---

<div class="post-metadata">

### Author: ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)
#### Post date: [January 31, 2017, 1:57pm UTC](https://discuss.bitrise.io/t/cache-flags/405/7 "2017-01-31T13:57:10Z")

</div>

Awesome, I love it!

Note: instead of `DEBIAN_FRONTEND=noninteractive apt-get install -qq -y jq > /dev/null` you can declare `jq` as a dependency for a step: [http://devcenter.bitrise.io/tips-and-tricks/install-additional-tools/#advanced-option-use-deps-in-bitriseyml](http://devcenter.bitrise.io/tips-and-tricks/install-additional-tools/#advanced-option-use-deps-in-bitriseyml)

```auto
deps:
  brew:
  - name: jq
  apt_get:
  - name: jq

```

You can add `brew` as well, to make it work on Mac too 😉

---

<div class="post-metadata">

### Author: ![viktorbenei](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.bitrise.io/viktorbenei/32/18_2.png) [@viktorbenei](https://discuss.bitrise.io/u/viktorbenei)
#### Post date: [January 31, 2017, 1:58pm UTC](https://discuss.bitrise.io/t/cache-flags/405/8 "2017-01-31T13:58:16Z")

</div>

Btw this is quite similar to what the **Carthage** step does, but instead of getting the dependency list it uses the full “resolve” file as the cache indicator source, combined with Swift version.
