Precisions on how caching works

#1

Hi there :raised_hand:

I’m playing with Cache Push / Pull steps and I must admit I’m not sure to understand how update indicators are working.

Typically, let’s say I have a package.json file used to fill a node_modules directory.
For this, I defined following path configuration :

./node_modules/ -> ./package.json

I would like to :

  • Cache node_modules directory content
  • Update its content (with PUSH step) in the cache everytime package.json is altered
  • Load its content (with PULL step) from the cache only if my local package.json checksum differs from the one I used to PUSH node_module content.

During my tests, it appeared that the PULL step is systematically retrieving the cache content, regardless of if package.json has changed or not since my latest PUSH to the cache.
(it’s as if update indicators were used for PUSHes only, and never for PULLs)

Am I wrong in my assumptions ?

0 Likes

#2

Hello @frederic.camblor :wave:

You are correct, the Cache:Pull step loads whatever in the cache every time it runs, regardless of any updates. If it did not the virtual machine would have no cache to utilize at all, since every build starts with a clean environment and is destroyed after the end of every build.

You CAN, however, make the Cache:Pull step run conditionally, if you prefer: https://devcenter.bitrise.io/steps-and-workflows/disable-a-step-by-condition/

This way you could perhaps create an env var in a script step which checks if there are any differences in the package.json since the last build (which you can reach via an API call: https://api-docs.bitrise.io/), and set up a condition to only run the Cache:Pull step when there is one.

Hope this helps! :slight_smile:

0 Likes

#3

Thanks :slight_smile:

The idea would be to store package.json “state” (its checksum for example) into a build “resulting” variable, which may be accessible through the API ?

I’m not sure if build “resulting” variable are available through /apps/{app-slug}/builds/{build-slug} API
Maybe consider package.json as a generated artefact and retrieve it through /apps/{app-slug}/builds/{build-slug}/artifacts + /apps/{app-slug}/builds/{build-slug}/artifacts/{artifact-slug} ?

0 Likes