Key-based caching: skip step when cache is available

I’m exploring the new key-based caching functionality and I wonder if could further speed up our pipeline when we skip certain steps completely when the cache is available.

Let’s assume I’ve a 100% reproducible build step, and I want to cache the result based on a key that is derived from the build inputs (like checksum of a file). In this case it should be save to always use the result from the cache for the same key when it exists, without re-running the build step.

Let’s use the cocoapods usecase as an example (although I’m sure if the 100% reproducibility applies here).

        - restore-cocoapods-cache@1: {}
        - cocoapods-install@2: {}
        - save-cocoapods-cache@1: {}

When there is no change in the lock file and I run this workflow a second time, where have a cache hit, it will still run cocoapods-install and save-cocoapods-cache although it is unlikely that it will yield a different result. So it would be awesome to have an easy way to determine if there was a cache hit by restore-cocoapods-cache. In this case we could use run_if to skip cocoapods-install and save-cocoapods-cache.

        - restore-cocoapods-cache@1: {}
        - cocoapods-install@2:
            run_if: {{enveq "CACHE_RESULT" "MISS"}}
        - save-cocoapods-cache@1:
           run_if: {{enveq "CACHE_RESULT" "MISS"}}

The only missing part would be the additional output variable CACHE_RESULT from restore-cocoapods-cache and restore-cache.

Does this sound reasonable?

The Save Cache step will not upload anything if there are no changes to what is being cached so that step should complete in a few seconds.

The Cocoapods Install step still needs to be run however since it does more than just install the Pods.

Yes, that was probably a bad example. That’s why I also mentioned the more generic restore-cache and save-cache steps. In our case we have a plain script step that makes some expensive calculations without any side-effects. So when it is cached, we don’t need to run the calculation again.

You could create an indicator file and use its presence to set an env var if present when the cache is loaded

Yes, that’s what I’m currently doing. And it works fine so far. I was just suggesting a feature that would make it easier. If restore-cache could already tell me if there was a cache hit or not, I wouldn’t have to write my own script step which figures that out.

1 Like

Thanks for the additional context and feature request. We have logged this with the team

1 Like

Hello @mathias.schreck :wave:

The latest version of restore-cocoapods-cache (as well as the other cache steps) now exposes an output called BITRISE_CACHE_HIT. More details about this output in the README.

We hope this will be useful for your workflow :slight_smile:

Hi @oliver.falvai,

thank you so much. I’ve already changed our pipeline to use BITRISE_CACHE_HIT and it works great so far.
The only thing I’ve noticed, is that the restore-cache step doesn’t print its output variables. This would be nice to have because it makes debugging the pipeline easier. Currently the output of the step looks like this:

| (0) Restore Cache (Beta)                                                     |
| id: restore-cache                                                            |
| version: 1.1.0                                                               |
| collection:                |
| toolkit: go                                                                  |
| time: 2023-01-16T13:01:28Z                                                   |
|                                                                              |
 * [OK] Step dependency (zstd) installed, available.
- verbose: false
- key: {{ .Arch }}-cocoapods-cache-{{ checksum "Podfile.lock" }}
Evaluating key template: {{ .Arch }}-cocoapods-cache-{{ checksum "Podfile.lock" }}
Cache key: arm64-cocoapods-cache-xxx
Downloading archive...
Exact hit for first key
Archive size: 42MB
Downloaded archive in 1s
Restoring archive...
Restored archive in 2s
|                                                                              |
| ✓ | Restore Cache (Beta)                                          | 5.39 sec |