Key-based caching: skip step when cache is available

Feature Requests
1

I’m exploring the new key-based caching functionality and I wonder if could further speed up our pipeline when we skip certain steps completely when the cache is available.

Let’s assume I’ve a 100% reproducible build step, and I want to cache the result based on a key that is derived from the build inputs (like checksum of a file). In this case it should be save to always use the result from the cache for the same key when it exists, without re-running the build step.

Let’s use the cocoapods usecase as an example (although I’m sure if the 100% reproducibility applies here).

        steps:
        - restore-cocoapods-cache@1: {}
        - cocoapods-install@2: {}
        - save-cocoapods-cache@1: {}

When there is no change in the lock file and I run this workflow a second time, where have a cache hit, it will still run cocoapods-install and save-cocoapods-cache although it is unlikely that it will yield a different result. So it would be awesome to have an easy way to determine if there was a cache hit by restore-cocoapods-cache. In this case we could use run_if to skip cocoapods-install and save-cocoapods-cache.

        steps:
        - restore-cocoapods-cache@1: {}
        - cocoapods-install@2:
            run_if: {{enveq "CACHE_RESULT" "MISS"}}
        - save-cocoapods-cache@1:
           run_if: {{enveq "CACHE_RESULT" "MISS"}}

The only missing part would be the additional output variable CACHE_RESULT from restore-cocoapods-cache and restore-cache.

Does this sound reasonable?

2

The Save Cache step will not upload anything if there are no changes to what is being cached so that step should complete in a few seconds.

The Cocoapods Install step still needs to be run however since it does more than just install the Pods.

3

Yes, that was probably a bad example. That’s why I also mentioned the more generic restore-cache and save-cache steps. In our case we have a plain script step that makes some expensive calculations without any side-effects. So when it is cached, we don’t need to run the calculation again.

4

You could create an indicator file and use its presence to set an env var if present when the cache is loaded

5

Yes, that’s what I’m currently doing. And it works fine so far. I was just suggesting a feature that would make it easier. If restore-cache could already tell me if there was a cache hit or not, I wouldn’t have to write my own script step which figures that out.

1 Like
6

Thanks for the additional context and feature request. We have logged this with the team

1 Like
7

Hello @mathias.schreck :wave:

The latest version of restore-cocoapods-cache (as well as the other cache steps) now exposes an output called BITRISE_CACHE_HIT. More details about this output in the README.

We hope this will be useful for your workflow :slight_smile:

8

Hi @oliver.falvai,

thank you so much. I’ve already changed our pipeline to use BITRISE_CACHE_HIT and it works great so far.
The only thing I’ve noticed, is that the restore-cache step doesn’t print its output variables. This would be nice to have because it makes debugging the pipeline easier. Currently the output of the step looks like this:

+------------------------------------------------------------------------------+
| (0) Restore Cache (Beta)                                                     |
+------------------------------------------------------------------------------+
| id: restore-cache                                                            |
| version: 1.1.0                                                               |
| collection: https://github.com/bitrise-io/bitrise-steplib.git                |
| toolkit: go                                                                  |
| time: 2023-01-16T13:01:28Z                                                   |
+------------------------------------------------------------------------------+
|                                                                              |
 * [OK] Step dependency (zstd) installed, available.
Input:
- verbose: false
- key: {{ .Arch }}-cocoapods-cache-{{ checksum "Podfile.lock" }}
Evaluating key template: {{ .Arch }}-cocoapods-cache-{{ checksum "Podfile.lock" }}
Cache key: arm64-cocoapods-cache-xxx
Downloading archive...
Exact hit for first key
Archive size: 42MB
Downloaded archive in 1s
Restoring archive...
Restored archive in 2s
|                                                                              |
+---+---------------------------------------------------------------+----------+
| ✓ | Restore Cache (Beta)                                          | 5.39 sec |
+---+---------------------------------------------------------------+----------+