Cache contains way too much things

Archive Tooling Issues
1

Hi all,

I’m using the cache-push step to cache some Gradle and Android related stuff. However, my cache file(s) are around 6GB now and this does not seem to be right.

So I downloaded a cache sample and look at the metadata cache-info.json file and noticed that lots of files are being in the cache will I did not request them. There is also what looks like duplicated entries. Maybe I’m not understanding the cache file right, maybe I’ve misconfigured something, maybe there is a bug?

I would request your help to determine what’s wrong and how I can fix it.

So, here my cache-push step config:

    - cache-push@1.1.0:
        title: Push cache
        inputs:
        - cache_paths: |-
            $HOME/.android/build-cache
            $HOME/.gradle
            ./.gradle
        - ignore_check_on_paths: |-
            $HOME/.gradle/caches/*.lock
            ./.gradle/*.lock
            ./.gradle/*.bin

And here a sample of the cache-info.json file:

{
  "fingerprint": "93c87448d0c73d5e56b136a0858c91f50a31797d",
  "cache_contents": [
    {
      "destination_path": "/root/.gradle",
      "relative_path_in_archive": "c-0"
    },
    {
      "destination_path": ".gradle",
      "relative_path_in_archive": "c-1"
    },
    {
      "destination_path": "/root/.gradle",
      "relative_path_in_archive": "c-2"
    },
    {
      "destination_path": "/root/.m2",
      "relative_path_in_archive": "c-3"
    },
    ...
    "/bitrise/src/wear/build/generated/source/r/viper/debug/com/google/android/gms/tasks/R.java": {
      "fingerprint_source": "[/bitrise/src/wear/build/generated/source/r/viper/debug/com/google/android/gms/tasks/R.java]-[248B]-[0x644]-[sha1:41b0002be23442b34186da69915072edf09b40cd]"
    },
    "/bitrise/src/wear/build/generated/source/r/viper/debug/com/google/android/gms/wearable/R.java": {
      "fingerprint_source": "[/bitrise/src/wear/build/generated/source/r/viper/debug/com/google/android/gms/wearable/R.java]-[251B]-[0x644]-[sha1:d5718afae1245c2c24c2e2d6bec70ace6b8fb731]"
    },
   ...

And it goes like that forever. I don’t understand why /bitrise/src/app/build would be added to the archive? I thought only the paths specified would be cached?

I looked at the source code and it seems it inspect cache_paths plus bitrise_cache_include_paths but the first is suppose to be what I defined in cache_paths input and the other one should be empty. So I don’t see the problem.

Any help appreciated.

1 Like
2

Hey @mvachon !

I see your point here. The thing is the same path is included in the cache multiple times is a bug definitely. From the other side bitrise_cache_include_paths is for the built-in cache method, which should collect cacheable paths generated from the build. So basically you won’t need to configure manually anything.

I’ve put this issue to our internal issue tracker, hopefully it will be fixed by the next week or so.

For a quick workaround you can do two things:

  • use the previous version of the step, that ignores bitrise_cache_include_paths
  • simply clear BITRISE_CACHE_INCLUDE_PATHS and BITRISE_CACHE_EXCLUDE_PATHS env. vars. in for example a script step.

Thank you for the report, it’s a very good catch!
I will ping you instantly as soon as the issue is resolved!

1 Like
3

Just to be clear, @tamaspapik was referring to v0.9.4 of the Cache:Push step as the old version, that was the last version without the built in BITRISE_CACHE_.. handling. Releases · bitrise-steplib/steps-cache-push · GitHub

1 Like
4

The issue is fixed in the new version now(1.1.1). And also in the Gradle Runner step version(1.7.1) a new input is added, so you can chose which type of cache you need:

Please let me know if you encounter any issues. :rocket:

1 Like