How to cache Gradle dependencies

viktorbenei · January 9, 2017, 6:44pm

Setup

Put the Cache:Pull step after the Git Clone step
Put the Cache:Push step to the very end of the Workflow

If you use v1.0 or newer of the Cache steps and the latest Gradle Runner step then that’s all you have to do.

If you use an older Cache or Gradle Runner step:

Select the Cache:Push step and specify the following paths for caching (Cache paths input):
```
$HOME/.gradle
./.gradle
```
Select the Cache:Push step and specify the following paths for update-check ignore (Ignore Paths from change check input):
```
$HOME/.gradle/caches/*.lock
$HOME/.gradle/caches/*.bin
./.gradle/*.lock
./.gradle/*.bin
```

Example workflow

example:
  steps:
  - git-clone: {}
  - cache-pull: {}
  - gradle-runner:
      inputs:
      - gradle_file: "$BITRISE_PROJECT_PATH"
      - gradle_task: "$GRADLE_TASK"
  - cache-push:
      inputs:
      - cache_paths: |-
          $HOME/.gradle
          ./.gradle
      - ignore_check_on_paths: |-
          $HOME/.gradle/caches/*.lock
          ./.gradle/*.lock
          ./.gradle/*.bin

For more information about the Bitrise Build Cache see the DevCenter.

fchristysen · September 28, 2017, 5:04am

Hi Viktor,
I encounter an issue with the Cache:Pull step, the step seems failed to download or extract the cache archive, it produce this log

2017/09/28 04:19:46 => > Downloading and extracting cache archive …
2017/09/28 04:19:46 [!] Unable to download or uncompress cache: failed to extract tar archive, output: gzip: stdin: not in gzip format
tar: Child died with signal 13
tar: Error is not recoverable: exiting now, error: exit status 2, retrying…
2017/09/28 04:20:33 => [DONE]
2017/09/28 04:20:33 => Took: 47.71805638s
2017/09/28 04:20:33 => Finished

I’ve checked the Cache:Push step from the previous build and it finished successfully.

viktorbenei · September 28, 2017, 10:44am

Hi @fchristysen,

Can you please create an issue report at #issues:build-issues, or via email or the onsite chat?

taku · February 17, 2018, 11:34am

Hi, im trying to cache gradle with this below. but it seems it’s not working.
it takes the same amount of time every build.

  - cache-push:
      inputs:
      - cache_paths: |-
          $HOME/.gradle
          ./.gradle
      - ignore_check_on_paths: |-
          $HOME/.gradle/caches/*.lock
          ./.gradle/*.lock
          ./.gradle/*.bin

especially i want to cache $HOME/.gradle, which contains local caches of gradle.

does this have something to do with this?
http://devcenter.bitrise.io/faq/why-my-build-takes-longer-on-bitrise-than-on-my-mac/

viktorbenei · February 23, 2018, 5:40pm

@taku Without more info it’s really hard to say anything useful.

I’d suggest you to upgrade all the related steps in the workflow, and if that wouldn’t help please create a #issues:build-issues report so that we’ll have enough info to debug it

taku · March 26, 2018, 11:45am

sorry, i figured it out. it was because PR. PR is not cashed, unless I set run_if: true

viktorbenei · March 26, 2018, 3:52pm

Indeed, the Cache:Push step is configured to not to push back cache in case of PR builds. In general we’d suggest you to keep that default behaviour, but of course if you have a use case which requires / benefits from pushing back from pull request builds the run_if: true option can be used

eric-grab · May 28, 2018, 8:07am

Hi @viktorbenei, does this eliminate cleanBuildCache or clean in Gradle build?
For example
Given

./graldew clean cleanBuildCache assembleDebug takes 20 mins for my app
./graldew clean assembleDebug takes 10+ mins for my app
./graldew cleanBuildCache assembleDebug takes 10+ mins for my app
./graldew assembleDebug takes <10 mins for my app
By using these cache-pull and cache-pull steps, what will my build time be?

bootstraponline · May 31, 2018, 1:02pm

You have to actually run a build to find out what the build time will be. I have noticed a good improvement by caching the gradle dependencies.

kawano_shohei · June 18, 2018, 10:43am

Hi @viktorbenei

The caching gradle system works fine on my Android project! Thanks!
I would like to know where I can find the logic of Bitrise for detecting when it can re-use the pulled cache or not; I assume it is somewhere in the Steps but I still can’t find it… sorry…

What I would like to do is to make sure NOT to re-use the cached data if certain project-specific file in the project(e.g. gradle file) is modified whenever changes are pushed.

Thanks in advance!

Shohei

godreikrisztian · June 28, 2018, 10:41am

Hi, thanks for the question.

I think you should add a script step right after the git-clone step.
This script should decide if the build should use cache or not.
You can use git diff command to determine which files changed. Use envman to export an environment variable which holds if cache is needed or not.

envman add --key USE_CACHE --value "true"

Use this env in the cache step’s run_if property (which controls if the step should run or not), like:

run_if: {{getenv “USE_CACHE” | eq “true”}} `

Let me know if you need any help to implement the script!

rafael.araujo · September 8, 2020, 12:28pm

Hi, there

I just added the PUSH stuff without any further confirmations and I think I still need to ask for Gradle cache. Is that right?
Does the cache work with a multi-module project?

Thank you!

bitce · October 9, 2020, 1:25pm

Hi @rafael.araujo! As long as you are using the Android Build step, Gadle depenencies should be cached automatically!

taso.dane · December 8, 2020, 5:14pm

Our build has around 6mins of Cache:Pull. That is a really long time. I’m not sure if adding 6 more minutes is worth it. I wonder if we are doing something wrong. What is the best way to check that?

Here is how our cache step looks like

_cache-build:
steps:
- cache-push@2:
inputs:
- is_debug_mode: ‘true’
- deploy-to-bitrise-io@1: {}

chaitanya.tanna · December 22, 2020, 1:32pm

Hi @taso.dane
Thank you for reaching out to Bitrise.
Can you please enable Support Access for your project and send us the build URL so that we can take a look at your WF and figure out why it is taking some time?

Thanks
Chaitanya

taso.dane · December 22, 2020, 2:04pm

Hi, I recently changed the caching configuration and deleted the old cache and it was better. Since then, ~1 week has passed and the cache is again became 10GB which is really big. That is the reason why the builds becomes slower.

The support access is enabled. Here is our URL: https://app.bitrise.io/app/2c2a6af38f8da808#/settings

chaitanya.tanna · December 23, 2020, 5:29pm

Hi @taso.dane
Thank you for the update.
I am checking this with the team.
In the meantime, can you manually clear the cache?

Thanks
Chaitanya

zeinab_abdelmawla · February 16, 2021, 8:12am

any updates regrading this issue

thi.nguyen · February 24, 2021, 3:22pm

Hi Taso!

It looks like like you have multiple steps caching and it could be leading to the larger caches.

Can you set the level of cache to none on those 3 android steps

Android Lint
Android Unit Test
Android Build for UI Testing

Thank you,

Thi

taso.dane · February 24, 2021, 3:54pm

Hi, thanks for the reply. I remembered this recently and I was going to also ping.

We have the step named _cache-build and it is included into only 1 workflow actually. Here is how it looks like in bitrise.yml. I’m not sure what I can change here. Any feedback will be appreciated.

workflows:
develop:
after_run:
- _setup-build-environment
- _run-lint
- _run-unit-tests
- _update-unit-tests-coverage
- _run-automation-tests-critical-path
- _deploy-dev-app-to-appcenter
- _deploy-to-bitrise-io
- _cache-build
- _slack_build_result

Topic		Replies	Views
How to cache Maven dependencies Caching build-cache	0	3375	January 9, 2017
Gradle Runner v1.6.0 Changelog step	0	708	May 26, 2017
Gradle Runner v1.9.8 Changelog	1	454	August 29, 2021
Cache pull failing for Android build Question & Answer	2	929	March 31, 2020
Cache-pull "failed to extract tar archive" Builds android	8	1958	October 13, 2017

How to cache Gradle dependencies

Setup

Example workflow

Related Topics