How to Cache the Source Code directory

How To Caching
1

Bitrise does not do automatic git/source code caching, unless you configure it for yourself. This is part of our Security policy too (that we don’t store your code and we never access it outside of the ephemeral build VMs): Redirecting…

You can of course configure the source to be cached, a very simple / minimal example:

workflows:
  primary:
    steps:
    - activate-ssh-key@3.1.1:
    - cache-pull@0.9.2: {}
    - git-clone@3.4.3:
        inputs:
        - reset_repository: 'Yes'
    - cache-push@0.9.4:
        inputs:
        - cache_paths: "$BITRISE_SOURCE_DIR"

​Basically you should add the Cache:Pull step before the Git Clone step, and a Cache:Push step after the Git Clone step.
If all you want to cache is the src dir then Cache:Push can be right after Git Clone, if you’ll want to cache other things then move it to be the last step in the workflow (or when all the things you want to cache are available).

So, once you added the Cache:Push and Cache:Pull steps:

  1. Select the Cache:Push step, and specify the Cache paths input of the step: $BITRISE_SOURCE_DIR
  2. That’s pretty much all what’s required, but it’s also a good idea to enable the Git Clone step’s “clear before checkout” option
    1. select the Git Clone step
    2. and search for the Reset repository option
    3. then set it to Yes.
    4. This will ensure that the step will clear out any previously checked out files before the new checkout (e.g. to remove files which got .gitignored, but were not in the previously cached state, and other similar edge cases).

If you have any questions just let us know! :wink:

2 Likes
2

Hi @viktorbenei,

Thanks for your post and sorry if my one is a silly questions but I’m newbie in Bitrise and for me it’s not clear how the Bitrise cache works. My questions are:

  1. In the example scripts reported previously in this page and in all the example I found on internet, when we call the cache-pull we don’t report any source/cache folder as input, but in cache push is required the cache_path, is the pull script pulling everything is cached?
  2. Is it possible to create several cache in the same script and in the same branch? For example one cache for node_modules, one for ruby gems, one for the cordova plugins… if it’s possible, is there an example how to do it? Can we keep it independents each other?
  3. Is it possible to define which folder we want to push/pull and where we store the cache?

Thanks in advance for your answers.

3

Hy there, sorry for the late reply

1, the cache pull will pull the cached things into the folder the previous step got them from (i don’t really understand how, it’s magic)

2, gems and cordova plugins are cached by default if you have the steps in, node_modules is not advised to be cached because it in most cases is filled with garbage dependencies and can cause errors.

3, you can define what to cache in the cache push step, we store it on our own servers

4

Hi, couple of questions here:

  1. You state that cache pull is before git clone step but documentation states something else: " You should not put the Bitrise.io Cache:Pull Step BEFORE the Git Clone Repository Step". A little bit confused here, can you explain which one is correct, or when to use one approach or the other?

  2. I tried saving the last commit sha in $BITRISE_CACHE_DIR and take it on the next build to use it for fastlane change log (release notes) like here. But $BITRISE_CACHE_DIR changes at each build so the file path is not found. Then tried the approach with $BITRISE_SOURCE_DIR like here but still the file is not found between builds even if Reset Repository is set to No. This are the logs:

    ==> Start
 ==> Switching to working directory: /Users/vagrant/git
 + file_path=/Users/vagrant/git/commit.txt
 + '[' '!' -f /Users/vagrant/git/commit.txt ']'
 + echo ' (!) File does not exist'
  (!) File does not exist
 + touch /Users/vagrant/git/commit.txt
 + echo 367ac9341ca07ba0234502446b2852274b0a7d9
 + envman add --key BITRISE_LAST_COMMIT_SHA --value 367ac9341ca07ba0234502446b2852274b0a7d9
 + exit 0
 ==> Script finished with exit code: 0

Not sure what i am missing here. Can you please help?

5

Hey there @mihaela.lorincz :wave:

Could you please send us the URL of the build and also enable Support Access on the Settings tab of the app (Enabling the Bitrise Support user for your app | Bitrise DevCenter), so that we may take a closer look? :slightly_smiling_face:

6

Hi @Roland-Bak . The support access is enabled and build link is: https://app.bitrise.io/build/470bbd8b38bdf6e3. Thanks for help

7

I’m still confused, should you put Cache:Pull before Git Clone or not?
These instructions contradict this documentation about caching.

8

Hi @GMosdall
Cache:pull step should after the Git clone step.
The cache is stored for the 7 days.

Thanks
Chaitanya

9

Can we get some clarification here, can Git LFS be cached using the Cache:pull step after the Git clone step? Does anyone have an example on how to cache Git LFS?

10

If anyone wonders in the future, adding .git/lfs to paths in the cache step worked for me.

11

Would you be so kind and share the steps of your bitrise.yml? It’s in Workflow editor, last tab, or committed to your repository. Hope I don’t need to reinvent the wheel.

I assume you configured it so it skips downloading LFS files if they are cached already, but if they are not cached or different, save it to cache, so next time they don’t need to be redownloaded.

Really, really Appreciate!