How to have a separate cache per workflow?

build-cache
#1

What I want is to have two workflows each with a different cache.

Using printenv on a macOS box I found $BITRISE_CACHE_DIR and $BITRISE_CACHE_API_URL, also some scripts use $BITRISE_CACHE_INFO_PATH but it’s not clear whether those variables can be changed to point to separate caches.

This is how I’ve setup bitrise.yml in my repository:

format_version: "4"

default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git

project_type: other

#trigger_map:
#  - push_branch: '*'
#    workflow: primary
#  - pull_request_source_branch: '*'
#    workflow: primary

app:
  envs:
    - SCONS_CACHE: $HOME/.scons_cache
    - SCONS_CACHE_LIMIT: 1024
    - CC: clang
    - CXX: clang++
    - GODOT_TARGET: osx
    - TOOLS: yes

workflows:
#  primary:
#    steps:
#      - activate-ssh-key@3.1.1:
#          run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
#      - git-clone@3.6.2: {}
#      - script@1.1.5:
#          title: Do anything with Script step
#      - deploy-to-bitrise-io@1.3.9: {}

  build:
    steps:
      - cache-pull:
          inputs:
            - cache_paths: |-
                $SCONS_CACHE
      - script@1.1.5:
          title: Build script
          inputs:
            - content: |-
                echo $BITRISE_CACHE_DIR
                echo $BITRISE_CACHE_API_URL
                echo $BITRISE_CACHE_INFO_PATH
                mkdir -p $SCONS_CACHE
                misc/bitrise/scons-local-osx.sh
                scons -j 2 CC=$CC CXX=$CXX platform=$GODOT_TARGET TOOLS=$TOOLS verbose=yes progress=no;
      - cache-push:
          inputs:
            - cache_paths: |-
                $SCONS_CACHE
0 Likes

#2

Hi @rraallvv,

Thanks for the question! :slight_smile:

What I want is to have two workflows each with a different cache.

Unfortunately that’s not possible. Cache storage is allocated for branches, not workflows. It’s automatic, every branch when pushes a cache to the API will create its own cache (will not overwrite another branch’s cache), but it’s always allocated by branch.

Related docs: http://devcenter.bitrise.io/caching/about-caching/

If you’d have any questions just let us know! :slight_smile:

0 Likes

#3

That said, checking your example bitrise.yml in your question, are you sure you need/want per workflow cache? It should definitely work with just per-branch cache, at least in the example I don’t see any reason why it might not.

If you have a specific error please copy paste the related build’s bitrise.io URL and we’ll check it ASAP! :slight_smile:

P.S.: in the example you have cache_paths in the “Cache:Pull” step too; that step does not have that input, the cache:pull step simply moves the caches to the original location, the one which was defined in the Cache:Push step.

In short you control the cache config (what to cache) in the Cache:Push step, which stores these infos in the cache archive; the Cache:Pull step simply reads that meta-info and uncompresses the cache archive to the original locations/paths.

0 Likes

#4

I was hoping maybe I could have separate caches for macOS and iOS for the same branch. I’ve seen parallel builds are not possible out of the box but I’ll try the workaround described by @koral. Regarding the separate caches issue, I’ll have to use separate folders for each build then. I’m going to remove the input in cache-pull. Thanks a lot.

1 Like

#5

Indeed, not yet available as a built in feature, but definitely planned for the near future. Please vote & comment at Parallel Builds: Seperate stacks for seperate workflows + trigger multiple workflows (builds) at the same time to bump its priority (although at this point it definitely is planned for early next year).

If you have any questions just let us know! :slight_smile:

0 Likes

#6

@viktorbenei thanks for the info.

1 Like

#7

Hey,

I would like to bump this thread with more sophisticated usage pattern:

I am using job with multiple workflows. The entry workflow is triggering few others via build-router-start. That is one per ios and one per android. I would love to be able to have separate caches per workflow. For instance: I want to cache Pods for ios workflow but I don’t care about pods in android flow.

Any way to achieve this today? Perhaps something changed since previous answer?

0 Likes

#8

Hi @max!

There weren’t any changes to the main feature, the link posted above by @viktorbenei is up to date, so you can check out the current detailed behaviour here: https://devcenter.bitrise.io/caching/about-caching/

But in short cache does not work per workflow, it does per branch, so you could only achieve this if you were to separate your ios and android code you build per branch too.

0 Likes