Workflow chaining and parameters

Question & Answer
1

Hi,

I’ve been looking into the possibility of storing bitrise.yaml in my repository and refactoring it to use chaining, making it far more maintainable.

One of the main questions to come out of this investigation is this: is it possible to define parameters a workflow requires? And if so, is it advisable (not too hacky)?

I can’t see anything in the docs about formally defining parameters, so I’m guessing it would all have to hinge rather informally on environment variables. i.e. the “calling” workflow sets an env variable that the chained workflow uses in some way.

Can anyone confirm my thinking here?

Thanks

1 Like
2

Hi @ch_kent,
thanks for the question!

Have you find our Use bitrise.yml from repository guide?

Regarding to:

I’m not sure if i understand correctly your question, but let’s say you have a workflow in your repository, which you want to run using the bitrise.io website ( either starting a build manually from the website or triggering a build through git events ), this can be done as described in the mentioned guide.

Let’s say your workflow in your repository contains steps, which have secret inputs, like Slack step’s Slack Webhook URL, you may not want to store this input’s value in your repository.

In this situation, first you set an env var as value for this input, for example SLACK_WEBHOOK_URL in your bitrise.yml in your repository. Then you define this env var in your ‘router config (bitrise.yml)’ on bitrise.io ( Workflow Editor/Env Vars or Workflow Editor/Secrets ).

This way your configs are:

bitrise.yml in your repository:

format_version: "4"
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git

...

workflows:
  primnary:
    steps:
    ...
    - slack:
        inputs:
        ...
        - webhook_url: $SLACK_WEBHOOK_URL
    ...

---
bitrise.yml on bitrise.io:

format_version: "4"
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git

...

app:
  envs:
  - SLACK_WEBHOOK_URL: https://hooks.slack.com/services/ABC/def

workflows:
  _run_from_repo:
    steps:
    - activate-ssh-key:
        run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
    - git-clone: {}
    - script:
        title: continue from repo
        inputs:
        - content: |-
            #!/bin/bash
            set -ex
            bitrise run "${BITRISE_TRIGGERED_WORKFLOW_ID}"
  ...

This way if a build started through the bitrise.io website the ‘router workflow’ will be performed and the build will be fulfilled with the SLACK_WEBHOOK_URL environment variable, so your Slack step in your repository will get its webhook_url input’s value from bitrise.io.

Also this config in your repo works well if you run it on your local machine, just define your secret env vars in a .bitrise.secrets.yml file (should be placed next to the bitrise.yml in your repo) and gitignore this file.

3

Thanks @godreikrisztian. This wasn’t quite what I was asking, but I have decided to migrate my bitrise config into my repository. So as I flesh this out, I may come up with a good example and post it here.

4

@godreikrisztian OK, here’s a good example…

Our configuration has several workflows, all of which want to set up the environment for later parts of the build. To achieve that, they currently all have something like this:

- script@1.1.4:
	 title: Configure Environment
	 inputs:
	 - content: |-
		 #!/bin/bash
		 # fail if any commands fails
		 set -e
		 # debug log
		 set -x

		 envman add --key PRODUCTION --value true
		 envman add --key RAYGUN_INTEGRATION --value true
		 envman add --key IOS_CONFIGURATION --value AppStore
		 envman add --key IOS_BUNDLE_IDENTIFIER --value com.blabla
		 envman add --key DEPLOY_COPY --value true
		 envman add --key TAG --value true

What I would like to do is define this as a utility workflow that is used by all the main workflows. But the problem is, each main workflow wants to set different values for each of the keys, so I end up with something like:

_configureEnvironment
	steps:
	- script
		inputs:
		- content: |-
			 #!/bin/bash
			 # fail if any commands fails
			 set -e
			 # debug log
			 set -x

			 envman add --key PRODUCTION --value $PRODUCTION
			 envman add --key RAYGUN_INTEGRATION --value $RAYGUN_INTEGRATION
			 envman add --key IOS_CONFIGURATION --value $IOS_CONFIGURATION
			 envman add --key IOS_BUNDLE_IDENTIFIER --value $IOS_BUNDLE_IDENTIFIER
			 envman add --key DEPLOY_COPY --value $DEPLOY_COPY
			 envman add --key TAG --value $TAG

Now my main workflows needs to be defined something like this:

- first
	envs:
	- PRODUCTION: true
	- RAYGUN_INTEGRATION: true
	- IOS_CONFIGURATION: AppStore
	- IOS_BUNDLE_IDENTIFIER: com.blabla
	- DEPLOY_COPY: true
	- TAG: true
	steps:
	- _configureEnvironment

- second
	envs:
	- PRODUCTION: false
	- RAYGUN_INTEGRATION: true
	- IOS_CONFIGURATION: AdHoc
	- IOS_BUNDLE_IDENTIFIER: com.blabla
	- DEPLOY_COPY: true
	- TAG: false
	steps:
	- _configureEnvironment

- third
	envs:
	- PRODUCTION: false
	- RAYGUN_INTEGRATION: false
	- IOS_CONFIGURATION: AdHoc
	- IOS_BUNDLE_IDENTIFIER: com.blabla
	- DEPLOY_COPY: true
	- TAG: false
	steps:
	- _configureEnvironment

In this case, it kinda works, but it’s not at all formal (what if the main workflow doesn’t set one of the env variables?). Moreover, this approach completely breaks down in other circumstances. For example, notifying Slack, which all my workflows do several times. All that differs between each Slack notification is the message and icon, so I want a Slack utility workflow that accepts only those parameters. But how can the primary workflows pass through different values for those parameters each time they run the utility workflow?

Thanks

5

Hi @ch_kent ,

to define default values for you environment variable, use the app level environments:

format_version: "2"
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git

app:
  envs:
  - APK_FILE_EXCLUDE_FILTER: "*-unaligned.apk"

workflows:
...

Four our gradle runner step, we have a similar configuration to yours. It has a common workflow (_common), which is defined as a utility workflow, and the main workflows are for branching the configuration:

workflows:
  test-multiple-tasks:
    envs:
    - SAMPLE_APP_GIT_CLONE_URL: https://github.com/bitrise-samples/sample-apps-android-sdk22.git
    - GRADLE_FILE: ./build.gradle
    - GRADLE_TASK: assembleDebug assembleRelease
    - GRADLEW_PATH: ./gradlew
    - APK_FILE_EXCLUDE_FILTER: "*-unaligned.apk"
    after_run:
    - _common
  
  test-subdir:
    envs:
    - SAMPLE_APP_GIT_CLONE_URL: https://github.com/bitrise-samples/sample-apps-android-sdk22-subdir.git
    - GRADLE_FILE: ./src/build.gradle
    - GRADLE_TASK: assembleDebug
    - GRADLEW_PATH: ./src/gradlew
    - APK_FILE_EXCLUDE_FILTER: "*-unaligned.apk"
    after_run:
    - _common

  test-no-gradlew-error:
    envs:
    - SAMPLE_APP_GIT_CLONE_URL: https://github.com/bitrise-samples/android-sdk22-no-gradlew.git
    - GRADLE_FILE: ./build.gradle
    - GRADLE_TASK: assembleDebug
    - GRADLEW_PATH: ""
    - APK_FILE_EXCLUDE_FILTER: "*-unaligned.apk"
    after_run:
    - _common

  _common:
...

You can use workflow ‘as a step’ with the workflow’s before_run and after_run property.

Your _configureEnvironment utility workflow is may just an example, but if not: you do not need that workflow, since it is the same as you would define workflow level environments:

workflows:
  test-multiple-tasks:
    envs:
    - SAMPLE_APP_GIT_CLONE_URL: https://github.com/bitrise-samples/sample-apps-android-sdk22.git
    - GRADLE_FILE: ./build.gradle
    - GRADLE_TASK: assembleDebug assembleRelease
    - GRADLEW_PATH: ./gradlew
    - APK_FILE_EXCLUDE_FILTER: "*-unaligned.apk"

Please let me know if i misunderstood your question, but currently this is the best way to run a workflow with different configurations.

6

Thanks @godreikrisztian. That much makes sense, but you didn’t address the main problematic example I gave at the end. Namely, what if I have a utility workflow for sending messages to Slack and I need to call it multiple times with different parameters from the same workflow?

For example, imagine I send a message to Slack when the APK is produced, then another when the IPA is produced. Each message needs to have different text and icon, so the utility workflow would take those as “parameters”. But how does the calling workflow pass different values for those parameters if it can only define its environment variables once?

Thanks

7

Hi @ch_kent,
So your requirement would be to define/override envs for utilty workflows, like this:

format_version: "2"
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git

workflows:
  wf_a:
    after_run:
    - common_slack
      envs:
      - MESSAGE: message_1
      - ICON_URL: icon_1
    - common_slack
      envs:
      - MESSAGE: message_2
      - ICON_URL: icon_2
    
  wf_b:
    after_run:
    - common_slack
      envs:
      - MESSAGE: message_3
      - ICON_URL: icon_3
    - common_slack
      envs:
      - MESSAGE: message_4
      - ICON_URL: icon_4
    
  _common_slack:
    envs:
    - MESSAGE:
    - ICON_URL:
    steps:
    ...
    - slack:
        inputs:
        - message: $MESSAGE
        - icon_url: $ICON_URL
    ...

Currently you can not solve this with utility workflows.

You may manage to define your slack massages in your common workflow (not in separated utility workflow for the slack messages) and define different env vars for the slack messages inputs, like this:

format_version: "2"
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git

workflows:
  wf_a:
    envs:
    - MESSAGE_1: message_1
    - ICON_URL_1: icon_1
    - MESSAGE_2: message_2
    - ICON_URL_2: icon_2
    after_run:
    - _common
    
  wf_b:
    envs:
    - MESSAGE_1: message_3
    - ICON_URL_1: icon_3
    - MESSAGE_2: message_4
    - ICON_URL_2: icon_4
    after_run:
    - _common
    
  _common:
    steps:
    ...
    - gradle_runner:
    ...
    - slack:
        inputs:
        - message: $MESSAGE_1
        - icon_url: $ICON_URL_1
    ...
    - xcode-archive:
    ...
    - slack:
        inputs:
        - message: $MESSAGE_2
        - icon_url: $ICON_URL_2
1 Like