From @viktorbenei on Thu Jun 16 2016 07:27:46 GMT-0400 (EDT)
Related UserVoice link: https://bitrise.uservoice.com/forums/235233-general/suggestions/10481289-allow-bitrise-yml-to-be-pulled-from-project-source
Summary and proposal
We know that version control for the configuration is quite important, but having the configuration stored separately has its own advantages, especially when you want to quickly iterate, or if you want to allow a UI (e.g. Workflow Editor on Bitrise.io) to edit it, without granting direct access to the repository (it should not have the possibility to change the code repository, as you can never know what might go wrong).
An alternative solution we’re thinking about is to give you direct
git access to the configuration (git repo hosted on bitrise.io only for the config). This way you could
git clone the configuration, change it,
git push it back and the new config will be used. This way the config would be under version control, but it would still be possible to quickly iterate on the config without changing the source repository, and would allow the Web UI/API to have access only to this configuration git repository, and not to the main code repository.
You could imagine the Workflow Editor to be similar to how you can edit & commit on GitHub.com - the change you make on the Web UI will still generate a commit, just like if you’d clone the repository, do your change and then push it.
You could define your
trigger_map in the configuration (repository) and let bitrise.io to read that before triggering the build.
If the trigger map have to be read from the code repository it means that bitrise.io have to git clone it before it could decide whether the build should be started at all. From a security & reliability point of view it would be way better to not to clone the whole code repository just to read the
Storing the configuration in a bitrise.io hosted git repository would mean that:
- the reliability of retrieving the configuration would only depend on the bitrise.io git repository, and not on a third party where the actual code is stored
- more secure isolation, as this would allow bitrise.io to stick to our current security policy: we never access your code/repository directly, only on the Build Virtual Machines and in the way you define it (in your bitrise.yml configuration).
Keeping the configuration in a separate repository is also useful if you want to use the same code repository multiple times. If the bitrise.yml config is defined in the code repository then it’s more complicated to define multiple configurations, using the same code repository.
A point against doing this as a bitrise provided git repository: the ability to be able to send Pull Requests for configuration changes is awesome, and implementing a PR review system is not an easy task.
On the other hand, getting the configuration from the code repository directly, in a service independent way (meaning: we can’t just use the GitHub etc. API to get the configuration, without actually git clone-ing the repository) would require us to store & poll the code repository, which would be even worse than the lack of configuration PRs…
Let us know what you think about this proposal!
(PROs and CONs) Store the config in the source code repository
- Version control
- Relatively simple setup (if you consider writing a YML manually - without the help of the UI - simple)
- You can use your source code hosting server’s Pull Request tools to do Pull Request reviews on the config change
- Not possible to implement workflow mapping (
trigger_map) without accessing the code before the build would start. Right now the source code repository is only accessed / cloned inside the build virtual machine.
- Impossible to integrate it with the Workflow Editor (web UI); that would require access to the source code repository (security concern), and it couldn’t use e.g. the GitHub API either, as we have to support more than just a “few” selected Git hosting providers.
- Possible “workaround” note (thanks to @kennethlynne): A possible workaround we would be happy with is with read only UI, so that if one wants to save a change to the bitrise file it would let us download the new config, or better yet: Create a PR for the change.
- Not trivial to decide what to do for a branch specific trigger. E.g. if the main config has a pattern for
feature/*, and you create a Pull Request on
feature/awhich also changes the configuration of the related workflow, should we use the config from the PR, or should we use the existing / main configuration (stored e.g. on the
- Doesn’t allow quick iteration of the configuration, you have to change your code (commit into your repository) to test a new configuration
- Requires access to the code outside of the build Virtual Machine, to be able to decide whether the build should start at all.
- Reliability: we’d depend on the source code hosting service, to be able to access the configuration.
(PROs and CONs) Store the config in a separate repository
- Version control
- High level overview, single config which covers what should happen on any branch / for any trigger
- Can be integrated with the Workflow Editor (web UI), through an API bitrise.io would provide
- Allows iteration of the configuration without changing the source code / commiting into the source code repository. Also allows you to use one version/revision of the code with a specific revision of the configuration. E.g. you originally build code X with configuration revision Y; if the configuration is stored in another repository you can rebuild the same build with another revision (e.g. latest) of the configuration.
- More granular access control - you can decide who should have access to the configuration; it’s not tied to the source code access of a user.
- Reliability: there’s no dependency on a 3rd party service to be able to read (and write) the configuration. This is especially important if you consider that someone might accidentally revoke the required access rights to the source code repository, so that bitrise.io won’t be able to access it anymore at all, until the issue is fixed.
- Allows projects where you don’t have a source code repository at all, and/or where you can’t store the configuration in the source code repository for one reason or another.
- Possibility of sharing configuration between projects; can be implemented in a way that more than one project can use the same config repository.
- Possible to use a workaround if you want to store the config in your source code repository, the way you can do it today (see the
Workaround in the meantimesection below).
- Requires additional setup, cloning the source code repository will not include the configuration automatically, you have to also clone the config repo - although that can be added as a submodule, for easier access.
- No easy way to provide Pull Request review feature for the configuration, unless we implement one ourself
Workaround in the meantime
Definitely planned, in the meantime you can do this by using a configuration on bitrise.io which clones your code and then runs bitrise (the CLI) again, using the bitrise.yml from your repository.
You can find a sample configuration at: https://github.com/bitrise-io/bitrise-yml-collection/blob/master/general/use-bitrise-yml-from-repo/bitrise.yml – you can just import this on bitrise.io (in the Workflow Editor – select bitrise.yml on the left side and just copy-paste the content & hit save).
This’ll add a bit of verbosity to your log, but this way you can define how your repository should be retrieved (if you need more than just a simple git-clone), and then it’ll use the bitrise.yml from your repository.
Of course, if you use this solution (or any solution where the code have to be cloned first) bitrise will only be able to check your trigger_map once the code is available, so every code push will trigger a build, because at that time the
trigger_map is simply not yet available for bitrise to check.
Copied from original issue: https://github.com/bitrise-io/bitrise.io/issues/41