Concerns about Step Development Guidelines

#1

Ok, so I am developing a step for Bitrise to send build notifications to Zulip, and I find myself fighting against the Step Development Guidelines. First, it is strongly advised (I’d daresay required) in the Guidelines that one does not use Environment Variables at all (except the ones for your step obviously). My main issue with this is that the environment variables I’m using are set by Bitrise, either at the start of the build or as the build progresses, notably BITRISE_BUILD_STATUS, PR, BITRISE_BUILD_NUMBER, BITRISE_BUILD_URL, BITRISE_GIT_TAG, BITRISE_GIT_COMMIT, BITRISE_GIT_MESSAGE, etc. While I do understand the concerns about using environment variables that probably shouldn’t be tampered with or accidentally overwriting a critical variable, there has to be some consideration given to the fact that, in some cases, allowing the user to set the variables could lead to VERY unintended side effects, and there is no way for the developer of a step to fix the value so it cannot be changed (at least there is no documented way). Additionally, the guidelines stress, again on the verge or requiring, that you don’t use any dependencies that cannot be obtained from aptitude or brew. Again, this seems counter-intuitive to me. Firstly, the core Bitrise code does this; in almost every single one of the Bitrise files I looked at while trying to figure out Go (which was a lost cause, Go makes no sense to me), a library is called from Github; so how do you guys expect developers to do what you couldn’t do yourself? Secondly, I am using Python (I know gasp, shock, horror I didn’t use Go or Bash), and I need 1, exactly one dependency from pip: requests. Why, because even the Python developers themselves, don’t recommend using the built-in urllib, see here. I get that you guys really like Go, Bash and Ruby, but not everyone does. I really wanted to use Typescript, becuase that’s what I know best, but it’s so unbelievably difficult to get anything working here, that I settled for Python, which I’m also not super familiar with.

0 Likes

#2

Hello there, you should calm down a little :upside_down_face:
obviously you can use TypeScript if you prefer that,
and you can use env vars as inputs as much as you like, it is just preferred that you don’t change them, instead export other env vars if needed with the modded values.

0 Likes

#3

Yeah, sorry about that…it is very frustrating though because the documentation is very lacking, and conflicting at times. I am gonna stick to Python because it’s simple to get working and I am actually growing quite fond of Python. Go would probably grow on me as well, since it is syntactically somewhat similar to Python, but it just makes no sense to me. The pip dependency shouldn’t be an issue correct? I am requesting python from brew and python3 as well as python3-pip from apt-get, and then calling pip3 from a bash script before launching my Python script.

1 Like

#4

If it works, we are totally happy with it. :upside_down_face:

0 Likes

#5

If you include the dependencies in the repo like we do that’s perfectly fine. Every one of our Go projects includes the Go deps in the standard vendor dir, directly in the repo.

0 Likes

#6

Can you please share a bit more detail, so that we can fix those?

0 Likes

#7

Can you please share an example? I’m not sure if I fully understand what you mean by users accidentally overwriting critical variables.

0 Likes

#8

The recommendation is to not to use is implicitely, instead make that an explicit input. E.g.:

  - project_path: $BITRISE_PROJECT_PATH

(source: https://github.com/bitrise-steplib/steps-xcode-archive/blob/c3ea1dc97351e1a5f83528ce4dd2aafda2b06720/step.yml#L33 )

This way the user can set this to any path s/he likes but the default value is read from the env var. In contrast if you’d simply hardcode $BITRISE_PROJECT_PATH to be used in your code, this makes this an explicit dependency for that input on the specified env var, and if the user wants to s/he can change the input’s value to something else.

0 Likes

#9

Sorry for the delayed response, I’ve been a bit busy. I’m a full-time OTR Truck driver here in the US, so sometimes I lack time to work on this stuff. Anyhow, the environment variables that I access probably should not be reconfigurable by the user, like the build number and stuff. While I can understand why that would be helpful in certain situations, I could also see that flooding a steps issues. My methodology is if a user shouldn’t change something, like say the git commit ID or message, or the build number, or the pull request status, they shouldn’t be able to. I assume (not because I genuinely feel this way, but it reduces issues on the back side) that the user is dumb and should have as little control as possible to make the thing function. As for including the dependency in the repo itself, I don’t know if that’s possible with Python, and if it is I don’t know how, so I have switched back to Typescript because I know how to do it with NPM. As for the documentation, bear with me for a bit and I will get back with you on that one, once I get some more time. In reference to the overwriting of environment variables what I meant was I assume that you guys didn’t want developers to access and accidentally change the value of an environment variable or poorly code a step in such a way that a user that uses Bitrise could accidentally change the value of an environment variable. Say for example, I wrote a step that had an output that was literally BITRISE_BUILD_NUMBER and then it added “lol” onto the end of it (silly step, I know, just demonstrating a point), and I poorly coded my step so that if the user pointed it to, let’s say, BITRISE_BUILD_URL, my code wouldn’t handle it correctly, and it would overwrite the BUILD_URL variable (mind you, that would be really REALLY bad coding, but again demonstrating a point). That was what I was getting at.

0 Likes

#11

Came up with a better solution, I will just use an undocumented option that Bitrise itself uses, is_dont_change_value

In the end, I just ended up caving in and doing it the way specified. I doubt this step will even get downloaded anyway.

1 Like

#12

No problem at all, we definitely know what it means to be time constrained :stuck_out_tongue:

Our general guideline that seemed to work over the years is to make this possible, but don’t make it simple. If someone actually needs to override some parameters like the build number that should be possible, to provide enough flexibility. A use case for this is if you want to run Parallel Builds, splitting a long build into multiple smaller builds (https://blog.bitrise.io/start-multiple-builds-with-the-same-trigger). In this situation you might want to use the same Build Number across all of these builds.

I definitely see your point, but based on our experience over the years the best solution seems to be to make it possible, just don’t make it simple. This way those users who actually know what and why to change can do that, while users who don’t want to acquire that deeper level of knowledge don’t have to.

As always this is a trade-off, and if it makes sense in your situation we won’t stop you. We have a few steps which work this way, e.g. the ones dependent on certain Ruby gems or tools. What we usually do is we run the related dependency manager as part of the step (e.g. bundle install in case of Ruby).
The main trade offs here are:

  1. that this will take time every time the step runs, as it’ll have to download those dependencies
  2. and that if there’s an issue on the dependency manager system the step will fail

We had issues mainly with the 2nd point and that’s why our recommendation (a strong one, but still a recommendation) is to include dependencies in the repo, whenever you can. That solves both of these issues.

Sure! We’re always happy to get feedback and to improve our system, including the documentation, whenever we can :wink: We actually have a team dedicated for the devcenter maintenance and they are always glad to receive feedback or highlights of what could be explained better/in more depth.

When you share a step into the Step Library we do a review on it, to ensure quality and security of the steps available in the library. Of course you can also use your own step without sharing it, by specifying the step repo’s URL in the bitrise.yml ( https://devcenter.bitrise.io/bitrise-cli/steps/#step-referenceid-format ). If/when you decide to share it with the bitrise community via the Step Lib that has a mandatory review step; without that the step isn’t available for other users.

0 Likes

#13

That’s a perfectly fine solution, though keep in mind the is_dont_change_value flag only means that the input won’t be editable on the UI, but users can still specify it in bitrise.yml directly (after all that is an input). Related documention: https://devcenter.bitrise.io/bitrise-cli/step-inputs/

0 Likes