Arrays in step outputs

Is there any convention or recommended way how to deal with cases when output need to be an array?
For example in official gradle-runner step there is an BITRISE_APK_PATH_LIST output which seems to be just file paths separated by pipe char |.

So when one of the paths contain | character (eg. app flavor name contains it) it becomes non-trivial to distinguish pipes in filenames from separators.
Isnā€™t it better to use for example a YAML/JSON array there?

Hey @koral! :wink:

Definitely would be, but you canā€™t do that with environment variables unfortunately. Env Vars are simple strings, no other data type is supported as an environment variable as far as I know.

Of course you can serialize data into string, e.g. serialize these infos as JSON and store that in an env var, then read and deserialize it in another step. If you have your own steps this is definitely possible, but that makes handling of these variables quite a bit more complex, e.g. handling JSON in Bash is not trivial at all unfortunately.

Why we chose the pipe char is because pipe should not be in the path. Pipes in shells are used for connecting commands, and AFAIK itā€™s considered as a really bad practice to include it in a fileā€™s name (due to the special meaning the char has in most shells).

I think the best solution might be to add a second output which would include the data in a JSON serialized form. That way simple steps which are only interested in the path list can be written in Bash, while the JSON output could include more info (e.g. mapping of paths) but would require a bit more complex logic to parse that output (json decoding).

WDYT @koral & @godreikrisztian?

Yeah, thatā€™s true. They are plain strings at low level: getenv

Maybe JSON as whole isnā€™t bash-friendly, but arrays only seem to be simple, for example with jq:

$ JSON_ARRAY='["foo|\"", ".", "|", "bar"]'
$ BASH_ARRAY=($(echo $JSON_ARRAY | jq '.[]' -r))
$ echo ${BASH_ARRAY[0]}
foo|"
$ echo ${BASH_ARRAY[1]}
.
$ echo ${BASH_ARRAY[2]}
|
$ echo ${#BASH_ARRAY[*]}
4

OK, understand.

Sounds good.

Just curious, are there any other official steps which use arrays as input and/or output?

1 Like

Sure!

Output: https://github.com/bitrise-io/steps-gradle-runner/blob/master/step.yml#L147

Input: https://github.com/bitrise-steplib/steps-google-play-deploy/blob/master/step.yml#L70

It seems that second one will fail if there is a pipe in APK filename: https://github.com/bitrise-steplib/steps-google-play-deploy/blob/master/main.go#L163-L165

Yes it will. How did you generate an APK file name like that?

I explained it here:

Here is the source code: https://github.com/DroidsOnRoids/android-bitrise-test/blob/master/lib/build.gradle#L23
and bitrise build having this ā€œissueā€ https://www.bitrise.io/build/a7f6486fff96fd23

1 Like

Ahh, so itā€™s the name of your flavor which includes the pipe char, not a ā€œdefaultā€ android thing.

Weā€™ll try to think about a better solution (I like the idea of a ā€œrichā€ output encoded into JSON more and more). In the meantime Iā€™d suggest you to use a different separator for your file name, if possible.

[quote=ā€œviktorbenei, post:8, topic:2200ā€]
In the meantime Iā€™d suggest you to use a different separator for your file name, if possible.
[/quote]:slight_smile:

This is not a real project but only playground to test our bitrise steps.
I wanted to consume BITRISE_APK_PATH_LIST in another step and realized that pipes may cause problems so I created this topic.

Ahh, got it :wink:

Sure, edge cases have to be covered and thatā€™s part of why we have this forum & this sub category :wink:

The thing here is that thereā€™s pretty much no char which is not allowed in a filename on a modern Unix system (AFAIK the only one is /), so we had to choose one which is unlikely to be in a path or any other value. Of course itā€™s impossible to select one which works for every case, pipe seems to be a ā€œsensible defaultā€ as a separator (but itā€™s not hardcoded, just a suggestion to use |, every step is free to choose any separator if required).