Unable to run custom step

Development
1

I’ve tried to build a step to download files from S3. It works well when run locally with bitrise run test but it doesn’t when run on bitrise.io. Or better, it does run, produces no output and doesn’t fail the build.
I’m a bit puzzled: the step’s output stops after bitrise installs the dependencies.

The step is available here: https://github.com/FutureWorkshops/bitrise-step-s3-download.git
I’ve basically copied bitrire’s upload script and tweaked it to download instead.

1 Like
2

That’s because of the is_skippable: true flag - https://github.com/FutureWorkshops/bitrise-step-s3-download/blob/master/step.yml#L17 (This flag is used for steps like Slack notification, where if that step fails you don’t want to mark the build as failed if everything else went OK)

How did you run this step? How did you reference it in your bitrise.yml config?

3

Could you please copy paste an example, with the log including the step’s header and footer?

4 
- git::https://github.com/FutureWorkshops/bitrise-step-s3-download.git@master:
    title: Download keychain
    inputs:
    - aws_access_key: "$AWS_ACCESS_KEY_ID"
    - aws_secret_access_key: "$AWS_SECRET_ACCESS_KEY"
    - s3_bucket: my.keychain.bucket
    - s3_filepath: "$KEYCHAIN_NAME"

And the output is:

Cloning into '/tmp/bitrise734915742/step_src'...
+------------------------------------------------------------------------------+
| (1) Download keychain (take 2)                                               |
+------------------------------------------------------------------------------+
| id: https://github.com/FutureWorkshops/bitrise-step-s3-download.git          |
| version: master                                                              |
| collection: git                                                              |
| toolkit: bash                                                                |
| time: 2017-03-21T14:55:11Z                                                   |
+------------------------------------------------------------------------------+
|                                                                              |
INFO[14:55:11] Start installing (git) with apt-get          
INFO[14:55:11]  * [OK] Step dependency (git) installed, available. 
INFO[14:55:11] Start installing (wget) with apt-get         
INFO[14:55:11]  * [OK] Step dependency (wget) installed, available. 
INFO[14:55:11] Start installing (awscli) with apt-get       
INFO[14:55:11] (awscli) isn't installed, installing...

And that’s the last line.

5

Can you drop in a Script step before this step with this content:

#!/bin/bash
set -ex
brew install awscli

and let us know how it goes? (It seems that for some reason the installation of awscli doesn’t finish. That said, if you’re downloading, you shouldn’t even need awscli, which is declared as a dependency at https://github.com/FutureWorkshops/bitrise-step-s3-download/blob/master/step.yml#L22)

6

Without the dependency it seems to be working again. It would be nice to have that as a bitrise managed dependency so I can easily deploy it for both Mac OS and linux.
You need awscli if the link is not public and you need to auth with a key/secret pair don’t you?

7

Did you try it with the Script step (brew install awscli)?

Ahh, you’re right, if it’s a private file you definitely have to use auth to get it, preferably through awscli.

8

Yes I meant: if I do brew install awscli instead of expressing the dependency it works. But I’d rather not manage that dependency manually.
Do you think this is something you guys can fix on your end?

Another thing that I noticed is that, when I use brew install awscli I don’t get any output neither from the awscli command nor the brew command, regardless if it fails or succeeds. The following is a sample output when it succeeds

Cloning into '/var/folders/90/5stft2v13fb_m_gv3c8x9nwc0000gn/T/bitrise275240977/step_src'...
+------------------------------------------------------------------------------+
| (1) Download keychain (take 2)                                               |
+------------------------------------------------------------------------------+
| id: https://github.com/FutureWorkshops/bitrise-step-s3-download.git          |
| version: master                                                              |
| collection: git                                                              |
| toolkit: bash                                                                |
| time: 2017-03-22T04:21:19-07:00                                              |
+------------------------------------------------------------------------------+
|                                                                              |

is there a way to pipe the result of the awscli command to the log output?

9

You don’t have to pipe it, every output generated by any tool/command should be in the log by default, unless the step hides it (e.g. by redirecting it into a file, instead of writing it to stdout/stderr).

10

That’s what I thought. Unfortunately it’s the case with awscli (works fine when testing locally).
Any idea where I should look? It seems to be an issue with all the steps I’m creating.

I get a lot of output if I use set -x but I don’t really want that level of debug. I can give you access to the project if that can help debugging the issue?

11

What’s the last line in the log if you have set -x in it?

12

I just tried it again an the output with set -x is the same as without (i.e. no output at all).
It seems to be random though. Sometimes I get full output sometimes I don’t.
Is there a way I can get some help debugging this? I can easily share the project with your support team.

13

Can you please enable Bitrise Support access to the related app and then send us the app’s URL in email / through the onsite chat?