Going 0 to publish with Flutter - Feedback, issues, workarounds

[daniel-v]

Iteration 67, got it working! (TL;DR version at the bottom)

I wanted to give you guys some feedback about my “first month with Bitrise”.

Background

I’m fairly comfortable in shell but I have limited DevOps experience. I put together a few Jenkins jobs and a couple of Ansible playbooks but that’s about it. My focus is usually around development.

As for Mobile Development: Flutter was my gateway drug. Been doing it for about half a year now professionally. I’m using Bitrise for one of my hobby projects as evaluation for the pro projects. (Been writing Dart apps since 2013)

Expectations

There are 2 big things I expected Bitrise to help me with:

1. Run automated test on push/PR and scream at me when I screw up
2. Create build artifacts and push them to Play Store (iOS coming later… maybe) - will be hooked up to a trigger on tag creation

Approach

I watched a couple of YT videos, looked around the internet, github for tutorials to get an idea what Bitrise is and how it work, how it’s different from other CI tools I’ve used. Maybe spent 3 days only getting the basics/concepts down.

Docs are pretty good if one has the patience to read them - things start to come together if you go over a couple of seemingly unrelated topics. What are steps, how they interact, envman, bitrise caching, test reporting and a bunch of others.

Let’s get dirty

The promise of Bitrise, sort of paraphrasing and expressing my understanding:

Ohh, we got these steps, you go online, throw them together and stuff works.

Khm… “Iteration 76”

Half of those, I imagine are because I’m new to Bitrise. I’m going to expand on the other half.

Starting off with Flutter project template

Just went through the steps, it works exceptionally well. Recognized that it’s a Flutter project,
preselected some defaults for me that I didn’t have to change.

I just went over project creation process to see if it jogs any memories. Right now
it uses outdated Step versions.

Tests

Flutter test step is simply unusable. Well, it was to me but not quite true… If you just want to run a quick test on push and report back to your version system that “oopsie” or “nice job, have a pat on the back from a bot”, I suppose it works.

I wanted, however, archive test result so that I get to see what failed on Bitrise UI. Why have a Test reports UI enabled if you can’t use it, right?

So the workaround is a utility workflow, _flutter_test with a single Script step:

#!/usr/bin/env bash
# do not fail if any commands fails
set +e
# debug log
set -x

flutter packages get
# run with --no-pub as it will screw up output with trash
flutter test --machine --no-pub > test-report.json
# cache exit code
TEST_EXIT=$?

# fail if any of the following commands fail
set -e
FL_TEST_XML_PATH=$BITRISE_TEST_RESULT_DIR/flutter
mkdir -p $FL_TEST_XML_PATH

# install junitreport to convert to xml
flutter packages pub global activate junitreport
flutter packages pub global run junitreport:tojunit --input $BITRISE_FLUTTER_PROJECT_LOCATION/test-report.json --output $FL_TEST_XML_PATH/TEST-result.xml
# make sure we have test-info.json for Deploy to Bitrise step
echo '{"test-name":"All Flutter Tests"}' > "$FL_TEST_XML_PATH/test-info.json"

exit $TEST_EXIT

Spent maybe 15 iterations and a several days to get this working. Most of my issues were around understanding and finding info on test-info.json, and figuring out what requirements Bitrise Push step has. It’s not perfect, since all test results will be under All Flutter Tests in the reporting UI but it’s good enough for me.

flutter test command also threw me a couple of curve balls with the json output or putting unwanted header text onto stdout and Test expecting JUNIT xmls.

Artifacts, building the app

Flutter build step has some weird defaults:

• output type: APK instead of appbundle (used in conjunction with bundletool to get apk)
• missing some default parameters: --build-number=$BITRISE_BUILD_NUMBER

Overall, it worked fairly well but I needed to introduce a script step beforehand:

#!/usr/bin/env bash
# fail if any commands fails
set -e
# debug log
set -x

flutter pub global activate pubspec_version
# expose pubspec build name to this step
export GOM_BUILD_NAME=`flutter pub global run pubspec_version:pubver get`
# and make sure it's available for any other step too
envman add --key GOM_BUILD_NAME --value "${GOM_BUILD_NAME}"

echo "Pubspec version: '$GOM_BUILD_NAME'. This will be the build-name parameter of flutter build command"

and make sure that the build parameters of build are:

--release --build-name=$GOM_BUILD_NAME --build-number=$BITRISE_BUILD_NUMBER

This way, I get to set versions in pubspec.yaml and let bitrise figure out the build name
and number for itself.

Leter, when I start doing proper tag based versioning, this step will still be needed, except build name will be coming from the parsed git tag name, instead of pubspec.

UI, little things

I never understood the “deploy to bitrise” stuff. Initially, I thought it was cool but after spending some time trying to figure it out, it seemed simpler to register the app on Play Store and push artifacts there.
Manage the channels/releases via Play Store. Maybe someone could enlighten me with some use-cases for it.

I missed the ability to set workflow input parameters when running one from the UI. How cool would it be that I could simply declare some workflow parameters, such as “deploy target” or “tag name” and the UI would offer me those when starting a build?

I was super happy with the workflow editor. I enjoyed using it way more than scripting yaml for travis/Gitlab CI. The experience was waaay better than Jenkins.

I’m used to writing Widgets. Widgets are the configuration of Elements and RenderObjects in Flutter. It offers composability very well. Well, I kind of wanted to create those Bitrise widgets so I get to compose my many Workflows as well. Utility workflows is the way to go but I spent more time figuring that out than I should have. Maybe a docs update could help newcomers. (Remember, I don’t have my own best practices figured out yet since this is the first project on Bitrise, some pointers from you guys could really help)

10 mins limit on a Flutter project is … very little. Some of the projects I work on uses native plugins and code and the build time is around 10 mins on a super-beefy build node. I know you guys need to make some $$$ too but that limit makes Bitrise limited to OS/hobby projects very limited. I’ll set up a developer account, that’s not an issue for me, but to some, it might be.

Summary, TL;DR

• project creation process for Flutter template uses outdated Step versions
• Flutter test step is very limited, doesn’t cover those use-cases that make Bitrise awesome
• Flutter build step parameters are lacking, needed to create some scripts as inputs for build step

Overall, pleasant experience, tho Flutter steps need some love… Congrats, you just read through the longest feedback I’ve ever written, you deserve a or 2