.jpeg)
In a panel discussion at this year’s inaugural Mobile DevOps Summit, Bitrise's Viktor Benei facilitated a conversation between Matt Robinson — iOS Build Engineer at Reddit, and Stephane Copin — Senior Software User Experience Engineer at Compass. They talked about how they use data from Bitrise Insights and other mobile tools to measure success, optimize workflows, and more.
How important is it to use data in day-to-day decision making?
You can’t improve what you can’t measure. Data guides mobile teams to make better feature decisions, bug improvements, and more. The right data can tell a mobile team what is working and what isn’t to optimize the user experience.
“Before we make a decision for a change in the app, we’ll try to look at the existing data to determine whether it will go in the right direction or not. And as the app or feature is released, we [use data to] evaluate whether that was the right call,” said Stephane Copin — Senior Software User Experience Engineer at Compass.
Using data gathered from Bitrise Insights, the mobile team at Compass were able to analyze their workflows and made small incremental changes. The Compass team collects data to measure the impact of internal and external improvement projects to optimize the overall experience — for both users and developers. Over time, they optimized their code and the project system itself. As a result, build time decreased from 16 minutes to 12 minutes.
Data can also help you identify potential problems before they become larger issues. Because of this, it’s important to give quick feedback to engineers when they make changes. Feedback loops help identify issues in code changes before release — helping you shift feedback forward in the development cycle.
What should you measure to optimize the app delivery process?
In order to optimize your app delivery processes, analyzing this data can help you better understand your app:
- App binary size: Matt says his team strives continuously to lower app size. Collecting binary size is informational and it’s helpful to track changes in binary size to ensure nothing is overly suspicious.
- Compile time: Mobile teams can monitor compile time to ensure resource consumption stays optimized.
- Build time: Stephane says his team monitors their build time to ensure it doesn’t get too high, saving developers time to work on other things.
- Credit usage: Stephane says his team monitors Bitrise credit usage per build.
- Test stability of UI and unit tests: UI tests can be flaky, so Stephane says his team measures the stability of UI tests to try to make sure all UI tests run and are up to date.
- Code coverage: Matt says his team takes a hard stance on code coverage, every pull request has to meet a certain threshold.
“One of the things that have become very important to us over the last couple of years is code coverage. We take a pretty hard stance on code coverage in our environment, where every pull request has to meet a threshold for the code that you’re touching. We even have a pretty granular way to describe [a situation where a] particular subset of the code is very important, and so it needs to have a higher threshold of coverage, otherwise, it can’t be merged [without meeting the code standard],” says Matt Robinson, iOS Build Engineer at Reddit.
What should you measure post-release to optimize app performance?
Following the release and related to the performance of your app, these metrics can help you optimize your app's overall performance:
- Crash rate: The Compass team monitors outages and crash rates so they can quickly remedy them.
- Error rates: The Compass team monitors error rates, such as API service failure rates.
- User-related metrics: The Compass team monitors feature usage, session time, etc.
- Apple phased release for iOS apps: Apple phase release for iOS apps releases apps in phases. It provides usage data from a release to a small subset of users to help the mobile team understand what’s happening in the back end. When something is off in production, the Compass team stops the phased release so they can fix it before it gets released to the entire user base.
How do you identify critical bottlenecks?
Identifying critical bottlenecks is one of the best ways to optimize mobile workflows. Data is key to helping you identify these bottlenecks. But, as Matt described, having a concrete relationship with a guiding star — such as reducing the amount of time engineers have to wait when they push code — helps you identify which bottlenecks are critical to address.
“We have 15 builds or more that run on builds that run per commit on a pull request. Those run on two different CI systems. There are three different levels of the way those build in a pipeline. It’s complex enough that unless you’re in that environment every day — do you necessarily know what’s happening? No, and you shouldn’t necessarily have to. But… at the end of the day… ultimately what you care about is how long an engineer has to wait after they push code and hit the merge button… everything that happens in between is just an implementation detail,” says Matt Robinson, iOS Build Engineer at Reddit.
About the panel discussion and speakers
The highlighted panel session — Mobile App Delivery: Accelerate and Excel Using Data - a Panel Discussion with Reddit and Compass — was part of Bitrise’s inaugural Mobile DevOps Summit in November. The Panel discussion is available to watch on-demand until the end of January 2023 in the Summit’s portal.
Matt Robinson is an iOS Build Engineer at Reddit — one of the world’s largest forum platforms. He works on the iOS platform team to ship better-performing code, easier. He focuses on the CI experience and the internal Bazel infrastructure. He previously worked as an engineer with Strava, Facebook, and Garmin.
🤖 Read the Bitrise and Reddit success story
Stephane Copin is a Senior Software User Experience Engineer at Compass — a software company that builds a mobile app that helps real estate agents. With over 10 years of experience in mobile, he works with the mobile CI/CD team, optimizes the mobile stack, and makes decisions to address user experience. Prior to Compass, Stephane worked as an engineer at Fueled.
🧭 Read the Bitrise and Compass success story
Viktor Benei is the CTO and Co-founder of Bitrise — the Mobile DevOps platform founded in 2014. Backed by Y Combinator, Viktor and the Bitrise founders lead a team to create a platform used by over 100,000 developers worldwide to maximize the impact of mobile apps.
