Pinch is thrilled to unveil its latest offering, Pinch Perfect – a platform designed for automated app testing. We had the privilege of speaking with Roeland Weve, CTO of Pinch, to delve into this latest development.
Roeland, as the founder and CTO of Pinch since its inception in 2011, could you shed some light on how apps were tested back then?
Back in those early days, we primarily relied on developers to test the apps themselves. You’d write a piece of code and then check on a device to see if it performed as expected. Sometimes our clients conducted minimal testing, but more often than not, they didn’t. They’d simply give us the green light to go live with an app update without thoroughly reviewing it. Needless to say, that approach didn’t always yield ideal results – you could even say it had its fair share of hiccups, haha.
Fast forward over 12.5 years. What’s the landscape like now?
In general, there’s been a greater emphasis on app testing over the years. However, there are still instances where apps with minimal testing go live, resulting in occasional mishaps. At Pinch, we’ve made significant strides since our early days to enhance and ensure the quality of our work. Every piece of code written is now reviewed and evaluated by two other team members, as it’s an integral part of our workflow. We also utilize tools like Sonarcloud to refine our code quality. Finally, a few years ago, we were fortunate to bring Kristina on board as our dedicated QA specialist. She has proven to be an invaluable addition to our team.
Finally?
Indeed, it took us quite some time to find the right candidate. We’re thrilled with her contributions, although we’re still actively seeking an additional team member for that role.
Does she evaluate developers’ code?
No, her focus is solely on what appears on the screen, aligning with the user’s experience. She meticulously devises comprehensive test plans, outlining what needs testing for each screen. Subsequently, she tests the apps on various iPhones and Android devices.
So, you’re introducing Pinch Perfect now. Does this mean it replaces manual testing?
No, not at all. They coexist harmoniously. Pinch Perfect serves as a valuable and complementary addition to manual testing. Manual tests can’t be executed at a high frequency, and humans are better at introducing unexpected variations during testing, something automation can’t replicate.
What does Pinch Perfect excel at, then?
With Pinch Perfect, you can efficiently test the essential functionalities or user flows within an app, which we collaboratively define with the client. We then create the test scripts and select the devices on which and the frequency with which tests should be conducted. Once the test runs, you automatically receive the results via Slack – a convenient feature. If the test identifies any issues with the app, we create a ticket for the development team, complete with a video showcasing the problematic segment.
I understand the importance of thorough testing before an app hits the stores. But once it’s available, shouldn’t the code remain unchanged?
That’s correct, but most apps are continuously evolving, with code changes occurring frequently. When an app (or an update) enters the store, it should ideally be free of bugs. However, we continue to develop new features, so it’s beneficial to test them as well. For apps that receive fewer updates, automated testing remains valuable, particularly if the app relies on an API. The API can undergo changes, or dynamic content within the app might cause unexpected issues.
Could you elaborate on how you set up Pinch Perfect?
For creating test scripts and executing tests, we opted for an existing platform called Maestro to avoid reinventing the wheel. However, we’ve built our environment around Maestro, allowing us to provide additional value. Maestro has become an integral part of our CI/CD* process. When we want to send an app to a client for approval, the developer can choose to run the Maestro tests first, specifying which devices to use, and so on. We’ve also developed our integration with Slack and the inclusion of video recordings of tests that uncover issues. All in all, we’ve established a robust framework that truly enhances app quality and maintains continuous vigilance.
Can you share some results with us?
We conducted tests using the NRC Nieuws app and achieved impressive results. We ran the tests nightly, and on several occasions, we identified issues that we would never have spotted before. For instance, a problem arose when a magazine was published in the app one morning – something that hadn’t happened in a while. Thanks to Pinch Perfect, we could address the issue immediately. In addition, we detected various bugs in their early stages.
Can apps not developed by you also utilize Pinch Perfect?
Certainly, they can. Essentially, all we need is the app itself. There is some initial coordination required to determine what needs testing, on which devices, and at what frequency. After that, everything operates automatically, including notifications, reports, and, if necessary, tickets for the app development team.
And what about the cost of Pinch Perfect?
Haha, great question. The cost varies depending on the number of tests and other factors, but you can anticipate it to be around €1500 per month per app.
*Continuous Integration (CI) and Continuous Delivery (CD) refer to a way of working within software teams in which the handling of code changes is managed through a series of automated steps. This sequence of automated steps is also known as a CI/CD pipeline. The CI/CD pipeline ensures that development teams can make code changes more frequently and reliably (with fewer errors).
