How is QAs role changing within engineering teams? Hear what some tech leaders are thinking about
Posted 8 months ago by Anke Corbin
We recently hosted a Peer 2 Peer Tech Leader roundtable discussion in the Rocky Mountains. The group is composed of engineering leaders from a wide variety of software tech companies ranging from smaller 4-6 person engineering teams at startups to engineering leaders at FANG companies, and organizations in the government sector responsible for over 400 engineers. The primary goal for the roundtables is to provide a forum to network with peers, to share experiences, and to learn from each other.
For each meeting a discussion topic is agreed upon – the most recent topic was ‘Changes in QA and How to Guarantee Quality and Efficiency’. The topic is very context dependent and most of the attendees were sharing from their perspective as SaaS businesses with frequent release cycles and some discussion about a combined SaaS and on prem deployment with slower release cycles.
Here are some take-aways from the event:
Engineering doing less more frequently enabled by DevOps
The evolution of DevOps enables continuous deployment (especially for SaaS businesses), typically several times per day. Engineering teams are doing less, within smaller releases, more frequently. This not only allows real-time reaction to issues, but can help create a culture of development teams unafraid to make mistakes. Bugs are typically caught earlier, and innovation can come faster.
“A reality check: If you create on prem releases, your release cycles may get somewhat longer. The length of the cycle is impacted by other factors, like regulatory requirements too. When working with SaaS apps, you can consider pushing multiple releases per day. Really small releases. Not big upgrades.”
Gabor Puhalla, Co-Founder and CEO profiq
New dev tools improve site observability allow for fine-grained feature roll-back
New tooling evolved that enables the ability to roll-back at the feature level (aka feature flags) easily and fractionally. Field reporting and observability have improved greatly. These tool kits help build a safety net for developers. Software that can report bugs, coupled with rollbacks, can create new opportunities for refactoring without having to rewrite whole sections of code.
“I think the thing that catches you more than anything is field observability. Observability is probably one of the biggest changes that I’ve seen in my career. If you can get your software to tell you when there’s a bug, instead of having somebody report it, that changes everything. That’s the stuff that saves us more than the test suite. The test suite allows us to refactor more aggressively.”
Rob Pinna, Co-founder and VP Engineering Serenity App
The number of dedicated QA roles is decreasing
The number of dedicated QA roles across organizations is decreasing. QA is not disappearing, just more distributed across the engineering and project teams. Companies are moving the responsibility for the quality from testing to development and, actually, the line between the two is becoming blurry. The product teams are helping to identify tests to execute and engineering is setting up end-to-end and functional testing. There is less of a need for a dedicated QA team to catch bugs before deployment, when engineers and developers are working responsively to, and fixing, issues as they arise.
“We work with the federal government, and we rarely see QA roles on their projects. QA is typically integrated into the roles on the team – engineers, product, design, data, research, etc. Everyone on the team owns quality. Across our team of 400+ practitioners, we only have a few dedicated purely to QA. It’s really important to us that everything is automated as much as we possibly can across the entire stack.”
Kathy Keating, VP Technology Ad Hoc
QA is no longer the release police
QA is no longer the ‘release police’ since responsibility for quality and testing is now pushed across product and the engineering organization. With continuous release, it’s no longer as feasible to wait for a QA team’s approval.
Accurate release scoping includes time for QA
A key requirement is scoping releases correctly. Developers need to include QA into the project scope since it’s no longer a separate function at many companies. This means not putting off testing until coding is done, but integrating, and actually building into the development plan, so QA works side-by-side with the coding. Some testing like UI testing can be very complicated to automate and estimate time for. By keeping the UI thin you can automate through API testing, and respond to issues, without having to worry about changing the UI often. Having a well-designed and functional UI up-front can mean better bug response in the future.
“With team members I make the philosophical point that the engineers write the bugs. So, you can’t blame someone else later for the bug in production, you wrote the bug. And then on the same point, I emphasize to them as I always do, that, since they own quality, they need to size the work in a way that allows them the time that they need to do it properly. They can’t complain later, if they told us this was a three, then they say later on,’Well, I didn’t have time to write tests,’ Well, then it wasn’t a three- that’s your opportunity to push back against the schedule. And if you don’t take it, you can’t complain later.”
Craig McDonald, CTO ToolWatch
Independence vs. interdependence for teams
A developer testing her own code doesn’t always lead to the best quality. This can be addressed by distributing testing, so some developers are writing and others are testing the same code. Creating this interdependence takes the pressure off individual coders having to test their own code, without putting testing off until later in development.
“It requires us to think about testing first in everything that we do, and ensuring that we have the developers on staff to be able to support the process. We’ll often have one developer writing the test and another developer writing the code so that we have the proper checks and balances in the process to ensure highest quality. They work together as a team.”
Kathy Keating, VP Technology Ad Hoc
Benefits for setting up CI/CD pipeline for startups outweigh costs
It is viable for small startup teams to implement CI/CD right from the beginning. Startups don’t necessarily have to maintain the rigorous update schedule of larger companies, but starting these practices early on can be much easier and more cost-effective than trying to implement them later in your company’s development.
“Not having a CI/CD pipeline is like a poverty trap. There are a number of ways to deploy a simple CI/CD pipeline. You can run it slower, and don’t have to release several times per day. And you can improve over time.”
Craig McDonald, CTO ToolWatch
In conclusion, QA, like all fast-moving technologies, is changing rapidly. We hope you enjoyed the synopsis of our discussion with tech leaders. If you’d like to learn more about participating in an upcoming Peer 2 Peer Tech Leader Roundtables, please reach out to [email protected].