At Daily, we’ve built some great tools to help people and companies communicate in real-time on the internet. One key component of these deliverables is ensuring that we’re releasing with the fewest issues possible. As a test engineer at Daily, part of my job is to iterate on our testing processes to ensure we're continuously improving. One of my priorities is making sure we're adhering to shift-left testing practices.
This article covers what shift-left testing is, why it matters, and some ways to start shifting left at your own company.
What is shift-left testing?
The term shift-left testing is referencing the traditional software development life cycle (SDLC) where testing happens as the last thing before release, and suggests we shift “left” up the SDLC to perform testing activities in earlier phases. This shortens the feedback loop, allowing for faster iteration when the content is most relevant and freshest in the creator’s mind.
In order for that to make sense, we need to align on what testing is; it’s not just using an app to find issues. Instead, testing is a thought process–asking questions and validating assumptions to understand both intended and actual functionality of software.
Why does it matter?
I used to work as a quality assurance (QA) tester at a game company that was very traditional in how they tested their software. The first look QA had at any new work was when it slid across our desks for validation. Test automation was non-existent, and developers manually tested their own code… sometimes. Some developers were consistent, and others only did it when they could be bothered.
One day, we were working on adding social features to the game. Unfortunately, when it crossed my desk, there were many issues. It was full of bugs, which is common when testing is always thought of as a final task. But aside from the bugs, it was also very confusing to use and really failed at being a feature our users would want to engage with.
So I took my concerns to the stakeholders. After lengthy debate, they agreed that it needed more work before it could be released. Factoring in the delays–the time it took them to come to that conclusion, the time needed to put the feature behind a feature flag so we could get other features out, the redesign, refactoring, and retesting–it was almost 2 months later when that feature was finally released.
As you can imagine, the impact of that is a lot of lost time and money for the company, in addition to delays to all the other features they wanted to add. That’s an unfortunately common result of leaving testing activities to the end on a product with continuous support and development. But how could they have saved money and avoided those costly delays? By shifting left.
How to shift left
Communicate earlier and more often
When having early discussions about a feature, solicit input from all disciplines that have a stake in its development. Bring in the designers, requirements managers, developers, testers, and even support teams to build a much more complete picture of both the work required to implement the feature and how it will impact the user experience. Solicit feedback and thoughts, even as early as when you’re just gathering requirements. For our example above, if the test and dev teams were included in some of the initial design discussions, the critical questions could have been asked and prevented the entire need for a rewrite.
Write testable code & automation
Write automated tests to ensure that you’re not introducing regressions. Build the infrastructure to facilitate fast feedback when developing. To do this, you’ll have to make sure to write testable code–be it code that can be unit tested well, proper abstractions to help with integration tests, or end products with hooks for UI testing.
In our example of traditional testing above, part of the issues encountered with the whole scenario could have been resolved by having automated test infrastructure. At the time, automation in games was nearly unheard of although now modern game engines provide APIs that give the developers the power to write unit tests. To shift left, the developers in my first example could have focused on writing code they could exercise with unit tests. This would have enabled them to utilize test automation and minimize bugs well before delivery to the test team. Additionally, some combination of devs and testers could build out automation on other levels, to build in safety nets.
Run your tests
It's not uncommon for companies to write tests but not run them often–especially in the case of end-to-end or UI tests. But this doesn’t help you achieve the fast feedback shifting left is meant for. So make sure the automated tests are set to automatically run, and often. This is typically done in continuous integration (CI) on pull requests (PRs) with failures blocking merges, but could also be done on commit via pre-commit hooks to get fast and frequent feedback from the tests. To eliminate issues with long-running tests (which typically tend to be end-to-end tests), perform regularly-scheduled test runs not tied to PRs–such as nightly regression runs.
Use automated tooling to find other issues faster
Sometimes the issues we encounter aren’t related to the feature code itself, but instead non-functional elements that impact it–such as security, accessibility, or performance. There are several tools that can help you find non-functional issues with your software, many of which are free and straightforward to set up. Some of these include:
- Linters. Flag common coding errors, deviations from standard code practices, and even search for things like memory leaks and security vulnerabilities in code.
- Dependency checkers and security scanners. Identify security issues via vulnerable dependencies or coding practices that leave your code open to exploitation.
- Accessibility scanners. For UI-based applications, ensure that your application is meeting basic accessibility standards. This will help prevent potential lawsuits for violation of the American Disabilities Act (in the US) and provide a better experience for users in need of accessibility features.
Impact of shifting left
In my earlier example, had the company had a more shift-left approach to their development cycle they would have seen many benefits:
- Lower costs. They spent a lot of money redoing a feature that was nearly ready for release.
- Improved efficiency. Having test automation in place would have provided several benefits:
- Allowed developers to quickly test their code as they developed and refactored the feature. This would’ve helped ensure that the functionality remained intact without the delays of going through a test team.
- Allowed for better, more efficient fixes. Failures could be raised sooner, often before context switching occurred and while the code was still fresh in mind.
- Reduced much of the manual test team’s workload. This would have allowed them to review the feature faster, focusing on areas that needed human eyes.
- Improved quality. Frequent testing at all levels leaves fewer opportunities for bugs to slip by unnoticed to production.
These are the specific ways that my previous company would have seen improvements had they shifted left in these particular ways. When it comes to other companies, shifting left may look a bit different. Not all elements described here will work in all situations precisely as described. However, if you take the overall testing philosophy–asking questions to reveal intended and actual behavior of an application, as well as its impact on a user's experience–and apply it early in your development process, you may find yourself addressing big issues much more efficiently. This helps in delivering quality software that users love.
If you have any questions or thoughts, head over to peerConnection, our WebRTC community, to keep the conversation going.
Never miss a story
Get the latest direct to your inbox.
