In-Sprint test automation even possible? Try these 5 easy steps.

Automation is definitely a buzzing word in today’s software development and testing life cycle. The software industry has been shifting to Agile rapidly and accelerating the “shifting left” concept using well-defined DevOps culture. Waterfall model (where testing has been done at the end of the delivery cycle) is breathing it’s last. Testing needs to be accomplished as soon as possible to ensure the stability of the product automatically, i.e, without any manual intervention.

Long gone those days where a manual tester validates the same functionality in five different browsers where one can simply run a single script that can simultaneously validate the functionality in any browsers. Enter in-sprint test automation, an ideal development scenario where the entire testing process — from creation and implementation to execution and reporting — happens in one sprint. To many, in-sprint automation is a unicorn or leprechaun. Ask most Agile experts, and they’ll share countless reasons as to why it doesn’t work, asserting that automation must always be at least one sprint behind, or worse, separated into an independent team.

Here are the 5 easy steps that can assure in-sprint automation in your team.

Develop no barrier between manual and automation tester

Scrum teams often comprised of 6–8 members, including scrum master and product owner. It will not take long to guess that the team includes 1–2 test engineers to maintain the quality of the delivery. It has been seen that the primary task of the test engineers remains limited to document the test artifact and manually test the product against the artifacts. The test scenarios written in the sprint timeframe usually picked up in the following sprint (which crashes soon because of the time crunch) or push to regression suite to be automated later.

A lot of times automation engineers sit on a separate team where they work through the regression suite backlog. So, the organization needs to maintain a separate team to look after the regression test suite and gradually push those regression test cases in the continuous testing pipeline (CT). This is a un-necessary expense carried by many organizations. But what if they can automate the regression suite incrementally in the sprint itself? That is easily possible if there is no clear separation between the manual tester and the automation tester concept.

The mindset should be clearly set from the beginning in a scrum team that testing starts from requirement analysis for a functional flow to automating the functional flow. There should not be a separate automation specific team which only clears out the regression backlog. Deploying automation engineers directly in the scrum team will send a clear message about the mindset of the organization and enhance the “shifting left” principle.

The message should be clear from the beginning — test automation is one of the primary focuses of the sprint and not an afterthought.

Developers must check-in code frequently in a sprint

This is the most important tactic that makes in-sprint automation possible. It’s also the most overlooked. Leaders and team members tend to only focus on testing once it’s become a bottleneck — a symptom of the problem — rather than look at the root problem — developers failing to check-in.

Developers often wait until sprint end to check-in codes because they don’t want to affect other people too early and they don’t want to get blamed for breaking the build. Sometimes developers pick a user story to be worked upon which is dependent on other user stories. So, they use to wait for another developer to check in first then they synch up their story branch and check-in their code. This often leads to testing time crunch. Testers barely get time to test the user story manually in the sprint. How can we expect the stories/features to be automated in the same sprint?

This is why it is so imperative that developers start deploying code to the test environment early and regularly within the sprint. They should not wait until the feature is complete, but rather until it’s complete enough for testers to have something to work with. The good practice is to deploy testable codes in a separate virtual machine, often called a “Virtual Dev Machine”. So that, the tester can start writing automation codes (at least page objects and reusable functions). In this way, the tester no longer needs to wait for the ultimate deployed build at the end of the sprint to start writing automation codes. The tester can write test codes incrementally and at the end of the sprint, he/she needs to point the automation code to actual feature build and run the end to end automated flow. It may not get passed in the first attempt due to the DOM element get changed. The tester needs to just re-structure their code slightly and that’s it. It will save everyone’s time and tester will be ready with the automation suite in the very sprint itself.

Test engineers should be treated as a multifunctional contributor

It might seem like a no-brainer, but it’s a frequent miss among scrum teams. Every team member in a scrum should be multi-functional. The test engineer especially needs to get involved in every phase of a sprint starting from the requirement analysis phase to sign off a user story in a sprint. They should have an abstract idea around the technology and codes used in the development process so that they think about the possible vulnerable areas. In parallel, they also need to understand how to automate the functional test scenarios around the user story. This means that they must be highly skilled in both testing processes as well as code writing.

In a Potentially Releasable Product Increment (PRPI), a scrum team can work on a single feature or multiple features. In this whole time frame, the scrum teams deploy builds frequently. If the test engineer fails to automate the functional flows of the previous sprint, then he/she needs to validate the regression effect on the functionalities manually each time. It will not only eat up maximum time from the test engineer’s bucket but also slows down the whole team drastically.

So, the key point is to treat test engineers as multi-function individuals — they should be capable of writing and executing manual test cases as well as automated tests in the team’s programming language of choice. A standout automation engineer should be able to quickly write new tests along with all corresponding automation features while keeping up with the quick pace of agile development.

Tests should be concise, descriptive and feature-focused

In the age of agile development, when product owners define a feature, it actually meant a deliverable and releasable piece of the software. That means the feature should work well in integration with other parts of the software without breaking anything. Keeping this in mind, all levels of testing should be structured and well maintained. This can only be achieved if and only if the test engineer designs the right framework to do the right job. There should be a well-designed unit test framework, API testing framework and persona-based E2E acceptance test framework.

In each sprint of the releasable PI cycle, testers should refrain from writing long workflows that verify multiple components at once. Instead, a good automated test case should verify only one feature or sub-feature at a time and should be very explicit as to what steps the test is completing.

The way of expressing tests is also a key factor. The team should choose a common language to define the acceptance criteria. There are already tools like “Gherkin” that are already available in the market which greatly helps to ramp up the team in understanding the acceptance criteria better — what needs to be developed, what needs to be tested. By writing clearly defined steps and expected results, the writing process is shortened and the automation engineer can focus on the actual automation rather than wasting time trying to figure out what the test is set to verify.

The tester needs to work very closely with the developer to understand what can be covered in unit tests and what needs to be included in functional test cases. At the end of the release cycle, the test engineer can build-up longer end to end workflows by using the small workflows developed in each sprint.

Include automated testing to the Definition of Done

The scrum team often forget to add automation as a DOD (definition of done) criteria. As the release cycles get shorter in Agile and DevOps, it becomes necessary to automate the testing so that these new features/enhancements are easily tested as part of regression testing in the upcoming cycles. The test scenarios need to be automatically validated in a continuous testing pipeline to check whether one scrum team’s development is breaking others. So, keeping automation as a DOD criterion in the sprint level ensures that the team is correctly contributing to DevOps culture.

Another benefit of including automation as a DOD item is that it will continue to be a topic of discussion in post-sprint meetings as team members reflect on what went well and what could be improved. This way, they can continually improve the process until in-sprint automation is achieved.

Parking Lot

Yes, in-sprint automation can be difficult but it is not impossible. With the correct processes in place, teams can successfully achieve it, thereby decreasing the time required for deployment and improving the quality of their product. As with anything that is difficult, it takes time to become proficient. But if you implement the right series of actions and stay the course, the benefits will be well worth the hardship. And who knows, you might just catch a leprechaun along the way.