Issue Reproduction: Why Reproducing Bugs Matter

I have always compared the work of a tester to the work of a detective. As you may already know, our job as testers is not just about finding software bugs, reporting, and validating them, it's also very important to dig deeper to find out the root cause, to think about all the details that could affect bugs, and to provide the best steps and all the necessary evidence for bug reproduction. In this blog article, you will find out everything you need to know about bug reproduction - what it means, why it is important, and how to do it effectively.

What is Bug Reproduction?

For bug reproduction, there is no need for a complicated definition as it speaks for itself - recreating the scenario that generated the bug. Bug reproduction refers to the steps needed to reproduce a bug that a software tester has encountered while testing the software. The aim of bug reproduction is to ensure that the bug can be reproduced on the same device and environment where the bug was first encountered by whoever follows the reproduction steps - be it a software developer or another software tester.

Why is Bug Reproduction Necessary?

Mainly, bug reproduction helps developers identify where the problem lies, how it can be fixed, and how to prevent similar bugs from appearing in the future. It also gives any interested party a chance to see what is going wrong and why.

By going through other testers’ bug reports, you can get benefits such as learning from the approach of others and expanding your thinking, being inspired by other test scenarios to think of other ways and things to test when you are out of ideas, and even to learn the product itself if you are a newbie.

The Bug Reproduction Process

Before reporting a bug, you should ask yourself the following questions about the issue detected:

• Is the bug reproducible? For example, is this reproducible all the time or maybe only once per a certain number of test runs? It can also affect the overall bug priority - how soon this bug should be fixed - and severity - which indicates the seriousness of the defect in the product functionality. Maybe the bug repro is so small that you need to hold on to the report to gather more information about it.
• What was the environment in which it was found? It's ideal to make sure this can be replicated on all test environments, as maybe it's just environment-specific.
• What actions were performed that triggered the bug? Try to weed out any redundant actions.
• What was happening around the time the bug occurred? Maybe it is possible to collect log files, stack traces, network requests, and responses while the bug is being reproduced to get additional information about what is happening, that may provide additional evidence.
• Can it be replicated on multiple phone OS? As we already know very well, the bug may not be due to the mobile app build, but also due to the fault of the backend.

It's not a secret that mistakes could happen in the testing itself, and no one wants to report a false positive bug - a test case failing while the software under test does not actually have a bug. For example, this can happen if the system is not installed correctly or if all the test pre-conditions have not been met. So be careful with it.

A general good practice is to ask a colleague to reproduce it. With such an approach, it is possible to reveal errors in testing itself, detect possible issues with the devices of a particular tester, or it may even discover new clues that can be added or considered before the report.

How to write a clear Bug Report

Bug Report - this is the basis on which we build everything necessary for bug reproducibility. A good bug report should clearly describe the bug and show the necessary steps to reproduce the bug. When writing a bug report, be sure to add a:

• Summary - A short, descriptive title of the bug.
• Description - A detailed description of the bug.
• Client - Build number, Environment.
• Repro - Reproducibility of test runs as % or ratio.
• Preconditions - Steps that need to be performed to get the test environment ready.
• Steps - Exact steps to reproduce the bug.
• Actual result - What actually happened?
• Expected result - What should happen?
• Devices - Phone or other devices used for testing.
• Notes - Other valuable observations, info for attached files, etc.

Tips and Tricks

Understandability

Although different companies have their own standards and rules for reporting bugs, one thing is clear - it must be understandable not only for ourselves but for everyone reading it so that it can be reproducible by anyone. It could be your fellow QA engineer, developer, or any one of the stakeholders. Unified bug ticket layout and style make it easier for any reader to orient themselves from the perceived information.

Accurate steps

It is equally important to ensure accurate steps. At the same time, try to find out the shortest way possible to reproduce this issue. Maybe you found this bug in one of your test cases, but it doesn't mean all steps caused it. Finding out whether this bug can be reproduced with simpler steps will make the work of not only developers easier, but also for you when you have to validate it later on. This will save time for everyone.

All the possibilities

Don't forget to check all possible aspects that could directly cause this software bug, and specify them in the bug's ticket. Check whether the test environment affects it, whether specific pre-conditions can contribute to it, and whether it can be repeated on all test platforms. Think as broadly and comprehensively as possible.

All these things should be remembered even when you have to reproduce a problem reported by another person. Collect all possible information about the steps, the environment, used test data, granted privileges that might be specific to this bug, and then try to isolate the bug itself.

Evidence

Provide all the possible evidence to reproduce the bug. It could be screenshots, screen recordings (ideally, if you can, enable a touch indicator for distinct actions that you made), log files, or anything that you think provides evidence or reveals the root cause. Don't forget to name the evidence files accordingly based on the main information about the test run, for example, the mobile phone model, build number, test environment, etc. Even this small thing helps others to oversee the issue in general.

Conclusion

The topics covered help to clarify the bugs and ensure the best way we can collaborate with developers to reach the common goal - to assert a better quality for the software product. Practice makes perfect, so don't limit your thinking, and expand your knowledge whenever it is possible. Hopefully, these tips will be useful to you, and you will be the best Sherlock Holmes in the project.

Are you struggling to create stable tests that deliver reliable results? Contact us to find out how we can help you improve your software testing process and ensure test stability.