Types of Black Box Testing Techniques

The main goal of software testing is to assess the quality of software and reduce the risk of it failing. You might think that you can test everything and assess the quality of every single detail. But is it actually possible? According to one of the seven testing principles as set out by the ISTQB, exhaustive testing is impossible, which means that not everything can be tested. That is why we use testing techniques to help us identify test conditions, design better test cases, and select the best test cases to execute. There are different types of testing techniques, but in this blog article we will focus on black box testing and elaborate on several types of black box testing techniques.

What is black box testing and what are some techniques?

Black box testing is a software testing method used to test a system without having any prior knowledge of the internal structure of the software under test. Instead, the focus is on how the system works as a whole. Specifically, this testing approach focuses on the input that goes into the software and the output that is produced to see whether the expected results are achieved.

To better explain how this works, let’s look at a simple example. Let’s say we want to test the Calculator application on our phone. We open the application, enter the numbers we want to calculate as well as the operator and equal sign, and wait for the output. Another example would be testing a social application. Black box testing in this case would be inputting a username and password, while the expected output would be getting logged into the application. In both examples, we input information and wait for the output without looking at the code.

There are various black box testing techniques that can be used to validate a system against predefined requirements. These techniques use the use cases, user stories, specifications, product requirements documentation, and software requirements specifications as a test basis to define what needs to be tested and how to properly create test cases.

Black box testing techniques apply to all levels of testing, as well as functional and non-functional testing types. There are four main black box testing techniques: equivalence partitioning, boundary value analysis, decision table testing, and state transition testing.

Equivalence partitioning (EP)

Equivalence partitioning is a technique that divides test objects into partitions or classes that are treated and tested in the same way. This technique is used for testing a range of values, inputs, and outputs. There are valid and invalid equivalence partitions. Valid equivalence partitions contain only the valid values that are accepted by the testing object. In contrast, invalid equivalence partitions contain only invalid values that are rejected by the testing object. When using this technique to design test cases, there are several important things that you need to remember. First of all, every single value belongs only to one of the partitions. Secondly, you must always test valid and invalid partitions. Finally, you must test invalid partitions individually because otherwise, they will mask the results. If there is a need, you can divide the partitions into subpartitions.

To achieve 100% coverage of the testing object with this technique, you must test at least one representative member from each partition. You can do that by taking one value from each defined valid and invalid partition. You will not increase coverage by testing two values from the same partition.

Example of equivalence partitioning:

The discount is calculated depending on the total amount of the shopping cart. If the total amount is in the range of $100–$200, the discount is 10%. If the total amount is in the range of $201–$500, the discount is 20%. If the total amount is more than $500, the discount is 30%. In this scenario, we can identify three valid partitions and one invalid partition for the amount under $100.

To apply equivalence partitioning, we can take one value from each partition: $140 from the first valid partition, $370 from the second valid partition, $530 from the third valid partition, and $78 from the invalid partition. Now we have four test cases and have achieved 100% coverage because all defined partitions are covered.

Boundary value analysis (BVA)

This technique is an extension of equivalence partitioning and is used to test the boundaries of the valid and invalid partitions. These boundaries are breaking points that are most likely to be incorrect. Also, both these black box testing techniques—equivalence partitioning and boundary value analysis—can be combined. There are two types of boundary value analysis, two-value and three-value boundary analysis. We use two-value boundary analysis to test the minimum and maximum values of the boundary. In comparison, we use three-value boundary analysis to test the values before the boundary, at the boundary, and just above the boundary.

Example of two-value boundary analysis:

Let’s look at the same discount scenario from earlier. The discount is calculated based on the total amount of the shopping cart. If the total amount is in the range of $100–$200, the discount is 10%. If the total amount is in the range of $201–$500, the discount is 20%. If the total amount is more than $500, the discount is 30%. With a simple illustration, we can define the boundaries very easily.

To apply two-value boundary analysis we will test the minimum and maximum value of each boundary: $99, $100, $200, $201, $500, and $501. We have six test cases and we will achieve 100% coverage by testing them because all defined boundaries are covered.

Example of three-value boundary analysis:

To apply three-value boundary analysis, we will test the following values: $99, $100, $101, $199, $200, $201, $499, $500, and $501. This gives us nine test cases and ensures 100% coverage, as all defined boundaries are covered.

If we use a combination of equivalence partitioning and boundary value analysis in this example, we will test $78, $140, $370, and $530 (equivalence partitioning), as well as $99, $100, $200, $201, $500, and $501 (boundary value analysis).

Decision table testing

The decision table testing technique is used to visually present the different combinations of inputs and outputs and is more focused on business rules. We use decision table testing in situations where different combinations of test input conditions result in different outputs. With this technique, we identify the conditions and the resulting actions of the testing object and present them in a table. Often conditions are the inputs, usually at the top, and actions are the outputs, usually defined at the bottom of the table. The first column of the table is an explanation of the conditions and actions. Every other column in the table represents one decision rule. When the condition or action is TRUE we use Y, T, or 1. When the condition or action is FALSE we use N, F, or 0. When the value doesn’t matter we use – or N/A.

There is always a way to collapse the table by deleting the columns that don’t affect the outcome or the ones that have impossible combinations of conditions. To achieve 100% coverage we need to test each rule in the table at least once.

Example of decision table testing:

The discount is calculated depending on the total amount of the shopping cart and whether the customer is a member. If the total amount is more than $200 and the customer is a member, the discount is 10%. If the customer is not a member, there is no discount regardless of the total amount in the shopping cart. To use the decision table testing technique, we first need to identify the conditions. In this case, the conditions are the total amount in the shopping cart and whether a customer is a member or not. Next, we need to identify all combinations of these conditions. We will mark them with Y and N.

The next step is to identify the correct actions/outcomes for each combination. We will use Y and N to mark them.

As the final step, we write test cases for each rule in the table. In this table we have four rules which means we will have four test cases:

• Test case 1 for Rule 1: The total amount is more than $200 and the customer is a member, the outcome will be a 10% discount off the total amount in the shopping cart.
• Test case 2 for Rule 2: The total amount is more than $200, but the customer is not a member. There will be no discount.
• Test case 3 for Rule 3: The customer is a member, but the total amount is not more than $200. There will be no discount.
• Test case 4 for Rule 4: The total amount is not more than $200 and the customer is not a member. There will be no discount.

State transition testing

State transition testing is a black box testing technique that is used when a system responds differently to the same input and depends on current conditions and previous history. The previous history of the system is called the state, while an event is an occurrence that can result in a transition. A state transition diagram or state transition table is a representation of the system’s states and events that are causing transitions between states. One event can cause different transitions to the same state.

An example of an event would be a user entering a pin into an ATM. The event causes a change in the state, which results in an action from the test object. A possible action in this case would be the ATM screen displaying either the available services or an error message, depending on whether the user input is correct or not.

We can use this technique to design test cases that can evaluate whether the test object executes the valid transitions and blocks the invalid ones successfully. State transition testing is used within the embedded software industry for testing screen-based applications, such as an ATM, or for testing any other transitional software.

We can achieve 100% coverage by testing every transition and state at least once. Tracking the coverage is supported by a state transition diagram or state transition table. Covering all transitions is better than covering all states.

Example of state transition testing:

After shopping, we put the receipt in the parking machine that calculates whether we get a coupon for free parking. To get a coupon, the receipt must be in good condition and have an amount that is equal to or greater than $200. First, the machine checks if the receipt is damaged. If the receipt is damaged, the machine will reject it. If the receipt is not damaged, the machine checks if the total amount is equal to or greater than $200. If the total amount is under $200, the receipt will be rejected. If the total amount is equal to or greater than $200, we will get a coupon for free parking.

In this example, we have three test cases for state coverage and also three test cases for transition coverage:

Test case 1: Enter the receipt > Receipt is damaged > Receipt is rejected.

Test case 2: Enter the receipt > Receipt is not damaged > Receipt is < $200 > Receipt is rejected.

Test case 3: Enter the receipt > Receipt is not damaged > Total amount is => $200 > Get coupon.

The bottom line

The great thing about these black box testing techniques is that you can use them to test one thing but in so many different ways, based on what you need specifically. If used correctly, these techniques will help you identify test conditions and help you design better test cases. And in turn, you will improve your testing process and the quality of your system.

Do you need some software testing done or would like to see these black box testing techniques in action? Contact us with your project details and we’ll make sure your product checks all the quality boxes.