Key Challenges in Testing Call Screening Services

Are you tired of answering calls to find out that you've picked up a spam call, or maybe you've been trying to avoid a certain type of call? If that's the case, call screening is exactly what you're looking for. It lets you find out who's calling without answering the phone, after which, you can choose to answer the call, reject the call, find out more information about the call, or perhaps just block the caller to avoid receiving a call from that number ever again.

How does it work?

Now let's review how exactly call screening works. Everything starts with setting up the call screening service. As this process is not the same for all call screening services we will not dive too deep into the details, but generally, the process can go from toggling just one settings option to having a full onboarding process, which may involve choosing an AI (artificial intelligence) voice, or, perhaps, setting up call forwarding for it to work. For more information on how to set up the call screening service that you've chosen, we suggest searching for a setup guide on the service provider's website.

After you've successfully set it up, you can already start using it. All you need to do is receive an incoming call for which you'll have the possibility to screen it. Selecting the screen call button on the incoming call screen lets the call be picked up by an AI assistant.

After selecting the option to screen the call, the caller will hear the AI assistant introducing itself, after which the caller can say the message that he wants the assistant to deliver to you.

The message that the caller said will be displayed to you as text, and, depending on which provider's call screening service you are using, you may have quite a few options available to choose how the call will be handled. Options such as:

• answer the call
• reject the call
• reject the call with a reason (e.g. “Call me later”)
• block the caller
• ask some extra questions

The functionality explained above is manual call screening, where you receive a call and make the choice of whether to screen the call or not. Some call screening providers also offer automatic call screening, which decides whether to screen the call or not in your place.

Now, you probably have a question of how the software can decide in my place what to do with each call. Although it depends on quite a few aspects, it's pretty straightforward. Usually, if such a feature is available, you may have the possibility to choose which calls the software screens automatically, whether these are spam calls, phonebook contacts, or maybe unknown callers, the choice is yours.

What are the differences between call screening service providers?

Since these services are still quite new, and there aren't that many options available at the moment, let's take a look at the call screening solutions that some of the more popular brands are offering. Although the main functionality for all of them is the same, there are a few differences that might be the deciding factor for you.

Bixby text call

Here comes the call screening service that Samsung is offering. Bixby can now answer calls in your place.

Availability

Unfortunately, the “Bixby text call” service is limited to Samsung devices that have Samsung One UI 5.1 and above for English models, and One UI 4.1.1 and above for Korean models. If you don't own a Samsung phone that matches the above criteria, and you're not willing to get a new phone, then this is probably not the choice for you. If you do own a Samsung device matching the mentioned criteria, the service is completely free and does not require you to pay for it. It doesn't seem to have any regional limitations either.

Functionality

The functionality of this one is pretty straightforward. After receiving a call you can let Bixby take the call for you. While Bixby is taking the call, you will be presented with a chat type of view where every response that the caller says to Bixby is displayed in text format. From there you can answer the call yourself, end it, or interact with the caller by typing a custom response or selecting one of the default quick response presets. Whether you type your response or use the default presets, Bixby will say that response to the caller. And after the call screening is finished you can find the transcript of the conversation in the recent calls list by selecting to see more info about the specific call.

Having checked this out myself, I am a little bit disappointed with this feature from Samsung. After testing it out and doing some research, I found that it's known to struggle with accents, and often fails to accurately transcribe what the caller said. But other than that, the rest of the functions are working smoothly, all buttons are responsive and I didn't have any issues with getting the call picked up. Text is transcribed instantly after the caller has finished the whole sentence.

I wish it had more voice lines in terms of informing the caller about what's going on, otherwise, the caller can keep speaking and never know whether the message was delivered or not because Bixby doesn't say anything. The same happens when a call ends, it just ends in silence and the caller might as well assume that something went wrong, unless the user at the other end typed something like “Bye”.

Customization

As for customization options, there's not much to talk about. Three more voices can be downloaded and used, in addition to the default one.

You can change the language, which, at the moment, is limited to Korean, English, and German.

The last customization option is the possibility to edit the quick responses, which is not a big surprise, considering that within the call you can type your own response on the go.

Google call screening

Similarly to Samsung, Google has also added call screening functionality to their “Google Assistant”. Worth mentioning is that Google developed this before Samsung, so this was one of the first publicly available solutions. Let's see how it stacks up to the competition.

Availability

If you were disappointed about the fact that Samsung's service was limited only to their own devices, you'll be even more disappointed about Google's service, as it not only has device limitations but is also limited to certain regions. As you may have already guessed, it's available only on Google Pixel devices. In addition, it's available only in these countries - Australia, Canada, France, Germany, Ireland, Italy, Japan, Spain, the UK, US. Although, regardless of the region, this service is available for free as a built-in feature within the phones of this brand.

Functionality

Since it has region limitations I cannot check this fully myself but I will give a brief overview based on the official information that Google has provided and the reviews and videos that are made by users who have access to this feature.

After you receive a call you have options to answer the call, screen the call, or reject the call. After selecting to screen the call Google Assistant will answer the call for you and you'll see a chat type of view, similarly as it was for Bixby. After the caller says something, Google Assistant will display it to you in the chat and you will also hear the playback of the caller saying the message. From this point you can either answer the call, end the call, or interact with the caller by selecting one of the quick response messages. Unfortunately, it's not possible to type your custom message. Besides manual call screening, Google also offers automatic call screening, but it has a limitation. It's available only for users in the US where it's used for fighting against the common problem of spam robocalls. Automatic call screening will screen all calls for you and based on the response that the caller gives it'll determine whether it's a spam call or a real call. If it determines it is a spam call it hangs up the call but if it determines it is a real call it will show you what the caller is saying. And, of course, the transcript after the call is available in the list of recent calls. So you can freely go through every screened call.

In terms of transcription accuracy, and recognition of accents, the “Google call screening” service is known to be the better service according to the reviews found on the internet, if we're comparing it to “Bixby text call” which we reviewed previously. The text is transcribed instantly after each word is finished, instead of transcribing it only after the whole sentence is finished.

The voice lines are pretty good. We can hear the AI informing the caller if the call has ended on the other, without specifying a reason. Although, there's still something missing that I wish they had added. It's the AI informing the caller about when his message has been sent, otherwise, the caller could be confused about why there's silence and nothing is happening resulting in the call ending too early.

Unfortunately, this time we don't have a chance to type our answers on the go. The responses within the live call screening are limited only to the quick responses that Google has provided.

Customization

There are quite a few customization options available. One of them is the possibility to change the voice of the AI, thankfully not limited only to the default one.

Since we can automatically screen calls, we also have specific settings for that. Within these settings, we can choose what to do with spam calls, possibly faked numbers, first-time callers, and private or hidden numbers. For each call type, there are 3 choices. You can let the call ring and decide by yourself what to do with the call, automatically screen the call, or you can also choose to silently decline the call.

Then we have another new customization option. We can save the audio of the conversation. I find this nice because, even if the transcription is really good, there may still be some cases where it transcribes something inaccurately, and, if that happens, you can just listen back to the audio. So, it not only screens the calls but also records them.

iOS Live voicemail

Since we already reviewed some of the call screening services for Android, let's take a look at what's available for iOS. Apple announced their new iOS 17 update which includes the new call screening feature, named “Live voicemail”. We'll be reviewing specifically this feature because for iPhones there aren't really any other options besides some paid call screening services, which we will not cover in this article, as we’re only reviewing the free-to-use options.

Availability

It'll be available on all Apple phones that will support the new iOS 17 update. Initially, the “Live voicemail” feature will be available only in the US and Canada however, they are planning to add more regions in the future.

Functionality

Since I cannot check this myself fully due to region limitations, I'll give a brief overview of how it functions compared to the Android counterparts by going with what Apple has publicly provided and what users that have access to this in beta have shown in the form of videos or articles.

Overall, this is pretty much the same old voicemail that iPhones had previously with the only difference being that, for this feature, the text is displayed on the incoming call screen while the caller is speaking, after the call has been sent to voicemail. In case you cannot read the transcript, the audio of the voicemail will be available after the call, as it is for the standard voicemail. But, if you manage to read the live transcript, and decide that you want to answer the call, you can do so.

Additionally, it has something in common with the call screening services that are available on Android. It will automatically block any calls that are marked as spam by the carrier, which might be useful to some. But, we have to keep in mind that the calls that are not marked as spam by the carrier will still go through.

Unfortunately, that's where it ends, because that's all you can do with it. It can't do custom replies, or even quick replies like it was possible on the Android counterparts. So, if you've sent a received call to voicemail, and the caller provides an unclear reason for the call, or maybe they don't specify who they are, you just can't get any more information out of that call besides answering the call.

Customization

This will be short, because, sadly, this feature on iOS doesn't offer any customization options. You can only toggle this service on or off.

What are the key challenges of testing a call screening service?

Since testing, overall, is not something you can sum up in a few sentences, let's look at some of the key points that QA should focus on during the manual testing of a call screening service.

Stability

It is very important to verify that the call screening service you're working on is stable and won't fail after a few calls, as it highly relies on the backend, which is unlikely to be well made on the first attempt. It's almost certain that something will fail, requiring that the backend configuration undergo some fine-tuning and re-testing multiple times until the expected stability is achieved.

To achieve stability, the QA should focus on running the tests around the whole call screening functionality, which could include making lots of calls, and checking if they're always screened when expected. It could also include testing the length of the call to verify that it doesn't fail at certain points, and can even test things, such as, what happens when the caller's message is long, what happens when the connectivity of the mobile network is low, or when a different connection type is used (e.g. Wi-Fi, 2G, 4G).

There are a lot of scenarios, and they are highly dependent on the structure of the project, but the key is to pay attention to the call screening process itself since the whole project is built around that.

Quality

Another important aspect for the QA in such a project is to pay attention to the quality, such as checking how accurate the transcription is when the caller speaks, whether it supports multiple languages - all of which must be tested, and for each language, check how well it transcribes it when a different accent is used. These all play a huge part in why someone is using the product, so, if the transcription is inaccurate, it may lead to miscommunication and overall dissatisfaction for the users involved.

Compatibility

While quality and stability are important, both of these should be tested alongside compatibility, to verify that the call screening works not only on one device, or OS version but on most of them. Even if it works fantastically on one device, it may work horribly on dozens of others. That's why the QA must test multiple devices that are different from each other in at least one aspect, whether it's screen size, OS type, OS version, device model, or device manufacturer. Suppose a project is expanding to multiple different countries or carriers. In that case, it's also crucial to verify that these carriers will work just as well as the ones that are already verified.

Sanity and regression testing

It's necessary to have the QA team execute sanity testing for each change - new features or bug fixes. This has to be done before regression testing to ensure that the change in the specific area has not introduced any new issues, and is working. In this way, the testing will be more efficient, and QA engineers doing the regression tests won't have to spend extra time dealing with issues that could've been caught and fixed earlier.

For regression testing, it's necessary to execute tests with various devices, network configurations, and carriers to ensure that any changes made haven't affected existing functionality in other areas. As call screening is highly dependent on the backend, there's a high chance that something might not work as well as it did before, whether it's a carrier-related issue, or perhaps the quality of transcription has regressed. That's why it's important to have a QA engineer, or, for a larger project, a team of QA engineers executing regression tests daily.

The bottom line

After understanding what it is, how it works, and the key differences between some of the available call screening services, it should be clear that the main idea of this service is to prevent spam calls or any calls that you may not want to pick up. The key differences between the different call screening services are the customization options and the fact that some of them may perceive certain accents or languages better than others.

I've had experience with call screening service for a rather long time now, and I find it to be a really useful and neat feature. If you're fortunate enough to use it on your device, and in your region, I strongly suggest trying it. It's a rather new feature that, in my opinion, has huge potential. I'm sure there's way more to come in the future, so stay tuned.

Key takeaways

• Call screening lets you identify who's calling without answering, allowing you to decide whether to answer, reject, block, or gather more information about the call.
• Manual call screening gives you control over each call, while automatic screening handles spam or predefined call types on your behalf, enhancing convenience and efficiency.
• Call screening services like Bixby (Samsung), Google Assistant (Pixel), and iOS Live Voicemail have unique functionalities and restrictions, such as device and regional availability, transcription accuracy, and customization options.
• Testing call screening services should focus on stability, quality, compatibility, and the accuracy of transcription, especially with accents and different languages. Sanity and regression testing are critical to ensuring consistent performance.
• Customization options, such as editing quick responses or choosing AI voices, vary by provider, and accessibility is limited by device compatibility and regional availability.

Ready to enhance your app with cutting-edge call screening technology? Contact us to learn how our expert engineers can help you build a reliable, user-friendly solution tailored to your needs.