IVR Test

IVR: Interactive Voice Response
NLU: Natural Language Understanding
TFN: Toll Free Number
DTMF:  Dual Time Multi Frequency
ANI:  Automatic Number Identification
ACD: Automatic Call Distributor
TTS: Text to Speech
CLI: Calling Line Identifier
CTI: Cloud Telephony Integration
DNS
ARU: Audio Response Unit
MPSR: Multi Pass Speech Recognition
PESQ: Perceptual Evaluation of Speech Quality
PSST: Post Speech Silence Timeout
AX: Agent Experience

CYARA

  CYARA is a IVR Automation Testing tool

  By using CYARA we can create Test Cases, Blocks, Campaigns and Execute the Test Cases.

To Creation TC :  Test Cases Prepared by using Pre-Coded Prompts ( Steps ) with the help of Call Flow..

Campaign : Design the test for Execution

Prompt Analyzer :  ( To verify the Text) Analyze the Prompt or Step.

What is IVR System? (Interactive Voice Response )

IVR is an automated technology that allows interaction with a human being (caller) with the help of voice input and DTMF input (Dual-tone multi-frequency) using the keyboard.

How a phone can be connected to the computer system ?

•  using DTMF. 
•  Using Speech Recognition

 By Using the tone of every key on a telephone keypad, the phones are connected to a computer system. These are known as “Dual-tone multi-frequency (DTMF)” signals. 

DTMF tones are entered using a telephone keypad.

“Speech Recognition”. Here, the caller provides input to the IVR system using his clear voice so that IVR can interpret the input correctly and provide accurate information.

IVR system provides an appropriate voice response to the caller’s DTMF input that is called as “Audio Response Unit (ARU)”. It is a device which provides information to the caller based on the input received from the caller and the information received from the database.

“Automatic Call Distributor (ACD)” is a technology that distributes customer calls, in the order they arrive, to the next available appropriate agent.

IVR application is a tree structure just like the folders and files structure in the Windows system. And this structure in the IVR is called a call flow diagram.

Text To Speech (TTS) is a system that converts normal language text into speech. TTS is a computer generator speech that speaks information like news, email, etc.

IVR Services : 

1. On-premises IVR Services.  Eg :  Call Centers.

2. Cloud Based IVR Services    Eg : 

An IVR system intelligently routes calls to the right department using an automatic call distributor (ACD) and gathers information about the customer so agents are well-prepared for the interaction.

Test Types : 

1. In Bound :  Customer to Agent (where agents/ business representatives receive incoming calls from customers or clients. Eg : to address inquiries, provide support, and handle customer concerns, contributing to positive customer experiences.)

inbound call center are typically familiar with key metrics including Average Handle Time (AHT), First Call Resolution (FCR), Customer Satisfaction (CSAT), Call Abandonment Rate, Occupancy Rate, Call Quality, and Agent Adherence.

2. Out Bound :  Agent to Customer  (where agents initiate outgoing calls to customers or prospects.)  Eg :  telemarketing, sales calls, lead generation, market research, and customer follow-ups,

agents connect with the maximum number of potential prospects within a set period. Outbound call center software is a solution that facilitates and manages outgoing calls made by agents.

Usage of IVR : 

1. Manual Errors:

2. Time-consuming process:

Authentication  Types :  Last 4 digits of card / Account number / DOB / Mail Id / Mobile Number / ZIP Code /  

Input Options : 

1. DTMF : Follow the Caller Instructions Eg : Press 1 for English or 2 for Telugu or 3 for Hindi etc..

2. Voice : Speech Recognition Eg : Here, the caller provides input to the IVR system using his clear voice so that IVR can interpret the input correctly and provide accurate information.

TAGS :- 

• ALPHA NUMERIC
• BYPASS RECOGNITION ( It won’t allow Speech Recognition)
• BARGAIN
• CHOICE  ( GM / GAF / GE / Etc….)
• CURRENCY  ( $ /   / etc..)
• DATE
• DIGITS ( Range of Numbers)
• END CALL ( If No Audio Within Threshold)
• FAX
• IGNORE
• INSERT
• LANGUAGE
• OPTIONAL
• OPTIONAL SPEECH  ( Either it will play or not. Eg : Either will play in Business Hours or After Business hours ).
• OPTIONAL AUDIO LENGTH
• TIME
• WAIT FOR HANGER
• WILDCARD

NLU / Non NLU : ( Natural Language Understanding)

NLU : Before Authentication ( tell me why are you calling )

Non – NLU :  After Authentication 

Voice Quality : 

1. MPSR : Multi Pass Speech Recognition 

2. PESQ : Perceptual Evaluation of Speech Quality

MPSQ :  We can give anything in Plain text.

PESQ :   We can give Audio file as a Input.

               Note : Always we can prefer MPSR

Replay Type : 

1. Audio File  eg : Pre Recorded Voice (We have to give input through Audio File)

2. DTMF   Eg : 1,2, 3, 4, …. Some text entered with the help of keypad. (Alphanumeric)

3. Speech   Eg : Numbers or Words (Numbers means need to be enter digits with space.)

Expect to hear :  What we need to play.

Replay with : Replay for Expect for some value.

Min / Max Pause time : (This is related to Waiting Time). It is used in very rare cases.

it is waiting time between the Steps.

It will wait after PSST specified value (so it is depending on PSST).

Eg : Step 1 : PSST --- 1 sec here it will wait 1 sec before playing next step.

        Step 2 : Min/Max/Pause time ---3 to 5 sec (3/5 sec) here it will wait between 3 to 5 sec.

Note : Min/Max/Pause time will start after the PSST.

Minor /Major threshold: This is related to Response Time.

Her we will check the time taken for Response Time.

Eg : 

1. Min & Max is 3/5 sec means 

here If the Response Time taken below 3 sec means we will get Success Status.

        Or     If the Response Time taken between 3 to 5 sec means we will get Satisfactory Status.

        Or     If the Response Time taken more than 5 sec means we will get Failure status.

       But No need to bother about when we will get Failure status response.

Major / Minor Confidence Threshold :- 

    Fail / Satisfactory / Success.

ASR is able to Understandable Speech.

Confidential level : Either it is Understandable or Hearable.

(Threshold of expect to hear Recognition).

Confidential Level Major / Minor Confidential Level

89.2 Success 60/80

99.7 Success 60/80

59.5 Failed 60/80

50.8 Failed 60/80

70.4 Satisfied 60/80

Here getting success status whenever Confidential level more than 80 

Response Time Minor / Major Threshold Level

0.58 Success 3/5

7 Failed        3/5

3 Success 3/5

4 Satisfied 3/5

2 Success 3/5

Here getting Response Time as Success whenever The Response Time is Less than 3

After you create a Test Case, the Test Case must be validated to ensure it runs successfully under normal conditions. 

If the results are unsatisfactory, make modifications to the test script until a satisfactory result is obtained. 

You must validate Test Cases before applying them to a Campaign

Failures :

• TFN Connection Issue
• PSST time Issue
• Confidential level Issue
• Response Time Issue
• Call not connecting within time period  : to Increase the Major/Minor Threshold level

Campaign Issues :-

Port are exceeded 

Parameterization : 

We can do the parameterization in testcase Screen it self but we can run only one iteration there it self.

In Campaign we can run multiple iterations by using parameterization.

Port Issue : 

       For Exceeded port issue, we can increase the Port Numbers or we can wait till to complete existing Execution to run present one.

Concurrent Ports :

      100 Ports ---------- 30 call

      Max CAPS ---------- 20 means sending 20 Calls Per Sec

Alarm Setting flow :   Alarm settings  Campaign Mapping  Alarm Mapping to Campaign Campaign Execution

Blocks : 

 Blocks means Reusable

Eg : -There are some common steps for 5 test Cases. These Common steps we can reuse for all the Test cases by using Blocks

Steps :-

Go to Create Test Case

We can use Description of TC which is created as Block test Case Previously.

Convert to Block to Normal Test Case.

here we need use Flattren Step tab. For unblock the that Blocked Step to avoid modifying by Others.

Block Details : 

Channel

Test Type

Name

Folder Path

Description 

Click on new Step button :- Here we will get New Step POPUP ( This is like New Test case Steps. It is same as like New Test Case).

 So. This block we can use in Test case 

Go to Test case and Click on New Test Case and Enter 

Call Explorer : 

It is used in Cloud CYARA

It is Nothing but to call to Agent with the help of Browser instead of using Mobile keypad.

Pulse Campaign :  For Alarm profile

 Click an Alarm Profile

 set the alarm schedule

                        Time

Mon                        -----

Tue                         -----

Wed                        -----

Thu                         -----

Fri                         -----

Sat                         -----

Sun                         -----

Either total week or any specific day 

We can set it as Mobile and also MAIL to get the mail or SMS

Mapping to Campaign as same like as Campaign settings as design schedule.

Import / Export TC’s : 

To change the TFN in TC instead of changing in every time we will use Import/Export option settings.

Eg : TFN is changed in every Application / TC

go to select Action

Select Export

Select Export Type  1. XML 2. XML spread sheet screenshot

Click on OK ( here the file is downloaded).

Do the Modification ( by replace the values or change the values manually).

Save it

Go to tool and import it for specific TC.

Questions :-

What are the key challenges in IVR testing?

Some of the key challenges in IVR testing include:

    Verifying the accuracy of speech recognition and voice prompts

    Dealing with different accents and languages

    Testing the system’s response to various user inputs

    Ensuring compatibility with different telephony infrastructure

    Handling high call volumes and concurrent users

How do you handle different scenarios in IVR testing?

In IVR testing, it is important to test the system’s response to different scenarios, such as:

Valid inputs: Testing the system’s ability to recognize and respond appropriately to valid inputs.

Invalid inputs: Testing the system’s handling of invalid or unexpected user inputs.

Error conditions: Testing the system’s response to errors, such as network failures, timeouts, and system crashes.

Interrupted calls: Testing the system’s ability to handle interrupted or abandoned calls.

Integration with other systems: Testing the integration of the IVR system with other backend systems, such as CRM or database systems.