Tests with everyday sentences as test material

Multiple sentence-level speech perception tests in noise are available in English. An overview of the most well-known tests is presented in Table 1.

This review will use these tests to demonstrate the different characteristics of sentence-level speech perception tests in noise, since most of the

validation data and reference values for NH listeners were easily available in English, unlike for many speech perception tests in different languages.

Due to variation in the development and validation processes, not all the information in Table 1 was available for all the tests. Where possible, data not reported in the original publication have been retrieved from

subsequent studies on the tests.

Even though all the tests in Table 1 use everyday sentences, the content and difficulty of the test material varies significantly. The Hearing in Noise

44

Test (HINT (Nilsson, Soli and Sullivan, 1994)), for example, uses very simple and redundant sentences (“The chicken laid some eggs.” or “A ball broke the window.”) based on Bamford-Kowal-Bench (BKB) sentences, which in turn were based on natural language samples from school-age children with HL (Bench, Kowal and Bamford, 1979). BKB-SIN uses sentences from the same original BKB material but presents them in background noise (‘BKB-SIN Test’, 2005). On the other hand, AzBio (Spahr et al., 2012) and the Perceptually Robust English Sentence Test Open-set (PRESTO (Gilbert, Tamati and Pisoni, 2013)) use longer and more complex sentences with less contextual cues (e.g., “Only the ants' coiled carcasses remained after the extermination.” from AzBio, or “A moth zig-zagged along the path through Otto's garden.” from PRESTO (Plotkowski and Alexander, 2016)).

Speaker characteristics also vary from a single trained speaker (Nilsson, Soli and Sullivan, 1994) to multiple speakers with distinct dialects (Gilbert, Tamati and Pisoni, 2013).

The type of background noise affects speech perception and test

difficulty. Fluctuating and informational maskers (i.e., maskers that contain recognizable speech) impair speech perception the most for HI listeners (MacPherson and Akeroyd, 2014; Plotkowski and Alexander, 2016). Most of the previously discussed tests use babble noise as their background noise (Table 2). HINT, QuickSIN, BKB-SIN, and AzBio have a designated

background noise each, and it is included in the test material. The test material for PRESTO contains only the speech recording, which can be then combined with different types of background noise (Faulkner et al., 2015;

Shader et al., 2020; DeRoy Milvae, Alexander and Strickland, 2021).

45

Table 1. Overview of the speech material of commonly used English speech perception tests in noise. Test nameTest material Speaker and style

No. of test lists

No. of sentences / list

No. of unique test sentences

Difficulty Examples of test sentences Hearing in Noise Test (HINT) (Nilsson, Soli and Sullivan, 1994)

Short everyday sentences based on Bamford- Kowal-Bench sentences Male speaker; clear pronunciation 25 10250 Easy

The dog played with a stick. The children helped their teacher. QuickSIN (Killion, Niquette and Gudmundsen, 2004)

Institute of Electrical and Electronics Engineers sentences (low- context sentences) Female speaker121 6 126

Easier than AzBio, more difficult than HINT

The lake sparkled in the red hot sun. Fake stones shine but cost little.3 BKB-SIN (‘BKB-SIN Test’, 2005)

"Americanized" Bamford-Kowal- Bench sentences Male speaker 18 Lists 1-8: 10 Lists 9-18: 8 160 Easy She spoke to her son. The little baby is sleeping.4

46

Table 1. Continued. 1) The material includes 3 additional list pairs that are equal in intelligibility, if used in pair 2) 15 test lists released as a commercial CD; 8 separate lists used in the MST 3)Quick Speech in Noise (QuickSIN); 4)Minimum speech test battery for adult cochlear implant users, 2011; 5) Plotkowski and Alexander, 2016 Test nameTest material Speaker and style

No. of test lists

No. of sentence s / list

No. of unique test sentencesDifficultyExamples of test sentences AzBIo Sentences (Spahr et al., 2012)

Diverse everyday sentences from newspapers, magazines etc.

2 male and 2 female speakers; "normal conversational pace and volume"

152 20 300 Difficult

He cut out his carbohydrate intake and lost weight. She was top banana in the shock department.4 Perceptually Robust English Sentence Test Open-set (PRESTO) (Gilbert, Tamati and Pisoni, 2013)

TIMIT sentences (phonetically diverse, low- context sentences)

A different speaker for each sentence in a list (9 males, 9 females), at least 5 different dialects in a list

19 18342 Difficult

See, he's already snapping it up, the dumb jerk. With a whirling jump, it could get into gear.5

47

The speech perception tests in noise that use everyday sentences can be divided into two groups based on the test procedure they use. Some tests (AzBio (Spahr et al., 2012)) use the same fixed presentation level throughout the test and present the results as a percentage of correct answers. Other tests (HINT (Nilsson, Soli and Sullivan, 1994), BKN-SIN (‘BKB- SIN Test’, 2005)) present the test material at multiple SNRs and report results as an SRT. Sometimes, the same speech material is used with both test procedures. For example, HINT (Nilsson, Soli and Sullivan, 1994) was originally developed to be used as an adaptive test, with the SNR changing based on participants’ answers. However, especially in the US, HINT has frequently been used with a fixed presentation level, either without noise (Francis, Yeagle and Thompson, 2015; Shearer et al., 2017) or with a fixed background noise, for example at +10 dB SNR (Reiss, Perreau and Turner, 2012; Dillon et al., 2016).

The test paradigms of QuickSIN (Killion, Niquette and Gudmundsen, 2004) and BKB-SIN (‘BKB-SIN Test’, 2005) use fixed, decreasing SNRs. Both tests start at a relatively easy SNR, but the SNRs decrease (i.e., become more difficult) with fixed step sizes, irrespective of the listener’s responses.

The results are reported as an SRT that is based on the number of correct answers and calculated using the Spearman-Kärber equation. Speech perception tests in noise that use a single, fixed presentation level or multiple but predetermined presentation levels have less requirements for the test equipment. The whole test material can be recorded into one audio file, as the sentences are always played in the same order at the same SNR. Since the test procedure remains the same irrespective of the listener’s answers, the full recording can just be played as such through an audiometer. Tests that use adaptive test procedures, where the

presentation level is based on previous answers, typically need separate, possibly expensive, software to calculate the next presentation level.

48 Table 2. Overview of test practices of commonly used English speech perception tests in noise. Test nameBackgroun d noiseTest procedure Scoring No. of scored items / test list Practice lists Hearing in Noise Test (HINT) (Nilsson, Soli and Sullivan, 1994)

Spectrally matched speech- shaped noise

Adaptive test protocol with 4dB steps for the first 4 sentences and 2 dB steps for the rest Sentence scoring, SRT is the average SRT of sentences 5 – 11 10

3 practice lists, no official recommendation on the use of practice lists QuickSIN (Killion, Niquette and Gudmundsen, 2004)

Four-talker babble fixed to speech material Six fixed SNR with the noise increasing in 5 dB SNR steps (25 to 0 dB SNR) after each sentence

Keyword scoring. SRT calculated using the Spearman-Kärber equation: SRT = 27.5dB SNR - (No. of correctly recognized keywords)

25

3 practice lists, no official recommendation on the use of practice lists BKB-SIN (‘BKB-SIN Test’, 2005)

Four-talker babble fixed to speech material Fixed SNRs starting at +21 dB with the SNR decreasing by 3 dB SNR after each sentence Keyword scoring. SRT is calculated using the Spearman-Kärber equation: SRT = 23.5 dB SNR - (No. of correctly recognized keywords). The final list pair score is the average of the two list scores

Lists 1-8: 31 Lists 9-18: 25 One list (half of a list pair) is recommended.

49

Table 2. Continued. SNR, signal-to-noise ratio; dB SNR, decibel signal-to-noise ratio; SRT, speech recognition threshold

Test nameBackground noiseTest procedure Scoring

No. of scored items / test list

Practice lists AzBIo Sentences (Spahr et al., 2012)

Ten-talker babble Fixed presentation typically at 0, +5, or +10 dB SNR Word scoring, results expressed as the percentage of correctly recognized words

135–157 No recommendation Perceptually Robust English Sentence Test Open-set (PRESTO) (Gilbert, Tamati and Pisoni, 2013)

No designated background noise

Not specified. Sentences have been presented at different SNRs (i.e., 0, -3 dB SNR) Word scoring, results expressed as the percentage of correctly recognized words

76No recommendation

50

Using a fixed presentation level for a speech perception test in noise has its drawbacks. A fixed presentation level predisposes to floor or ceiling effects (i.e., the test being too difficult or too easy), especially if the same test material is used to assess hearing before and after rehabilitation (Gifford, Shallop and Peterson, 2008; Dornhoffer et al., 2021). With adaptive test procedures, floor effects are possible if the listeners have very poor hearing, but there is no risk of ceiling effects as the adaptive test procedure can decrease the SNR to beyond the SRTs of NH listeners.

Despite these well recognized drawbacks, tests with a fixed presentation level are still extensively used, especially in the US (Carlson et al., 2018;

Wick et al., 2020; Lundberg et al., 2021; Murr et al., 2021), where FDA criteria and many insurance coverage criteria are based on speech

recognition scores from sentence-level speech perception tests presented at a fixed level (‘MED-EL Premarket Approval’, 2001; ‘Medicare National Coverage Determinations Manual 50.3 – Cochlear Implantation’, 2005;

‘Nucleus cochlear implants, Physician’s Package Insert’, 2020).

HINT is the only test in Table 2 to use sentence scoring. The other tests use word or key word scoring with a variable number of scored items per test list (see Table 2). Research papers published after the original

publications have found some of the original test materials to be too heterogeneous and have recommended revisions and omissions of some of the original test lists (Schafer, Pogue and Milrany, 2012; Faulkner et al., 2015). Therefore, the number of individual test lists in active use may differ from the number reported in the original publication. However, it is not standard practice in research papers to report whether the testing was carried out using the original or a revised list selection.

Comparison of test reliability related characteristics (i.e., the slope of the psychometric function, test-retest difference, intra- and interindividual variation, and normative reference values) is difficult since different

information has been reported for different tests (see Table 3). For HINT, for example, reliability is reported as the standard deviation of differences in repeated measures within subjects (Nilsson, Soli and Sullivan, 1994),

51

which is not used for any of the other tests. For HINT, QuickSIN, BKB-SIN, and AzBio, the 95% confidence intervals are reported for different

numbers of test lists (Nilsson, Soli and Sullivan, 1994; Killion, Niquette and Gudmundsen, 2004; ‘BKB-SIN Test’, 2005; Spahr et al., 2012). However, for HINT these data are available only for NH listeners, and for QuickSIN only for HI listeners (Killion, Niquette and Gudmundsen, 2004). Tests that use fixed presentation levels have no uniform practice for reporting reference values, and reference values are reported at different SNRs (i.e., 0 dB SNR or +10 dB SNR) for different tests (see Table 3). For PRESTO, reference values are reported for different SNRs and for different types of background noise. All the reference values reported in the original publications are based on binaural presentation. Monaural data is

available from later studies; for example, Wilson et al. (Wilson, McArdle and Smith, 2007) evaluated HINT, BKB-SIN, and QuickSIN monaurally, and Plotkowski et al. (Plotkowski and Alexander, 2016) evaluated PRESTO in NH and HI listeners using monoaural presentation.

Table 3 demonstrates why speech perception test results are always test-specific, and why direct comparison between results from two different speech perception tests in noise should be avoided. No current, official reference data are available for HI listeners for any of the speech perception tests in noise discussed here. Very few studies have compared two or more different speech perception tests in noise in the same,

clinically representative patient population (Wilson, McArdle and Smith, 2007; Jansen et al., 2012; Dillon et al., 2016). No official recommendations exist on how to compare results obtained with different types of speech perception tests in noise, even though the problems related to the lack of uniform performance data are well recognized (Adunka et al., 2018;

Prentiss, Snapp and Zwolan, 2020; Biever, Amurao and Mears, 2021).

52

Table 3. Overview of test characteristics related to test reliability for commonly used English speech perception tests. ^Tes

t name

Reference value for NH listeners

SD in NH listeners Slope of the psychometric function 95% Confidence interval for NH listeners

Critical difference for comparison (95% CI) NH listeners

Critical difference for comparison (95% CI) HI listeners Hearing in Noise Test (HINT) (Nilsson, Soli and Sullivan, 1994)

-2.92 dB SNR0.78 dB SNR10.6%/dB SNR21 list: 2.41 dB SNR 2 lists: 1.49 dB SNRNot reportedNot reported QuickSIN (Killion, Niquette and Gudmundsen, 2004)

2 dB SNR1.25 dB SNR10.8%/dB SNR3 Not reportedNot reported1 list: 3.9 dB SNR 2 lists: 2.7 dB SNR BKB-SIN (‘BKB-SIN Test’, 2005)-2.5 dB SNR 0.8 dB SNR11.9%/dB SNR31 list: 1.6 dB SNR 2 lists: 1.1 dB SNR1 list: 2.2 dB 2 lists: 1.6 dB 1 list: 4.4 dB 2 lists: 3.1 dB

53

Table 3. Continued. NH, normal-hearing; SNR, signal-to-noise ratio; dB SNR, decibel signal-to-noise ratio; SRT, speech recognition threshold. 1 Schafer, Pogue and Milrany, 2012; 2 Soli and Wong, 2008; 3 Wilson, McArdle and Smith, 2007.

Test nameReference value for NH listeners SD in NH listeners Slope of the psychometric function 95% Confidence interval for NH listeners Critical difference for comparison (95% CI) NH listeners

Critical difference for comparison (95% CI) HI listeners AzBIo Sentences (Spahr et al., 2012) SNR 0 dB: 72%1 SNR -3dB: 48%1SNR 0 dB: 9.1%1 Not reported4.7% SNR 0 dB: 1 list: 18% 2 lists: 12%1

SNR +10 dB: 1 list: 23.3%, 2 lists: 15.5%1 Perceptually Robust English Sentence Test Open-set (PRESTO) (Gilbert, Tamati and Pisoni, 2013)

Babble at 0 dB SNR: 71%.

Babble at 0 dB SNR: 6.2%.Not reportedNot reportedNot reportedNot reported

54