Fluency measures and L2 speaking proficiency levels

The relationship between automatic fluency measures, including speed, breakdown and repair measures (e.g., articulation rate, silent pause ratio, the number of repairs per clause), and overall proficiency levels has gained rising interest. So far, only a few studies have investigated which fluency measures can distinguish between proficiency levels (e.g., Tavakoli, Nakatsuhara, & Hunter., 2020; Tavakoli, Kendon, Mazhumaya, et al., 2023), and found that several temporal fluency measures (e.g., articulation rate, speech rate) and some breakdown fluency measures (e.g., frequency of mid-clause/end-clause pauses, length of mid-clause/end-clause pauses) can do so but repair fluency measures cannot.
The current study advances this line of research by testing which fluency measures can distinguish between IELTS speaking proficiency levels (Understanding and explaining IELTS scores n.d.). Using a data set taken from a larger project investigating the reliability of commonly used automatic complexity, accuracy and fluency (CAF) measures in L2 English oral data (Wu et
al., 2023), we examined if the 27 automatic fluency measures obtained by Praat (de Jong et al. 2021) and CLAN (MacWhinney 2000) would differ across IELTS band levels 5.0, 5.5, 6.0 and 6.5 (CEFR higher B1 to B2 levels) in the IELTS Speaking Test of 49 L1 Chinese learners of English. These band scores were based on speaking samples collected from a mock test using published IELTS Speaking Test materials (e.g., Cambridge University Press, 2016), which were rated by two expert raters who had over ten years of experience in IELTS training.
Our analysis (see Table 1) showed that two temporal and two breakdown fluency
measures (articulation rate, length of run, silent pause ratio, and mid-clause silent pause ratio) could differentiate between lower IELTS levels (5.0 and 5.5) and the highest level (6.5) (c.f. Tavakoli, Kendon, Mazhumaya, et al., 2023). For the remaining measures, no comparisons between levels reached significance (effect sizes ranging from .01 to .21). Overall, none of the measures can differentiate neighbouring IELTS proficiency levels.