Introduction
AI-assisted speech coaching refers to software systems that use automatic speech recognition (ASR), natural language processing (NLP), and acoustic signal analysis to evaluate and provide feedback on a person's spoken communication, typically covering measures such as filler-word frequency, speaking pace, pause patterns, vocal pitch and volume, articulation, and in some systems lexical or grammatical content. The category sits at the intersection of several more established fields. Its analytical methods come from speech and language technology research, particularly automatic disfluency detection and computer-assisted pronunciation training (CAPT), while its behavioral framing comes from the coaching and professional-development literature, particularly executive and workplace coaching. As of the mid-2020s, no single Wikipedia-citable source defines "AI speech coaching" as a discrete category; the practice instead appears piecemeal across the natural language processing, computer-assisted language learning, and digital coaching literatures. This article surveys that literature to describe the category's methods, typical applications, and the evidence available on its effectiveness.
The professional coaching field that AI speech coaching tools operate alongside is itself large and still expanding. The International Coaching Federation's (ICF) 2023 Global Coaching Study, based on 14,591 responses from coach practitioners and coaching-skilled managers across 157 countries, estimated 109,200 active coach practitioners worldwide in 2022, a 54% increase over the 2019 estimate, generating an estimated $4.564 billion in global annual revenue, a 60% increase over 2019 [1]. The profession has also begun to formally define the AI-assisted portion of its own field: in April 2021 the ICF convened a working group of technology, coaching, and standards-development specialists that, in 2024, published an Artificial Intelligence Coaching Framework and Standards document setting out technical and ethical requirements, including data encryption, algorithmic transparency, bias mitigation, and disclosure obligations, specifically for AI-mediated coaching systems [2]. That a professional body active in executive and workplace coaching has moved to formally standardize AI-assisted coaching, rather than leave it as an unregulated adjacent practice, is itself evidence that the category described in this article had, by the mid-2020s, become established enough to require governance.
Definition and Taxonomy
AI-assisted speech coaching systems can be organized along two largely independent axes: when feedback is delivered relative to the act of speaking, and what kind of signal the system analyzes to produce that feedback.
Real-Time Versus Asynchronous Feedback
Real-time (or in-situ) systems analyze speech as it is produced and surface feedback during the speech act itself. An early research example is Rhema, a wearable interface built for Google Glass that used automatic speech recognition to track a speaker's volume and speaking rate and displayed live visual cues during an ongoing presentation [3]. Asynchronous (or after-action) systems instead record a full speech sample and generate a feedback report once the speech is complete. An example from the research literature is a multimodal assessment framework that extracted audiovisual nonverbal behavior features from recorded presentations and modeled their correlation with expert evaluators' judgments of speaking performance [4].
At least one controlled study has compared the two approaches directly. Chollet, Marsella, and Scherer (2022) used an interactive virtual-audience public-speaking training system to compare real-time feedback, delivered through the reactions of virtual audience characters during a speech, against after-action feedback delivered as a personalized report once the speech was over. They found the two forms of feedback served different functions: live feedback made the training experience more immersive and increased participants' motivation to keep using the system, while after-action feedback produced positive training outcomes specifically when it included personalized elements. The authors described the two feedback types as complementary contributors to training efficacy rather than one being straightforwardly superior to the other [5]. This finding is a useful caution against assuming that real-time feedback is inherently more pedagogically effective simply because it is more technically demanding to build.
NLP-Based Versus Acoustic-Signal-Based Analysis
A second, largely orthogonal axis concerns what the system measures. NLP-based (or "language-based") approaches first transcribe speech to text using automatic speech recognition and then apply natural-language-processing techniques, such as keyword spotting or sequence classification, to the resulting transcript to identify disfluencies, filler words, or characteristics of word choice and sentence structure. Acoustic-signal-based approaches instead operate directly on the audio waveform, extracting features such as pitch (fundamental frequency), energy, spectral characteristics, and pause duration, without requiring an intermediate transcription step.
Research comparing the two approaches has found that acoustic methods can outperform transcript-based methods for disfluency detection specifically because they are not limited by transcription error. In a 2023 study of frame-level automatic disfluency detection, Romana, Koishida, and Provost reported that an acoustic-based approach that does not require transcription as an intermediate step outperformed the ASR language approach, and that multimodal architectures combining both acoustic and language information outperformed either unimodal approach on its own [6].
Because the two axes are largely independent, a given system's position in this taxonomy does not fully determine its design elsewhere. A real-time system can rely on acoustic features alone, on an NLP pipeline running against a live, incrementally updated transcript, or on both; an asynchronous system has the same range of choices, with fewer latency constraints on how much processing it can perform. In practice, most research prototypes and applied systems described in the literature combine elements of both signal types rather than occupying a single quadrant of the taxonomy, reflecting the multimodal performance advantage reported by Romana, Koishida, and Provost [6].
Technical Methods: Detecting Fillers and Disfluencies
The behaviors most AI speech coaching systems are built to detect, filled pauses such as "um" and "uh," word or phrase repetitions, false starts, and prolongations, are collectively termed disfluencies in the linguistics and speech-technology literature. Foundational psycholinguistic research by Clark and Fox Tree (2002) analyzed filled pauses in spontaneous speech corpora and argued that "uh" and "um" are not simply performance errors but conventionalized signals speakers use to announce upcoming minor or major delays, for instance while searching for a word or deciding what to say next [7]. This is a relevant nuance for how the category should be understood: automated filler detection targets a linguistic phenomenon with a genuine communicative function, not merely a defect to be eliminated outright.
Building automated detectors for these phenomena requires labeled data at scale. Zhu, Caceres, and Salamon (2022) introduced PodcastFillers, a dataset of naturally occurring podcast audio containing more than 35,000 annotated filler words and 50,000 annotations of other non-lexical sounds such as breaths, laughter, and repetitions, and used it to benchmark a detection pipeline combining voice activity detection, automatic speech recognition, and a downstream classifier, reporting that leveraging ASR substantially outperformed a simpler keyword-spotting baseline [8]. Other work has approached the same problem with deep neural architectures operating on audio directly: Chatziagapi and colleagues (2022) combined convolutional and recurrent neural network models over acoustic features with ASR-derived information to detect filled pauses, reporting improved classification performance from combining the two signal types over using either alone [9].
A typical detection pipeline in this literature proceeds in stages: audio capture, voice activity detection to isolate speech segments from silence, automatic speech recognition to produce a time-aligned transcript, extraction of acoustic features such as pitch contour, energy, spectral coefficients, and pause duration from the raw waveform, and a final classification stage that assigns each candidate segment to a disfluency category, filled pause, repetition, prolongation, or fluent speech, before that information is passed to a feedback-generation layer. These methods have begun to move from research benchmarks into applied educational tools. A 2024 study conducted at Escuela Superior Politecnica del Litoral in Ecuador built an automatic feedback system for oral presentation training using OpenAI's Whisper, a transformer-based automatic speech recognition model trained on 680,000 hours of multilingual audio [10], to detect filler words and filled pauses in Spanish-language student presentations. The authors reported that Whisper "demonstrates promise" for this task, while also noting that general-purpose ASR systems, because they are typically trained to smooth over disfluencies for other purposes, can require additional annotated data or fine-tuning to reliably flag them as a coaching signal rather than noise [11]. This illustrates how NLP-based and acoustic-signal-based methods are combined in practice rather than treated as mutually exclusive design choices.
Applications in Professional and Language-Skill Development
Two adjacent bodies of research inform how these methods are applied. The first concerns public speaking and presentation training more broadly, exemplified by systems such as Rhema [3] and multimodal assessment and virtual-audience frameworks developed in speech and human-computer-interaction research [4][5], which target delivery-level skills such as pacing, volume modulation, filler-word frequency, and nonverbal behavior largely independent of a speaker's first language. These systems are typically situated in contexts adjacent to executive and workplace coaching, such as presentation training, interview preparation, and general professional communication skill-building, though as discussed below, direct outcome comparisons against traditional executive coaching delivery are not yet established in the literature.
The second is computer-assisted pronunciation training (CAPT), a longer-established subfield of computer-assisted language learning concerned specifically with second-language learners' pronunciation and, increasingly, fluency. Early CAPT research emphasized that the technology's pedagogical value depends on interface and curriculum design choices, not simply on the sophistication of the underlying speech-recognition engine [12]. A 2024 systematic review of the CAPT literature found that AI-based tools were consistently associated with improvements in pronunciation accuracy and, in some studies, broader speaking fluency, while also noting substantial variation in study quality and design across the field [13]. More recent longitudinal research on learner responses to AI-based pronunciation feedback has examined not just whether feedback improves performance but how learners appraise and act on that feedback over repeated sessions, a relevant qualifier given that automated feedback systems typically generate far more feedback than a human tutor would in the same amount of time, and learners do not act on all of it equally [14]. Together, these two bodies of research indicate that automated fluency and pronunciation feedback has become a recurring feature category within language learning applications, rather than an isolated experimental add-on.
Comparison With Human-Delivered Coaching
No large controlled study has directly compared AI-delivered speech coaching against human speech coaches on standardized outcome measures; this specific comparison remains an open area of the research literature as of the mid-2020s. Evidence is more developed for two related, broader questions: how effective human-delivered coaching is in general, and how digital or AI-supported delivery compares with self-directed digital tools when a human coach is added to the loop.
On human-delivered coaching, two independent meta-analyses restricted to randomized controlled trials have found moderate positive effects. De Haan and Nilsson (2023), analyzing 39 coaching RCT samples (total n = 2,528) drawn from studies published between 1994 and 2021, reported a best-estimate standardized effect size of g = 0.59 for workplace coaching across leadership and personal outcomes, describing this as a moderate effect while noting evidence of publication bias in the underlying literature [15]. Nicolau, Candel, Constantin, and Kleingeld (2023), analyzing 20 randomized controlled trials of executive coaching, reported an overall effect of Hedges' g = 0.43 on coachee outcomes, with effects varying by outcome category: g = 0.73 for behavioral outcomes, g = 0.52 for personal characteristics, and g = 0.34 for attitudes, and a notably larger effect for goal attainment specifically (g = 1.32) [16].
On the marginal contribution of human coaching within a digital delivery format, a pilot randomized controlled trial of a digital behavior-change platform among young adults at risk for depression and obesity found that participants who received human coach support alongside the app-based tools showed substantially larger reductions in depressive symptoms (Cohen's d = -1.07) than those using a self-care-only version of the same platform (d = -0.32), a between-group effect of d = 0.45. Anxiety symptoms showed a similar pattern (d = -1.02 for the coached condition versus d = -0.12 for self-care, a between-group effect of d = 0.50) [17]. This trial did not involve speech coaching specifically, but it is one of relatively few controlled comparisons isolating the marginal value of human coaching within an otherwise similar digital delivery format, and its result, that adding a human coach substantially increased effect sizes relative to the technology alone, is directly relevant to how AI speech coaching tools are typically positioned: as an adjunct that can operate with or without a human coach in the loop, rather than as a demonstrated substitute for one.
This positioning is consistent with how the coaching profession's own standards body has framed the category. The ICF's working group described AI coaching as "a resource in support of learning, development, and performance that can be used by individuals, teams, and organizations independently or in conjunction with human-to-human coaching activities," and stated that its particular interest was in "how these technologies can best augment human coaches," rather than in evaluating AI systems as freestanding replacements for them [2].
Efficacy Evidence From Digital Coaching and Behavior-Change Research
Because controlled outcome studies specific to AI speech coaching are still limited, the broader literature on digital coaching and AI-mediated behavior-change tools provides the most relevant available efficacy evidence. A 2020 systematic review in the Journal of Medical Internet Research examined 31 studies of AI conversational agents used in health care, including chatbots, embodied conversational agents, and speech-recognition-based systems, and found the evidence was mostly positive or mixed. Usability and satisfaction performed well in the large majority of studies that assessed them (27 of 30, and 26 of 31, respectively), and positive or mixed effectiveness was reported in three-quarters of studies that measured it (23 of 30), though the review also concluded that study quality was often limited and that improved study design and reporting were needed across the field [18].
Randomized controlled trials of app-based coaching interventions outside the speech domain have generally reported statistically significant symptom improvements. A trial of a text-based mental health coaching application among 392 participants (197 intervention, 195 control) at a Malaysian university found statistically significant reductions in depression (P = .02) and anxiety (P = .002) scores over three months relative to a waitlist control, with the largest effects observed among corporate-employee participants [19]. These findings are broadly consistent with the digital behavior-change platform trial described above [17]. Together, they suggest that digitally delivered coaching interventions, speech-specific or otherwise, can produce measurable behavioral and psychological benefits, particularly when paired with some form of human oversight or support, though effect sizes and study quality vary considerably across the literature, and direct evidence for speech-delivery-skill outcomes specifically remains sparse. Extrapolating general digital-coaching efficacy findings to speech coaching specifically should therefore be done cautiously until dedicated outcome studies are available.
Privacy and Data Considerations
AI speech coaching systems process a category of data, recorded human speech, that raises privacy considerations distinct from text-based inputs. Under the European Union's General Data Protection Regulation (GDPR), voice recordings constitute personal data, and processing that extracts features capable of uniquely identifying a speaker (a "voiceprint") can additionally qualify as biometric data subject to the heightened processing conditions of Article 9, which restricts the processing of special categories of personal data [20]. In the United States, the Illinois Biometric Information Privacy Act (740 ILCS 14) explicitly enumerates "voiceprint" among the biometric identifiers it regulates, requiring organizations to obtain informed written consent before collection, publish a retention and destruction schedule, and provide written notice when biometric data is destroyed [21].
A separate line of speech-technology research has focused on technical mitigations for these risks. The VoicePrivacy initiative, launched in 2020, organizes a recurring community challenge aimed at developing and benchmarking speaker anonymization techniques, such as pitch scaling, vocal tract length normalization, and voice conversion, designed to suppress speaker-identifying characteristics in an audio signal while preserving the underlying linguistic content [22]. How widely such anonymization techniques have been adopted in commercially deployed speech coaching products has not been independently studied.
The coaching profession's own standards have also begun to address data handling and disclosure specifically for AI systems. The ICF's 2024 framework requires that AI coaching systems "implement encryption for all data during storage and transmission to ensure confidentiality and security," and establishes baseline and advanced tiers of data-protection requirements applicable to any system that processes client data, including audio [2]. The same framework separately addresses a related but distinct concern, transparency about the nature of the interaction itself, requiring that a client "should know when they're interacting with an algorithm and not a person," on the stated rationale that people behave differently when they believe they are speaking with a human and that undisclosed automation can damage trust; it further requires providers to disclose the general limits of the system's capabilities and to take steps to identify and mitigate bias arising from training data or design choices [2].
FAQs
What distinguishes AI-assisted speech coaching from general-purpose voice assistants or dictation software?
AI-assisted speech coaching systems are built specifically to evaluate the manner of a person's spoken delivery, such as filler-word frequency, pacing, pauses, and pronunciation, rather than to execute commands or transcribe speech to text as an end in itself. They typically combine automatic speech recognition with additional acoustic or natural-language analysis layers whose output is diagnostic, evaluative feedback intended to change a speaker's future behavior, a design goal that general dictation and voice-assistant software does not share.
Is there direct evidence that AI speech coaching tools are as effective as a human speech coach?
No large controlled study has directly compared AI-delivered speech coaching against human coaches on standardized outcome measures. The available evidence instead comes from adjacent literatures: meta-analyses of human-delivered executive and workplace coaching report moderate positive effect sizes, and separate trials of digital coaching platforms in other domains have found that versions including human coach support produced substantially larger improvements than self-directed technology-only versions. Together these findings suggest AI tools are currently better evidenced as complements to human coaching than as demonstrated substitutes for it, a framing that professional coaching bodies have also adopted in their own published standards.
What happens to the audio recordings these systems collect?
Practices vary by provider and are not comprehensively documented in the independent research literature. Because recorded speech can qualify as personal data, and in some jurisdictions as regulated biometric data when processed to extract a voiceprint, frameworks such as the European Union's GDPR and Illinois' Biometric Information Privacy Act impose specific obligations, including consent, retention limits, and security requirements, on organizations that collect and process it. Separately, speech-technology researchers have developed anonymization techniques that can strip speaker-identifying characteristics from audio while preserving its linguistic content, though how widely such techniques are deployed in commercial speech coaching products has not been independently verified.