Perceiving and Modeling the Scope of Question Focus in Korean

Six stimulus sets were created. For each set, a three-way ambiguous sentence containing a CPF was used. All of the ambiguous sentences followed the template in Figure 2. Open and polar question stimuli were created from recordings collected as part of Jones (2016), from eight native speakers of Seoul Korean (six female and two male) aged between 18 and 35, studying at the University of Oxford. For each recording, participants were presented with a screen showing contextual information and the ambiguous sentence. They were asked to read the sentence aloud as a question in such a way that it made sense in context, and such that a hearer would be able to infer the context. Where the target was an open question, the context was “You know some, but not all, details of an event.” and where the target was a polar question, the context was “You don’t know whether an event happened.”

A native speaker of Seoul Korean reviewed the recordings from Jones’ experiment, and for the open and polar question types, selected the recording that most clearly portrayed the associated meaning, resulting in recordings made by five female speakers and one male speaker. The native speaker identifying the utterance types was naïve to the experiment and the underlying research. The native speaker also had broad training in linguistics but very minimal training in syntax and/or prosody.

Statement stimuli were recorded at the time of this experiment by a further female native speaker of Seoul Korean who was asked to read the sentence aloud so that it would be unambiguously understood as a statement. All of the above-mentioned stimuli were recorded digitally with a sampling rate of 44.1 kHz using a professional-grade microphone in a sound-attenuated room.

Using Praat (Boersma & Weenink, 2023) and scripts provided by Lennes (2017), the selected recordings were divided into audio segments following Row 2 of the template in Figure 2. From these segments, a series of incrementally longer utterances was generated, each utterance adding one segment. Thus, for participants, the first constituent and the intermediate constituents were each presented as word-length fragments, and the CPF and verb were presented as fragments that increased one syllable at a time.

Participants accessed the stimuli via the PsyToolkit website (Stoet, 2010, 2017), after confirming that they were native speakers of Korean and giving their consent to participate. Stimuli were divided into three cohorts using a Latin square design such that each participant was presented with six trials, two of each stimulus type (open, polar, statement). These trials were presented in a random order. Within each trial, the individual utterances were presented in turn, in increasing length (see Figure 6, which illustrates the similar procedure for Experiment 2). After each presentation, participants selected a checkbox to indicate whether they had heard a statement, an open question, or a polar question, or whether they did not know. Participants then had to click a button to confirm their choice and move to the next item.

In total, 26 participants started the experiment, of whom 12 completed all six trials and a further participant completed three trials. The remaining 13 participants dropped out of the experiment during the first trial; we discuss this further in Section 3.3. All participants were native speakers of Seoul Korean residing in Korea or the United States, recruited through social networks. Further demographic information was not collected.

Participants were recruited through Prolific.¹ All participants reported their first language as Korean. A total of 124 participants completed the experiment, of which 85 identified as female, 38 as male and 1 did not state a gender. The age range of participants was 18–68 years with a mean age of 32.08 years and an interquartile range of 25–36 years. In total, 102 participants were born in Korea, 17 in the United States, 3 in Canada, 1 in Germany, and for 1 participant, these data were unavailable. In total, 57 participants were residing in the United States, 26 in Korea, and 19 in Canada. A total of 14 participants were residing in other English-speaking countries, and eight were residing in other countries. Participants were paid GBP 1.75 for participation, which represented a payment of GBP 12.00 per hour at the median length of time to carry out the experiment.

Twenty-one sets of stimuli were generated by a native speaker of Seoul Korean, recorded in a sound-attenuated room at a sampling frequency of 44.1 kHz. Each set consists of the same ambiguous sentence, read aloud three times with the speaker asked to produce the sentence as a statement, an open question, or a polar question, respectively, as in Example (5):

(5) hakchangsicel ttay nwukwu-lul mollay sarangh-ayss-eyo

school.days during who/someone-obj secretly love-pst-pol

a. “(I) secretly loved someone when I was at school.” (statement)

b. “Who did you secretly love when you were at school?” (open question)

c. “Did you secretly love someone when you were at school?” (polar question)

Once the recordings were generated, we then created the corresponding TextGrids in Praat (Boersma & Weenink, 2023) using a script readily available online.² Working within Jun’s (2005) description of the Korean AP, we followed Jones (2016) in assuming that focus is associated with expanded pitch range. The schematic diagram in Figure 3 shows an idealized version of the prosodic patterns that we are assuming; the placement of phrase boundaries in the naturally produced stimuli is shown in Table 4. All 63 naturally produced stimuli had an AP boundary after the first constituent, and no stimuli had AP boundaries within either the CPF or the verb. In total, 23 of the 63 stimuli had a boundary pattern matching the schematic diagram (pattern F in Table 4).

The remaining 40 stimuli showed some variation from the idealized pattern in the placement of AP boundaries after the first constituent. Table 5 shows how the presence or absence of AP boundaries at specific points during the stimuli was distributed between the different categories. Open question stimuli were significantly different from the other two categories in the presence of an AP boundary immediately after the CPF $(p<.01),$ but this was not deterministic: for polar question and statement stimuli, eight stimuli in each category did not have an AP boundary after the CPF. There were no differences between the three categories for boundary placement during or after the adverbial.

In 10 of the 21 stimulus sets, the AP boundary patterns were identical across all three categories. Of the remaining 11 sets, 10 had polar questions and statements patterning together; one set had open and polar questions patterning together; and one set had open questions and statements patterning together.

Having recorded and analyzed the baseline stimuli, we proceeded to generate test stimuli by manipulating the F0 contour in the region of interest, which was the CPF for open question stimuli and the verb for polar question stimuli. Stimuli for declarative statements were not manipulated. All manipulations were based around F0 measurements from the entire AP that included the region of interest. Up to four points were measured, depending on how Jun’s T-H . . . L-H tone pattern was realized. An example of the measurement points for one AP in one stimulus is shown in Figure 4.

From each baseline open and polar question utterance, we created a set of five test stimuli. The sets had four equal steps between the full extent of pitch expansion in the region of interest and a baseline F0 contour measured in the corresponding region of the comparator stimulus. For open questions, the comparator was the polar question, and for polar questions, the comparator was the open question. All manipulations reduced the height of the F0 peaks in the regions of interest, and the minimum and maximum values of those peaks matched F0 values that were naturally produced.

For open questions, the region of interest was not at the edge of the sentence, and so initial and final F0 were the same for original and manipulated stimuli. For polar questions, the verb was in focus, and so the region of interest was the AP containing the verb. Because this AP was also IP-final, the AP-final H tone was replaced by the LH% boundary tone associated with questions. Because the statement had a sentence-final HL% boundary tone, the pitch expansion contrast for polar questions was with the corresponding open question. Again, we created test stimuli that have four equal steps in the region of interest, shown in Figure 5.

For polar questions, we followed Jones (2016) and assumed that there was also an expanded pitch range at the final LH% tone. We therefore also created four equal steps in the utterance-final verb and particle. Here, the extent of manipulation was the natural F0 difference between open questions and polar questions.

The aforementioned variants were produced by manipulating the F0 contour using Praat (Boersma & Weenink, 2023) using the following procedure:

For the open and polar questions, the start-maximum F0 range for the LH% tune $T$ was calculated in the same way, Formula (2).

The start-end F0 range for APs $E$ was calculated by subtracting the starting log F0 from the ending log F0, Formula (3).

The F0 contour of the natural open question was streamlined to remove all points except the start, the maximum, the minimum (if this was not also the end of the phrase), and the end of the phrase. The maximum point was then changed to the revised F0. Four variants were produced for each open question, with the proportion $x$ being equal to 0%, 25%, 50%, and 75%, respectively.

We expected that manipulation might reduce the audio quality, and thus, the intelligibility of the stimuli. All 168 manipulated stimuli were validated by asking native speakers of Seoul Korean $(n=9)$ to judge their intelligibility on a five-point Likert-type scale (1 = completely unintelligible, 5 = completely intelligible). The mean acceptability across all stimuli was 4.54, but 16 of the 105 stimuli had a mean score below 4.0, and these were excluded from the results.

Once the full set of manipulated utterances was ready, the individual gating stimuli were prepared, with one set of stimuli for each manipulated utterance. The stimuli were segmented using Praat following the model in Figure 6; there were five segments for each stimulus. The open question region of interest with the CPF was first presented in Stimulus 2, and the polar question region of interest was first presented in Stimulus 4. Only in Stimulus 5 did participants hear the tune associated with either a question or a declarative statement.

Following segmentation, the gating stimulus files were produced using a script amended from the Speech Corpus Toolkit for Praat (Lennes, 2017).

Participants were presented with stimuli via a website written using OpenSesame (Mathôt et al., 2012) and jsPsych (de Leeuw et al., 2023), which was powered by a JATOS server (Lange et al., 2015) hosted at the University of Groningen. After giving consent to participate, participants were shown instructions, which included explanations for what statements, open questions, and polar questions are, respectively. Having confirmed that they had read the instructions, participants continued to the data collection screen. Four buttons were presented in a horizontal row at the center of the screen with labels in Korean acik molukessta “Don’t know yet”; kaypanghyeng cilmun “Open question”; phyeyswayhyeng cilmun “Closed question”; cinsul “Statement.” Below the buttons was the question etten mwuncangul tutko issnayo? “What sort of sentence are you listening to?,” and at the top of the screen was a bar showing progress through the experiment.

Stimuli were played automatically when the page loaded, and once the participant had made a choice, the page re-loaded to play the next stimulus. It was not possible for participants to replay the stimuli.

Because each stimulus set contained 11 members (five variants of open questions, five variants of polar questions, and one declarative statement), participants were randomly allocated to one of 11 cohorts. Each cohort heard one member of each of the stimulus sets in a Latin Square design, a total of 21 trials with no repetition of stimulus sets. During the experiment, each participant was presented with a mixture of open questions, polar questions, and sentences, and for the open and polar questions, there was a mixture of the five variant levels of prosody. The order of presentation of the stimulus sets was random for each participant.

For each trial, participants were presented with the five stimuli in the utterance set, in increasing order of length. Once all five stimuli had been heard, the next utterance set was presented. Four times during the experiment, at the end of an utterance set, participants were asked a question to confirm they were paying attention, in line with guidance from Prolific. The question was a multiple-choice question, and the question included the answer that was required to be given. Participants who answered two or more of these attention questions incorrectly were excluded from the study.

In this section, we present the raw experimental data. Participants who failed the attention checks as described above were excluded (2/126). All data points from the remaining 124 participants were included in the analysis.

Did participants accurately disambiguate the stimuli? Figure 7(a) shows how participants disambiguated open question stimuli after the whole of the stimulus had been heard, and Figure 7(b) shows the same for the polar question stimuli.

Open question stimuli were reasonably reliably disambiguated (range = 79%–84%). However, polar question stimuli were disambiguated much less reliably (range = 29%–55%). Only with 100% of natural prosody were more than 50% of the stimuli reported as polar questions. For all other manipulations, there was a preference to report the stimuli as open questions.

Statement stimuli had no prosodic variants, so Figure 8 shows how participants’ responses to these stimuli changed over time. In some trials, statements were identified as open questions at the CPF and following adverbial, and at the verb, the responses were spread between don’t know, statement, and open question. However, by the end of the stimulus set, statement stimuli were being reliably identified.

How did prosody affect disambiguation during the utterances? Figure 9(a) shows the proportion of open question stimuli that were correctly identified with different levels of natural prosody, and Figure 9(b) shows the same for polar question stimuli.

For the open questions, accuracy increases as more of the stimulus is heard, with accuracy increasing most strongly after Segment 3 and a smaller increase in accuracy from Segment 2 to Segment 3. There is some gradient effect associated with the proportion of natural prosody at the CPF and the subsequent adverbial, but this disappears by the end of the utterance. It appears that Segment 2 is more important in disambiguation than Section 3. However, this is not to the extent that would support a claim that expanded pitch range at the CPF (Segment 2) is unambiguously associated with open questions. If this were the case, we would have expected a higher level of disambiguation at Segment 2 modulated by the proportion of focus prosody present.

For the polar questions, disambiguation seems to begin at the verb (Segment 4), but the highest level of disambiguation takes place once the sentence-final LH% tone at the particle -yo has been heard (Segment 5). This is later than the theory would predict, although there does appear to be a gradient effect of prosody in the correct disambiguation.

Figure 10(a) shows how prosody affected the incorrect disambiguation of open question stimuli during the repeated presentations. Figure 10(b) shows the same for polar question stimuli.

For open questions, there is a slight increase toward the end of the sentence but no discernible gradient effect of prosody. For polar questions, some participants identify the stimulus as an open question as soon as the CPF is heard (Segment 2), and misidentifications continue to increase as more of the stimulus is heard. In this case, there appears to be a gradient effect of prosody at Segment 5 but not earlier, with increasing natural prosody leading to fewer inaccurate disambiguations. A gradient effect would be in line with theoretical predictions, but the level of incorrect disambiguations is not, particularly the continuing increase at Segments 4 and 5, once the verb has been heard.

Five generalized additive models (GAMs; Hastie & Tibshirani, 1990) were constructed using the packages mgcv (Wood, 2011) and itsadug (van Rij et al., 2022) within R (R Core Team, 2018). GAMs were chosen because the nature of the experimental variables does not satisfy the requirement of independence necessary to use a linear mixed-effects model. However, their results must be interpreted with caution. Two GAMs were constructed for each of the open and polar question categories, respectively, with the dependent variable being a binary predictor of whether the stimulus had been correctly identified. Fixed effects were segment and variant. Random effects were participant and item in relation to segment and order in relation to participant. For each category, one of the models also included a fixed effect of the interaction between segment and variant. For the statement category, where there was no manipulation of the stimuli, and therefore, no variants, one GAM was produced with segment as a fixed effect and the same random effects as the other models. Models were visually inspected to check for structure in the random effects; none was found.

The maximal formula used for the models is shown at (6). The left-hand side term correct is a derived Boolean that is true when a participant correctly disambiguated the stimulus. The right-hand side terms (a)–(b) represent fixed experimental effects, term (c) represents a possible interaction between the fixed effects, and terms (d)–(f) represent the random effects of participant, item, and order of presentation of the stimuli, respectively. The terms in (g) are the other parameters of the calculation:

(6) gam(correct ~ s(variant, k = 4)                   (a)

          + s(segment, k = 4)                   (b)

          + ti(variant, segment, k = 4)             (c)

          + s(segment, participant, bs = “fs,” m = 1, k = 4) (d)

          + s(segment, item, bs = “fs,” m = 1, k = 4) (e)

          + s(participant, order, bs = “fs,” m = 1, k = 20), (f)

          data = dataset, family = binomial, discrete = TRUE) (g)

For statements, right-hand side terms (a) and (c) were not used in the model because no pitch manipulation was used in preparing statement stimuli, and the value of the parameter $k$ in (f) was set to 10 because of the smaller number of data points compared with the other stimulus types. The statement model explained 67.3% of the variance with a highly significant effect of segment $(p<.001).$

Figure 11 shows the effect of the segment on participants’ correct identification of statements. By the region of interest, Segment 5, statements are being correctly identified as expected. The model predicts a slight reduction in accuracy at Segment 2, when the CPF is heard.

The pairs of models with and without the fixed interaction between segment and variant were compared. The amount of deviance explained was similar between pairs (63.0% for the open category, 51.8% and 51.7% for the polar category). Using the Akaike information criterion (AIC), for the polar category, the model with the interaction between variant and segment was preferred (AIC 2464 vs. 2481), and so this model was selected. For the open category, the model without the interaction was slightly preferred (AIC 3400 vs. 3396), and the probability of an interaction was not significant $(p=.790).$ Accordingly, for the open category, the model without an interaction was chosen, omitting the right-hand side term (c).

The open category model explained 66.9% of the variance in the data, with a significant $(p=.041)$ effect of variant and a highly significant effect of segment $(p<.001).$ The polar category model explained 51.7% of the variance in the data. This is lower than the open question model but not unreasonable. For the polar category, the model found no significant effect of variant alone $(p=.090),$ but highly significant effects of both segment $(p<.001)$ and the interaction between segment and variant $(p<.001).$

Figure 12(a) shows the effects of segment and variant according to the preferred model on participants’ correct prediction of open question stimuli, and Figure 12(b) shows the same for polar question stimuli. For open questions, the stimuli were correctly identified significantly above chance levels (0.25) after Segment 4. There is no evidence of an effect of the variant on the time that stimuli were correctly identified. In other words, there is no evidence that expanded pitch range played a role in participants’ decisions.

For polar questions, the stimuli were never significantly correctly identified above chance levels. However, there is evidence of an interaction between variant and segment; in other words, a greater amount of expanded pitch range increased the likelihood that participants would correctly identify the stimuli. However, it was only at Segment 5 and with 75% or higher expanded pitch range that correct identifications were at chance levels; earlier in the sentence for all levels of pitch expansion, and at Segment 5 for 50% of lower pitch expansion, participants were significantly more likely to identify the stimulus incorrectly (as an open question, a statement, or unknown).

Although Jones (2016) takes expanded pitch range as applying across a number of syllables in the focused constituent, the method we used to construct the stimuli used the F0 peak within the relevant AP, streamlining the contour between this point and the boundaries of the phrase. This approach is more in line with the F0 peak as described by Yun and Lee (2022), where a positive association was seen between the height of the F0 peak and an open question reading. However, we did not see a reduction in open question interpretation as the F0 peak at the CPF decreased, which would have been predicted by Yun and Lee’s results.

It is also possible that post-focus compression is a necessary element of prosody, alongside expanded pitch range. In a similar case involving disambiguating wh-interrogatives from wh-declaratives in Mandarin, Yang et al. (2020) found that open questions showed a more compressed F0 range relative to their declarative counterparts. The stimuli for open questions all had the natural prosody of an open question after the AP containing the CPF. Thus, even for the variants where the expanded F0 range had been removed, the subsequent F0 range compression may have been detectable. We did not control for this, and so the question remains open for further investigation.

Our study did not set out to explore the role of AP boundaries in disambiguation, but it is possible to make some inferences about their impact. If the placement of AP boundaries is the crucial determiner of disambiguation, then we would expect to see no gradient effect in either the open or polar categories, because only the pitch peaks within the regions of interest were manipulated, and the low tones at phrasal edges were unchanged. For open questions, we would also expect to see successful disambiguation at or shortly after the region of interest Segment 2; the study design means that there is time to fully process the sentence fragment before making a decision. However, a gradient effect was observed in the polar category, and for the open category, it was not until Segment 4 that disambiguation reached chance levels. The data therefore suggest that AP boundaries are not crucial in disambiguation.

Our design assumed that there is no preferred reading for CPFs, but the data suggest that there is a degree of lexical preference for interpreting them as open questions rather than indefinite pronouns. For statement stimuli, where there was no manipulation of natural prosody, the 10%–15% of participants who identified an utterance type at the CPF or the subsequent adverbial largely thought the utterance was an open question. Even when the sentence-final boundary tune had been heard, just more than 10% of participants continued to identify the statement stimuli as open questions. For the polar category, at least 75% of natural prosody at the verb, Segments 4 and 5, was required to bring disambiguation up to chance levels, and even with full natural prosody at the verb, an open question reading was as likely as a polar question reading. Within the limits of the study, it was not possible to carry out a corpus investigation to explore this further.

Cross-linguistic evidence shows that context interacts with prosody in disambiguating ambiguous utterances. Snedeker and Trueswell (2003) studied syntactic ambiguity in prepositional phrase attachment in English and found that speakers produced strong prosodic differences when contextual information was insufficient to disambiguate between syntactic structures. These prosodic cues significantly contributed to listeners’ ability to disambiguate. However, when speakers were unaware of the ambiguity, they produced weaker prosodic cues, making it more difficult for listeners to rely on them for disambiguation. Similarly, Hansen et al. (2023), investigating prosodic grouping in coordinated name sequences in German, examined how prosodic cues such as F0 range, final lengthening, and pause signaled internal grouping within three-name sequences. Using a gating paradigm, they tested whether listeners could predict these groupings based on boundary-related prosodic information. They found that only minimal prosodic information related to grouping was necessary; most of the listeners were able to disambiguate after the first name before the grouping information was available. Interestingly, listeners used different disambiguation strategies: some preferred to wait for as much information as possible, whereas others started the identification process early on. In our study, where utterances were presented out of the blue with no supporting context, we also found late disambiguation, which underscored the interpretation that listeners tried to wait for as much information as possible before they attempted disambiguation. Moreover, we saw that listeners relied little on the prosodic information from the CPF; instead, the lexical meaning of the CPF appeared to have biased the listeners’ interpretation toward questions. These findings echo Song et al. (2022), showing that prosodic features are not the only factors in disambiguating Korean polar and wh-questions, as subtle lexical meaning can also influence the interpretations. A similar phenomenon has also been reported in other languages (e.g., see Zhang, 2018, p. 146, for Tianjin Mandarin).

Our sample size is relatively large, and this may have contributed to unexpected patterns being revealed. However, we note that our pilot study $(n=16)$ also showed differences in successful disambiguation between the different stimulus categories. The online environment for the experiment is less controlled than testing participants in the laboratory, but this may also more closely reflect how people are using language in their daily lives.

One possible confound is that throughout our study, participants potentially forgot the meanings of the different terms open question, polar question, and statement. Although we provided an explanation and relevant examples at the start of the study to acclimatize participants to the key terms, we recognize these terms are still technical in nature, thus potentially not very accessible to the everyday speaker. Therefore, it could be that over the course of the experiment, participants possibly defaulted to a particular response, and in this case, to the open interpretation for questions, which would have resulted in later responses showing a different pattern to earlier responses. However, this is not borne out by our statistical modeling. In our preferred models, order was treated as a random effect, and its inclusion as such did not add structure to the residuals. There was no evidence to suggest that order should have been included as a fixed effect.

An aspect that may have prevented the early identification of open questions was the presence of statements in the stimuli. Because the defining distinction in Korean between statements and questions is the utterance-final tune (HL% for statements vs. LH% for questions), which in the polite speech style is associated with the particle -yo, participants may have waited until Segment 5 to make their decision, even if they had formed a strong hypothesis having heard F0 variation against declination at Segment 2, the CPF. We did not expect this to happen, and it was not seen in the pilot study, but restricting the experiment to a choice between open and polar questions would remove this as a potential confound.