Matching—Language and gesture conveying the same viewpoint

This section presents how speakers’ speech-accompanying gesture collaborates with the speech in expressing the same viewpoint.

From Table 11, we found that matching expressions where language and gesture convey the same viewpoint occur most often when character viewpoint is adopted in both channels. Example (25) illustrates this situation. In Example (25), the conversational topic is about F1’s friend—Chāchā, and her first experience of riding a motorcycle in a mountain area.

(25) |~~~~~~~~~~~~*******************************************************

[a] [b]

F1: Chāchā shuō wǒ zhēnde shì yìzhí zài gēnzhe chē wǒ hěn jǐnzhāng Chacha say 1SG really COP all the time at follow car 1SG very nervous [a] From Chāchā, both hands move from back of the head to front of the chest, with all

digits curling into the fists; fists facing outward opposed to the speaker’s body ((1) to (2) in Figure 12)

[b] At wǒ, both hands that had been made into the fists start wriggling and rotating left-and-right successively. The body slightly jumps up-and-down and then bends to the front ((3) to (4) in Figure 12)

F1: ‘Chacha says, “I’m really…I have been keeping up with other motorcycles. I’m nervous.”’

Figure 12. Collaborative expressions of character viewpoint in language and gesture

In talking about the past event, F1 used voiced direct quote to enact Chacha’s speech—what Chāchā said at the time when the event took place, but superimposed with a special voice quality—a trembling and nervous voice that shows Chacha’s attitudes and emotions. Therefore, in speech, we find that F1 is attempting to enact

Chacha’s part in the original event.

In talking about this event, F1 also produced a speech-accompanying gesture to represent Chacha’s action of gēnzhe chē ‘riding the motorcycle’. In the concurrent speech where F1 starts to directly quote what Chāchā has said (wǒ zhēnde shì yìzhí

zài gēnzhe chē wǒ hěn jǐnzhāng), F1’s use of the gesture collaborates with the speech

to act as Chāchā in the original scene of the past event by enacting her action of riding. In representing Chacha’s action, F1’s hands are Chacha’s hands, wriggling and

rotating left-and-right as if holding the handles of the motorcycle (Line [b], (3) to (4) in Figure 12); F1’s body is Chacha’s body, slightly jumping up-and-down and then bending to the front, like sitting unsteadily on a saddle (Line [b], (3) to (4) in Figure 12). The enactment of the role of Chāchā is dispersed over F1’s body, not only through the use of hand movement, but also bodily movements. In addition, the gesture with longer stroke duration that lasts for 3.02 seconds is performed with the use of both hands, and the wriggling and rotating movement is done successively several times. The interactions of four complex gestural features—the use of both hands, longer stroke duration, the repetition of the stroke phase, and the involvement of other parts of the body, taken together suggest that F1 is as though inside the original scene of the past event, acting as Chāchā who was the character involved in the event. In this case, F1’s use of speech and speech-accompanying gesture together help F1 to reconstruct the scene of the past event and express the character viewpoint.

Matching expressions might also occur while observer viewpoint is adopted in both language and gesture. Example (26) illustrates this situation. In Example (26), the conversational topic is about F1’s landlord Liánpànmā and her reactions when she saw a group of firefighters come to her house:

(26) |~~~~~~~~~~~~~~************************-.|

[a] [b] [c]

F1: nàge Liánpànmā jiù yuǎn yuǎn de guòlái

that Lianpanma just far far DE come

[a] From nàge, right hand rises from the Center-Center place to slightly up-and-right side of speaker’s front of the chest; the thumb, index and middle finger bend and the ring and little finger curl into the palm. The palm faces downward ((1) to (2) in Figure 13) [b] From jiù to the first segment of lái, all digits of the right hand bend, with the palm

facing downward. Right hand slightly moves from right side of speaker’s front of the chest inward to speaker’s body at Center-Center place ((3) in Figure 13)

[c] At the second segment of lái, right hand starts to move down to the waist level ((4) in Figure 13)

F1: ‘Then that Lianpanma was coming from far away.’

Figure 13. Observer viewpoint in language and gesture

In the descriptions of this past event, speaker F1 simply uses a plain statement that is without either expressing her personal attitudes or comments or enacting the character’s speech or thought to talk about the event. F1 is talking about the event like an observer, plainly conveying the spatio-temporal information of the event without acting like a person involved in the original event.

Concerning the speech-accompanying gesture, F1 made an inward hand movement (Line [b], (3) in Figure 13) from Periphery to Center-Center space,

representing the character of this event—Liánpànmā and her action of guòlái ‘came’.

In performing this gesture, F1 is using her right hand and all of the digits to represent the character of the event—Liánpànmā as the whole of the hand. Lianpanma’s body is represented with the focus on F1’s right hand, and F1’s inward movement suggests the line of the trajectory that Liánpànmā moves along. The inward movement is performed only once, and there is no involvement of other body parts to suggest that F1 is acting in the role of the character Liánpànmā. Three simple gestural features—the use of a single hand, no repetition of the same stroke, and no involvement of other body parts, taken together suggest that F1 is depicting the past event like an outside-the-event observer who does not seek to get involved in the original scene. In this example, therefore, the gesture collaborates with the accompanying speech in expressing observer viewpoint.

5.6.2 Mismatching—Gesture conveys different viewpoints from those conveyed

in language

While speakers in talking about third-person past events might express the same viewpoint through both linguistic and gestural modalities, mismatching cases where language and gesture convey different viewpoints are more commonly observed in the current data. In this section, we will see all possible mismatching expressions of

linguistic and gestural viewpoint in the data.

Among all mismatching expressions of linguistic and gestural viewpoint, the representation of speaker viewpoint in language but other viewpoints in gesture is the most commonly seen mismatching case in the current data. This suggests that while speakers are paying attention to the maintenance of the conversations and their current role as a speaker within the ongoing conversations in the descriptions of third-person past events, their speech-accompanying gestures are performed to represent the propositional contents of the past event. The performance of such speech-accompanying gestures might structure and represent the information regarding the events from either an observer viewpoint or a character viewpoint.

Matching expressions where speaker viewpoint is adopted in the language with observer viewpoint in the gesture can be seen in Example (27). In Example (27), F1 is talking about how a firefighter came over from the fire engine to the house to remove the wasp’s nest. finger slightly extended and points down; other digits bend and curl into a fist ((1) to (2) in Figure 14)

[b] From tā to jiù, right hand slightly lifts up and moves to speaker’s right side of the front chest ((3) in Figure 14)

[c] At chūlái, right hand moves down and forward quickly to Center-Center place, with all digits slightly bent and the palm faced downward ((4) in Figure 14)

F1: ‘Then he got out.’

Figure 14. Speaker viewpoint in language, observer viewpoint in gesture

In talking about this event, F1 injects an utterance-final discourse marker a into the plain statement she has made, suggesting F1’s attitude toward the event and involvement within the ongoing conversation as a current speaker. However, a gesture which represents the character of this event—the firefighter’s action of chūlái ‘got out’ suggests that F1’s speech-accompanying gesture is mainly focused on the propositional content of the past event. In this hand movement, F1 is using her right hand and the index finger to represent the character—the firefighter, and the movement of the firefighter’s body in getting out of the fire truck. The representation of the firefighter is only given by use of F1’s right hand; F1’s use of this gesture without any other bodily movement suggests that she is not attempting to act as the firefighter in the past scene of the event. With a stroke duration that lasts for only 0.64

seconds, the hand movement is done only once. With the interaction of these simple gestural features—the use of a single hand, shorter stroke duration, no repetition of the same hand movement, and no other bodily movements involved, F1’s gesture suggests that observer viewpoint is adopted. From this example, we can see that while speakers in describing third-person past events might use accompanying speech to show their concern about the ongoing conversation and reveal their current role as a speaker; they might use gesture at the same time to represent the propositional content of the event like an outside-the-event observer.

On the other hand, speakers’ speech-accompanying gesture might also represent the past events as if the speaker were a character in the event when speaker viewpoint is adopted in language. This situation can be seen in Example (28). In Example (28), M1 is talking about how his two superiors were helpless when a colleague came to them complaining about not being invited to a banquet.

(28) |************************************************************** the front chest (Periphery), with all digits extended and palms facing upward ((1) to (2) in Figure 15)

[b] At the first mention of liǎng, both hands slightly lift up; the lift-up action is done twice ((3) in Figure 15)

[c] At de, hands move back to Center-Center place, with both palms faced inward toward the speaker’s body ((3) in Figure 15)

M1: ‘What they can only do is fling up their hands.’

Figure 15. Speaker viewpoint in language, character viewpoint in gesture

In talking about this past event, M1 superimposes a laughing quality over the clause, suggesting that M1 is currently speaking on behalf of himself as a speaker in the conversation and revealing his own feelings toward this event. Laughter that serves to show M1’s own attitude toward the event suggests that he is also adopting the speaker viewpoint in his language.

However, M1’s speech-accompanying gesture produced within this event meaning liǎng shǒu yì tān ‘flinging up hands’ suggests that the gesture refers to the propositional content, rather than the speaker’s attitudes or comments that serve to reveal the speaker’s current role as a speaker within the conversation. In representing the characters of the past event—two superiors in M1’s company and their action of

liǎng shǒu yì tān, the production of the speech-accompanying gesture is dispersed

over M1’s body. M1’s hands are the two superiors’ hands, flinging up to two sides of the speaker’s front of the chest (Line [a], (2) in Figure 15); his body is the two superiors’ bodies represented in a single body, shrugging the shoulders slightly. Both hands move from the Center-Center to Periphery space in making the gesture. While the same hand movement is held for a long period as the post-stroke phase, it is done only once. The stroke phase also holds a longer duration which lasts for 2.1 second.

Combining the performance of four complex gestural features—gestural space that encompasses large area, the use of both hands, longer stroke duration, and shoulder-shrugging as other bodily movement, M1’s gesture suggests that character viewpoint is being conveyed. The gesture does not collaborate with the speech which is expressing speaker viewpoint that aids the maintenance of the ongoing conversation within the description of the same event.

The collaborative expressions of observer and character viewpoint within an event seems contradictory, but speakers indeed create this situation in talking about third-person past events. In addition, most of the cases happen when observer viewpoint is adopted in language, but character viewpoint in gesture (37 out of a total of 39 collaborative expressions of observer and character viewpoint in either language or gesture). Speakers might use language to plainly describe the past events in the role

of an outside-the-event observer, but at the same time use gesture to suggest that they are acting as a character in the past event. Example (29) illustrates this situation. In Example (29), the conversational topic is about the reaction of F1’s father when he heard how F1 talked to her teacher on the phone. F1 is talking about how her father really wanted to grab the phone at that time.

(29) |~~~~~~~~~~***************************

[a] [b]

F1: wǒ bà jiù shuō tā chāo xiǎng bǎ diànhuà qiǎng -guòlái 1SG father just say 3SG very think BA telephone grab-come

[a] At chāo xiǎng, left hand moves outward and downward from front of the speaker’s left eye to speaker’s own front space, with all digits slightly curled ((1) to (2) in Figure 16) [b] At bǎ, left thumb and little finger extend wide, and all other digits are curled against the

palm; the left hand moves toward the speaker’s body near her left ear ((3) in Figure 16) F1: ‘My father said that he really wants to grab the phone.’

Figure 16. Observer viewpoint in language, character viewpoint in gesture

In talking about this event, speaker F1 does not directly quote what her father said at the time when the event took place, instead, she simply reports what her father said with a plain statement. F1’s use of the pronoun tā ‘he’ in the quoted speech suggests that she is using indirect reported speech, without attempting to enact the character of this event—her father’s speech.

Within the description of this event, F1 also produced a speech-accompanying gesture that represents her father’s action of qiǎng diànhuà ‘grab the phone’. In representing this action, F1’s left hand is made into a ‘Y’ shape—the thumb and the little finger extend wide but others curl into the palm, like the holding of the handset of a telephone (Line [b], (2) in Figure 16). In making this hand shape, F1 is suggesting that she is temporarily acting as her father in the original scene of the past event by making her hand as her father’s hand performs the action of ‘grabbing’. The hand movement is made at the Periphery space exclusively, and the space used for the inward movement to the speaker’s body near her ears clearly suggests F1 is enacting a role inside the original scene of the past event. The stroke phase also has a longer stroke duration that lasts for 0.8 seconds. The performance of these complex gestural features taken as a whole suggests that F1 is expressing a character viewpoint in gesture, which is different from that conveyed simultaneously in the accompanying speech concerning the description of the same event.

Collaborative expressions of character viewpoint in language, but observer viewpoint in gesture seldom occur in the current data. Since when character viewpoint is adopted in language, the speaker’s use of a speech-accompanying gesture often collaborates with the speech in expressing character viewpoint as well. However, we can still see how speakers might express an observer viewpoint through the gestural

channel when character viewpoint is adopted in language. Example (30) is an instantiation. In Example (30), the conversational topic is about the F1’s boyfriend’s oral defense in his PhD program. In the clausal event being examined, F1 is talking about how her boyfriend had prepared so many snacks for the oral defense that the some teachers even asked whether they could take some home:

(30) |~~~~~******-.-|

[a] [b] [c]

F1: hěnduō lǎoshī jiù shuō ei zhè kěyǐ názǒu ma many teacher just say PRT this can take QST

[a] At shuō, right hand rises slightly from the table in front of the speaker, with the index finger slightly extended outward to the space which is opposed to the speaker’s body;

other digits curl inwards to the palm ((1) in Figure 17)

[b] At ei, right index finger slightly points to the to the space in front of the speaker’s body that is as if opposite to the speaker’s body ((2) in Figure 17)

[c] At zhè, right index finger bends back but remains extended; other digits curl against to the palm ((3) in Figure 17)

F1: ‘Many teachers have asked, “Can we take these away?”’

Figure 17. Character viewpoint in language, observer viewpoint in gesture

In talking about this past event, F1 acts as the teachers who are involved in the original scene of the past event—the oral defense, and attempts to enact their speech.

She directly quotes what the teachers said at that time (ei zhè kěyǐ názǒu ma), and

superimposes a somehow mincing and pretend-courteous way to mimic the speech of the teachers. Therefore, F1 is expressing character viewpoint in speech when describing this event.

F1 in talking about this event also produces a spatial gesture that helps orientate the use of zhè—which means the snacks in the original scene of the past event. In making this spatial gesture, F1 simply produces a short and quick pointing movement with a stroke duration that lasts for only 0.296 seconds. In the previous section on the types of gesture used in representing viewpoints (see section 5.2), we noted that the five gestural features—gestural space, handedness, stroke duration, frequency, the involvement of other body part also apply to the analysis of spatial gesture in representing viewpoints. In this example, the performances of the gestural features do not suggest that F1 is attempting to enact a role as characters in the past event—the teachers who adjudicated in the oral defense. The gestural space is narrowed to the Center-Center space; the hand movement in making the pointing gesture does not encompass large area due to the fact that it is presented swiftly and is done only once.

In doing the pointing, the spatial gesture is performed with only the use of a single hand. No other bodily movements accompany the pointing gesture. It is therefore more appropriate analyze F1’s spatial gesture as expressing observer viewpoint in this case. In sum, in this example, while F1 is expressing character viewpoint through the

use of speech, her gesture suggests that she is acting more like an observer outside the past event.

5.7 Summary

This chapter presents the quantitative study of gestural viewpoints and the collaborative expressions of viewpoints in language and gesture. Concerning the distribution of gestural viewpoints in the descriptions of third-person past events, speakers’ use of gestures most often express character viewpoint. Different from the performance in language where speakers seldom enact characters’ speech or thoughts, speech-accompanying gestures very often enact characters’ deeds or actions that make it seem as if the speakers have walked into the scene of the past events. On the other hand, speakers rarely ever make use of gesture to suggest their role as a current speaker within an ongoing conversation, whether by making comments or interacting with other co-conversationalists through the gestural channel.

In discussing the distributional pattern of gestural viewpoints, we have also seen how each viewpoint is represented through the use of five gestural features as coding principles—gestural space, handedness, stroke duration, frequency, and the involvement of other body parts. By observing how gestures in representing different viewpoints perform these gestural features, the statistics from the Chi-square tests

suggest that the correlations between each gestural feature and gestural viewpoints are all significant. This clearly suggests that the five gestural features recognized in this study could serve as indicative criteria in identifying different gestural viewpoints.

This chapter also presents the use of different gesture types in representing the three viewpoints in the current data. In McNeill’s gestural study (1992), he mainly focused on how gestural viewpoints might be inferred from iconic gesture. However, in the current study, we have suggested that in addition to iconic gestures, metaphoric and spatial gestures are also able to express three different viewpoints. The present study also finds that 92.1% of character viewpoint in gesture is represented through iconic gesture. On the other hand, iconic gestures only account for 51.9% of all gestures that represent observer viewpoint. Metaphoric gestures and spatial gestures

This chapter also presents the use of different gesture types in representing the three viewpoints in the current data. In McNeill’s gestural study (1992), he mainly focused on how gestural viewpoints might be inferred from iconic gesture. However, in the current study, we have suggested that in addition to iconic gestures, metaphoric and spatial gestures are also able to express three different viewpoints. The present study also finds that 92.1% of character viewpoint in gesture is represented through iconic gesture. On the other hand, iconic gestures only account for 51.9% of all gestures that represent observer viewpoint. Metaphoric gestures and spatial gestures