In this section, the findings in Chinese will be compared to those in other languages, to understand whether the linguistic-imagistic patterning of motion in a verb-serializing/
path-and-manner-incorporating language differs from those in satellite-framed/manner-incorporating and verb-framed/path-satellite-framed/manner-incorporating languages. Given that the use of hands and arms to convey information along with speech is indispensable and prevalent in multimodal communication (Kendon, 2004; McNeill, 1992, 2000), the variation in cross-linguistic representations, if there is any, can further suggest a cross-linguistic-imagistic
16 typology of motion.
The comparison rests upon the studies in Özyürek and Kita (1999), Kita and Özyürek (2003), Özyürek et al. (2005), and Kita et al. (2007), which used the same cartoon stimuli to elicit narrations in English, Turkey, and Japanese, with comparable quantitative data.
The studies obtained the following results:
English speakers typically used a manner verb and a path particle or preposition (e.g., he rolled down the hill) to express the two pieces of information within one clause. In contrast,
Japanese and Turkish speakers separated manner and path expressions over two clauses with path in the main clause and manner in the subordinated clause (e.g., he descended as he rolled)…. In depicting how an animated figure rolled down a hill after having
swallowed a bowling ball, Japanese and Turkish speakers were more likely to use separate gestures, one for manner and one for path, and English speakers were indeed more likely to use just one gesture to express both. (Özyürek et al., 2005, pp. 222-223)
Aside from these recurrent linguistic-imagistic patterns, the data in (Özyürek et al., 2005) also showed that English speakers could gesture like Turkish speakers when the former chose to talk about only path or only manner. Kita et al. (2007) further confirmed the influence of clause types on gesture types in English elicited narrations: “when the concurrent speech syntactically linked Manner and Path expressions tightly within a single clause, gesture tended to express Manner and Path simultaneously. When the concurrent speech syntactically linked Manner and Path expressions in a less tight way in two separate clauses, gestures tended to separate Manner and Path information” (ibid, p.
1231). Since it is rare in Turkish to package both MANNER and PATH in one clause, and it is also unusual to express them in separate VPs in a multiple-VP clause to denote a single motion event in Chinese conversations (5 out of a total 375), it remains unknown whether Turkish speakers would express these two components simultaneously in one gesture if they chose to mention both within the same clause, or whether Chinese speakers would
17
produce separate gestures when they chose multiple-VP constructions. Substantial work needs to be done across different languages to determine the influence of speakers’ online choice of various linguistic patterns on gestural representation.
According to the predominant linguistic-imagistic packaging patterns across different languages, English has been shown to differ from Turkish and Japanese. As to Chinese, it exhibits a third way in the coordination of motion information between speech and gesture. On the one hand, Chinese, just like English, frequently brings MANNER and PATH
up in speech. They are ‘high-manner-salient’ languages in Slobin’s term. However, there is a language-specific serial-verb encoding of these two components within the same clause in Chinese, and unlike English speakers, Chinese speakers seldom produce conflated gestures. Nor do they gesture like Turkish and Japanese speakers, who are likely to produce two separate gestures. The preference in Chinese is to talk about PATH
and/or depict it manually.5 Thus, language specificity is evidenced in the alignments of motion information in linguistic encodings and manual depictions among a satellite-framed language, two verb-satellite-framed languages, and a verb-serializing language.
Such linguistic-imagistic variation can further reveal language specificity in the conceptualization of motion. Because of the intimate connection between language and gesture, which has been supported by a considerable amount of evidence in different lines of research, conceptualization of an event includes both linguistic and imagery content.
Speech and gesture together in the real-time expression of motion provide more insights into what people think about motion. The findings in Chinese discourse with regard to the recurrent patterns in expressing motion in speech and gesture provide converging evidence in support of Chinese speakers’ habitual focus of attention on a certain aspect of
5 The studies in Turkish and Japanese showed that speakers would produce separate gestures for MANNER and PATH. However, it is not clear whether speakers would prefer gesturing a particular component when they just produce one gesture for the same motion event.
18
motion. Although it is beyond the scope of the study to discuss embodied cognition (Wilson, 2002; Shapiro, 2007; Barsalou, 2008; Glenberg, 2010), the recurrent focus of attention on PATH during online speaking in Chinese could reveal the salience of PATH in the conceptualization of motion embodied in people’s perceptual and bodily experiences in daily social interaction.
It is hoped that conversational data from other languages will be available to confirm language specificity in representing motion information in speech and gesture. Second, while the discourse factor of information state does not appear to affect the linguistic-imagistic representation of motion, the nature of the other factors that may influence the occurrence of gestures needs further research.
19 References
Allen, Shanley, Asli Özyürek, Sotaro Kita, Amanda Brown, Reyhan Furman, Tomoko Ishizuka, & Mihoko Fujii (2007). Language-specific and universal influences in
children's syntactic packaging of manner and path: A comparison of English, Japanese, and Turkish. Cognition, 102, 16-48.
Armstrong, David F. & Sherman E. Wilcox (2007). The Gestural Origin of Language.
Oxford: Oxford University Press.
Barsalou, Lawrence W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.
Brown, Amanda & Marianne Gullberg (2008). Bidirectional crosslinguistic influence in L1-L2 encoding of manner in speech and gesture: A study of Japanese speakers of English. Studies in Second Language Acquisition, 30 (2), 225-251.
Brown, Amanda & Marianne Gullberg (2010). Changes in encoding of path of motion after acquisition of a second language. Cognitive Linguistics, 21, 263-286.
Chen, Liang (2007). The Acquisition and Use of Motion Event Expressions in Chinese.
Lincom GmbH, München, Germany.
Chen, Liang & Jiansheng Guo (2009). Motion events in Chinese novels: Evidence for an equipollently-framed language. Journal of Pragmatics, 41 (9), 1749-1766.
Choi, Soojung & James P Lantolf (2008). Representation and embodiment of meaning in L2 communication: Motion events in the speech and gesture of advanced L2 Korean and L2 English speakers. Studies in Second Language Acquisition, 30 (2), 191-224.
Chui, Kawai & Huei-ling Lai (2008). The NCCU corpus of spoken Chinese: Mandarin, Hakka, and Southern Min. Taiwan Journal of Linguistics, 6 (2), 119-144.
Chui, Kawai (2005a). Structuring of information flow in Mandarin Chinese. Journal of Chinese Linguistics, 33 (1), 34-67.
Chui, Kawai (2005b). Topicality and gesture in Chinese conversational discourse.
Language and Linguistics, 6 (4), 635-654.
Chui, Kawai (2009). Linguistic and imagistic representations of motion events. Journal of Pragmatics, 41 (9), 1767-1777.
20
Fadiga, L., L. Fogassi, G. Pavesi, & G. Rizzolatti (1995). Motor facilitation during action observation: A magnetic stimulation study. Journal of Neurophysiology ,73, 2608-2611.
Gallese, Vittorio & George Lakoff (2005). The brain’s concepts: The role of sensory-motor system in conceptual knowledge. Cognitive Neuropsychology, 22, 455–479.
Glenberg, Arthur M. (2007). Language and action: creating sensible combinations of ideas. In Gaskell Gareth (Ed.), The oxford handbook of psycholinguistics (pp. 361-370). Oxford, UK: Oxford University Press.
Glenberg, Arthur M. (2010). Embodiment as a unifying perspective for psychology.
WIREs, 1, 586-596.
Gullberg, Marianne, Henriette Hendriks, & Maya Hickmann (2008). Learning to talk and gesture about motion in French. First Language, 28 (2), 200-236.
Huang, Shuanfan & Michael Tanangkingsing (2005). Reference to motion events in six western Austronesian languages: Toward a semantic typology. Oceanic Linguistics, 44, 307-340.
Kendon, Adam (2004). Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press.
Kita, Sotaro & Asli Özyürek (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48, 16–32.
Kita, Sotaro, Asli Özyürek, Shanley Allen, Amanda Brown, Reyhan Furman, & Tomoko Ishizuka (2007). Relations between syntactic encoding and co-speech gestures:
Implications for a model of speech and gesture production. Journal of Language and Cognitive Processes, 22, 1212–1236.
Lakoff, George (2008a). The neuroscience of metaphoric gestures: Why they exist. In Alan Cienki & Cornelia Müller (Eds.), Metaphor and gesture (pp. 283–289).
Amsterdam/Philadelphia: John Benjamins.
Lakoff, George (2008b). The Political Mind: Why You can't Understand 21st-century American Politics with an 18th-Century Brain. USA: The Penguin Press.
Langacker, Ronald W. (2008). Metaphoric gestures and cognitive linguistics. In Alan Cienki & Cornelia Müller (Eds.), Metaphor and gesture (pp. 249–251).
Amsterdam/Philadelphia: John Benjamins.
21
McNeill, David (1992). Hand and Mind: What Gestures Reveal about Thought. Chicago:
University of Chicago Press.
McNeill, David ( 2000). Language and Culture. Cambridge: Cambridge University Press.
McNeill, David & Susan D. Duncan (2000). Growth points in thinking-for-speaking. In McNeill, David (Ed.), Language and culture (pp. 141-161). Cambridge: Cambridge University Press.
McNeill, David & Elena T. Levy (1993). Cohesion and gesture. Discourse Processes, 16 (4), 363-386.
Núñez, Rafael E. & Eve Sweetser (2006). With the future behind them: Convergent evidence from language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30, 401–450.
Núñez, Rafael E. (2008). A fresh look at the foundations of mathematics: Gesture and the psychological reality of conceptual metaphor. In Alan Cienki & Cornelia Müller (Eds.), Metaphor and gesture (pp. 93–114). Amsterdam/Philadelphia: John Benjamins.
Özyürek, Asli, & Sotaro Kita (1999). Expressing manner and path in English and Turkish:
Differences in speech, gesture, and conceptualization. In Martin Hahn & Scott C. Stoness (Eds.), Proceedings of the twenty first annual conference of the cognitive science society (pp.
507-512). Mahwah, NJ: Lawrence Erlbaum Associates Inc.
Özyürek, Asli, Sotaro Kita, Shanley Allen, Amanda Brown, Reyhan Furman, & Tomoko Ishizuka (2008). Development of cross-linguistic variation in speech and gesture:
motion events in English and Turkish. Developmental Psychology, 44 (4), 1040-1054.
Özyürek, Asli, Sotaro Kita, Shanley Allen, Reyhan Furman, & Amanda Brown (2005).
How does linguistic framing of events influence co-speech gestures? Insights from cross-linguistic variations and similarities. Gesture, 5, 215–237.
Özyürek, Asli, Roel M. Willems, Sotaro Kita, & Peter Hagoort (2007a). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 19 (4), 605–616.
Özyürek, Aslı & Spencer D. Kelly (2007b). Gesture, brain, and language. Brain and Language, 101 (3), 181–184.
Shapiro, Larry (2007). The embodied cognition research program. Philosophy Compass, 2, 338–346.
22
Slobin, Dan l. (2000). Verbalized events: a dynamic approach to linguistic relativity and determinism. In Susanne Niemeier & Rene Driven (Eds.), Evidence for linguistic relativity (pp. 107-138). Amsterdam: John Benjamins Publishing.
Slobin, Dan l. (2004). The many ways to search for a frog: linguistic typology and the expression of motion events. In Sven Strömqvist & Ludo Verhoeven (Eds.), Relating events in narrative: Typological and contextual perspectives (pp. 219-257). Mahwah, NJ: Lawrence Erlbaum Associates.
Talmy, Leonard (1985). Lexicalization patterns: semantic structure in lexical forms. In Timothy Shopen (Ed.), Language typology and syntactic description (Vol. 3, pp. 57-149). Cambridge:
Cambridge University Press.
Talmy, Leonard (1991). Path to realization: a typology of event conflation. Proceedings of the Seventeenth Annual Meeting of the Berkeley Linguistics Society, February 15-18, (pp. 480-519). Berkeley, CA: Berkeley Linguistics Society.
Wilcox, Sherman (2008). Sign and gesture: Towards a new paradigm. In Alan Cienki &
Cornelia Müller (Eds.), Metaphor and gesture (pp. 273–275).
Amsterdam/Philadelphia: John Benjamins.
Wilson, Margaret (2002). Six views of embodied cognition. Psychonomic Bulletin &
Review, 9, 625–636.
23 Appendix
Gesture and speech transcription conventions
Transcription of speech [ ] speech overlap ...(N) long pause
... medium pause
.. short pause
(0) latching
@ laughter
<L3 L3> code-switch to Taiwanese
Transcription of gesture
Kendon’s (2004) transcription conventions for gesture were adopted.
| gesture phrase boundaries
~ ~ ~ preparation phase
~ ~ ~ pre-stroke hold
**** stroke phase
**** post-stroke hold -.-.- recovery phase / gesture phase boundaries
In examples, the description of the manual movement is given under the line of accompanying speech.
The time code shown at the bottom of each panel in the figures is expressed in hours: minutes: seconds.
milliseconds.
Abbreviations of linguistic terms
1PL first person plural
1SG first person singular 2SG second person singular 3SG third person singular
3PL third person plural
BC backchannel CL classifier
COP copula verb
NEG negative morpheme
PRF perfective aspect
PRT discourse particle
24
Enlargement of Figure 1. Gestural depiction of running continuously.
25
Enlargement of Figure 2. Gestural depiction of upward movement.
26
Enlargement of Figure 3. Gestural depiction of springing toward a target.