Schemata as the primary
modelling system of culture:
Prospects for the study
of nonverbal communication
Jui-Pi Chien
Institute of Foreign Languages and Literatures, National Taiwan University No. 1, Sec. 4, Roosevelt Rd., Taipei 10617, Taiwan
e-mail: jpchien@ntu.edu.tw
Abstract. The art historian Ernst Gombrich asserted that both artists and
view-ers are constrained by specific cultural mental sets within which they operate: he used “schemata” to refer to both actual sketches and our mental sets. In mapping the mechanism of culture, Juri Lotman situated both language and the “schematized image” at the centre of semiospheres: they function to introduce unity and elimi-nate contradictions. Schemata have obviously served as both the cause and the result of human knowledge, some loosely constructed yet highly condensed forms of art in different domains. Theorizing at a time of a hypertrophy of scientific and lin-guistic models, Lotman and Gombrich appear to have converged on theorizing our means of connecting people across cultures. They each put forward their own per-spectives concerning the interplay between culture, art and pictorial representation. This paper serves to: (1) examine how Lotman and Gombrich interpreted Jakobson’s scheme of communication; (2) reveal the paradox of schemata and nonverbal com-munication latent in their writings; (3) soften the paradox by drawing on Lotman’s conceptulization of play as an emotional structure. It is argued that the kind of emo-tive potential defined from the perspecemo-tive of beholders or addressees is crucial to our survival in the global village.
Keywords: schemata, modelling system, communication, art, play, emotion
1. Introduction
Cybernetics, information theory, mathematics and thermodynamics contributed to the formation of a new phase of Soviet semiotics – the Tartu-Moscow School launched a systematic survey on aspects of culture in the 1950s and 1960s (Shukman 1978; Ivanov 1978). In the 1960s and 1970s Juri Lotman conducted intensive seminars and collabo-rated with his colleagues in devising general models of communication (Lotman et al. 1973). Specifically, Lotman problematized Roman Jakobson’s communication scheme in order to (1) make distinctions between poetic language (internal communication) and natural language (external communication); (2) define the idea of modelling sys-tems; (3) come up with an advanced view on the correlations between dialogues and the expansion of cultural spheres (Lotman 2001). In parallel to such a scientific study of cul-ture, there also emerged a new way of appreciating pictorial representation. Between the 1950s and the 1970s, Ernst Gombrich gave a series of lectures, trying to advocate a psy-chology of appreciating paintings and diagrams. Gombrich’s efforts were more or less in conjunction with Lotman’s: they both valued the mental process of interpreting codes as part of their theoretical underpinnings. Although Gombrich did not name Jakobson in his lectures, he did acknowledge the latter as a source of inspiration (in the 1950s) while editing a preface to his own anthology (Gombrich 1984: x). Gombrich used terms such as “cryptogram” and “conundrum” to theorize our task of appreciating paintings or drawings. More often than not, he used “the beholder’s share” and “the painter’s com-merce” to measure the success of encoding and decoding messages. Arguing against some stereotypical impressions which confuse art with either resemblance of or simili-tude with nature, Gombrich redefined art as several conditions of “illusion”. In his terms, illusions come around when we make efforts to overcome certain binary oppositions: we are supposed to deal with some cognitive disparities between the real and the imagi-nary; what we have learned and what we see; the finite and the infinite; the rough and the neat, etc. These discrepancies not only define different types of art in history, but also demonstrate the dynamics or tension among the six elements listed in Jakobson’s communication scheme.
2. Emotive potential, variations of code and
a dynamic model of communication
Context Message
Addresser Addressee Contact
Code
Horizontally, the energy (or source of information) in Jakobson’s scheme is supposed to flow from the left to the right; from the addresser to the addressee. Vertically, fol-lowing the sequence given by Jakobson, the context takes on the referential function; the message takes the poetic function; the code the metalingual one; the contact the phatic one. According to the linear, mathematical and informational model, a safe reception of the message should be every bit our concern – information scientists therefore did not find it a problem to contain some unruly noises occurring in the process of transmitting messages. Some of Jakobson’s contemporary linguists even tried to ignore the variable of emotion so as to make the reception of message as ideal and well-conditioned as possible (Jakobson 1960: 353). However, Jakobson fought against such a linear and mechanistic approach, which he thought not only ignores the emotive potential of a message, but also renders addressees as passive participants in any sort of communication. Even though there have been “programmatic, planning [and] normative endeavors” in any verbal culture, Jakobson (1960: 354–355) insisted that we should reconsider how emotions have been at work between the two ends of communication. He began his inquiry by coming up with a couple of thought provok-ing questions: “What makes a verbal message a work of art?” […] “What is the empiri-cal linguistic criterion of the poetic function? In particular, what is the indispensable feature inherent in any piece of poetry?” (1960: 350, 358).
In the 1970s Lotman appears to have answered Jakobson’s questions by way of illustrating another linear scheme which serves to make subtle distinctions between poetic and daily languages:
Context Contextual Displacement
I Message 1 Message 2 I Code 1 Message 1
Figure 2. Lotman’s scheme of internal communication (Lotman 1977b: 101).
External communication is constructed on the following scheme: given a code, a text is introduced that is encoded in the code’s system, transmitted, and decoded. Ideally, the text coincides at entrance and exit; in practice, a decrease in informa-tion occurs. In this scheme, the code constitutes the constant, and the text is the variable. Internal communication is constructed on the following scheme: given a text that is encoded in a certain system, another code is introduced and the text is transformed. The code constitutes the variable, and the texts differ at entrance and exit; an increase in information occurs in the text due to its interaction with the new code (Lotman 1977b: 99–100).
Rather than being conditioned by a precisely defined model, message here is likely to alter dynamically. Lotman conceived the emotive potential of the message (a poetic
text) as a process of introducing new codes, and such a transformation (visualized in Figure 2) obviously leads to an increase of information (as shown in the distinction made between Message 1 and Message 2 in Figure 2). According to the controlled model, the code is defined as a set of shared rules of encoding and decoding which enable the communication of messages. A major difference in codes or contexts may lead to the breakdown of a system. However, Lotman’s idea that the code can be a variable puts forward a dynamic and evolving model of communication: (1) the code has a meaning-generating potential in its own right; (2) it may change and restructure the model of communication. Lotman here highlighted the code as a variable on the addressers’ end in order to fully explain the formation of a poetic text. Nevertheless, there are occasions when the variations of code are also enacted in the addressees in the form of a series of emotional reactions (both pleasant and unpleasant) to the addressers’ messages. So let us apply Lotman’s scheme to revising our understanding of the actual functioning of the addressee in Jakobson’s scheme so as to create a truly dynamic model.
In his essay on communication-modelling systems, Lotman illustrated one more linear scheme:
Text in an artificial language, for text in a poetic text example in the language of street signals natural language
Figure 3. Lotman’s scheme of communication-modelling systems (Lotman 1977a: 97).
Lotman suggested that the notion of evolution could be employed to explore the cor-relations between these three types of communication systems. Perceived in the evo-lutionary context, the variations of code appear to have derived from the complexities of character found in both addressers and addressees. Because of an increasing free-dom to use and combine codes, texts are becoming specialized and personal. The more esoteric a text was when it was encoded, the more rules or artificial languages we need to help with decoding and understanding it. Although Lotman arranged the three systems in a sequential fashion (ranging from ease to difficulty of understand-ing), he suggested that it is more effective to construe the links between them as a cycle in which a process of translation is constantly alive:
The act of exchanging information ceases to be a passive transfer of a message that is adequate unto itself from one bloc of memory to another and becomes a translation, in the course of which the message is transformed and the striving for adequacy enters into dramatic conflict with the impossibility of its complete real-ization. The act of communication begins to include the aspect of tension within itself. (Lotman 1977a: 97–98)
Such a definition more or less renews the ideal and mechanistic concept of commu-nication. Lotman observed that the changing codes which trigger the transformation (here construed as translation) of messages not only find their lives within poetic (I-I) communication, but also in communications in general. In addition, every act of exchange is underlined by some kind of paradox – the fact that codes in address-ers and addressees can be somewhat similar yet alarmingly different – which creates a certain tension between systems and some modelling primacy of one system over another:
Poetic language and natural language are particular manifestations of more gen-eral systems that are in a state of continual tension and mutual translation, and at the same time are not wholly mutually translatable; therefore the question of the primacy of one or the other communication-modeling system is determined by what is translated into what. (Lotman 1977a: 98)
Lotman’s renewed concept of communication indicates that some process of transla-tion sustains addressers and addressees in an eternal tug of war. Therefore, a cer-tain coexistence of translatability and untranslatability is natural (even some kind of a precondition) both within and between systems. The functioning of the code in Jakobson’s scheme (Figure 1) is thought to be “metalingual”. How can we possi-bly integrate this concept into Lotman’s thoughts about modelling and the continual tension therein? Some logical reasoning is required: (1) when A is translated into B, metatexts are generated in B; (2) these metatexts enable B to model A; (3) B is thus defined as the secondary modelling system, while A is the primary one. Moreoever, Lotman also valued direction as a decisive factor in making distinctions between typologies of culture. Thus, metatexts can be generated either between systems within a culture or at the borders between cultures. Most intriguingly, they can self-reduplicate within a system itself: “The possibility of self-reduplication of metalan-guage formations on an unlimited number of levels, along with the introduction of ever-new objects into the sphere of communication, forms culture’s reserve in infor-mation” (Lotman, Uspensky 1978: 229).
Because of such self-reproduction of metatexts, a culture has the potential to generate at least three modelling systems (as shown in Figure 3).1 So it appears that the system in Figure 3 that secures the least efforts of theorizing is artificial lan-guage. It is thought to present no difficulty of communication: “the first link implies a single meaning for everyone” (Lotman 1977a: 97). Compared with the other two systems, codes in this system are collective rather than personal: they can be traf-fic signs or diagrams used in sciences that are generally shared between addressers
1 Metaliterature, metapainting, metanovels, metacinematography provide some examples of
and addressees. Nevertheless, in a couple of essays also written in the 1970s Lotman envisioned another scheme – some concentric spheres (Figure 4) – to engage with certain intricacies of culture that may go beyond the linguistic level (Lotman 2001). This scheme draws on constant oscillations flowing between the center and the periphery of spheres which rigorously illustrate the merits of dramatic conflicts and differences between systems. According to our interpretation, there is a play on the value of artificial language: it appears as the least complicated system in the lin-ear scheme (Figure 3) while occupying the centre of culture (quite complicated) in the spherical scheme (Figure 4). The power what we call artificial language is explained as follows:
And yet culture requires unity. In order to fulfill its social function, culture has to appear as a structure subject to unified constructive principles. This unity comes about in the following manner: at a specific stage in the development of culture, there comes a moment when it becomes conscious of itself, when it creates a model of itself. The model defines the unified, the artificially schematized image, that is raised to the level of a structural unity. When imposed onto the reality of this or that culture, it exerts a powerful regulating influence, preordaining the construction of culture, introducing order, and eliminating contradiction. (Lot-man, Uspensky 1978: 227).
Lotman conceived the nonverbal quality of artificial language as an artificially “schema-tized image” that regulates diversities in a specific culture. Some sort of structural unity occurs at a rather late stage when a certain culture is becoming aware of itself (Lotman, Uspensky 1978: 222–223). However, such a unifying power works only within the same culture, community, field or domain. Lotman’s assertion that the artificial language or schematized image implies a single meaning for everyone is ideal: it becomes prob-lematic when addressers and addressees are communicating across times or cultures. Considered in such a context, the system of artificial language (containing mainly non-verbal or visual elements) may still present certain difficulty of understanding.
information information onculture schematized image nonculture entropy culture n
3. Schemata and nonverbal communication
How can Gombrich be useful for our inquiry into the ambivalence of the schematized image emerging from Lotman’s revision of Jakobson? To begin with, Gombrich devel-oped two theses in his major writings: (1) artists cannot draw beyond their learned devices, just as much as viewers cannot recognize any pictorial elements beyond their knowledge; (2) we are somewhat bold and creative on our first encounters with pictures – “making comes before matching” (Gombrich 1984: 116; 1994: 150–151). Appreciating these theses through Jakobson’s scheme, we find that artists create within certain contexts (their concepts about the function and purpose of art), and use certain codes (lines, shapes, colours and perspectives) available to them to repre-sent something within a proper medium (canvas, poster and website, etc.). Viewers gather messages through the mediums, and apply their knowledge about contexts and codes (both preconceived and learned) to analyze some given representations.
Context Message Artists iewers Contact Code v
Figure 5. Gombrich’s scheme of pictorial communication (adapted by the author).
The ways in which artists and viewers perceive the given code generate the making of pictures and meanings, while their knowledge about the implied context induces them to match their thoughts with what they see. The gap between code and context serves to explain not only certain conflicting interests between artists and behold-ers, but also changing conditions of illusion in the evolution of art history. When code and context are unfamiliar, certain pictorial messages such as impressionist and ancient paintings appear ambiguous. On the other hand, when the gap between them is made minimal, viewers are likely to understand what artists attempted to convey on the basis of the verisimilitude of a message. We may draw on examples such as com-mercials, magazines and posters, in which designers are thought to have been more sympathetic with their viewers than painters of high art (Gombrich 1984: 234–239). When arguing for his theses, Gombrich favoured examples drawn from sketch books and certain geometrical shapes such as circles, triangles and squares which were thought to be extremely effective in terms of pictorial communication. Moreover, he made distinctions between our ways of perceiving verbal and nonverbal messages by drawing on a Pompeii mosaic:
[L]ike verbal messages, images are vulnerable to the random interference engi-neers call noise. They need the device of redundancy to overcome this hazard. It is this built-in safeguard of the verbal code that enables us to read the inscription Cave Canem without hesitation even though the first e is incomplete. As far as image recognition is concerned it is the enclosing contour that carries most of the information. (Gombrich 1994: 141)
The easier it is to separate the code from the content, the more we can rely on the image to communicate a particular kind of information (Gombrich 1994: 147).
Seen from the perspective of the addressee, knowledge of verbal and nonverbal com-binations of shapes serves as a firm grid of meaning making, while variations of code (manifested in certain forms of noise, speech ambiguity and emotive potential) remain a hindrance to perfect communication. Gombrich appears to have implied that those geometrical shapes deprived of the variations of code can be much more effective than words in conveying certain kinds of messages to addressees. This is once again a paradox since we cannot dispense with certain feelings or ambiguities on our numerous encounters with artistic texts. Variations of code may serve to sug-gest or conjure up something unexpected in our mind instead of channelling us into specific ideas. They can actually work wonders as we perceive them, giving rise to constantly changing new discoveries.
Intriguingly, one of the strengths of Lotman’s spherical scheme (Figure 4) is to deal with our communication in the visual arts. This model serves to explain how we apply our intelligence to combining and recombining shapes:
The importance of spatial models created by culture lies in the fact that, unlike other basic forms of semiotic modeling, spatial models are constructed not on a verbal, discrete basis but on an iconic continuum. Their foundations are visually visible iconic texts and verbalization of them is secondary. This image of the uni-verse can better be danced than told, better drawn, sculpted or built than logically explicated. The work of the right hemisphere of the brain here is primary. But the first attempts at self-description of this structure inevitably involve the verbal level with the attendant semantic tension between the continual and the discrete semiotic pictures of the world. (Lotman 2001: 203)
Even though Lotman still retained the idea that natural language normally serves as the primary modelling system – the tool of meaning making – he put forward a new sense of modelling which is not linguistic at all but appears all the more powerful across cultures. His idea that modelling can be based on nonverbal and nondiscrete entities suggests that schemata or geometrical shapes are entitled to function as the primary modelling system as well – to model our innermost feelings and sensations for the arts. Nevertheless, even after merging Lotman and Gombrich on the spherical
scheme, we still cannot solve the paradox of schemata and nonverbal communication. According to Lotman, schematized images are created in accordance with our growth of consciousness, while in the case of Gombrich, our consciousness is preceded by the formation of schemata. Schemata are thought to function both at the periphery, modelling stimuli or information from the world, and at the centre, regulating and binding understanding. They obviously serve as both the cause and the result of our knowledge, some loosely constructed yet highly condensed forms in culture.
4. Art, play and emotional structure
How can Lotman’s concept of play serve to soften the paradox of schemata? In the 1960s, on top of Jakobson’s scheme, Lotman recognized some play-like elements in our way of dealing with the arts. The strength of introducing the concept of play into communication-modelling systems is in recognizing certain “emotional struc-tures” that addressees develop while communicating with addressers (Lotman 2011). Lotman made the point that addressees tend to make a larger story out of the infor-mation they receive: beholders and readers not only work to increase the amount of information already contained in the arts, but also to change their mindsets in the process of observing and interpreting. The heuristic function of play is to observe how addressees on the one hand work to integrate diverse feelings (mainly about overcoming fear or anxiety on the first encounters), and on the other to increase their chances of winning the game of interpreting (Lotman 2011: 253). Following the spirit of play which is about creating (1) the co-existence of contrary states of conscious-ness, i.e. some sort of multiperspectives; (2) alternations between diverse visual or conceptual patterns; (3) the bond between addressers and addressees, we are likely to conceive schemata as a system for shaping or scuplturing feelings which enables addressees to gain the greatest possible amount of empathy, harmony and insights from the kind of art or culture they are engaging with. By way of communicating with the spirit of play, we simply enjoy being tricked and concentrate on enhancing our imagination and wellbeing. In this sense, we no longer worry about whether the variations of code (highly charged with our emotions) are arbitrary or motivated, a disturbance or redundancy in communicating. We definitely need such emotive-cognitive composites to outcross different cultures and to push for new discoveries. Meanwhile, they enhance our chances of survival by way of increasing the amount of shared experiences and emotions in the global village.2
2 An early draft of this paper was presented at the conference dedicated to the 80th
Anniversary of Juri Lotman in 2002. Th e author revised the manuscript in July 2013 in light of her recent work on evolutionary aesthetics and a reading of Marshall McLuhan’s Th e Medium is the Message.
References
Gombrich, Ernst Hans 1984[1960]. Art and Illusion: A Study in the Psychology of Pictorial
Representation. New Jersey: Princeton University Press.
Gombrich, Ernst Hans 1994[1972]. The visual image: Its place in communication. In: Gombrich, Ernst Hans, The Image and the Eye: Further Studies in the Psychology of Pictorial
Representation. London: Phaidon, 137–161.
Ivanov, Vyacheslav V. 1978. The science of semiotics. New Literary History 4(2): 199–203. Jakobson, Roman 1960. Closing statement: Linguistics and poetics. In: Style in Language.
Cambridge: Press of Massachusetts Institute of Technology, 350–377.
Lotman, J. M. 1977a. Primary and secondary communication-modeling systems. In: Lucid, Daniel P. (ed., trans.), Soviet Semiotics: An Anthology. Baltimore: Johns Hopkins University Press, 95–98.
– 1977b. Two models of communication. In: Lucid, Daniel P. (ed., trans.), Soviet Semiotics:
An Anthology. Baltimore: Johns Hopkins University Press, 99–101.
– 2001[1990]. Universe of the Mind: A Semiotic Study of Culture. (Shukman, Ann, trans.) London: I. B. Tauris.
– 2011[1967]. The place of art among other modelling systems. Sign Systems Studies 39(2/4): 249–270.
Lotman, Juri M; Uspensky, Boris A. 1978. On the semiotic mechanism of culture. New Literary
History 4(2): 211–232.
Lotman, J. M.; Uspenskij, B. A.; Ivanov, V. V.; Toporov, V. N.; Pjatigorskij, A. M. 1973. Theses on the semiotic study of cultures (as applied to Slavic texts). In: Van der Eng, Jan; Grygar, Mojmir (eds.), Structures of Text and Semiotics of Culture. The Hague: Mouton, 57–83. Shukman, Ann 1978. Soviet semiotics and literary criticism. New Literary History 4(2): 189–197.
Схемы как первичная моделирующая система: перспективы изучения невербальной коммуникации Историк искусства Эрнст Гомбрих утверждает, что как художники, так и зрители стеснены специфическими культурными ментальными конфигурациями, внутри которых они действуют: он использует термин «схемы» для обозначения конкретных зарисовок и наших ментальных конфигураций. Описывая механизм культуры, Ю.М. Лотман помещает в центр семиосферы как язык, так и «схематический образ»: они способствуют введению единства и устранению противоречий. Схемы явно служат причиной и результатом человеческого познания, являясь некими свободно констру-ируемыми, но чрезвычайно сконденсированными формами искусства разных типов. Похоже, Лотман и Гомбрих, создававшие свои теории во времена гипертрофии науч-ных и лингвистических моделей, пришли к близким мнениям относительно способов и средств, объединяющих людей «поверх культур». В настоящей статье предпринимается попытка (1) исследовать, как Лотман и Гомбрих истолковывали якобсоновскую схему коммуникации; (2) показать парадокс схем и невербальной коммуникации, латентно присутствующий в их трудах; (3) смягчить этот парадокс, опираясь на лотмановскую концепцию игры как эмоциональной структуры. В статье утверждается, что эмоцио-нальный потенциал, определяемый перспективой зрителей или адресатов, необходим для нашего выживания в «глобальной деревне».
Skeemid kui kultuuri esmane modelleeriv süsteem: väljavaateid mitteverbaalse kommunikatsiooni uurimiseks
Kunstiajaloolane Ernst Gombrich väitis, et nii kunstnikke kui kunstipublikut piiravad konk-reetsed mõttelised kultuurikonfiguratsioonid, mille raames nad tegutsevad. Ta kasutas ter-minit “skeemid” (“schemata”), osutamaks nii tegelikele visanditele kui ka meie mõttelistele konfiguratsioonidele. Kultuurimehhanisme kaardistades paigutas Juri Lotman nii keele kui ka “skematiseeritud kujundi” semiosfääri keskmesse: need funktsioneerivad ühtsustamise ning vasturääkivuste kõrvaldamise huvides. Skeemid on ilmselt toiminud nii inimteadmiste põh-juse kui ka tulemusena lõdvalt konstrueeritud, ent äärmiselt kondenseeritud kunstivormidena erinevates valdkondades. Näib, et Lotman ja Gombrich, kes teoretiseerisid täppis- ning keele-teaduslike mudelite hüpertroofia ajal, tulid välja lähedaste teooriatega inimesi kultuuriüleselt siduvatest vahenditest. Kumbki esitas enda perspektiivi kultuuri, kunsti ja pildilise kujutamise omavahelisele toimimisele. Käesolevas artiklis püütakse (1) uurida, kuidas tõlgendasid Lotman ja Gombrich Jakobsoni kommunikatsiooniskeemi; (2) näidata skeemide ja mitteverbaalse kommunikatsiooni paradoksi, mis on latentselt kohal nende kirjutistes; (3) leevendada seda paradoksi, lähtudes sellest, kuidas Lotman kontseptualiseerib mängu emotsionaalse struk-tuurina. Artiklis väidetakse, et vaatajate või adressaatide perspektiivist määratletud emotiivne potentsiaal on meie maailmakülas ellujäämiseks hädavajalik.
