Iconicity and textual flow

Another line of “textual reality”, besides the iconic instantiation of grammatical metaphors, is defined by Halliday as the iconic flow of textual information, which is constructed by Information values, i.e. the relative idea of Given or New information of a text. Specifically, he states:

There is a movement from a given Theme (…) to a rhematic New (…) (T) his movement in time construes iconically the flow of

information.

(Halliday 2002 [1990]:175)

This definition of textual reality is based on Halliday’s theory on textual pattern.

As claimed, the clausal structure in English consists of two intertwined information lines, namely the couplet of Theme and Rheme, which is speaker-oriented, as well as Given and New, which is listener-oriented. The two information lines are simultaneous in nature: As the Theme and the Rheme realized via the “segment position” in a clause, the Given and the New accompany prosodic contrasts. Therefore, it is possible that the

two interrelated systems combine, resulting in an overlap of the Theme and the Given, together serving as the starting information or “ground” (as contrary to “figure” in

gestalt psychology) of an expression.

For instance, in the sentence I think it looks great, the front part I think serves as

expression. Meanwhile, the part it looks great remains more “unpredictable” and “news worthy”, which expresses the speaker’s evaluation. Without this part, the sentence fails to give out its essential “punch line”. It could thus be counted as the Rheme, or the New of the sentence. In this case, the overlap of Theme and Given, as well as that of Rheme and New, applies to Halliday’s definition of a default, or “unmarked” instance of textual information. The proceeding of a text, then, is made possible by the spatial as well as meaningful tension between the Given and the New. This is one of the core principles that constitute the grammar of natural languages (2002 [1990]:173). The default

arrangement, as noted, consists of a Theme or Given information at the sentence initial, and a follow-up of Rheme or New. Concerning the writing directionality of most Western languages, it is often the case that such “default” information flow of linguistic texts lines up from the left, to its right-branching sentential parts:

Halliday’s Information system is extended into the realm of the multimodality.

Specifically, social semioticians Kress and van Leeuwen (1996; 2006) studied the

compositional pattern of visual texts in general. They have proposed a general pattern, or ‘visual grammar’, across most western visual layouts. The proposed diagram for

visual compositions is as illustrated:

Figure 2.1 Kress and van Leeuwen (2006:179)

This diagram predicts the general textual composition of visual materials in the context of Western cultures: From the left to the right, there is a pattern of Given to New;

from the top to the bottom, there is a pattern of Ideal to Real. It is claimed that a

Given-New information pattern could be observed across most Western visuals, which is postulated to be influenced by the writing-directionality in most Western languages,

following Halliday’s view. Usually, there would be a left-placing of Given information, while the more “surprising”, unexpected content featured at the right, and labeled as the

New. The circle in the diagram stands for the sometimes centralized visual focus observed. Kress and van Leeuwen claimed that a centralized visual composition would

still follow the overall tendency suggested by this diagram, which is exclusively designed for the two-dimensional nature of pictoriality.

Besides the criteria of Ideal-Real and Given-New, which are based on Information values of in-text pictorial areas, Kress and van Leeuwen mentioned two alternative

principles which also play a role in influencing visual compositions. The two principles are Saliency and Framing, respectively. While Saliency refers to the perceptually salient qualities of a visual, which is often created by stronger contrast in color, tone, size or sharpness of pictorial compositions (Kress and van Leeuwen 1996:212), Framing refers to the compositional separation of textual areas, which is an important feature of applied visual layouts.

Kress and van Leeuwen’s (1996; 2006) model, while inspiring in nature, consists of a mixture of visual genres, ranging from composite pictorials such as magazine layouts, films, photography, picture books, and single pictorials like paintings, graphic designs, and advertisements. The realm of paintings has not been systematically focused on in this model. This gap prompts the current need to study modern paintings in a systematic way. In the following section, we will briefly introduce the basic

characteristics of the set of pictorials in wait for systematic discussion, and with the utmost essentialness to be involved in cognitive multimodal studies.

2.4 Modern Western nonrepresentational paintings as autonomous pictorials This study specifically focuses on a selection of nonrepresentational modern Western paintings. According to Luhmann (2000), after the Renaissance, painters in the Western world slowly gained their autonomous in art creation, shedding away the traditional burden, for instance to paint for religious duties. Since the late 19^th century to the early 20s, the outburst of artistic movements such as Impressionism, Fauvism and Cubism opened up a brand new extent of artistic freedom, leading to the heydays of Modern Art.

Art movements mushroomed, each striving to defy the traditional ways of depiction.

The paintings created in the Modern Art era, which strongly coincides with the

blooming of the mentioned artistic movements around 1900 till the late 70s (concerning the life spans of the featured artists), revolutionized the way we see and think visually.

Besides its expressive autonomy, modern Western paintings are also comparatively autonomous due to their default forms: They are almost always presented on the flatness of a rectangular canvas, as commented by Schapira (1969):

“We take for granted today as indispensable means the rectangular form of the sheet of paper and its clearly defined smooth surface on which one draws and writes (…) the smooth prepared field is an invention of a later stage of humanity.”

(Schapiro 1969:9)

In a sense, the somewhat conventionalized form of modern painting pieces strengthens their independence. They are considered single pieces, ready to be displayed or

purchased, rather than serving as an illustration for other communicative purposes. We could say that the autonomy of a modern painting is co-shaped by the “conventionalized”

use of canvases. Such materiality distinguishes “works of art” away from “crafts”, such as pictures painted on vases, walls or furniture. Summing the above, modern paintings, especially ones that originated in the mentioned contexts (20^th century Western modern Art movements), are ideal to be seen as independent “texts” conveyed in a pictorial mode. This stands as the basis for our assumption. In the next chapter, we will introduce how we examine these paintings via a cognitive multimodal approach.

Chapter 3

A cognitive multimodal model

This chapter introduces a methodology for textually analyzing paintings. Our model combines Hiraga’s (2005) grammatical metaphors (GM) with Kress and van Leeuwen’s (1996) visual compositional principles, serving to analyze paintings in a multimodal approach. We aim to find out whether non-representative paintings qualify the definition of a ‘reality of text’, and constitute a structurally-arranged text genre. The following illustrates the present methodological design, the adopted models, as well as details of data selection.

3.1 Adopted models

Iconicity remains one of the central characteristics of linguistic grammar, not only in the figurative case of poetic texts (Jakobson 1960; Hiraga 2005), but also in scientific writings (Halliday, 1990). This phenomenon is nonetheless never fully explored in modalities outside language. In this chapter, we adopt related models for the exploration of iconicity to a corpus of nonrepresentational modern Western paintings.

Two foci line up in our examination of paintings. First, the existence of iconic grammatical metaphors is investigated across a spectrum of nonrepresentational

Western modern paintings via Hiraga’s (2005) model. Second, the pattern of an iconic textual flow across painting-layouts is analyzed via the semiotic model of Kress and van Leeuwen (1996).

Hiraga’s linguistic model defines and lists out instances of iconic GMs, and Kress and van Leeuwen’s visual model helps provide a working definition for the visual counterparts of GI in painting-plates. The pictorial GM in this study, then, is defined when the visual forms and the multimodal semantic cues of a painting align with each other, creating a comprehensive meaning of the whole artwork.

Some of the GMs in Hiraga (2005) could directly be mapped to the examination of pictorials in Kress and van Leeuwen (1996), for instance the GMs of form quantity, distance and similarity. As noted in 2.2, there are iconic rules that only apply to

linguistic data, such as the iconic principle on linearity, and how linguistic politeness is instantiated through the in-text distance of lexical items. These iconic rules would be excluded in this study. While the politeness issue is clearly missing in the case of

paintings, the linearity issue too needs to be “hedged” in order to fit our visual data now, as implied in Chen and Su (2014).

Instead, in the two-dimensional case of paintings, Kress & van Leeuwen’s (1996) visual-oriented model could complement Hiraga’s (2005) linguistic model, for it focuses on an overall compositional flow of paintings via a four-part diagram (Fig. 2.1), rather

than a linear one. In the aspect of textual directionality, it is more flexible and visually-oriented than Hiraga’s model. In this study, both models are adopted and adjusted in generating a method suitable for multimodal exploration.

3.2 Data

Our data consists of 90 nonrepresentational modern Western paintings created in the 20^th century, including 30 figurative paintings, 30 semi-figurative paintings, and 30 abstract paintings, following the categorization in Arnheim (1969). The concepts of figurative and abstract are relative: they constitute two ends of a continuum, and could never be put in a clear-cut dichotomy. The inclusion of all styles of paintings ensures that a

comprehensive data pool is covered. Moreover, the three-group sampling expects to bring forth possible cross-group comparisons in compositional principles.

Each sampling group contains paintings created by one specifically representative painter. For the quality of the data, it is ensured that these representatives remain productive painters. The sampled paintings are selected out of three chronological painting albums of the representative artists, excluding practices of representational artwork, sketches, prints or other miscellaneous scripts. After the exclusion, the amount of samples is balanced to approximately thirty in each of the three painting groups.

The pure reality-simulative artworks are excluded. They would not constitute idealistic examples of expressive paintings since a representational intention, or restrictions bound to photography or real-world referents are involved. Instead,

nonrepresentational paintings are selected, with the pictorial contents generated via the painter’s free association. In this sense, paintings are autonomously expressive in its creation process and creative form. In the following, we will illustrate details of the sampled data.

1) Figurative painting samples: Dali

Thirty of Spanish artist Dali's paintings are representative as figurative, nonrepresentational modern Western paintings, for they are:

i. Figurative in style; the viewers can easily identify and name the painted elements;

ii. Nonrepresentational in intention, in which the painted events are mainly created via the process of free association, so they may lean closer to expressive texts rather than to real-world snapshots.

There are no abstract parts in Dali’s paintings-- every line and pile of paint constructs part of a figurative object. We can always name his painted items, even though his art does not rely on the tradition of real-life painting. His paintings thus serve ideally as our figurative samples.

2) Semi-figurative painting samples: Miro

Thirty of Spanish artist Miro's paintings are representative as being semi-figurative samples. The paintings are selected for being:

i. Semi-figurative in style: The intention of abstraction has not yet fully erased the identifiable semanticity of depicted items, and the pictorial items still carry degrees of semanticity. However, they are abstracted to some extents that demand more reading effort.

ii. Nonrepresentational in intention: While arranging non-abstract elements like human, animals, stars and birds, Miro would almost always simplify these elements. His artistic elements became so abstracted in form and

nonrepresentational in intention, that they became sign-like (Corbella, 1993).

Note that for this sampling group, possible candidates include semi-abstract styles of artwork such as ones done by Cubists. However, in most Cubists’ work, even if the depicted objects are abstracted and transformed, deliberately flattened till they stuck between figurativeness and mere abstraction, certain extents of real-world

representation are involved. To be able to select paintings that are most generative and autonomous in expressive processes, we exclude these pictures.

3) Abstract painting samples: Pollock

American artist Pollock's 30 paintings stand for ideal samples for being

“non-representation” and “abstract” modern Western paintings. His paintings are:

i. Abstract in style; the viewers cannot identify nor name any of the pictorial elements;

ii. Non-representative in intention, in which neither real-world referents nor a will

to represent objects could be found.

Pollock’s art pieces are selected due to their highly intuitive and impromptus nature

known as “action painting”. The “forms” in these paintings stand for nothing but themselves—the visual pleasure they arouse rather than any concrete referents.

The excluded candidates for this category are the abstract paintings done by the Minimalists. It is indeed the case that the inclination to feature only geometric shapes or color layers on canvases fits utmost the criterion of being nonrepresentational.

Nevertheless, minimalist abstract paintings undergo conceptual planning processes.

Even if the color-field paintings are done in an impromptu way, they do not necessarily contain the same extent of expressive intention most lyrical abstract paintings, such as ones done so intuitively done by Pollock, would have.

3.3Analysis procedure

We proposed an analysis procedure for examining the grammatical iconicity and textual flow of modern paintings. This procedure consists of three major examination steps designed based on the two models mentioned. A detailed introduction follows in 3.3.1 to 3.3.4, respectively.

3.3.1 Preliminary: Figures and titles

It is crucial to note that paintings preserve traces or processes of affordance during production, e.g. how painters physically make sense of the painting medium (paints

and textures) while they create. For every viewer alike, there is an initiative step to tell

“what is what” in a painting. After this process, the basic “happening” in the targeted

painting are more attentively viewed. Before we start our analysis, which concerns the multimodality of the current data, we will explain here how the iconic GMs and the judgment of information zones are achieved in a two-dimensional visual text. The key, we believe, lies in a mechanism which is often subconsciously applied: our instinct to the separation of figure and ground.

The notion of figure and ground has a psychological backbone: We as viewers could usually tell the attention focus of a painting, and even find such process effortless most of the time. Such attention foci are called figures in perceptual psychology, and

the rest of the painting, which is usually less-noticed and “backgrounded” in our attention, is termed grounds. The notion of figure and ground has been proposed in visual psychology and cognitive sciences:

“The figure is located inside an outline (a closed visual edge), it has a form and is more or less like an object (…) In experiments, it is more easily identified and named, more easily linked to semantic, aesthetic and emotional values.”

“The ground (…) is more or less formless, more or less

homogeneous and is perceived as extending behind the figure.”

--Aumont (1997:46)

To identify the figure zones in two-dimensional paintings is mainly a perceptual process. In figurative and semi-figurative sample groups, as the figurativeness of depicted foci mostly stand as figures, which are attention zones opposing the surrounded grounds. In the more challenging case of abstract samples, it is still possible to tell specific layers or zones of Pollock’s paintings being figures. That is to say, even the seemingly wild action paintings created by Pollock are actually

governed by the cognitive principle of figure-ground distinction. Again, this may be due to the fact that even Pollock himself would subconsciously make sense of his creation during the rapid painting process. Some zones or layers of paints thus stand out, even just slightly, over the other zones or layers of Pollock’s abstract paintings.

Figure 3.6 shows the figure zones of one of Pollock’s painting. We consider the red parts to be the figure zones, since the blackness segregates densely into a unity, a gestalt area, at the center of the painting.

The following is a demonstration of figure-ground separation to a figurative sample, a semi-figurative sample, and an abstract sample. The outlined are the figures, and the rests stand for grounds.

Figure 3.1 The birth of liquid desires Figure 3.2 Figure zones to The birth of liquid desires

Figure 3.3 Character and bird Figure 3.4 Figure zone for Character and bird

Figure 3.5 Number 32 Figure 3.6 Figure zone for Number 32

Besides the visual-perceptual basis, painting titles are also an essential part of the multimodal whole of an artwork. In Kress and van Leeuwen (1996), visual

representational structures are divided into two major types, namely Narrative and Conceptual (1996:56). The narratives pictorials are those that contain actions or visual

linkages between visual participants within an image, while the conceptual pictorials being ones that are more motionless, meaningful in the appearance of the visualized itself. In categorizing paintings following such a distinction, their titles serve as a helpful semantic cue other than the mere visual content presented in the paintings.

Moreover, modern paintings are created by artists with a brand new extent of autonomy over their artwork. They are pieces of work in wait for specific

acknowledgment from viewers, art critics, exhibition curators, art dealers, and even from art museums. In this respect, the titles of modern paintings constitute a crucial part of the art piece as a symbolic, multimodal, and contextualized whole. We thus co-analyze painting titles and the painted images, so that the denoted content could be more comprehensively interpreted.

In our three sample groups, painting titles generally inform the viewers not only the expressed content, and also help trigger richer interpretation or appreciation to the painting. For instance, Dali, a skilled figure-ground reversion expert, entitled a

painting with the keyword Voltaire. In the painting, the close-up of Voltaire is formed

by the smaller outlines of three maids in black-and-white dresses. If not the title, the viewers might miss out the alternative, gestalt grasp of the Voltaire outline (Fig. 3.7).

The same title-image alignment could be commonly found in semi-figurative samples as well, such as a Miro painting entitled Portrait II (Figure 3.8). Of course, it is possible that many viewers could interpret the human figure the geometric zones in this painting indicate. Nevertheless, it is the title that more firmly fixated the

semantics to the pictorials.

In abstract paintings, particularly, it is often the case that they do not own a

In abstract paintings, particularly, it is often the case that they do not own a