Adopted models

Iconicity remains one of the central characteristics of linguistic grammar, not only in the figurative case of poetic texts (Jakobson 1960; Hiraga 2005), but also in scientific writings (Halliday, 1990). This phenomenon is nonetheless never fully explored in modalities outside language. In this chapter, we adopt related models for the exploration of iconicity to a corpus of nonrepresentational modern Western paintings.

Two foci line up in our examination of paintings. First, the existence of iconic grammatical metaphors is investigated across a spectrum of nonrepresentational

Western modern paintings via Hiraga’s (2005) model. Second, the pattern of an iconic textual flow across painting-layouts is analyzed via the semiotic model of Kress and van Leeuwen (1996).

Hiraga’s linguistic model defines and lists out instances of iconic GMs, and Kress and van Leeuwen’s visual model helps provide a working definition for the visual counterparts of GI in painting-plates. The pictorial GM in this study, then, is defined when the visual forms and the multimodal semantic cues of a painting align with each other, creating a comprehensive meaning of the whole artwork.

Some of the GMs in Hiraga (2005) could directly be mapped to the examination of pictorials in Kress and van Leeuwen (1996), for instance the GMs of form quantity, distance and similarity. As noted in 2.2, there are iconic rules that only apply to

linguistic data, such as the iconic principle on linearity, and how linguistic politeness is instantiated through the in-text distance of lexical items. These iconic rules would be excluded in this study. While the politeness issue is clearly missing in the case of

paintings, the linearity issue too needs to be “hedged” in order to fit our visual data now, as implied in Chen and Su (2014).

Instead, in the two-dimensional case of paintings, Kress & van Leeuwen’s (1996) visual-oriented model could complement Hiraga’s (2005) linguistic model, for it focuses on an overall compositional flow of paintings via a four-part diagram (Fig. 2.1), rather

than a linear one. In the aspect of textual directionality, it is more flexible and visually-oriented than Hiraga’s model. In this study, both models are adopted and adjusted in generating a method suitable for multimodal exploration.

3.2 Data

Our data consists of 90 nonrepresentational modern Western paintings created in the 20^th century, including 30 figurative paintings, 30 semi-figurative paintings, and 30 abstract paintings, following the categorization in Arnheim (1969). The concepts of figurative and abstract are relative: they constitute two ends of a continuum, and could never be put in a clear-cut dichotomy. The inclusion of all styles of paintings ensures that a

comprehensive data pool is covered. Moreover, the three-group sampling expects to bring forth possible cross-group comparisons in compositional principles.

Each sampling group contains paintings created by one specifically representative painter. For the quality of the data, it is ensured that these representatives remain productive painters. The sampled paintings are selected out of three chronological painting albums of the representative artists, excluding practices of representational artwork, sketches, prints or other miscellaneous scripts. After the exclusion, the amount of samples is balanced to approximately thirty in each of the three painting groups.

The pure reality-simulative artworks are excluded. They would not constitute idealistic examples of expressive paintings since a representational intention, or restrictions bound to photography or real-world referents are involved. Instead,

nonrepresentational paintings are selected, with the pictorial contents generated via the painter’s free association. In this sense, paintings are autonomously expressive in its creation process and creative form. In the following, we will illustrate details of the sampled data.

1) Figurative painting samples: Dali

Thirty of Spanish artist Dali's paintings are representative as figurative, nonrepresentational modern Western paintings, for they are:

i. Figurative in style; the viewers can easily identify and name the painted elements;

ii. Nonrepresentational in intention, in which the painted events are mainly created via the process of free association, so they may lean closer to expressive texts rather than to real-world snapshots.

There are no abstract parts in Dali’s paintings-- every line and pile of paint constructs part of a figurative object. We can always name his painted items, even though his art does not rely on the tradition of real-life painting. His paintings thus serve ideally as our figurative samples.

2) Semi-figurative painting samples: Miro

Thirty of Spanish artist Miro's paintings are representative as being semi-figurative samples. The paintings are selected for being:

i. Semi-figurative in style: The intention of abstraction has not yet fully erased the identifiable semanticity of depicted items, and the pictorial items still carry degrees of semanticity. However, they are abstracted to some extents that demand more reading effort.

ii. Nonrepresentational in intention: While arranging non-abstract elements like human, animals, stars and birds, Miro would almost always simplify these elements. His artistic elements became so abstracted in form and

nonrepresentational in intention, that they became sign-like (Corbella, 1993).

Note that for this sampling group, possible candidates include semi-abstract styles of artwork such as ones done by Cubists. However, in most Cubists’ work, even if the depicted objects are abstracted and transformed, deliberately flattened till they stuck between figurativeness and mere abstraction, certain extents of real-world

representation are involved. To be able to select paintings that are most generative and autonomous in expressive processes, we exclude these pictures.

3) Abstract painting samples: Pollock

American artist Pollock's 30 paintings stand for ideal samples for being

“non-representation” and “abstract” modern Western paintings. His paintings are:

i. Abstract in style; the viewers cannot identify nor name any of the pictorial elements;

ii. Non-representative in intention, in which neither real-world referents nor a will

to represent objects could be found.

Pollock’s art pieces are selected due to their highly intuitive and impromptus nature

known as “action painting”. The “forms” in these paintings stand for nothing but themselves—the visual pleasure they arouse rather than any concrete referents.

The excluded candidates for this category are the abstract paintings done by the Minimalists. It is indeed the case that the inclination to feature only geometric shapes or color layers on canvases fits utmost the criterion of being nonrepresentational.

Nevertheless, minimalist abstract paintings undergo conceptual planning processes.

Even if the color-field paintings are done in an impromptu way, they do not necessarily contain the same extent of expressive intention most lyrical abstract paintings, such as ones done so intuitively done by Pollock, would have.

3.3Analysis procedure

We proposed an analysis procedure for examining the grammatical iconicity and textual flow of modern paintings. This procedure consists of three major examination steps designed based on the two models mentioned. A detailed introduction follows in 3.3.1 to 3.3.4, respectively.

3.3.1 Preliminary: Figures and titles

It is crucial to note that paintings preserve traces or processes of affordance during production, e.g. how painters physically make sense of the painting medium (paints

and textures) while they create. For every viewer alike, there is an initiative step to tell

“what is what” in a painting. After this process, the basic “happening” in the targeted

painting are more attentively viewed. Before we start our analysis, which concerns the multimodality of the current data, we will explain here how the iconic GMs and the judgment of information zones are achieved in a two-dimensional visual text. The key, we believe, lies in a mechanism which is often subconsciously applied: our instinct to the separation of figure and ground.

The notion of figure and ground has a psychological backbone: We as viewers could usually tell the attention focus of a painting, and even find such process effortless most of the time. Such attention foci are called figures in perceptual psychology, and

the rest of the painting, which is usually less-noticed and “backgrounded” in our attention, is termed grounds. The notion of figure and ground has been proposed in visual psychology and cognitive sciences:

“The figure is located inside an outline (a closed visual edge), it has a form and is more or less like an object (…) In experiments, it is more easily identified and named, more easily linked to semantic, aesthetic and emotional values.”

“The ground (…) is more or less formless, more or less

homogeneous and is perceived as extending behind the figure.”

--Aumont (1997:46)

To identify the figure zones in two-dimensional paintings is mainly a perceptual process. In figurative and semi-figurative sample groups, as the figurativeness of depicted foci mostly stand as figures, which are attention zones opposing the surrounded grounds. In the more challenging case of abstract samples, it is still possible to tell specific layers or zones of Pollock’s paintings being figures. That is to say, even the seemingly wild action paintings created by Pollock are actually

governed by the cognitive principle of figure-ground distinction. Again, this may be due to the fact that even Pollock himself would subconsciously make sense of his creation during the rapid painting process. Some zones or layers of paints thus stand out, even just slightly, over the other zones or layers of Pollock’s abstract paintings.

Figure 3.6 shows the figure zones of one of Pollock’s painting. We consider the red parts to be the figure zones, since the blackness segregates densely into a unity, a gestalt area, at the center of the painting.

The following is a demonstration of figure-ground separation to a figurative sample, a semi-figurative sample, and an abstract sample. The outlined are the figures, and the rests stand for grounds.

Figure 3.1 The birth of liquid desires Figure 3.2 Figure zones to The birth of liquid desires

Figure 3.3 Character and bird Figure 3.4 Figure zone for Character and bird

Figure 3.5 Number 32 Figure 3.6 Figure zone for Number 32

Besides the visual-perceptual basis, painting titles are also an essential part of the multimodal whole of an artwork. In Kress and van Leeuwen (1996), visual

representational structures are divided into two major types, namely Narrative and Conceptual (1996:56). The narratives pictorials are those that contain actions or visual

linkages between visual participants within an image, while the conceptual pictorials being ones that are more motionless, meaningful in the appearance of the visualized itself. In categorizing paintings following such a distinction, their titles serve as a helpful semantic cue other than the mere visual content presented in the paintings.

Moreover, modern paintings are created by artists with a brand new extent of autonomy over their artwork. They are pieces of work in wait for specific

acknowledgment from viewers, art critics, exhibition curators, art dealers, and even from art museums. In this respect, the titles of modern paintings constitute a crucial part of the art piece as a symbolic, multimodal, and contextualized whole. We thus co-analyze painting titles and the painted images, so that the denoted content could be more comprehensively interpreted.

In our three sample groups, painting titles generally inform the viewers not only the expressed content, and also help trigger richer interpretation or appreciation to the painting. For instance, Dali, a skilled figure-ground reversion expert, entitled a

painting with the keyword Voltaire. In the painting, the close-up of Voltaire is formed

by the smaller outlines of three maids in black-and-white dresses. If not the title, the viewers might miss out the alternative, gestalt grasp of the Voltaire outline (Fig. 3.7).

The same title-image alignment could be commonly found in semi-figurative samples as well, such as a Miro painting entitled Portrait II (Figure 3.8). Of course, it is possible that many viewers could interpret the human figure the geometric zones in this painting indicate. Nevertheless, it is the title that more firmly fixated the

semantics to the pictorials.

In abstract paintings, particularly, it is often the case that they do not own a

semantically concrete title. Even so, the title directs us to be able to focus on the

“entity-like” status of visual Gestalts. When they accompany semantic titles, as in the

below example, the title helps greatly in hinting a certain way of painting appreciation.

In Pollock’s Out of the web (Fig. 3.9), the red zones already catch our attention in the first glimpse. However, adding in the information of the verbal cue, we are more likely to observe the vectoring that relaxes from left to right, as it matches the “out of the web” concept in the title. In this respect, painting titles appear as cognitively essential in the process of painting appreciation, serving as a more dominant interpretation within the numerous possible ones.

Figure 3.7 The slave market with the disappearing bust of Voltaire

Figure 3.8 Portrait II

Figure 3.9 Out of the web

3.3.2 Examination 1: Iconicity in grammatical metaphors

Hiraga’s (2005) set of GMs serves as our examination criteria for pictorial

MEANING. In this study, we will further examine the selected paintings by the same five GMs found applicable in Modern paintings based on Chen and Su’s (2014) pilot study. Based on these cognitive linguistic principles, we propose a set of extended working definition for the current multimodal data. In the following, we list out instances of iconic GMs defined in figurative, semi-figurative and abstract paintings.

1) SIMILARITY OF FORM IS SIMILARITY OF MEANING Hiraga (2005:177-178) defines similarity of form as a flexible range of

form-resemblance, including shared lexical elements, i.e. word affinity phenomena.

For instance, many words featuring gl- imply lightness, e.g. glitter, glisten, and glow.

In the same vein, we define similarity of form in terms of the family resemblance of shapes or outlines between figure zones done by the same artist.

For instance, in Dali’s The Persistence of Memory (Fig. 3.10), several clocks are depicted. Even though they are twisted in different ways, the fact that they all stand for prototypical locks is unquestionable: the clocks are fully depicted with pointers, rims and clock-faces. Thus, the clocks in this painting are considered similar in form, matching their meaning similarity.

Similarly, in Miro’s Woman and bird at sunrise (Fig. 3.11), the two major pictorial elements stand for “birds”, and are arranged in similar forms. They clearly

differ from the smaller pictorial elements at the lower left, which stands for

“human”—it is, contrastingly, vertically arranged and depicted with stronger

emphasis on his body. The birds are similar in form, and similar in meaning, hence applying the GM SIMILARITY IN FORM IS SIMILARITY IN MEANING.

In abstract paintings, however, the meaning contrasts do not exist, and this GM could not be applied. There is no semanticity embedded in the pictorial elements in abstract paintings. In fact, from GM 1) to 3), the abstract group could not be defined by them.

2) DIFFERENCE IN FORM IS DIFFERENCE IN MEANING

Difference of form in Hiraga (2005) is defined based on the sharedness of lexical

components. For instance, the light-related words coined with gl- usually share nothing in common with words coined with fl-, which often suggest movement, such as flip, flit, or flap (Hiraga 2005:178). Thus, for multimodal data, we identify instance of different forms as pictorial units that clearly distinctive from each other in formal features; and usually, the distinction of pictorial forms is actively created to avoid semantic confusions.

As an example, in Dali’s Swans reflecting elephants (Fig. 3.12), the pictorial elements above water stand for swans, and their reflections became elephants. The swans and elephants are distinctive in their shapes even though they are closely placed, and are supposedly symmetrical in reality. This is an example of

DIFFERENCE IN FORM IS DIFFERENCE OF MEANING.

In the semi-figurative Woman and bird in the night (Fig. 3.13), two similarly sized pictorial units are clearly divergently formed: The upper one features thicker outlines and has no emphasis of facial parts, while the lower one clearly featuring a head and two exaggerated eyes. Thus, the bird on top and the person beneath it clear belong to two distinct denotations, and are different in form and meaning.

3) SEMANTIC RELEVANCE IS CLOSENESS

This iconic GM concerns the actual distance between pictorial items in a painting. It is similar to Lakoff and Johnson’s (1980:126-33) STRENGTH OF EFFECT IS CLOSENESS. Hiraga instantiates this GM with the below example (2005:181):

a) Mary doesn’t think he’ll leave until tomorrow.

b) Mary thinks he won’t leave until tomorrow.

The stronger iconicity of proximity in b) is created by the shorter semantic proximity in the sentence structure.

The madonna of port lligat (Fig. 3.14) is an instantiation of SEMANTIC

RELEVANCE IS CLOSENESS in our figurative samples: The animated or semantically relevant elements are placed closer, as other elements grouped at the peripheral. The two sets of elements are not randomly mixed in the painting-plate.

In a semi-figurative sample,The day's awakening (Fig. 3.15), animated characters are placed at the left-center, as other non-human elements arranged at the reams of the painting. Semantically closer elements are compressed closer.

4) MORE CONTENT IS MORE FORM

In Hiraga’s original definition, this GM refers to the effect word repetition creates in strengthening the power and rhythm of Japanese poetry. The amount (spatial size) of words matches and enhances its intended expressive effect.

In the case of images, it is reported (O’Toole 1994; Kress and van Leeuwen 1996) that some of the visual-perceptual facets would affect the vividness of pictorial parts, including size, extent of sharpness, and tonal or color contrasts. QUANTITY IS IMPORTANCE could be instantiated either by the repetition or larger size of certain figures, or by a stronger visual-perceptual saliency. In this respect, the

linguistic-based QUANTITY IS IMPORTANCE could be imported to a multimodal-based SALIENCY IS IMPORTANCE.

In Dali’s The spectre of sex appeal (Fig. 3.16), the largest pictorial element is the giant woman body at the leftward center. Its size makes it extensively eye-catching, and highlights the semantics of the title. This is an instance of MORE CONTENT IS MORE FORM, as the giant woman in this painting serves as the attention center as well as the textual focus of the multimodal piece.

Sometimes, the importance of content would be reflected both in the size of the targeted form, and the number of it. In Miro’s Woman and birds in the night (Fig.

3.17), the largest elements are the “faces” entangled at the center area. Their size and their coupling status declare and strengthen their textual importance.

Similarly, in abstract paintings, bigger sized gestalt shapes occupy more textual space, therefore a more protruding textual importance. Sometimes, both sizes and elemental numbers multiply the importance. A Pollock painting called Blue Poles (Fig.

3.18) is a vivid example.

5) MARKED MEANING IS MARKED FORM

Linguistically speaking, the markedness of form contrasts a default, more basic linguistic arrangement. For example, English past tense is often featured with overt semantic markers (Hiraga 2005:184), thus considered more marked comparing to their present simple tense counterparts.

In our multimodal analysis, the markedness of form would be defined according to the visual facets of perceptual saliency and its meaningful status within the

In our multimodal analysis, the markedness of form would be defined according to the visual facets of perceptual saliency and its meaningful status within the