The Influence that Person's Sensibility Receives from Mixed-Reality based on a
Tanka (a traditional Japanese poem of 31 syllables)
Hiroshi Juzoji, Muhammad Athar Sadiq, Isao Nakajima, Yasumitsu Tomioka *
Tokai University School of Medicine, Office Tomioka*
juzoji@is.icc.u-tokai.ac.jp
Abstract
We chose a poem depicting the moon and a mountain scene at night, written by Yakamochi Otomo in the Man’yo-shu that is thought to represent Japan’s oldest diary. The particular Otomo’s poem was chosen because it allows easy identification of the “where”(location) and the “when”(time) described by the poem. EEG measurement were taken as subjects were presented with Yakamochi Otomo’s poem in three different formats: 1) an oral presentation; 2) when viewing a TV screen image; 3) when viewing a three panel wide-vision screen (as a Mixed-reality image). According to results provided by fast Fourier transforms and frequency analysis, the presentation of Mixed-reality displayed on the three panel wide-vision screen was most effective in creating a sense of immersion and a state of relaxation.
1. Purpose
We set out to evaluate the sensitivity and emotional response of subjects as determined by EEG (Electroencephalogram) readings by visually reproducing scenes onto a three panel wide-vision screen adapted from a tanka poem written by Yakamochi Otomo. This poem is also thought to represent Japan’s oldest diary or journal. For this research, we chose a poem depicting the moon and a mountain scene at night, written by Yakamochi Otomo and recorded in the Collection of Myriad Leaves (Man’yo-shu, Man’yo collection of poems). The particular Otomo poem was chosen because it allows easy identification of the “where” (location) and the “when” (time) described by the poem. The night-scene depicted in the poem was visually reproduced onto a three panel wide-vision screen in order to evaluate the effects induced by brain activity when recounting or recalling the past. The subject’s EEGs was first converted by a fast Fourier transform, and then evaluated through frequency analysis.
2. Background
2-1. Research on Sensitivity
Human thinking entails rationalization based on knowledge or intellect. We simultaneously perceive our experiences in both this intellectual way, and in sensuous and emotional terms. Humans recognized the presence of
these two faculties long before Greek civilization. The former has been termed “Logos” and the latter “Pathos." From the earliest times, one way of sharing experience has been poetry – including the tanka, a traditional Japanese form of 31 syllables –music, painting, theater, and religion, to some extent. These methods of non-intellectual communication were inherited by contemporary society, and together help define what we know as a specific culture. On an individual and personal level, such works are often inspired by the desire to share various feelings and emotions. Over time, and with advancing technology, this desire to communicate has evolved into media forms such as theater, movies, and television. We regard the psychological extremity of sensitivity sharing this type of communication equivalent to a state of mass-intoxication. Gamelan music indigenous to the island of Bali, resonating over a frequency of 50-100 kHz, is said to induce a neuro-physiological state of intoxication. Similarly, the Kecak dance in Bali, with its hand motions (fever-pitched motion of hands) of a hundred or so naked figures dancing in the dark is said to have the power to induce intoxication in even the audience watching the dance. These states of intoxication occur as the result of an irresistible psychological urge to empathize with the masses. In medical terms, the phenomenon can be regarded as a transient case of reactive psychosis [1].
There are numerous methods by which one can experience the type of emotional communication described above, from tanka to Kecak dance. However, we believe a projection of Mixed-reality produced by a synthesis of multiple camera images and computer-generated virtual reality space projected onto a three panel wide-vision screen represents a much more effective form of such communication than reading a book or watching television. We began by comparing the effectiveness of two methods. The first method entails reading of a tanka poem, which conveys minimal information, while the second involves recreating a scene mentioned in the tanka poem onto three panel wide-vision screen.
2-2. EEG and Fast Fourier Analysis
EEGs (electroencephalograms) are recordings of electrical activity of the brain, as detected by silver electrodes positioned on the scalp. The electrical potentials involved fall in the range of 5-300µV, at frequencies of 5-30 Hz. No examination surpasses the EEG’s sensitivity
in representing direct brain activity in terms of space and time. However, because EEGs monitor the total electric potential of numerous neuron cells from the surface of the scalp, spatial separation and frequency analysis are critical in evaluating EEG results. We currently use a spectrum analysis method of frequency analysis for the relationship between electric potential and frequency data. This frequency analysis is called a discrete Fourier transform, using algorithms derived from the fast Fourier transform. Here we are most interested in the beta wave’s (13-30 Hz) PCA (Principal Component Analysis) and alpha attenuation, the electric potential ratio of alpha waves (8-13 Hz) when eyes are shut and open. Past research has demonstrated that the former (beta wave) indicates concentration/attention span, while the latter (alpha wave) indicates a state of comfort and relaxation [2].
2-3. Information on Collection of Myriad Leaves
Ancient Japanese history is classified into the Yayoi,Asuka, Nara, and Heian periods. Any writing dating
before the Nara period, public or private, has been found only in the form of engravings on tombs or stones (a total of 20 artifacts). The oldest extant paper documents are stored at Shousou-in in Nara prefecture. The number of artifacts totals approximately 12,000 pieces. Valuable historical literature such as “Kojiki” and “Nihon-sho-ki” are among these documents, as well as Collection of Myriad (ten thousand) Leaves for waka or tanka poems.
The typical citizens around the time of the Nara period could not read or write; the only individuals capable of writing were bureaucrats. Nor was there a tradition of a personal diary or journal, even among the educated class. One of the first such is a compilation of poems edited by Yakamochi Otomo, entitled Collection of Myriad Leaves, which may also be interpreted as a diary or journal written by Yakamochi Otomo [3].
We thought it would be possible to reproduce the night scene described in a poem using a computer if we could identify the location and the date described in the Collection of Myriad Leaves, and if we could find a poem in Collection of Myriad Leaves describing the shape and the location of the moon.
2-4. Reproduction of Time
If we enter an arbitrary time in hour / minute / date / month / year and an arbitrary location in latitude / longitude / height, astronomical simulator can reproduce the constellations visible on that particular night. However, this work also involves various adjustments involving date / month settings, since Europe in the Middle Ages relied on the Julian calendar, which set 365.25 days per year. Our current calendar system, the Gregorian calendar, sets 365.2422 days per year. A conversion is needed to accurately recreate the oldest dates. The essential
difference between Julian and Gregorian calendars is the handling of leap years. Europeans added 10 days to the date October 5, 1582, making it October 15, 1582, to make dates consistent with the present system of leap years. Europe converted to the Gregorian calendar after making this adjustment. Japan converted from the old lunar calendar to the Gregorian calendar on January 1, 1873. The Japanese lunar calendar originated in China and is based on the Julian calendar. However, the two may differ by several days. Since dates in Collection of Myriad Leaves are based on the old lunar calendar, we need a formula to convert past dates to the present Gregorian calendar [4].
3. Method
3-1. System Overview
An overview of the evaluation system is shown in figure 1. A three panel wide-vision CRT screen (hardware configuration providing a 135˚ angle of view consists of IBM compatible PC with clock speeds of 3GHz) and three LCD projectors (800 x 600 pixels, full-color RGB). The system is designed to examine EEG readings and frequency analysis using oral presentations, single-panel CRTs, and three panel wide-vision screen (135˚).
Figure 1. System outline
3-2. Examination Method
We chose five subjects whose fields of specialization excluded literature. Subjects were asked to draw straws to determine the order in which they were examined, then asked to stand in the waiting room so that the contents of the examination would remain unknown. Each subject was given a presentation of a Collection of Myriad Leaves poem in the following sequence of presentations, from 1) to 3). The subject was also asked to open and close their eyes from time to time to permit readings to be obtained. 1) Listening to an oral presentation
2) Viewing a television screen
3) Viewing a three panel wide-vision screen
3-3. EEG Examination
Silver electrodes were used to measure electric potential, as shown in figure 2. The electrodes were placed in contact with the subject’s scalp as single-pole conductors throughout the subject’s scalp (evenly spaced and retaining left / right symmetry).
Figure 2. Electrode layouts
A. PCA (Principal Component Analysis)
PCA is designed to evaluate wave synchronism from a number of electrodes. When synchronism is uniform, the signals are considered to be integrated. When synchronism is off and random, the signals are considered to be non-integrated. An epileptic fit may be regarded as an extreme example of the former. In such instances, the PCA determines that the brain is concentrating on processing information [5].
B. Alpha attenuation
When a subject’s eyes are shut, the subject’s alpha wave activity increases. A comparison of open and shut status implies that levels of awareness [5]. In other words, comparing the alpha attenuation of a subject during testing (test load) can help indicate how well a subject has adapted to our designed environment.
alpha attenuation =
3-4. Mixed-reality
A. Virtual Reality:An astronomical simulator was used to reproduce the night sky on a three panel wide-vision screen. Software was used on PCs configured for the Windows XP environment, and graphic board that has three outputs were installed to display the image on three screens. B. Reality:
Present night scene actually described in the poem was taken with a triple-lens camera and recorded onto a VCR. The image was then loaded onto PC following editing. The primary reason for this shoot was to capture certain content difficult to reproduce via virtual reality, such as mountains casting a shadow onto foregrounds or grass fields blowing in the wind. These images were then
combined with VR. C. Sound
This sensory component would aid in transporting the listener back to a night sky in the distant past when listening to a poem (factual) written about a certain past time and date (simulated). Some interpretive comments and some sound effects were added to the presentation.
3-5. Selected Poem
In recreating the moon and the starry skies of a certain time and date (simulated) in the past on the astronomical software, the following poem has been selected.
“It must be dark, and late at night, because already the moon sinks toward the ridge of Mt. Futagami.”
The author is Yakamochi Otomo, born in 718 A.D.The poem was written in the year 746. In the preceding year of 746, in March, he had won a promotion within the government. But after three months, in June 746, he was sent to Ecchu area as Defense Minister, the northern part of Japan. He apparently composed the poem there, while recalling the city far away.
3-6. Measurement Results
Measurements were taken for beta wave’s PCA and alpha attenuation, the electric potential ratio of alpha waves (8-13 Hz) when eyes are shut and open. Both measurements were taken from the three different presentation formats of Yakamochi Otomo’s poem: 1) Listening to an oral presentation (eyes shut); 2) when viewing the TV screen; 3) when viewing the three panel wide-vision screen.
Table 1. Result by presentation formats
EEG (Power Spectrum) PCA (Principal Component Analysis) alpha attenuation (Eyes shut / open) 1) Listening 2) TV screen 3) Three panel wide-vision screen small medium large small small Large
4. Discussion
4-1. Measurement Analysis
In general, alpha waves appear in the parietal region when the eyes are closed, fading when the eyes are opened. However, we also see wave frequencies quite similar to alpha waves when a person watches a dimly-lit screen while deep in a state of delusion. The alpha-wave activity has been observed to increase as the display area of the night sky grows larger (a three-panel screen compared to a single CRT).
alpha wave: eyes open alpha wave: eyes shut
Even when we compare the alpha attenuation, the effect of the three panel wide-vision screen is significant, coming close to presenting the subject with the impression that he or she is back in the Nara period. The EEG results seem to concur with our findings. On the other hand, the oral presentation indicated low values even in PCA (Principal Component Analysis). The oral presentation alone appears to be too diffuse, failing to capture the subject's full attention.
These findings suggest that the Mixed-reality image projected onto the three panel wide-vision screen provides a deeper sense of immersion for subjects.
4-2. Lunar Calendar in the Days of Man’yo-shu
Yakamochi’s poem describes the moon nearing the ridges of Mt. Futagami, which also implies that the time is late at night. If we can find a poem with a detailed description of the shape and position of the moon, it would be possible to convert lunar calendar dates into Gregorian calendar dates.According to research conducted by Japanese literary experts, we can say the following about Yakamochi’s poem:
Time: August 27, year 746. Converted in to solar calendar.
Location: Fushiki-furukofu, Takaoka City, Toyama prefecture.
Academic theories point to a location around the present Fushiki meteorological station area.
What the poem fails to reveal: 1) The shape of the moon
2) The time at which the moon sets over the horizon The position of the moon during this season as it sets falls approximately between west-by-southwest and southwest. If we can assume an angle of elevation of 15˚, the moon should be in a southwesterly location one hour before setting.
According to computer simulations, the observation point must be to the north and to the east of Mt. Futagami’s peak in order for the moon to be viewed both in a southwesterly direction and over Mt. Futagami itself. The Fushiki meteorological station falls within these parameters. According to our calculations, the moon sets over the horizon at 9:50pm, which seemingly conflicts with Yakamochi’s verse, which gives the time as follows: “it must be late in the dark night－." If we can assume a later time for the moon to set – perhaps midnight – it would be reasonable to assume the date is not August 27, but several days later, around September 1.
Figure 3. Map around the Fushiki Figure 4. Simulation of starry sky
5. Conclusion
We evaluated the subject’s sensitivity and emotional changes through measurements of EEG. EEG measurements were taken as subjects were presented with Yakamochi Otomo’s poem in three different formats: 1) an oral presentation; 2) when viewing a TV screen image; 3) when viewing a three panel wide-vision screen (as a Mixed-reality image). According to results provided by fast Fourier transforms and frequency analysis, the presentation of Mixed-reality displayed on the three panel wide-vision screen was most effective in creating a sense of immersion and a state of relaxation.
6. Acknowledgements
We would like to express our deep appreciation to the staff curator at the Man’yo Historical Museum of Takaoka City, Collection of Myriad Leaves research authority Souzou Kurokawa, and Takaoka city council member Eiichiro Mukai for their invaluable advice and assistance.
7. References
1. K.Kaketa: Modern Psychiatry Summa 6B, Nakayama Shoten 2. T.Okuma: Clinical Electroencephalography, Igaku-shoin 1991 3. N.Kojima, M.Kinoshita, H.Higashino: Man’yo-shu, Shogakukan 1996
4. M.Uchida: Source book of Japanese lunar calendar, Yuzankaku 1975
5. H.Araga, T.Musha: A new analysis of sensitivity by EEG measurement, NF Circuit Design Block
