Subjective experiment

Ten automotive audio methods proposed in Sections II and III are compared via the following subjective listening experiments, according to a modified double-blind Multi-Stimulus test with Hidden Reference and a hidden Anchor (MUSHRA) [38].

The experiment cases are described in Table II. In Experiment I, four songs in two-channel PCM format involving various instruments with significant dynamic

variations were chosen to be the test materials. In Experiments II to IV, four 5.1-channel movies in Dolby Digital format were used. Both timbral and spatial qualities are considered. The loudness of each reproduced signal was adjusted to the same level by measuring the sound pressure level at each seat with a monitoring microphone.

Eight subjective attributes employed in the tests, including preference, timbral attributes (fullness, brightness, artifact) and spatial attributes (localization, frontal image, proximity, envelopment) are summarized in Table III. Forty subjects participated in each experiment. The subjects participating in the tests were instructed with definitions of the subjective indices and the procedures before the listening tests. The subjects were asked to respond after listening in a questionnaire, with the aid of a set of subjective indices measured on an integer scale from −3 to 3.

Positive, zero, and negative scores indicate perceptually improvement, no difference, and degradation, respectively, of the signals after processing with the audio spatializers. The order of the attributes is randomized except that the index preference is always the last question. On the average, it took approximately forty minutes to finish an experiment. In order to access statistical significance, the scores were further processed by using the MANOVA. If the significance level is below 0.05, the differences among all methods are considered statistically significant and then examined further by the Fisher’s LSD post-hoc test.

4.2.1 Experiment I

In this experiment, three methods for the listening positions at the FL and RR seats (representing the ‘extreme’ cases) and the two-channel input, including the UDWD method, the upmixingHIF1 method and upmixingPIF1 method are evaluated.

Apart from these three methods, a hidden reference (H. R.) and an anchor (An.) are added into the comparison. The case in which two-channel stereo input signals are

fed to the respective front and rear loudspeakers is used as the hidden reference. The signal obtained by summing and lowpass filtering (with 4 kHz cutoff frequency) the two-channel input signals is used as the anchor that is also fed to all loudspeakers.

Since the methods upmixingHIF1 and upmixingPIF1 are devised for single listener, this experiment is separated into two parts: the front-left seat and the rear-right seat.

In FL position, the MANOVA output indicates that only the index artifact exhibited no significant difference among all methods in timbral quality (F = 1.08262, p>0.367). However, in spatial quality, the indices localization (F = 1.8456, p >

0.154) and proximity (F = 2.57037, p>0.067) exhibited no significant difference among all methods. Figures 26(a) and (b) show the means and spreads (with 95%

confidence intervals) of the grades of each subjective index. The x-axis and y-axis represent the method and grade, respectively. The results of Fisher LSD post hoc test indicted that the grades of the UDWD method and the upmixingPIF1 method are significantly higher than those of the hidden reference and the upmixingHIF1 method in preference and brightness. In fullness, the grades of the inverse filter-based approaches (upmixingHIF1 and upmixingPIF1) are significantly lower than those of hidden reference and the UDWD method. In the spatial attributes, the proposed approached are all significantly outperform the hidden reference in frontal and envelopment, but there are no significant different among the methods in localization and proximity.

In the RR position case, Figures 26(c) and (d) show the means and spreads (with 95% confidence intervals) of the grades of each subjective indices. The results of post hoc test reveal that the grade of the upmixingPIF1 method is significantly higher than those of the other approaches in preference, notwithstanding the grades of the UDWD and the upmixingHIF1 methods are significant higher than the hidden reference. In brightness and fullness, result similar to the case of the FL position is

obtained. The proposed approaches receive lower grade in fullness, but higher grade in brightness. In terms of artifact, the grade of the UDWD method is the highest among all the approaches. This implies that no artifacts are audible during the UDWD processing. In localization, frontal and proximity, the proposed methods are all significant higher than the hidden reference; whereas only the UDWD and upmixingHIF1 methods perform significantly well to the reference. To summarize, the UDWD method and upmixingPIF1 method are the preferred choices for position FL and RR respectively, because of their rendering performance in preference and spatial quality.

4.2.2 Experiment II

The DWD, HIF1 and PIF1 methods and the unprocessed 5.1-channel reproduction are compared in this experiment. Because only four loudspeakers are available in this car, the center channel of the 5.1-channel input is attenuated by -3 dB and mixed into the frontal channels to serve as the hidden reference. In addition, the four-channel signals are summed and lowpass filtered (with 4 kHz cutoff frequency) is used as the anchor. Fifteen listeners participated in the test for the front left and rear right seats.

Figures 27(a) and (b) show the means and spreads (with 95% confidence intervals) of the grades of all attributes for all methods for FL position, whereas Figs.

27(c) and (d) show those for RR position. For the FL position, the results of the post hoc test indicate that the grades of the HIF1 method in preference and fullness are significantly higher than those of the DWD and the PIF1 methods. In brightness, only the grade of PIF1 methods is significantly higher than the hidden reference, and there is no significant different among the DWD method and the HIF1 method.

Further, there is no significant difference among methods in the attribute artifact, localization, proximity and envelopment. In frontal, the inverse filter-based methods

are significantly higher than the hidden reference and the DWD method.

In the RR position, there is no significant difference among all the methods in fullness, artifact and localization. However, the grades of the inverse filtering-based methods are significantly higher than those in others in preference and brightness.

In addition, grades of all the proposed methods are significantly higher than the grade of the hidden reference but in frontal and proximity. Finally, only the HIF1 method significantly outperform to the hidden reference. In general, all grades received are higher for the rear seat than for the front seat. In particular, the HIF1 method received the highest grades in most attributes, especially in spatial attributes. A low computation complexity substitute would be the PIF1 method since it received the highest grade in many attributes as well.

4.2.3 Experiment III

Experiment III is intended for evaluating the methods designed for two-listener mode and 5.1-channel input. Four methods are compared in this experiment, including the DWD method, the HIF2 method, the HIF2a method and the PIF2a method. The hidden reference and the anchor cases are the same with those in experiment II. Figures 28(a) and (b) show the means and spreads (with 95%

confidence intervals) of the grades of the first four and the last four attributes, respectively. The post hoc test reveals that there is no significant difference between the DWD method and HIF2a method, while the grades of both are significantly higher than the hidden reference in overall preference. In fullness and proximity, there is no significant difference among all proposed methods. In brightness, result similar to the experiment II is obtained, the inverse filtering-based methods receive significant higher grades than the hidden reference but there is no significant difference among these methods. The grade of artifact obtained using the HIF2 method is very low, implying that some artifacts are audible. The reason might be the nature of

non-square inverse filter design. In frontal and localization, the grades of all proposed methods are significantly higher than the hidden reference. Finally, the HIF2a method performs best in envelopment among all methods. To conclude, the HIF2a method might be the best choice for spatial quality. It is noted that the result is contrary to our expectation that more inverse filters (HIF2) should mean better performance. In terms of computation complexity and rendering performance, the DWD method is the adequate approach for the two-passenger mode.

4.2.4 Experiment IV

In this experiment, methods developed for four-listener mode, including the DWD method and the PIF4 method, were compared. The hidden reference and the anchor cases are the same with those in experiment II. The means and spreads (with 95% confidence intervals) of the grades of the attributes are shown in Figs. 29(a) and (b). The results of MANOVA output indicate that there is no significant difference among the methods in the attributes artifact and envelopment. Further, the results of the post hoc test show that there is no significant difference between the DWD method and the PIF4 method in preference and proximity, but the grades of these two methods are all significantly higher than the hidden reference. In the attributes brightness, localization and frontal, the PIF4 method receives the significantly highest grade. Overall, the PIF4 method does not significantly outperform the DWD method in both timbral and spatial quality. Similar result can be obtained that the inverse filtering-based approaches do not outperform the DWD method in multi-listener mode.