Chapter 6 Statistics5The purpose of this chapter is to introduce the concept of statistical analysis of data from experiments on perceptual evaluation of sound. The aim is to introduce the general strategy for such analysis and the various statistical methods available and to provide guidance on method to be used in specific cases. The aim is not to give a detailed mathematical description of the procedures, but rather to present the methods such that qualified questions can be asked to a professional statisticianif needed. If you are not a professional statistician or do not have one on your team, it is recommended that arrangements be made such that one can be consulted on a regular basis. It is especially important that the statistical parts of an experimental design be discussed thoroughly before any subjects are invited to participate in the experiment.The focus of this chapter is on hypothesis testing and to a lesser degree on the estimation of the various variables. This is motivated by the fact that the purpose of most listening tests is to detect and quantify significant differences between the loudspeakers, for example, and not (yet) to quantify the relationship between the dependent and independent variables.A multitude of books exist on the various statistical analysis procedures, to which the interested reader is referred. A selection that the authors have found useful, covering most of the issues discussed, includes 155, 163, 253, 268,328,396.The sections have been organised to reflect the order of the processes that an experimenter follows: statistical design of the experiment, testing of the statistical assumption of the obtained data and, finally, the detailed statistical analysis. This organisation might not be optimal when seen from a purely statistical point of view, but as this book addresses the experimenter and not the statistician, the needs of the former have prevailed.The importance of detailed statistical planning of the experiment and subsequent analysis of the results is often not acknowledged by audio engineers (nor practitioners from other disciplines as well). The physical variables (the test setup, the stimuli, etc.) are usually controlled to an acceptable degree; however, the results are often only reported by simple mean values without confidence intervals, for example. Such a lack of statistical information may lead to incorrect scientific or application decisions.The basic purpose of a listening test is to objectively quantify the variations in auditory impression caused by the presented stimuli. The variation in the stimulus set is created by the controlled combinations of the experimental independent variables. The research question asked by the experimenter is whether the variation in the subjective impression is a result of this intended variation or is it more likely to be a random variation. The purpose of the statistical planning of the experiment and analysis of the subjective responses is to provide a statistical validation of the answer to this question.The statistical analysis is based on the assumption that the t, ith score, Yt,i provided by a subject consists of two parts: one part t representing the reaction to the ith repetition of the stimuli t defined by the controlled experimental variables (the treatments) and the other, an error part Et,i, representing a random variation caused by arange of uncontrolled variables:Yt,i = t + Et,i(6.1)The random variation caused by uncontrolled variables is related to the subject themselves and/or the experimental setup and environment. The variables related to the subject are physiological variables such as variations in sensitivity, background level of intrinsic neural activity, and so on, and cognitive variables such as mood, expectation, interpretation and bias effects (see Section 4.2.4). The variables related to the experimental setup and the environment in which the setup is established are, for example, changes in the temperature of the room, in the lighting conditions in the room, background noise, and so on. Toole 421423 and Bech 26 have more detailed discussions of these variables and their influence in relation to listening tests (see also chapter 5). Keppel and Wickens 253 (Section 2.1) provide a more general discussion of the variables that cause the random fluctuations in the subjects responses.The variables contributing to the score of a subject can, in general, be classified as follows:Permanent or stable abilities. This includes, for example, the subjects hearing threshold for the attribute under study or the general impression of a loudspeaker that remains constant during the experiment.The treatment effect. This is the influence of the variables controlled by the experimenter.Internal variability. This includes the variables related to temporary changes (during the experiment) in the subjects physiology and psychology.Externa