Assessment of multimedia quality relies heavily on subjective assessment, and is typically done by human subjects in the form of preferences or continuous ratings. Such data is crucial for analysis of different multimedia processing algorithms as well as validation of objective (computational) methods for the said purpose. To that end, statistical testing provides a theoretical framework towards drawing meaningful inferences, and making well grounded conclusions and recommendations. While parametric tests (such as t test, ANOVA, and error estimates like confidence intervals) are popular and widely used in the community, there appears to be a certain degree of confusion in the application of such tests. Specifically, the assumption of normality and homogeneity of variance is often not well understood. Therefore, the main goal of this paper is to revisit them from a theoretical perspective and in the process provide useful insights into their practical implications. Experimental results on both simulated and real data are presented to support the arguments made. A software implementing the said recommendations is also made publicly available, in order to achieve the goal of reproducible research.
Perceptual Visual Quality Assessment: Principles, Methods, and Future Directions
Cardiff University(卡迪夫大学);University of Klagenfurt(克拉根福大学);Shanghai Jiao Tong University(上海交通大学);University of Nantes(南特大学);University of Texas at Austin(德克萨斯大学奥斯汀分校)
Comparative Studies of Unsupervised and Supervised Learning Methods based on Multimedia Applications
IEEE Computational Intelligence Society(IEEE计算智能学会);Institute of Electrical and Electronics Engineers(电气和电子工程师协会);Blekinge Institute of Technology(布莱金厄理工学院)
Blind Quality Assessment of 3D Dense Point Clouds with Structure Guided Resampling
University of Waterloo(滑铁卢大学);Tencent MediaLab(腾讯媒体实验室);Ningbo University(宁波大学);Shanghai Jiao Tong University(上海交通大学);Nanyang Technological University(南洋理工大学)
Modeling Beyond MOS: Quality Assessment Models Must Integrate Context, Reasoning, and Multimodality
R&D Lab F-Initiatives(F-Initiatives研发实验室);Université d’Orleans(奥尔良大学);Université Sorbonne Paris Nord(巴黎-索邦大学);University of Dubai(迪拜大学);IULM University(IULM大学)
Generalised Score Distribution: A Two-Parameter Discrete Distribution Accurately Describing Responses from Quality of Experience Subjective Experiments