CATALOG DESCRIPTION: Fundamentals of visual, acoustic, and tactile perception; display devices; perceptual models for image, video,
acoustic, and tactile signal analysis, compression, quality evaluation, and understanding; multimodal signal processing and perception; content-based retrieval; sense substitution.

REQUIRED TEXT: None

OPTIONAL TEXT(s):

  1. Stephen E. Palmer, "Vision Science: Photons to Phenomenology" MIT Press, 1999
  2. Eberhard Zwicker and Hugo Fastl, "Psychoacoustics: Facts and Models", Springer, Information Sciences, third edition, 2006
  3. (Book Chapter) Roberta L. Klatzky and Susan J. Lederman, "Touch", (Chapter 6) in "Handbook of Psychology, Volume 4: Experimental Psychology", A. F. Healy and R.W. Proctor, eds., John Wiley and Sons, second edition, 2003

COURSE DIRECTOR: Thrasyvoulos N. Pappas

COURSE GOALS: To provide a thorough background in the fundamental principles and techniques in digital media processing, analysis, and compression, and how they relate, on the one hand, to acquisition and display devices, and on the other, to human perception. Overview of the current technologies and standards, and directions for future research.

PREREQUISITES BY COURSES: EECS 359 or equivalent or permission of the instructor.

DETAILED COURSE TOPICS:

Visual Imaging and Perception
+ Basics of the human visual system
+ Perceptual effects and illusions
+ Multiscale decompositions
+ Subband analysis and wavelets
+ Markov random fields
+ Perceptual quality metrics
+ Structural similarity metrics
+ Quality evaluation for scalable compression
+ Perceptually lossless compression
+ Structurally lossless compression
+ Bilevel image compression
+ Incremental parsing
+ Color processing and perception
+ Texture analysis and synthesis
+ Shape
+ Adaptive clustering
+ Perceptual color-texture segmentation
+ Spatio-temporal segmentation
+ Background subtraction
+ Joint segmentation and motion estimation
+ Semantic classification and retrieval
+ Digital image halftoning

Acoustic Perception
+ Distortion Measures
+ Perceptually lossless compression
+ Objective and subjective evaluation of loudness, sharpness, and roughness

Tactile Display and Perception
+ Fundamentals of human tactile perception
+ Haptic perception of real and synthetic materials
+ Tactile dimensions: roughness, regularity, directionality
+ Tactile devices
+ Signal processing for tactile display

Multimodal Processing and Perception
+ Roughness in sound and vision
+ Joint perception of visual, acoustic, and tactile signals

HOMEWORK ASSIGNMENTS: Each student will study and present in class two or three papers on a course topic to be selected in consultation with the instructor.

PROJECT: A bibliographical search or computer type project is required. The purpose of this project is to enhance the understanding
of a topic covered in class or to investigate a topic not covered in class. A final written report and a presentation are required.

GRADES: Homework - 40%
Class participation - 20%
Project - 40%

COURSE OBJECTIVES: When a student completes this course, s/he should be able to:
1. Understand the fundamentals of visual, acoustic, and tactile perception.
2. Have a basic understanding of display devices and their models.
3. Understand perceptual models for image, video, acoustic, and tactile signal analysis, with applications to compression, quality evaluation, retrieval, and understanding.
4. Understand the basics of multi-modal perception, interaction, and sense substitution.
5. Apply the acquired knowledge to specific electronic media applications.
6. Be prepared for advanced research or development in this area.