This book presents the state-of-the-art in visual media coding and transmission Visual Media Coding and Transmission is an output of VISNET II NoE, which is an EC IST-FP6 collaborative research project by twelve esteemed institutions from across Europe in the fields of networked audiovisual systems and home platforms.

VISNET II Researchers xiii Preface xv Glossary of Abbreviations xvii 1 Introduction 1 2 Video Coding Principles 7 2.1 Introduction 7 2.2 Redundancy in Video Signals 7 2.3 Fundamentals of Video Compression 8 2.3.1 Video Signal Representation and Picture Structure 8 2.3.2 Removing Spatial Redundancy 9 2.3.3 Removing Temporal Redundancy 14 2.3.4 Basic Video Codec Structure 16 2.4 Advanced Video Compression Techniques 17 2.4.1 Frame Types 17 2.4.2 MC Accuracy 19 2.4.3 MB Mode Selection 20 2.4.4 Integer Transform 21 2.4.5 Intra Prediction 22 2.4.6 Deblocking Filters 22 2.4.7 Multiple Reference Frames and Hierarchical Coding 24 2.4.8 Error-Robust Video Coding 24 2.5 Video Codec Standards 28 2.5.1 Standardization Bodies 28 2.5.2 ITU Standards 29 2.5.3 MPEG Standards 29 2.5.4 H.264/MPEG-4 AVC 31 2.6 Assessment of Video Quality 31 2.6.1 Subjective Performance Evaluation 31 2.6.2 Objective Performance Evaluation 32 2.7 Conclusions 35 References 36 3 Scalable Video Coding 39 3.1 Introduction 39 3.1.1 Applications and Scenarios 40 3.2 Overview of the State of the Art 41 3.2.1 Scalable Coding Techniques 42 3.2.2 Multiple Description Coding 45 3.2.3 Stereoscopic 3D Video Coding 47 3.3 Scalable Video Coding Techniques 48 3.3.1 Scalable Coding for Shape, Texture, and Depth for 3D Video 48 3.3.2 3D Wavelet Coding 68 3.4 Error Robustness for Scalable Video and Image Coding 74 3.4.1 Correlated Frames for Error Robustness 74 3.4.2 Odd–Even Frame Multiple Description Coding for Scalable H.264/AVC 82 3.4.3 Wireless JPEG 2000: JPWL 91 3.4.4 JPWL Simulation Results 94 3.4.5 Towards a Theoretical Approach for Optimal Unequal Error Protection 96 3.5 Conclusions 98 References 99 4 Distributed Video Coding 105 4.1 Introduction 105 4.1.1 The Video Codec Complexity Balance 106 4.2 Distributed Source Coding 109 4.2.1 The Slepian–Wolf Theorem 109 4.2.2 The Wyner–Ziv Theorem 110 4.2.3 DVC Codec Architecture 111 4.2.4 Input Bitstream Preparation – Quantization and Bit Plane Extraction 112 4.2.5 Turbo Encoder 112 4.2.6 Parity Bit Puncturer 114 4.2.7 Side Information 114 4.2.8 Turbo Decoder 115 4.2.9 Reconstruction: Inverse Quantization 116 4.2.10 Key Frame Coding 117 4.3 Stopping Criteria for a Feedback Channel-based Transform Domain Wyner–Ziv Video Codec 118 4.3.1 Proposed Technical Solution 118 4.3.2 Performance Evaluation 120 4.4 Rate-distortion Analysis of Motion-compensated Interpolation at the Decoder in Distributed Video Coding 122 4.4.1 Proposed Technical Solution 122 4.4.2 Performance Evaluation 126 4.5 Nonlinear Quantization Technique for Distributed Video Coding 129 4.5.1 Proposed Technical Solution 129 4.5.2 Performance Evaluation 132 4.6 Symmetric Distributed Coding of Stereo Video Sequences 134 4.6.1 Proposed Technical Solution 134 4.6.2 Performance Evaluation 137 4.7 Studying Error-resilience Performance for a Feedback Channel-based Transform Domain Wyner–Ziv Video Codec 139 4.7.1 Proposed Technical Solution 139 4.7.2 Performance Evaluation 140 4.8 Modeling the DVC Decoder for Error-prone Wireless Channels 144 4.8.1 Proposed Technical Solution 145 4.8.2 Performance Evaluation 149 4.9 Error Concealment Using a DVC Approach for Video Streaming Applications 151 4.9.1 Proposed Technical Solution 152 4.9.2 Performance Evaluation 155 4.10 Conclusions 158 References 159 5 Non-normative Video Coding Tools 161 5.1 Introduction 161 5.2 Overview of the State of the Art 162 5.2.1 Rate Control 162 5.2.2 Error Resilience 164 5.3 Rate Control Architecture for Joint MVS Encoding and Transcoding 165 5.3.1 Problem Definition and Objectives 165 5.3.2 Proposed Technical Solution 166 5.3.3 Performance Evaluation 169 5.3.4 Conclusions 171 5.4 Bit Allocation and Buffer Control for MVS Encoding Rate Control 171 5.4.1 Problem Definition and Objectives 171 5.4.2 Proposed Technical Approach 172 5.4.3 Performance Evaluation 177 5.4.4 Conclusions 179 5.5 Optimal Rate Allocation for H.264/AVC Joint MVS Transcoding 179 5.5.1 Problem Definition and Objectives 179 5.5.2 Proposed Technical Solution 180 5.5.3 Performance Evaluation 181 5.5.4 Conclusions 182 5.6 Spatio-temporal Scene-level Error Concealment for Segmented Video 182 5.6.1 Problem Definition and Objectives 182 5.6.2 Proposed Technical Solution 183 5.6.3 Performance Evaluation 187 5.6.4 Conclusions 188 5.7 An Integrated Error-resilient Object-based Video Coding Architecture 189 5.7.1 Problem Definition and Objectives 189 5.7.2 Proposed Technical Solution 189 5.7.3 Performance Evaluation 195 5.7.4 Conclusions 195 5.8 A Robust FMO Scheme for H.264/AVC Video Transcoding 195 5.8.1 Problem Definition and Objectives 195 5.8.2 Proposed Technical Solution 195 5.8.3 Performance Evaluation 197 5.8.4 Conclusions 198 5.9 Conclusions 199 References 199 6 Transform-based Multi-view Video Coding 203 6.1 Introduction 203 6.2 MVC Encoder Complexity Reduction using a Multi-grid Pyramidal Approach 205 6.2.1 Problem Definition and Objectives 205 6.2.2 Proposed Technical Solution 205 6.2.3 Conclusions and Further Work 208 6.3 Inter-view Prediction using Reconstructed Disparity Information 208 6.3.1 Problem Definition and Objectives 208 6.3.2 Proposed Technical Solution 208 6.3.3 Performance Evaluation 210 6.3.4 Conclusions and Further Work 211 6.4 Multi-view Coding via Virtual View Generation 212 6.4.1 Problem Definition and Objectives 212 6.4.2 Proposed Technical Solution 212 6.4.3 Performance Evaluation 215 6.4.4 Conclusions and Further Work 216 6.5 Low-delay Random View Access in Multi-view Coding Using a Bit Rate-adaptive Downsampling Approach 216 6.5.1 Problem Definition and Objectives 216 6.5.2 Proposed Technical Solution 216 6.5.3 Performance Evaluation 219 6.5.4 Conclusions and Further Work 222 References 222 7 Introduction to Multimedia Communications 225 7.1 Introduction 225 7.2 State of the Art: Wireless Multimedia Communications 228 7.2.1 QoS in Wireless Networks 228 7.2.2 Constraints on Wireless Multimedia Communications 231 7.2.3 Multimedia Compression Technologies 234 7.2.4 Multimedia Transmission Issues in Wireless Networks 235 7.2.5 Resource Management Strategy in Wireless Multimedia Communications 239 7.3 Conclusions 244 References 244 8 Wireless Channel Models 247 8.1 Introduction 247 8.2 GPRS/EGPRS Channel Simulator 247 8.2.1 GSM/EDGE Radio Access Network (GERAN) 247 8.2.2 GPRS Physical Link Layer Model Description 250 8.2.3 EGPRS Physical Link Layer Model Description 252 8.2.4 GPRS Physical Link Layer Simulator 256 8.2.5 EGPRS Physical Link Layer Simulator 261 8.2.6 E/GPRS Radio Interface Data Flow Model 268 8.2.7 Real-time GERAN Emulator 270 8.2.8 Conclusion 271 8.3 UMTS Channel Simulator 272 8.3.1 UMTS Terrestrial Radio Access Network (UTRAN) 272 8.3.2 UMTS Physical Link Layer Model Description 279 8.3.3 Model Verification for Forward Link 290 8.3.4 UMTS Physical Link Layer Simulator 298 8.3.5 Performance Enhancement Techniques 307 8.3.6 UMTS Radio Interface Data Flow Model 309 8.3.7 Real-time UTRAN Emulator 312 8.3.8 Conclusion 313 8.4 WiMAX IEEE 802.16e Modeling 316 8.4.1 Introduction 316 8.4.2 WIMAX System Description 317 8.4.3 Physical Layer Simulation Results and Analysis 323 8.4.4 Error Pattern Files Generation 324 8.5 Conclusions 328 8.6 Appendix: Eb/No and DPCH_Ec/Io Calculation 329 References 330 9 Enhancement Schemes for Multimedia Transmission over Wireless Networks 333 9.1 Introduction 333 9.1.1 3G Real-time Audiovisual Requirements 333 9.1.2 Video Transmission over Mobile Communication Systems 335 9.1.3 Circuit-switched Bearers 339 9.1.4 Packet-switched Bearers 348 9.1.5 Video Communications over GPRS 350 9.1.6 GPRS Traffic Capacity 351 9.1.7 Error Performance 354 9.1.8 Video Communications over EGPRS 357 9.1.9 Traffic Characteristics 357 9.1.10 Error Performance 358 9.1.11 Voice Communication over Mobile Channels 359 9.1.12 Support of Voice over UMTS Networks 360 9.1.13 Error-free Performance 361 9.1.14 Error-prone Performance 362 9.1.15 Support of Voice over GPRS Networks 362 9.1.16 Conclusion 363 9.2 Link-level Quality Adaptation Techniques 365 9.2.1 Performance Modeling 365 9.2.2 Probability Calculation 367 9.2.3 Distortion Modeling 368 9.2.4 Propagation Loss Modeling 368 9.2.5 Energy-optimized UEP Scheme 369 9.2.6 Simulation Setup 370 9.2.7 Performance Analysis 372 9.2.8 Conclusion 373 9.3 Link Adaptation for Video Services 373 9.3.1 Time-varying Channel Model Design 374 9.3.2 Link Adaptation for Real-time Video Communications 379 9.3.3 Link Adaptation for Streaming Video Communications 389 9.3.4 Link Adaptation for UMTS 396 9.3.5 Conclusion 402 9.4 User-centric Radio Resource Management in UTRAN 403 9.4.1 Enhanced Call-admission Control Scheme 403 9.4.2 Implementation of UTRAN System-level Simulator 403 9.4.3 Performance Evaluation of Enhanced CAC Scheme 410 9.5 Conclusions 411 References 413 10 Quality Optimization for Cross-network Media Communications 417 10.1 Introduction 417 10.2 Generic Inter-networked QoS-optimization Infrastructure 418 10.2.1 State of the Art 418 10.2.2 Generic of QoS for Heterogeneous Networks 420 10.3 Implementation of a QoS-optimized Inter-networked Emulator 422 10.3.1 Emulation System Physical Link Layer Simulation 426 10.3.2 Emulation System Transmitter/Receiver Unit 428 10.3.3 QoS Mapping Architecture 428 10.3.4 General User Interface 438 10.4 Performances of Video Transmission in Inter-networked Systems 442 10.4.1 Experimental Setup 442 10.4.2 Test for the EDGE System 443 10.4.3 Test for the UMTS System 445 10.4.4 Tests for the EDGE-to-UMTS System 445 10.5 Conclusions 452 References 453 11 Context-based Visual Media Content Adaptation 455 11.1 Introduction 455 11.2 Overview of the State of the Art in Context-aware Content Adaptation 457 11.2.1 Recent Developments in Context-aware Systems 457 11.2.2 Standardization Efforts on Contextual Information for Content Adaptation 467 11.3 Other Standardization Efforts by the IETF and W3C 476 11.4 Summary of Standardization Activities 479 11.4.1 Integrating Digital Rights Management (DRM) with Adaptation 480 11.4.2 Existing DRM Initiatives 480 11.4.3 The New ‘‘Adaptation Authorization’’ Concept 481 11.4.4 Adaptation Decision 482 11.4.5 Context-based Content Adaptation 488 11.5 Generation of Contextual Information and Profiling 492 11.5.1 Types and Representations of Contextual Information 492 11.5.2 Context Providers and Profiling 494 11.5.3 User Privacy 497 11.5.4 Generation of Contextual Information 498 11.6 The Application Scenario for Context-based Adaptation of Governed Media Contents 499 11.6.1 Virtual Classroom Application Scenario 500 11.6.2 Mechanisms using Contextual Information in a Virtual Collaboration Application 502 11.6.3 Ontologies in Context-aware Content Adaptation 503 11.6.4 System Architecture of a Scalable Platform for Context-aware and DRM-enabled Content Adaptation 504 11.6.5 Context Providers 507 11.6.6 Adaptation Decision Engine 510 11.6.7 Adaptation Authorization 514 11.6.8 Adaptation Engines Stack 517 11.6.9 Interfaces between Modules of the Content Adaptation Platform 544 11.7 Conclusions 552 References 553 Index 559

This book presents the state-of-the-art in visual media coding and transmission Visual Media Coding and Transmission Transmission is an output of VISNET II NoE, which is an EC IST-FP6 collaborative research project by twelve esteemed institutions from across Europe in the fields of networked audiovisual systems and home platforms. The authors provide information that will be essential for the future study and development of visual media communications technologies. The book contains details of video coding principles, which lead to advanced video coding developments in the form of Scalable Coding, Distributed Video Coding, Non-Normative Video Coding Tools and Transform Based Multi-View Coding. Having detailed the latest work in Visual Media Coding, networking aspects of Video Communication is detailed. Various Wireless Channel Models are presented to form the basis for both link level quality of service (QoS) and cross network transmission of compressed visual data. Finally, Context-Based Visual Media Content Adaptation is discussed with some examples. Key Features: Contains the latest advances in this important field covered by VISNET II NoE Addresses the latest multimedia signal processing and coding algorithms Covers all important advance video coding techniques, scalable and multiple description coding, distributed video coding and non-normative tools Discusses visual media networking with various wireless channel models QoS methods by way of link adaptation techniques are detailed with examples Presents a visual media content adaptation platform, which is both context aware and digital rights management enabled Contains contributions from highly respected academic and industrial organizations Visual Media Coding and Transmission will benefit researchers and engineers in the wireless communications and signal processing fields. It will also be of interest to graduate and PhD students on media processing, coding and communications courses.