DI04000051: Multimedia Technologies
GTU Course

Lecture Slides

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

1 / 533

Table of Contents I
1

Lecture 1: Fundamentals of Multimedia

2

Lecture 2: Fundamentals of Multimedia

3

Lecture 3: Fundamentals of Multimedia

4

Lecture 4: Fundamentals of Multimedia

5

Lecture 5: Multimedia Software Tools and Team Roles

6

Lecture 6: Fundamentals of Multimedia

7

Lecture 7: Image Editing

8

Lecture 8: Image Editing
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

2 / 533

Table of Contents II
9

Lecture 9: Image Editing

10

Lecture 10: Image Editing

11

Lecture 11: Image Editing

12

Lecture 12: Image Editing

13

Lecture 13: Image Editing

14

Lecture 14: Image Editing

15

Lecture 15: Image Editing

16

Lecture 16: Image Editing
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

3 / 533

Table of Contents III
17

Lecture 17: Image Editing

18

Lecture 18: Image Editing

19

Lecture 19: Image Editing

20

Lecture 20: Image Editing

21

Lecture 21: Video and Animation

22

Lecture 22: Video and Animation

23

Lecture 23: Video Basic Terminology and Concepts

24

Lecture 24: Video and Animation
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

4 / 533

Table of Contents IV
25

Lecture 25: Video and Animation

26

Lecture 26: Video and Animation

27

Lecture 27: Video and Animation

28

Lecture 28: Video and Animation

29

Lecture 29: Video and Animation

30

Lecture 30: Video and Animation

31

Lecture 31: Video and Animation

32

Lecture 32: Video and Animation
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

5 / 533

Table of Contents V
33

Lecture 33: Video and Animation

34

Lecture 34: Multimedia Information Sharing

35

Lecture 35: Multimedia Information Sharing

36

Lecture 36: Multimedia Information Sharing

37

Lecture 37: Multimedia Information Sharing

38

Lecture 38: Multimedia Information Sharing

39

Lecture 39: Multimedia Information Sharing

40

Lecture 40: Augmented Reality and Virtual Reality
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

6 / 533

Table of Contents VI
41

Lecture 41: Augmented Reality and Virtual Reality

42

Lecture 42: Augmented Reality and Virtual Reality

43

Lecture 43: Augmented Reality and Virtual Reality

44

Lecture 44: Augmented Reality and Virtual Reality

45

Lecture 45: Augmented Reality and Virtual Reality

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

7 / 533

Introduction: What is Multimedia? I
Definition of Multimedia: Derived from the words Multi (many)
and Media (distribution mediums or tools used to store and deliver
information).
Core Concept: The combination of text, graphics, artwork, sound,
animation, and video delivered to users by computer or other
electronic means.
Interactive Multimedia: When the end-user is given operational
control over what elements are presented and when they are delivered.
Hypermedia: A structure of linked elements through which users can
navigate interactively (e.g., clickable links, graphics, and media
elements).
Linear vs. Non-linear:
Linear : Projects that start and run through to completion without
navigation control (e.g., traditional cinema or television).
Non-linear : Projects where user interaction controls navigation and
content flow (e.g., interactive games, courseware).
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

8 / 533

Core Components of Multimedia: Part 1 I
Text:
Most basic and common medium of communication.
Includes titles, menus, navigation elements, body text, and headlines.
Typography plays a crucial role (font selection, size, tracking, leading,
and legibility).

Sound & Audio:
Enhances context, ambiance, and emotional engagement.
Types include speech/voiceovers, music, background scores, and
sound effects (SFX).
Digitized audio formats: WAV, MP3, AAC, and MIDI control data.

Images & Graphics:
Visual representations used to convey information quickly and vividly.
Bitmap/Raster : Pixel-based images (JPEG, PNG, GIF) ideal for
complex continuous-tone photographs.
Vector Graphics: Mathematical descriptions of lines and shapes
(SVG) scalable without loss of resolution.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

9 / 533

Core Components of Multimedia: Part 2 I
Animation:
Illusion of motion created by consecutive static images displayed
rapidly (Persistence of Vision).
2D Animation: Flat vector or raster motions (e.g., keyframing, path
animation).
3D Animation: Modeling, rigging, texturing, rendering, and lighting in
three-dimensional space.

Video:
Photographic images recorded in sequence and played back at
standard rates (24, 30, or 60 frames per second).
High data bandwidth requirements necessitate spatial and temporal
compression (codecs like H.264, H.265/HEVC, AV1).

Interactivity & GUI:
User interface elements allowing direct control, navigation, and user
input feedback.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

10 / 533

Key Attributes and Classification I

Time-Based vs. Static Media:
Static (Time-Independent): Text and graphic elements do not change
over time.
Continuous (Time-Dependent): Audio, video, and animation rely on
strict temporal synchronization.

Digital Representation:
Analog signals converted to digital bits (0s and 1s) via sampling and
quantization.
Enables non-destructive editing, rapid duplication, and error-free
network transmission.

Quality Factors:
Resolution, bit depth, sampling frequency, frame rates, and bitrates
directly impact fidelity and file storage requirements.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

11 / 533

Multimedia Past & Present: Early History I

Pre-Digital Foundations:
Early combinations of media included illuminated manuscripts, slide
shows with synchronized audio tracks, and cinema.

1960s – Memex & Xanadu:
Vannevar Bush’s Memex concept (1945): Microfilm-based system for
linked retrieval.
Ted Nelson (1963): Coined terms Hypertext and Hypermedia;
initiated Project Xanadu.

1970s – 1980s: The Computer Revolution:
Alan Kay’s Dynabook concept at Xerox PARC leading to Graphical
User Interfaces (GUIs).
Launch of Apple Macintosh (1984) and HyperCard (1987)
democratizing personal multimedia authoring.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

12 / 533

Hypermedia and Hypertext Evolution I

Hypertext:
Text containing links to other passages of text, allowing
non-sequential reading paths.

Hypermedia Extension:
Broadens hypertext links to include images, audio streams, video clips,
and executable scripts.

Node & Link Architecture:
Nodes: Individual information units (pages, frames, or media clips).
Links: Associated pointers that connect nodes bidirectionally or
unidirectionally.

Challenges in Hypermedia:
Cognitive overload and the “Lost in Hyperspace” problem resolved via
breadcrumbs and sitemaps.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

13 / 533

The World Wide Web (WWW) and the Internet I

Tim Berners-Lee & CERN (1989–1991):
Invented the World Wide Web using HTML, HTTP, and URI
specification.
Connected global hypermedia documents across distributed TCP/IP
networks.

Browser Wars & Media Integration:
NCSA Mosaic (1993): First browser to display inline images along
with text.
Rise of plug-in architectures (Flash, QuickTime, RealPlayer) for rich
media playback in the late 1990s.

HTML5 Standard:
Native <video> and <audio> tags eliminated dependence on
proprietary third-party browser plug-ins.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

14 / 533

Multimedia in the New Millennium I

Broadband Infrastructure:
Transition from dial-up modems to high-speed fiber optics, 4G, 5G,
and satellite internet.
Enabled real-time high-definition (HD) and 4K video streaming.

Web 2.0 & User-Generated Content:
Platforms like YouTube, Vimeo, Twitch, and social networks
empowered users to create and publish media globally.

Mobile Multimedia:
Smartphones and tablets equipped with high-resolution displays,
multi-touch screens, and powerful mobile GPUs.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

15 / 533

Next-Generation Frontiers: AR, VR, and AI I

Immersive Technologies:
Virtual Reality (VR): Fully synthetic 3D environments replacing the
real world view.
Augmented Reality (AR): Digital graphics overlaid onto real-world
environments.
Extended Reality (XR): Convergence of physical and virtual
environments.

AI-Driven Multimedia:
Generative AI models for text-to-image, text-to-video, voice synthesis,
and dynamic content adaptation.
Real-time automated compression, upscaling, and noise reduction.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

16 / 533

Where to Use Multimedia? Applications I

Education & E-Learning:
Interactive courseware, virtual laboratories, computer-based training
(CBT), and simulation software.

Business & Industry:
Corporate presentations, video conferencing, product catalogs,
interactive kiosks, and digital marketing.

Entertainment & Gaming:
Video games, interactive storytelling, streaming video on demand
(VOD), and digital cinema.

Medicine & Healthcare:
3D medical imaging (MRI, CT scans), surgical simulations, and
remote telemedicine.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

17 / 533

Delivering Multimedia: Storage vs. Web Delivery I

Optical Storage Media (Physical Delivery):
Compact Discs (CD-ROM: ∼ 700 MB), Digital Versatile Discs
(DVD-ROM: 4.7 − 8.5 GB), Blu-ray Discs (25 − 50 GB).
High transfer rates without bandwidth constraints, ideal for
high-bitrate offline applications.

Online Network Delivery:
Web-based downloading vs. progressive downloading vs. adaptive
bitrate streaming (e.g., HLS, DASH).
Content Delivery Networks (CDNs) distribute media servers globally
to minimize latency and buffering.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

18 / 533

Technical Trade-offs and Best Practices in Delivery I

Bandwidth vs. Quality:
Balancing compression rates against loss of visual or acoustic fidelity.

Latency and Buffer Management:
Minimizing jitter, packet loss, and startup latency in live and
interactive media streams.

Cross-Platform Compatibility:
Utilizing open standard formats (HTML5, MP4, WebM, WebP) to
ensure seamless playback across diverse devices and operating systems.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

19 / 533

What is Multimedia? Definition & Core Concepts I
Etymological Roots:
Derived from Multi (numerous or multiple) and Media (medium,
channel of communication).
Represents the technical convergence of multiple sensory formats into
a unified computer system.

Formal Definition:
Any combination of text, art, sound, animation, and video delivered
by computer or other digital/electronic means.
A computer-controlled system that integrates information from
various media formats digitally.

Interactive vs. Passive Multimedia:
Linear (Passive): User observes without active control over content
sequence (e.g., film screening, traditional TV broadcast).
Non-linear (Interactive): User exercises navigation control,
influencing the sequence and output of content (e.g., video games,
interactive hypermedia).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

20 / 533

Fundamental Attributes of Multimedia Systems I

Computer-Controlled:
Multimedia content must be generated, stored, processed, and
presented via computational devices.

Integration:
Media components are combined into a single, cohesive interface
rather than stored independently without temporal or spatial
alignment.

Digital Representation:
Analog signals (sound, analog video) are sampled and quantized into
binary representations (0s and 1s) for digital manipulation and storage.

Interactivity:
Gives end-users control over what information is presented and when
it is displayed, fostering active engagement.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

21 / 533

Components of Multimedia: Categorization I

Classification by Time Dependence:
Discrete (Static / Time-Independent Media): Information where
space is the main dimension; temporal aspects do not alter validity
(e.g., text, static images, graphics).
Continuous (Dynamic / Time-Dependent Media): Information
dependent on continuous time flow; correct playback requires strict
temporal constraints (e.g., audio, video, animation).

Classification by Capture vs. Synthesis:
Captured Media: Derived from physical world phenomena (e.g.,
digital photographs, recorded microphone audio, digital video
recordings).
Synthesized Media: Computer-generated constructs created using
programmatic algorithms or design applications (e.g., vector graphics,
synthetic speech, 3D animations).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

22 / 533

Detailed Breakdown of Multimedia Elements I
Text:
Fundamental medium for precise information display, hypertext
navigation links, and system commands.

Audio:
Speech, musical scoring, and acoustic sound effects. Operates as
continuous data streams requiring consistent sampling rates (e.g.,
44.1 kHz for CD audio).

Images & Graphics:
Raster Graphics: Bitmap images composed of pixel grids (e.g.,
JPEG, PNG).
Vector Graphics: Mathematical shapes and paths scalable without
quality degradation (e.g., SVG).

Video & Animation:
Digital Video: Sequential frames of real-world captures stored with
synchronous audio tracks.
Animation: Sequential display of generated static frames creating the
optical illusion of motion.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

23 / 533

Early History of Multimedia: Pre-Digital Era I
Pre-Computer Foundations:
Traditional multi-sensory experiences: theatrical performances, opera,
illuminated manuscripts, sound motion pictures (1920s), and color
television (1950s).

Vannevar Bush and Memex (1945):
Theoretical desktop device concept for storing books, records, and
communications linked by associative trajectories.
Conceptual precursor to modern hypertext and web navigation
systems.

Early Digital Milestones:
Sketchpad (Ivan Sutherland, 1963): First interactive computer
graphics system utilizing a light pen interface.
Mouse and Graphical UI (Doug Engelbart, 1968): ”The Mother of
All Demos” introduced hypertext links, video conferencing, and mouse
navigation.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

24 / 533

Hypertext, Hypermedia, and Non-linear Content I

Coining Hypertext:
Ted Nelson (1965, Project Xanadu) coined hypertext as
non-sequential writing allowing user branching.

From Hypertext to Hypermedia:
Hypertext: Non-linear linked text documents enabling non-sequential
reading paths.
Hypermedia: Extension of hypertext incorporating audio, video,
graphics, and interactive elements within linked nodes.

Commercial Hypermedia Breakthroughs:
Apple HyperCard (1987): Popularized stacks of linked cards
containing text, graphics, and scriptable interactivity for personal
computers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

25 / 533

Multimedia and the World Wide Web (WWW) I

Invention of the Web (Tim Berners-Lee, 1989–1991):
Introduced HTTP, HTML, and URLs to link distributed hypermedia
nodes globally across the Internet.

Graphical Web Browsers:
NCSA Mosaic (1993): First browser to render inline graphics
alongside text, driving explosive internet growth.
Netscape Navigator (1994): Introduced client-side scripting
(JavaScript) and multi-format web media plugins.

Network Infrastructure Expansion:
Evolution from dial-up modems (56 kbps) to broadband networks
enabled richer media transfer across global networks.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

26 / 533

Multimedia in the New Millennium I

Streaming Media Era:
Shift from downloadable files to real-time progressive playback (e.g.,
MP3 compression, Flash video, YouTube, Netflix).

Ubiquitous Mobile Computing:
Smartphones and tablets equipped with high-resolution displays,
multi-touch interfaces, and dedicated GPU pipeline hardware.

Rich Internet Applications & HTML5:
Elimination of proprietary browser plugins in favor of native HTML5
<video>, <audio>, and WebGL APIs.

Immersive Technologies:
Virtual Reality (VR), Augmented Reality (AR), 360-degree video, and
real-time ray-traced graphics integration.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

27 / 533

Where to Use Multimedia? Education & Business I

Education & Interactive Learning:
Computer-Based Training (CBT) and Web-Based Training (WBT).
Virtual laboratories, educational simulations, and interactive e-learning
portals (MOOCs).

Business & Corporate Operations:
Interactive product catalogs, digital kiosks, and online commerce
storefronts.
Corporate video conferencing, remote collaboration tools, and
dynamic pitch presentations.

Public Access & Information Systems:
Information kiosks at airports, museums, shopping malls, and trade
exhibitions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

28 / 533

Where to Use Multimedia? Entertainment, Medicine
& Engineering I

Entertainment & Digital Games:
High-definition video games with immersive spatial audio, real-time
physics engines, and cinematic cutscenes.
Streaming services for movies, music, and interactive multi-branch
storytelling.

Medicine & Healthcare:
3D medical imaging visualization (CT, MRI scans), tele-surgery
applications, and surgical training simulators.

Engineering, Architecture & Scientific Research:
Computer-Aided Design (CAD), virtual architectural walkthroughs,
and complex spatial dataset visualizations.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

29 / 533

Delivering Multimedia: Optical Media vs. Online
Delivery I
Offline Delivery (Optical / Physical Storage Media):
Formats: CD-ROM (650–700 MB), DVD-ROM (4.7–8.5 GB), Blu-ray
(25–50 GB), flash storage drives.
Advantages: High data transfer rates without network constraints,
consistent playback quality.
Disadvantages: Static content updates require physical
redistribution; higher unit cost per update.

Online Delivery (Network / Cloud Delivery):
Formats: Web streaming (HTTP Live Streaming / DASH), cloud
storage downloads, web apps.
Advantages: Instantaneous updates, global accessibility, on-demand
playback flexibility.
Disadvantages: Subject to network latency, packet loss, bandwidth
bottlenecks, and connectivity dependencies.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

30 / 533

Technical Challenges in Multimedia Delivery I

Data Rate and Storage Requirements:
Uncompressed HD/4K video streams demand gigabits per second,
necessitating advanced compression standards (e.g., H.264,
H.265/HEVC, AV1).

Bandwidth and Network Throughput:
Fluctuating network capacity can cause buffering delays, dropped
frames, or degraded resolution.

Quality of Service (QoS) & Latency:
Real-time continuous media requires strict latency bounds and minimal
jitter (delay variation) for smooth audio/video synchronization.

Cross-Platform Synchronization & Rendering:
Ensuring consistent presentation across diverse hardware
configurations, operating systems, and display dimensions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

31 / 533

What is Multimedia? I

Etymology: Multimedia originates from two Latin roots: Multi
(many or multiple) and Media (distribution tools or information
channels).
Technical Definition: The computer-controlled integration of
dynamic and static media—including text, graphics, audio, animation,
and video—stored, processed, and presented digitally.
Key Characteristics:
Digital representation of data enabling algorithmic manipulation.
Interactivity allowing dynamic user control over content navigation.
Multimodal communication engaging multiple visual and auditory
senses.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

32 / 533

Core Components of Multimedia I
Text:
Primary element for explicit alphanumeric communication.
Includes static text, formatted body content, and navigable hypertext.

Audio:
Speech, music tracks, ambient soundscapes, and sound effects.
Encoded digitally (e.g., PCM, MP3, AAC) with specified sampling
rate and bit depth.

Graphics and Images:
Vector graphics: Mathematically defined geometric primitives
(scalable).
Raster images: Pixel arrays defined by color depth and spatial
resolution.

Animation:
Rapid sequential presentation of static images creating the illusion of
motion (2D and 3D).

Video:
Sequences of recorded real-world frame captures synchronized with
spatial audio channels.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

33 / 533

Classification of Multimedia Systems I

Linear Multimedia:
Non-interactive content structure following a predetermined timeline.
User acts as a passive consumer without navigational interventions.
Examples: Feature films, traditional television broadcasts,
non-interactive presentations.

Non-Linear (Interactive) Multimedia:
Content flow governed by user input, choices, and navigation paths.
Examples: Video games, hypermedia applications, simulation software.

Media Temporal Properties:
Discrete (Static) Media: Time-independent elements (Text, Still
Images).
Continuous (Dynamic) Media: Time-dependent elements (Audio,
Video, Animation).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

34 / 533

Past and Present: Early History of Multimedia I

Pre-Digital Foundations: Early combinations of mechanical
projection devices, sound synchronized with celluloid films, and
analog broadcasting.
Memex Concept (1945):
Proposed by Vannevar Bush in As We May Think.
Conceptual electromechanical desk storing microfilmed literature
connected by cross-referencing associative trails.

Project Xanadu (1960s):
Conceived by Ted Nelson to create a global, deep-hyperlinked
electronic publishing network.

Early Digital Integration (1970s–1980s):
Transition from analog signals to digital encoding standards for audio
and raster displays.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

35 / 533

Evolution of Hypertext and Hypermedia I

Hypertext:
Term coined by Ted Nelson in 1963 for non-sequential, linked textual
information structures.
Replaces strict linear reading paths with associative node-link
topologies.

Hypermedia:
Extension of hypertext principles to encompass graphics, audio, video,
and executable nodes.
Links allow seamless traversal across distinct media modalities.

Commercial Milestones:
Apple HyperCard (1987): Popularized card-and-stack hypermedia
authoring on personal computers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

36 / 533

The World Wide Web and Internet Integration I

Invention of the WWW (1989–1991):
Tim Berners-Lee authored HTTP, HTML, and URL protocols at
CERN.
Transformed local desktop hypermedia into a globally distributed
architecture.

NCSA Mosaic (1993):
First widely deployed graphical browser rendering inline images
alongside HTML text.

Network Infrastructure Expansion:
Shift from low-speed analog dial-up modems to broadband
connections enabled rich online media transmission.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

37 / 533

Multimedia in the New Millennium I

Broadband and Streaming: High-speed connectivity facilitated
real-time audio and video delivery without full prior download.
Advanced Compression Standards:
Audio: MP3, AAC, FLAC.
Video: H.264/AVC, H.265/HEVC, AV1, enabling high-definition
playback under bandwidth constraints.

Mobile Multimedia Era:
Smartphones and tablets equipped with high-density displays,
hardware decoders, and mobile GPUs.

User-Generated Content (UGC):
Transition from centralized studio broadcasting to decentralized
video-sharing and streaming networks.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

38 / 533

Advanced Frontiers: VR, AR, and AI Multimedia I

Immersive Environments:
Virtual Reality (VR): Fully synthetic 3D spatial environments.
Augmented Reality (AR): Overlayering digital media onto the
real-world view.
Mixed Reality (MR): Real-time interaction between real and virtual
elements.

Spatial Audio: Object-based surround sound positioning matching
user head movements.
AI-Driven Multimedia:
Real-time neural upscaling, automated video synthesis, and generative
multimodal models.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

39 / 533

Where to Use Multimedia? Applications I

Education and E-Learning:
Interactive simulations, virtual dissection labs, and intelligent tutoring
systems.

Business and Enterprise:
High-definition video conferencing, interactive product catalogs, and
virtual trade shows.

Entertainment and Gaming:
Interactive 3D games, virtual cinema, and digital special effects
(VFX).

Medicine and Scientific Research:
3D anatomical visualization (CT, MRI scans), surgical training
simulators, and telemetry.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

40 / 533

Delivering Multimedia: Offline vs. Online Methods I

Offline Delivery (Physical Storage Media):
CD-ROM, DVD, Blu-ray, and flash storage drives.
High data throughput independent of network access, but static
content and physical distribution costs.

Online Delivery (Network Transmission):
Progressive Downloading vs. Adaptive Bitrate Streaming (HLS,
MPEG-DASH).
Requires Quality of Service (QoS) considerations: latency control,
jitter reduction, and buffer management.

Hybrid Delivery:
Local assets combined with dynamic cloud updates for optimized
performance.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

41 / 533

Summary and Key Takeaways I

Multimedia combines five basic media building blocks into integrated,
interactive computer-based systems.
The field evolved historical milestones from Bush’s Memex and
Nelson’s Hypertext concepts to the WWW and modern broadband
streaming.
Delivery paradigms shifted from physical optical media to dynamic,
adaptive streaming over high-speed networks.
Emerging paradigms in spatial computing, immersive VR/AR, and
generative AI represent the current frontier of multimedia technology.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

42 / 533

Overview of Multimedia Software Tools I

Role in Production Pipeline: Software tools enable creation,
editing, composition, and distribution of digital media assets.
Taxonomy of Multimedia Software:
Sound and Audio Processing (Music Sequencing, Digital Audio
Editors)
Visual Asset Creation (2D Graphics, 3D Animation, Video Editing)
Integration and Delivery (Authoring Software, Broadcasting Protocols,
AR/VR Engines)

Selection Criteria: System requirements, supported file formats,
non-destructive editing capability, workflow interoperability, and cost
structure.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

43 / 533

Music Sequencing and Notation Software I

MIDI (Musical Instrument Digital Interface): Protocol for
transmitting performance data (note-on, note-off, velocity, pitch
bend) rather than sampled audio waves.
Digital Audio Workstations (DAWs):
Combine MIDI sequencing with multi-track audio recording.
Popular Tools: Ableton Live, Logic Pro, Steinberg Cubase, FL Studio.
Key Features: VST/AU plugin support, virtual instrument synthesis,
MIDI automation.

Score Notation Software:
Focuses on standard musical notation layout for composition and
printing.
Popular Tools: Avid Sibelius, MakeMusic Finale, MuseScore.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

44 / 533

Digital Audio Editing Tools I

Waveform vs. Multi-track Editing:
Destructive Editors: Directly alter sample data in file storage.
Non-destructive Editors: Real-time preview layers without altering
original master audio.

Core Capabilities: Spectral editing, noise reduction, equalization
(EQ), dynamic compression, mastering.
Industry Standard Tools:
Audacity : Open-source, multi-platform waveform editor.
Adobe Audition: Advanced multi-track mixing and spectral frequency
analysis.
Avid Pro Tools: Industry standard for professional recording studio
recording and mixing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

45 / 533

Graphics and Image Editing Tools I

Raster vs. Vector Paradigm:
Raster (Bitmap): Pixel grids, resolution-dependent; ideal for complex
natural photographic imagery.
Vector : Mathematical paths (Bézier curves), resolution-independent;
ideal for logos, typography, line art.

Raster Editing Software:
Adobe Photoshop: Layer-based editing, color correction, masking,
compositing.
GIMP: Open-source alternative providing comprehensive image
manipulation.

Vector Graphics Software:
Adobe Illustrator : Scalable vector graphics design, SVG export,
typography control.
Inkscape: Open-source vector design software standard.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

46 / 533

Digital Video Editing Tools I

Non-Linear Editing (NLE): Allows random access to video frames
on a timeline without physical cutting.
Key Editing Operations: Cutting, trimming, color grading,
multi-camera sync, transition effects, audio ducking.
Leading NLE Platforms:
Adobe Premiere Pro: Deep integration with Creative Cloud ecosystem.
DaVinci Resolve: World-class color grading, Fairlight audio, and
Fusion visual effects.
Final Cut Pro: Optimized performance for macOS architecture.

Video Rendering and Codecs: Exporting timeline sequences into
distribution formats (H.264, HEVC/H.265, ProRes).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

47 / 533

Digital Animation Software I

2D Animation Software:
Vector/raster frame-by-frame animation and motion graphics.
Adobe After Effects: Compositing, keyframe motion graphics, particle
effects.
Toon Boom Harmony : Industry standard for professional 2D cartoon
production.

3D Animation Software:
Pipeline: Modeling → Texturing → Rigging → Animation → Lighting
→ Rendering.
Autodesk Maya: High-end 3D character animation and visual effects.
Blender : Open-source full pipeline 3D suite (Cycles/Eevee rendering).
Cinema 4D: Preferred engine for broadcast motion graphics.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

48 / 533

Multimedia Authoring Tools I

Definition: Software that binds discrete media assets (text, graphics,
audio, video, interaction) into an integrated interactive application.
Authoring Paradigms:
Card or Page-based: Organizes assets like pages in a book (e.g.,
ToolBook, HyperCard).
Icon or Flowchart-based: Visual programming connecting event nodes
and logic branching (e.g., Authorware).
Time-based: Syncs media along an explicit temporal timeline (e.g.,
Adobe Animate, Director).

Modern Equivalents: Web-based interactive engines, HTML5
authoring suites, dynamic eLearning platforms.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

49 / 533

Multimedia Broadcasting Tools and Protocols I

Streaming Infrastructure: Delivery of live or on-demand multimedia
over packet-switched networks.
Broadcasting Software:
OBS Studio (Open Broadcaster Software): Open-source scene
compositing and live video streaming.
vMix / Wirecast: Professional multi-bitrate live switching tools.

Network Protocols:
RTMP (Real-Time Messaging Protocol): Traditional ingest protocol.
HLS (HTTP Live Streaming) / DASH: Adaptive bitrate distribution
protocols.
WebRTC : Ultra-low latency real-time communication.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

50 / 533

Augmented Reality (AR) and Virtual Reality (VR)
Tools I

Immersive Paradigms:
Virtual Reality (VR): Fully synthetic digital environment replacing
physical reality.
Augmented Reality (AR): Overlaying digital assets onto real-world
camera feeds.

Development Engines:
Unity : C# scriptable engine dominant in cross-platform mobile
AR/VR.
Unreal Engine: C++ / Blueprints engine offering photorealistic
real-time rendering.

AR/VR SDKs and Frameworks: Apple ARKit, Google ARCore,
Meta Movement SDK, WebXR standard.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

51 / 533

Multimedia Production Team: Core Roles (Part 1) I

Project Manager: Oversees budget, schedule, resource allocation,
client communication, and overall quality assurance.
Content Specialist: Researches, gathers, and verifies domain-specific
subject matter information.
Instructional Designer / Writer: Crafts structural narrative,
scripts, storyboards, and pedagogical flow.
Art Director / Visual Designer: Establishes aesthetic themes, color
palettes, visual identity, and graphic layout.
Interface / UX Designer: Ensures intuitive navigation pathways,
accessibility, and user-centered interaction design.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

52 / 533

Multimedia Production Team: Technical Roles (Part
2) I

Audio / Video Engineer: Handles sound recording, editing, mixing,
video capture, compression, and codec optimization.
Multimedia Programmer / Developer: Writes core code,
integrates API services, implements interactive logic and physics.
3D Modeler / Animator: Builds 3D polygonal meshes, textures,
rigs, and animates assets.
Quality Assurance (QA) Tester: Conducts cross-platform testing,
stress testing, usability audits, and bug tracking.
Webmaster / Deployment Engineer: Manages server hosting,
CDN distribution, network security, and live streaming setups.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

53 / 533

Summary and Integrated Multimedia Workflow I

Interoperability: Modern multimedia relies on seamless cross-tool
asset pipelines (e.g., SVG vector graphics to 3D meshes to real-time
engines).
Compression vs. Quality Balance: Selecting optimal codecs and
bitrates across audio, video, and 3D media.
Collaborative Synergy: Successful multimedia production requires
harmonious synchronization between creative design, technical
engineering, and project management.
Future Trends: AI-assisted asset generation, spatial computing,
cloud-based collaborative editing, and ultra-low latency interactive
broadcasting.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

54 / 533

Music Sequencing and Notation Software I
MIDI and Music Sequencing Concepts:
Musical Instrument Digital Interface (MIDI) records control data (note
pitch, velocity, duration, channel) rather than raw analog waveforms.
Sequencers enable multi-track recording, editing, quantized timing
correction, and playback of MIDI data streams.

Digital Audio Workstations (DAWs):
Modern DAWs integrate MIDI sequencing, virtual instruments
(VST/AU plugins), and multitrack audio routing.
Key Industry Tools: Avid Pro Tools, Apple Logic Pro, Steinberg
Cubase, Ableton Live, and FL Studio.

Score Notation Tools:
Convert MIDI performance data into standard printed musical score
layout and sheet music.
Prominent Software: MakeMusic Finale, Avid Sibelius, MuseScore,
and Steinberg Dorico.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

55 / 533

Digital Audio Editing Software I

Core Capabilities:
Destructive vs. Non-destructive audio editing processing pipelines.
Multitrack recording, spatial mixing, equalization, dynamic range
compression, and audio mastering.
Sample rate conversion, bit-depth dithering, and audio format
encoding (WAV, FLAC, MP3, AAC).

Spectral and Restoration Editing:
Visualizing audio frequencies over time via 2D spectrogram analysis.
Removing background noise, clicks, power line hums, and sibilance
using spectral selection tools.

Key Software Tools:
Open-Source / Free: Audacity, Ardour.
Commercial / Professional: Adobe Audition, Sound Forge, WaveLab,
Reaper, iZotope RX.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

56 / 533

Graphics and Image Editing Tools I

Raster vs. Vector Graphics Paradigm:
Raster (Bitmap): Pixel-grid based, resolution dependent; ideal for
continuous-tone photographic imagery.
Vector : Mathematically defined geometric primitives (lines, curves,
polygons); resolution independent.

Raster Image Editors:
Features: Layer-based composition, masking, color channel
manipulation, spatial filtering, and non-destructive adjustment layers.
Software: Adobe Photoshop, GIMP, Affinity Photo, Corel Photo-Paint.

Vector Graphics Editors:
Features: Bezier curve path modeling, scalable typography rendering,
SVG export, pathfinder boolean operations.
Software: Adobe Illustrator, Inkscape, Affinity Designer, CorelDRAW.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

57 / 533

Digital Video Editing Software I

Non-Linear Editing (NLE) Architecture:
Random access to video frames on disk without altering underlying
raw source media files.
Multi-track video and audio timeline management with real-time
playback preview engines.

Essential NLE Operations:
Assembly, ripple, roll, slip, and slide trimming techniques.
Color grading (LUTs, vector scopes, primary/secondary wheels),
compositing, and audio synchronization.
Keyframing spatial transformations, speed ramps, and dynamic
transition effects.

Industry Standard NLE Suites:
Adobe Premiere Pro, Apple Final Cut Pro, Blackmagic DaVinci
Resolve, Avid Media Composer.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

58 / 533

2D and 3D Animation Tools I
2D Vector and Motion Graphics Software:
Frame-by-frame traditional animation vs. interpolated keyframe
tweening and vector morphing.
Software: Adobe Animate, Toon Boom Harmony, Adobe After
Effects, Synfig Studio.

3D Computer Graphics Pipeline Tools:
Stages: Polygonal Modeling, UV Unwrapping, Texturing, Skeletal
Rigging, Animation, Lighting, Rendering.
Software Suites: Autodesk Maya, Autodesk 3ds Max, Blender, Cinema
4D, Houdini.

Render Engines and Physics Simulations:
Ray-tracing and path-tracing engines (Arnold, Cycles, Redshift,
V-Ray).
Particle systems, rigid body dynamics, cloth, and fluid physical
simulations.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

59 / 533

Multimedia Authoring Systems I

Authoring Metaphors:
Card / Page Metaphor : Content organized as sequential screens or
interactive cards (e.g., ToolBook, HyperCard).
Icon / Flowchart Metaphor : Logic flow built with visual
drag-and-drop structural nodes (e.g., Adobe Authorware).
Time / Stage Metaphor : Content driven by a global timeline with
user-controlled actors and scripts (e.g., Adobe Director, Adobe
Animate).

E-Learning and Interactive Authoring Suites:
SCORM and xAPI compliant tools for packaging interactive
educational modules and quizzes.
Leading Applications: Articulate Storyline, Adobe Captivate, Lectora.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

60 / 533

Multimedia Broadcasting and Live Streaming I

Streaming Architecture and Encoding:
Real-time hardware and software video compression using H.264,
H.265 (HEVC), and AV1 codecs.
Protocol encapsulation: RTMP (Real-Time Messaging Protocol),
SRT (Secure Reliable Transport), HLS, and DASH.

Live Production Software:
Scene switching, multi-camera capture ingestion, chroma keying, and
lower-third graphic overlays.
Industry Platforms: OBS Studio (Open Broadcaster Software), vMix,
Wirecast, Streamlabs.

IP Video Routing Standards:
NDI (Network Device Interface) and SMPTE ST 2110 for low-latency
network audio/video transmission.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

61 / 533

Augmented Reality (AR) and Virtual Reality (VR)
Tools I
Immersive Technology Paradigms:
Virtual Reality (VR): Fully synthetic, isolated digital environment
immersion with head-tracked displays.
Augmented Reality (AR): Overlaying digital assets onto real-world
camera feeds with 6-DOF spatial tracking.

AR/VR Engine Platforms:
Unity (C# scripting, Universal Render Pipeline, XR Interaction
Toolkit).
Unreal Engine (C++ and Blueprints visual scripting, Nanite geometry,
Lumen lighting).

SDKs and Spatial Asset Pipeline:
AR Frameworks: Apple ARKit, Google ARCore, PTC Vuforia,
Qualcomm Snapdragon Spaces.
3D Spatial Audio processing, head-related transfer functions (HRTF),
and haptic feedback APIs.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

62 / 533

Multimedia Team Roles: Management and Content
Creation I

Project Manager:
Defines project scope, timeline schedules, budget constraints, resource
allocation, and milestone delivery.
Coordinates communication across cross-functional creative and
technical team members.

Subject Matter Expert (SME) and Content Specialist:
Ensures domain accuracy, provides core educational/business material,
and validates learning outcomes.

Instructional Designer and Writer:
Scripts narrative content, develops storyboards, designs learning
architectures, and crafts interactive prompts.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

63 / 533

Multimedia Team Roles: Creative, Technical, and
Quality I

Art Director and Graphic Designer:
Establishes visual style guides, typography, UI/UX aesthetics, layout
templates, and vector/raster assets.

Audio/Video Specialist and 3D Animator:
Captures high-fidelity media, performs noise cleanup, edits video
sequences, and builds 3D character models/animations.

Multimedia Programmer and QA Specialist:
Integrates assets into application code or game engine logic; enforces
cross-platform compatibility and usability testing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

64 / 533

Team Collaboration Pipelines and Skill Integration I

Asset Pipeline and Version Control:
Managing large binary files using Git LFS (Large File Storage),
Perforce Helix Core, or Plastic SCM.
Standardized directory structures, file naming conventions, and asset
metadata tracking.

Interoperability Standards:
Open interchange formats: Universal Scene Description (USD), glTF,
FBX, OpenEXR.

Core Competencies for Multimedia Professionals:
Technical versatility, creative problem-solving, cross-disciplinary
communication, and adaptability to evolving toolchains.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

65 / 533

Overview of Multimedia Software Tools I
Definition and Scope: Multimedia software tools encompass
applications used to create, edit, process, assemble, and deliver
diverse media elements (text, graphics, audio, video, and interactive
components).
Classification of Multimedia Tools:
Content Creation Tools: Software dedicated to generating individual
raw assets (e.g., digital audio workstations, vector graphics editors).
Editing and Post-Production: Applications for refining raw assets,
color grading, multi-track audio mixing, and visual compositing.
Authoring Systems: Frameworks for integrating diverse media into
cohesive, interactive software applications.
Distribution and Broadcasting: Systems for transmitting live or
pre-recorded media streams across network protocols.

Interoperability and Standards: Modern workflows rely on
standardized file formats (e.g., MIDI, SVG, WAV, MP4, USDZ) to
ensure seamless cross-platform asset transfer.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

66 / 533

Music Sequencing and Notation Software I
Digital Audio Workstations (DAWs) and Sequencers:
Software designed to record, edit, and play back digital audio streams
and MIDI (Musical Instrument Digital Interface) control data.
Key functionalities: Multi-track audio recording, MIDI event
sequencing, virtual instrument hosting (VST, AU plug-ins), and
dynamic automation curves.
Industry Examples: Logic Pro, Ableton Live, Steinberg Cubase, FL
Studio, Pro Tools.

Music Notation Software:
Tools tailored for score writing, sheet music typesetting, and academic
music publishing.
Core feature set: Standard staff notation rendering, lyric alignment,
sound-font playback engines, and MusicXML export/import
capabilities.
Industry Examples: Finale, Sibelius, Dorico, MuseScore.

MIDI vs. Audio Sequencing: MIDI stores compact performance
control event data (note-on, note-off, velocity, pitch bend), whereas
audio sequencing processes continuous sampled waveform data
streams.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

67 / 533

Digital Audio Editing and Processing Tools I
Audio Waveform Editors:
Specialized software for precision editing of single-track or multi-track
audio signals.
Key Operations: Trimming, noise reduction (spectral editing),
normalization, equalization (EQ), dynamic range compression, and
time-stretching.
Industry Examples: Adobe Audition, Sound Forge, Audacity,
WaveLab.

Digital Signal Processing (DSP) Plug-ins:
Sound transformation algorithms providing reverberation, delay,
spatialization, and psychoacoustic enhancements.
Real-time processing architectures: VST3, Audio Units (AU), and
AAX standards.

Spatial Audio and Multichannel Processing: Modern audio
software incorporates binaural encoding, Dolby Atmos object-based
mixing, and 3D ambisonic panning for immersive soundscapes.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

68 / 533

Graphics and Image Editing Software I
Raster (Bitmap) Image Editing:
Pixel-based image manipulation focusing on photo retouching, digital
painting, and color correction.
Key concepts: Layer management, alpha channels, masking, blend
modes, non-destructive adjustment layers, and color spaces (RGB,
CMYK, LAB).
Industry Examples: Adobe Photoshop, GIMP, Affinity Photo, Corel
PHOTO-PAINT.

Vector Graphics Editing:
Mathematical representation of visual primitives (Bézier curves, lines,
nodes, geometric shapes) providing complete resolution independence.
Applications: Logo design, typography, UI/UX asset creation,
technical diagramming, and SVG generation.
Industry Examples: Adobe Illustrator, Inkscape, CorelDRAW, Figma.

Hybrid Workflows: Integration of vector paths within raster editors
for precise selection masking and resolution-independent vector shape
layers.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

69 / 533

Digital Video Editing and Post-Production Tools I
Non-Linear Editing (NLE) Systems:
Non-destructive timeline-based assembly of video clips, audio tracks,
and graphic overlays.
Core Features: Multi-camera editing, ripple/roll edits, timeline
scrubbing, proxy workflow generation, and timecode synchronization.
Industry Examples: Adobe Premiere Pro, Apple Final Cut Pro,
DaVinci Resolve, Avid Media Composer.

Visual Effects (VFX) and Compositing:
Layer-based and node-based software for combining disparate visual
elements, motion tracking, keying (chroma keying), rotoscoping, and
particle generation.
Industry Examples: Adobe After Effects (layer-based), Nuke
(node-based), Blackmagic Fusion.

Color Grading and Finishing: High-dynamic-range (HDR) color
mapping, LUT (Look-Up Table) integration, and integrated
DAW-level audio finishing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

70 / 533

2D and 3D Animation Tools I
2D Animation Software:
Frame-by-frame traditional animation and vector bone-rigging
keyframe animation suites.
Features: Onion skinning, inverse kinematics (IK), sprite sheet export,
and vector morphing.
Industry Examples: Adobe Animate, Toon Boom Harmony, Moho,
Synfig Studio.

3D Modeling and Animation Packages:
Comprehensive environments for 3D polygon modeling, texturing,
skeletal rigging, keyframing, dynamics (particle and fluid physics), and
rendering.
Rendering Engines: Cycles, Arnold, Redshift, and V-Ray utilizing ray
tracing and path tracing pipelines.
Industry Examples: Autodesk Maya, Blender, Autodesk 3ds Max,
Cinema 4D.

Motion Capture (MoCap) Integration: Retargeting real-world
movement dataset tracking onto 3D character skeletons for realistic
motion output.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

71 / 533

Multimedia Authoring Paradigms and Tools I

Concept of Multimedia Authoring: Software environments that
assemble text, audio, video, graphics, and interactivity into an
integrated executable application.
Authoring Paradigms:
Card/Page-Based: Metaphor of discrete pages containing interactive
elements (e.g., HyperCard, ToolBook).
Icon/Flowchart-Based: Logic structures represented visually through
connected structural icons (e.g., Adobe Authorware).
Time-Based: Media elements arranged along a master timeline with
frame scripts (e.g., Adobe Director, Adobe Animate).
Object-Oriented/Scripting: Event-driven frameworks using
programming languages (e.g., Unity, HTML5/JS engines).

Key Capabilities: Asset management, inter-activity scripting,
cross-platform compilation, and runtime engine distribution.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

72 / 533

Multimedia Broadcasting and Streaming Systems I

Broadcasting Software Infrastructure:
Live production software for real-time video switching, audio mixing,
lower-thirds overlays, and virtual set chroma keying.
Industry Examples: OBS Studio, vMix, Wirecast, NewTek TriCaster.

Streaming Protocols and Architectures:
Real-Time Messaging Protocol (RTMP) and WebRTC for low-latency
ingest streams.
HTTP Live Streaming (HLS) and Dynamic Adaptive Streaming over
HTTP (DASH) for adaptive bitrate client delivery.

Codec Efficiency and CDN Integration: Encoders (H.264,
H.265/HEVC, AV1) compress video payloads prior to distribution
across Content Delivery Networks (CDNs).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

73 / 533

Augmented Reality (AR) and Virtual Reality (VR) I

Virtual Reality (VR) Development:
Fully immersive, computer-generated environments requiring
stereoscopic rendering and 6DoF (Six Degrees of Freedom) tracking.
Primary Engines: Unity, Unreal Engine, Godot Engine.

Augmented Reality (AR) Frameworks:
Superimposing digital assets onto physical real-world environments via
spatial mapping, plane detection, and optical marker recognition.
SDKs and Tools: ARKit (iOS), ARCore (Android), Vuforia, Spark AR,
Reality Composer.

Mixed Reality (MR) and Spatial Computing: Merging real and
virtual worlds where physical and digital objects co-exist and interact
dynamically in real time.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

74 / 533

Multimedia Production Team Roles I
Project Manager: Oversees development schedules, project scope,
budget management, resource allocation, and stakeholder
communication.
Multimedia Architect / UX Designer: Defines overall user
experience (UX), information architecture, interactive navigation, and
interface wireframes.
Audio / Video Specialists: Responsible for sound engineering,
acoustic scoring, video acquisition, lighting setup, color grading, and
edit cuts.
Graphics Designers and Animators: Create visual assets, 2D/3D
models, textures, interface icons, visual effects, and character
animations.
Multimedia Programmer / Developer: Writes core application
logic scripts, integrates API calls, manages event handlers, and builds
engine functionality.
Quality Assurance (QA) Tester: Conducts cross-platform usability,
performance, accessibility, and functional defect testing.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

75 / 533

Multimedia Skills Required in a Team I

Technical Competencies:
Mastery of domain-specific media creation suites (e.g., Adobe
Creative Cloud, Unity, Digital Audio Workstations).
In-depth knowledge of file compression formats, color management
profiles, and network bandwidth constraints.
Interactive scripting languages (JavaScript, C#, Python) and
database integration techniques.

Creative and Conceptual Capabilities:
Storyboarding, narrative design, visual aesthetics, typography
composition, and acoustic design principles.

Soft Skills and Team Collaboration:
Version control (Git) workflow adaptation, cross-disciplinary
communication, agile project iteration, and collaborative
problem-solving.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

76 / 533

Introduction to Digital Image Representation & Bit
Depth I
Digital Image Concept: A digital image is a two-dimensional grid of
discrete picture elements called pixels.
Bit Depth (bpp): Defines the number of bits used to represent the
color or intensity of each pixel.
Quantization Levels: A bit depth of n bits per pixel allows 2n
distinct intensity or color levels:
L = 2n
Dynamic Range: Higher bit depths provide higher dynamic range,
smoother tonal gradients, and reduced contouring artifacts.
Memory Implications: Bit depth directly determines the memory
required to store uncompressed image frames.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

77 / 533

1-Bit Images (Monochrome / Binary Images) I

Definition: Each pixel is represented by a single bit (n = 1).
Intensity Levels: Exactly 21 = 2 states (0 = Black, 1 = White).
Applications:
Text document scanning and Optical Character Recognition (OCR).
Line art, architectural schematics, and binary masks.
Halftoning previews and low-power monochrome displays.

Memory Storage: Extremely compact; 1 byte stores 8 individual
pixels.
Limitation: Cannot represent continuous shades of gray or subtle
continuous tones without dithering.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

78 / 533

8-Bit Gray-Level Images & Intensity Mapping I

Bit Depth: Each pixel is assigned 1 byte (8 bits).
Shades of Gray: 28 = 256 discrete intensity levels, typically ranging
from 0 (pure black) to 255 (pure white).
Monochrome Luminance: Represents brightness without
chromaticity (color information).
Perceptual Adequacy: 256 gray levels match human visual
perception under standard viewing conditions, preventing visible
step-like banding.
Common Use Cases:
Medical imaging (X-rays, CT scans).
Grayscale photography and computer vision feature extraction.
Alpha channels / transparency masks in graphic editing software.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

79 / 533

Image Data Types & Computer Memory Formats I

Unsigned 8-Bit Integer (uint8):
Range: [0, 255]. Standard format for mainstream display monitors and
web images.

Unsigned 16-Bit Integer (uint16):
Range: [0, 65535]. Used in high-precision scientific imaging and RAW
photography.

32-Bit Floating Point (float32):
Range: Typically normalized to [0.0, 1.0] or unbounded in High
Dynamic Range (HDR) workflows.
Provides high numerical precision for complex image processing
algorithms.

Signed Integer Types (int8, int16): Used in difference images
and motion estimation vectors.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

80 / 533

24-Bit Color Images (Truecolor / RGB Model) I

Architecture: Composed of three separate color channels: Red,
Green, and Blue (RGB).
Channel Depth: 8 bits per channel (3 × 8 = 24 bits per pixel).
Color Variations: Each channel yields 256 intensity levels, producing:
256 × 256 × 256 = 224 = 16, 777, 216 distinct colors
Human Perception: Often termed Truecolor because 16.7 million
colors exceed the human eye’s ability to distinguish subtle color
variations.
32-Bit Truecolor Extension: Stored as 32 bits per pixel (8-bit Red,
Green, Blue, plus an 8-bit Alpha channel for transparency).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

81 / 533

Memory Calculation & Pixel Grid Storage for 24-Bit
Color I
Storage Formula: For an image of resolution W × H pixels at b bits
per pixel:
W ×H ×b
Size (bytes) =
8
Example Calculation: Full HD Image (1920 × 1080) in 24-bit
Truecolor:
Pixels = 1920 × 1080 = 2, 073, 600 pixels
Size = 2, 073, 600 × 3 bytes = 6, 220, 800 bytes ≈ 6.22 MB
Data Layout:
Interleaved: RGB RGB RGB... stored sequentially in memory.
Planar : RRR... GGG... BBB... stored in distinct channel blocks.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

82 / 533

Higher Bit-Depth Images (10-bit, 12-bit, and 16-bit
Channels) I

Beyond Truecolor: Professional imaging workflows utilize bit depths
> 8 bits per channel to maintain fidelity.
30-Bit Color (10-bit per channel):
210 = 1, 024 levels per channel → Over 1.07 billion colors.
Prevents visible color banding in wide color gamuts (e.g., DCI-P3,
Rec. 2020).

36-Bit Color (12-bit per channel):
212 = 4, 096 levels per channel → 68.7 billion colors. Standard in
cinema capture.

48-Bit Color (16-bit per channel):
216 = 65, 536 levels per channel. Standard for digital photography
RAW editing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

83 / 533

High Dynamic Range (HDR) & Professional
Applications I

High Dynamic Range (HDR): Captures shadow and highlight
details simultaneously without clipping.
Precision Requirements:
Requires 10-bit to 16-bit per channel (or 32-bit floating point) to
prevent quantization artifacts during aggressive color grading.

Applications:
Digital cinematography and visual effects compositing.
Remote sensing and satellite multispectral imaging.
Astronomical imaging and industrial vision inspection.

Editing Latitude: High bit-depth allows severe contrast stretching
without revealing discrete quantization steps.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

84 / 533

8-Bit Color Images (Indexed Color Representation) I

Concept: Uses 8 bits per pixel, but each pixel value does NOT
directly represent grayscale intensity or individual RGB values.
Indexed Color Mechanics:
The 8-bit pixel value acts as an index (0 to 255) into a lookup table.
A maximum of 256 distinct colors can be displayed in a single image
frame.

Historical Context & Efficiency: Developed in early computer
graphics to conserve video memory while retaining colorful displays.
Common Formats: GIF (Graphics Interchange Format) and indexed
PNG.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

85 / 533

Color Look-Up Tables (CLUT) & Dithering
Techniques I

Color Look-Up Table (CLUT):
A palette array of size 256 × 3, storing full 24-bit RGB values for each
index entry.
The palette is customized for the specific image using color
quantization algorithms.

Color Quantization: Reduces a 24-bit image to an optimal 256-color
palette (e.g., Median Cut algorithm).
Dithering:
Applies spatial patterns of available palette colors to simulate missing
intermediate shades.
Floyd-Steinberg error diffusion dithering minimizes perceptual color
distortion.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

86 / 533

Comprehensive Comparison of Image Data Types I

Summary Table of Image Data Types:
Image Type
1-Bit Binary
8-Bit Grayscale
8-Bit Indexed
24-Bit Truecolor
48-Bit Deep Color

Bit Depth
1 bpp
8 bpp
8 bpp
24 bpp
48 bpp

Colors / Levels
2 levels
256 grays
256 (from 16.7M)
16,777,216
2.81 × 1014

Primary Use Case
Text, Line Art, Masks
Medical, Vision, Alpha
Web Graphics, GIF
Standard Photography
RAW Editing, Archival

Trade-off Summary: Higher bit depth increases tonal fidelity and
editing latitude at the cost of higher storage, memory bandwidth, and
processing overhead.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

87 / 533

Introduction to Graphics and Image Data Types I

Digital image representation relies on discrete spatial coordinates and
quantization levels.
A digital image is mathematically modeled as a 2D matrix F (x, y ),
where x and y denote spatial coordinates.
Bit depth (color depth) determines the number of distinct colors or
gray levels available per pixel.
Calculated as N = 2b , where b is the number of bits per pixel (bpp)
and N is the total available intensity values.
Selection of image data types affects memory consumption, dynamic
range, computational complexity, and visual quality.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

88 / 533

1-Bit Images: Binary and Monochromatic
Representation I

1-bit images store binary information where each pixel is represented
by a single bit (0 or 1).
Binary mapping: 0 typically represents black (off) and 1 represents
white (on), or vice-versa.
Memory requirements are minimal: 1 byte = 8 pixels.
Common applications include document scanning, line art, OCR
(Optical Character Recognition), and fax transmission.
Efficient raster formats include 1-bit TIFF, CCITT Group 4 fax
compression, and PBM (Portable BitMap).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

89 / 533

8-Bit Gray-Level Images I

8-bit gray-level images allocate 1 byte (8 bits) per pixel.
Provides 28 = 256 distinct shades of gray, ranging from 0 (pure
black) to 255 (pure white).
Human visual system (HVS) can distinguish roughly 30–100 shades of
gray simultaneously; 256 levels prevent contouring artifacts under
normal viewing.
Monochromatic intensity function I (x, y ) ∈ [0, 255].
Widely used in medical imaging (X-rays, CT scans preliminary
display), grayscale computer vision, and pattern recognition.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

90 / 533

Overview of Numerical Image Data Types I

Unsigned Integers (uint8, uint16): Range [0, 2b − 1]; standard for
direct pixel storage and rendering engines.
Signed Integers (int8, int16): Range [−2b−1 , 2b−1 − 1]; useful for
difference images, motion estimation, and gradient filtering.
Floating-Point (float32, double64): Normalized range [0.0, 1.0] or
arbitrary physical units; essential for high dynamic range (HDR)
calculations and linear color spaces.
Data type conversion requires scaling and clamping to prevent integer
overflow and underflow artifacts.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

91 / 533

24-Bit Color Images: Truecolor Systems I

24-bit color images use an additive RGB color model with 8 bits
assigned to Red, Green, and Blue channels.
Yields 224 ≈ 16.77 million potential unique color combinations per
pixel.
Spatial pixel structure consists of 3 distinct contiguous or planar bytes
per coordinate: (R, G , B).
Truecolor capability exceeds the color discrimination capacity of the
human visual system under standard conditions.
Common formats include JPEG, 24-bit PNG, BMP, and PPM
(Portable Pixel Map).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

92 / 533

Pixel Storage and Alpha Channels in Truecolor I

Interleaved memory storage vs Planar memory storage:
Interleaved (RGB RGB RGB...): Preferred for real-time display
hardware buffers.
Planar (RRR... GGG... BBB...): Advantageous for single-channel
image processing filters.

Extension to 32-bit (RGBA): Adds an 8-bit Alpha channel (α)
representing pixel opacity (0 = transparent, 255 = opaque).
Alpha blending formula: Cout = α · Cfg + (1 − α) · Cbg .

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

93 / 533

Higher Bit-Depth Images (10-bit, 12-bit, 16-bit, and
HDR) I

Modern imaging sensors capture data at higher precision: 10-bit
(1024 levels), 12-bit (4096 levels), or 16-bit (65536 levels per
channel).
High Dynamic Range (HDR) uses 16-bit half-float or 32-bit
single-precision floating-point values per channel.
RAW camera image formats store uncompressed or losslessly
compressed raw sensor readings prior to demosaicing.
Higher bit depth provides headroom for aggressive color grading,
exposure compensation, and contrast enhancement without
posterization.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

94 / 533

Applications of High Bit-Depth Data Types I

Medical Imaging (DICOM): 12 to 16-bit grayscale resolution is
essential for detecting subtle tissue density variations.
Remote Sensing & Satellite Imaging: Multi-spectral imagery
captured at 12/14 bits per channel for radiometric precision.
Professional Digital Cinematography: 10-bit/12-bit ProRes,
DNxHR, and RAW video formats ensure smooth gradients and wide
color gamut (Rec. 2020).
Scientific & Astronomical Photography: High precision reduces
accumulation of quantization noise during image stacking.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

95 / 533

8-Bit Color Images and Color Look-Up Tables
(CLUT) I

8-bit color (indexed color) reduces memory by using an 8-bit index
(0–255) pointing into a Color Look-Up Table (CLUT).
The CLUT (Palette) contains 256 entries, where each entry is a
24-bit RGB triple (3 bytes).
Storage footprint per pixel drops from 24 bits to 8 bits, plus a
768-byte header palette table.
Popularized by legacy graphics hardware and file formats like GIF
(Graphics Interchange Format).
Major limitation: A single image can only display up to 256 distinct
colors selected from the broader 24-bit palette.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

96 / 533

Palette Quantization and Dithering I

Color Quantization: The process of choosing an optimal subset of
256 colors to represent a 24-bit truecolor image.
Algorithms for quantization: Median Cut, Octree quantization,
K-Means clustering.
Dithering: Technique used to simulate missing colors by arranging
spatial patterns of available pixels.
Common dithering algorithms: Floyd-Steinberg error diffusion,
Ordered Dithering (Bayer matrix).
Trade-off: Dithering reduces visual band artifacts (posterization) but
adds spatial high-frequency noise.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

97 / 533

Comparative Analysis of Image Data Types I

1-bit Binary: 1 bpp, 2 colors, ultra-low storage, ideal for line art &
text.
8-bit Grayscale: 8 bpp, 256 gray levels, compact storage, ideal for
monochrome analysis.
8-bit Indexed Color: 8 bpp + 768-byte CLUT, 256 selectable colors,
ideal for simple graphics & Web GIF.
24-bit Truecolor: 24 bpp, 16.7M colors, standard photographic
rendering.
48-bit High-Depth RGB: 48 bpp (16-bit/channel), 281 trillion
colors, professional editing & archiving.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

98 / 533

Summary and Key Takeaways I

Image data types represent a fundamental balance between visual
fidelity, computational processing speed, and memory usage.
Bit-depth selection directly dictates quantization granularity and the
risk of banding or quantization noise.
Truecolor (24-bit) remains the baseline display standard, while higher
bit depths (10–16 bit floating point) dominate professional creation
pipelines.
Indexed color (8-bit color) optimized historical bandwidth and
storage, leveraging palettes and spatial dithering.
Matching data types to downstream application needs (storage,
display, analytics) is essential in digital image editing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

99 / 533

Overview of Digital Image Data Types I
Digital Image Concept: A discrete spatial array of intensity or color
samples (pixels).
Bit Depth (b): The number of bits allocated per pixel (or per color
channel) to represent visual information.
Quantization Range: A bit depth of b bits allows 2b distinct
intensity levels or colors.
Key Data Categories:
Binary (1-bit): Pure black and white.
Grayscale (8-bit): 256 shades of gray.
Truecolor (24-bit): 16.77 million colors (8 bits/channel × 3 channels).
High Bit-Depth (10–16 bits/channel): Deep color, HDR, and medical
imaging.
Indexed Color (8-bit color): 256 selected colors via a Color Map /
Look-Up Table (LUT).

Storage Impact: Higher bit depth increases dynamic range and
fidelity at the cost of memory bandwidth and file size.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

100 / 533

1-Bit Images (Monochrome / Binary) I
Definition: Each pixel is represented by a single bit (b = 1), yielding
21 = 2 states.
Pixel Representation:
Value 0: Represents black (or background).
Value 1: Represents white (or foreground).

Memory Calculation:
Image size: W × H pixels.
Total memory: (W × H)/8 bytes.
Example: A 1024 × 1024 binary image requires 131,072 bytes
(≈ 128 KB).

Primary Applications:
Document scanning, text OCR processing, line art, and architectural
blueprints.
Binary masks for image segmentation and thresholding operations.

Limitations: Complete lack of tonal gradations; continuous-tone
scenes appear harshly segmented unless dithered.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

101 / 533

8-Bit Gray-Level Images I
Structure: Each pixel holds b = 8 bits, allowing 28 = 256 intensity
levels.
Intensity Scale:
0: Pure Black.
255: Pure White.
1 . . . 254: Intermediate monochrome gray levels.

Psychovisual Justification: Human visual system (HVS) can
distinguish approximately 100–200 shades of gray under typical
viewing conditions; 256 levels generally prevent perceptible false
contouring.
Memory Footprint:
Exactly 1 byte per pixel.
Uncompressed size for W × H image = W × H bytes.

Core Uses:
Computer vision feature extraction (e.g., edge detection, texture
analysis).
Infrared, satellite, and single-channel sensor raw data processing.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

102 / 533

Image Data Types and Numerical Representations I
Unsigned Integer Formats:
uint8: Range [0, 255] – standard definition storage.
uint16: Range [0, 65535] – professional photography and medical
imaging (DICOM).

Signed Integer Formats:
int16 / int32: Useful for differential image processing, gradients,
and Laplacian filters.

Floating-Point Formats:
float32 / float64: Normalized values usually in [0.0, 1.0] or
arbitrary real ranges.
Mandatory for High Dynamic Range (HDR) imaging, optical flow
calculations, and deep learning models.

Data Conversion Risks: Clipping/saturation during downcasting
(e.g., float to uint8) and precision loss in integer division.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

103 / 533

24-Bit Color Images (Truecolor) I
Trichromatic Model: Represents color using Red, Green, and Blue
(RGB) primaries.
Bit Allocation:
8 bits for Red (256 levels).
8 bits for Green (256 levels).
8 bits for Blue (256 levels).
Total: 24 bits per pixel (3 bytes/pixel).

Color Space Capacity: 256 × 256 × 256 = 16,777,216 distinct
possible colors (≈ 16.77 million).
Memory Organization:
Interleaved (RGB RGB RGB...) vs. Planar (Separate R, G, B
channels).
32-bit RGBA adds an 8-bit Alpha channel for transparency masking.

Storage Calculation: A 1920 × 1080 Full HD image requires
1920 × 1080 × 3 = 6,220,800 bytes ≈ 5.93 MB uncompressed.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

104 / 533

Higher Bit-Depth Images I
Motivation: Exceeding standard 8-bit per channel limits to prevent
color banding in post-processing and retain dynamic range.
10-Bit and 12-Bit Systems:
10-bit per channel (30-bit Truecolor): 1,024 levels/channel (1.07
billion colors). Wide Color Gamut (WCG, Rec. 2020).
12-bit per channel (36-bit): 4,096 levels/channel. Standard in cinema
RAW formats (ProRes RAW, CinemaDNG).

16-Bit per Channel (48-Bit RGB):
65,536 levels per channel. Used in professional editing (Adobe
Photoshop, Lightroom, TIFF formats).

High Dynamic Range (HDR):
Uses 16-bit or 32-bit floating point representations (OpenEXR
format).
Captures extreme brightness contrasts (shadows to bright sunlight)
without highlight clipping.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

105 / 533

8-Bit Color Images (Indexed Color) I
Concept: Storage optimization for color images using an explicit
Color Map or Look-Up Table (LUT).
Architecture:
Palette/LUT: Contains up to 256 color entries, where each entry is a
full 24-bit RGB value (3 bytes).
Pixel Array: Each pixel stores an 8-bit index (0 . . . 255) pointing to an
entry in the LUT.

Memory Savings:
Pixel data: 1 byte/pixel (instead of 3 bytes/pixel for 24-bit Truecolor).
LUT overhead: 256 × 3 = 768 bytes (negligible).
Savings: Approximately 66.7% reduction in raw pixel memory.

Standard Example: Graphics Interchange Format (GIF) and indexed
PNG formats.
Trade-off: Limited palette causes posterization in continuous-tone
photography.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

106 / 533

Color Quantization and Bit Depth Reduction I

Color Quantization: The process of reducing the number of distinct
colors in an image while preserving visual fidelity.
Key Algorithms:
Uniform Quantization: Equal splitting of RGB color space (e.g., 3
bits R, 3 bits G, 2 bits B).
Median Cut Algorithm: Recursively splits color space along the axis
of maximum variance to form balanced clusters.
K-Means Clustering: Iterative clustering minimizing color distance
error in perceptual spaces (e.g., CIELAB).

Artifacts of Bit Reduction:
False Contouring / Banding: Sharp boundaries in smooth gradients
due to missing intermediate quantization levels.
Color Shift: Mapping out-of-palette colors to nearest neighbors
distorts hue.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

107 / 533

Dithering Techniques for Low Bit-Depths I
Purpose: Simulating missing shades or colors by spatially arranging
available pixels to exploit visual spatial integration.
Ordered Dithering:
Uses a threshold matrix (e.g., Bayer matrix) repeated across the
image.
Computationally fast and parallelizable, but creates repeating
geometric patterns.

Error Diffusion Dithering:
Quantization error at a pixel is distributed to neighboring unprocessed
pixels.
Floyd-Steinberg Algorithm: Distributes error to right (7/16),
bottom-left (3/16), bottom (5/16), and bottom-right (1/16).

Visual Trade-off: Reduces false contouring significantly, but
introduces high-frequency noise (”graininess”).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

108 / 533

Comparative Summary of Bit Depths I

Summary Table of Data Types:
Data Type
1-Bit Binary
8-Bit Grayscale
8-Bit Indexed
24-Bit Truecolor
30-Bit Deep Color
48-Bit High Depth

Bits/Pixel
1
8
8 + LUT
24
30
48

Unique Colors/Grays
2
256
256 (from 16.7M)
16.77 × 106
1.07 × 109
281.5 × 1012

Typical Applications
Text, Line Art, Masks
Medical, CV, Monochromatic
Web Graphics, GIF
Digital Photography, Web
HDR Displays, Editing
Archival, RAW Editing

Selection Criteria: Choose based on display target, sensor
capabilities, acceptable storage, and processing headroom.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

109 / 533

Practical Applications and Data Conversions I

Conversion Workflows:
24-bit RGB to 8-bit Grayscale: Luminance conversion formula:
Y = 0.299R + 0.587G + 0.114B.
Grayscale to 1-bit Binary: Otsu’s adaptive thresholding or error
diffusion dithering.
24-bit to 8-bit Indexed: Color quantization (Median Cut / Octree)
+ Palette mapping + Optional Dithering.

Storage and Transmission Protocols:
Lossless compression (PNG, TIFF) preserves bit-depth integrity.
Lossy compression (JPEG) operates on 8-bit YCbCr channels.

Conclusion: Understanding image data types ensures optimal
performance, accurate color fidelity, and efficient resource allocation
in digital image processing pipelines.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

110 / 533

Fundamentals of Digital Image Data and Bit Depth I
Digital Image Representation: An image is represented as a
discrete 2D grid of pixels (picture elements), I (x, y ), where spatial
coordinates (x, y ) map to numerical intensity or color values.
Definition of Bit Depth: Bit depth (b) determines the number of
bits allocated to store the color or intensity value of each individual
pixel.
Dynamic Range and Quantization:
A bit depth of b allows 2b distinct intensity or color levels per pixel.
Higher bit depth improves tonal resolution and minimizes visual
quantization artifacts such as false contouring.

Spatial vs. Color Resolution:
Spatial resolution specifies pixel density across dimensions (W × H).
Color/intensity resolution specifies dynamic range per pixel (b bits).

Data Size Formula: Uncompressed image size in bits is calculated
as:
Size = Width × Height × Bit Depth
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

111 / 533

1-Bit Monochromatic (Binary) Images I
Overview:
Bit depth b = 1, resulting in 21 = 2 possible states per pixel: 0
(black) or 1 (white).
Commonly referred to as binary, bilevel, or monochrome images.

Memory Storage Efficiency:
Exceptionally compact memory footprint: 1 byte of memory stores 8
adjacent pixels.
Ideal for low-bandwidth transmission and resource-constrained
environments.

Binarization via Thresholding:
Generated from grayscale images using global or adaptive thresholding:
(
1 if f (x, y ) ≥ T
g (x, y ) =
0 if f (x, y ) < T

Primary Applications:
Document scanning, Optical Character Recognition (OCR), line-art
rendering, architectural schematics, and binary mask generation in
computer vision.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

112 / 533

8-Bit Gray-Level (Grayscale) Images I
Overview:
Bit depth b = 8, providing 28 = 256 distinct gray levels ranging from
0 (pure black) to 255 (pure white).

Luminance and Visual Perception:
Encodes light intensity without chromaticity (color) information.
Aligns closely with human visual system sensitivity, which resolves
approximately 100–200 discrete gray levels.

Memory Representation:
Each pixel maps directly to 1 byte (8 bits) of memory storage.
Storage for an M × N image is exactly M × N bytes.

Conversion from RGB:
ITU-R BT.601 standard luminance conversion equation:
Y = 0.299R + 0.587G + 0.114B

Applications:
Preprocessing step for edge detection, texture analysis, feature
extraction, and medical diagnostic imaging.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

113 / 533

Core Image Data Types and Numerical Precision I
Unsigned Integer Formats:
uint8: Unsigned 8-bit integer (0 to 255). Standard format for web
graphics and display hardware.
uint16: Unsigned 16-bit integer (0 to 65, 535). Standard for high-end
photography and DICOM medical scans.

Signed Integer Formats:
int16: Signed 16-bit integer (−32, 768 to +32, 767). Utilized in
image difference calculations and transform-domain representations.

Floating-Point Formats:
float32 / float64: Single/Double precision floating-point numbers.
Essential in mathematical operations (e.g., Fourier transforms,
convolution filtering, HDR computing) to prevent rounding errors.

Normalisation and Scaling:
Integer values range over [0, 2b − 1], whereas floating-point values are
normalized to [0.0, 1.0].
Iuint8
Scaling transformation: Ifloat = 255.0
.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

114 / 533

24-Bit Color Images (Truecolor RGB) I
Color Space Structure:
Implements the additive RGB (Red, Green, Blue) color model.
Allocated bit depth: 8 bits per channel across 3 color channels
(8 × 3 = 24 bits total per pixel).

Color Gamut Capacity:
Supports 28 × 28 × 28 = 224 ≈ 16, 777, 216 possible color
combinations.
Exceeds the estimated 10 million distinct colors visible to the human
eye, earning the designation Truecolor.

Memory Footprint Calculation:
Requires 3 bytes per pixel.
Storage for a Full HD image (1920 × 1080):
1920 × 1080 × 3 bytes ≈ 6.22 MB (uncompressed).

32-Bit Alpha Extension (RGBA):
Includes an additional 8-bit alpha channel (α) for encoding opacity
and transparency blending.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

115 / 533

Channel Memory Layouts and Storage Architectures I

Interleaved Channel Ordering (Pixel-Sequential):
Color values for each pixel are stored contiguously:
R1 G1 B1 R2 G2 B2 . . . RN GN BN .
Standard organization for graphics framebuffers, display controllers,
and image formats (BMP, JPEG, PNG).

Planar Channel Ordering (Band-Sequential):
Each color channel is stored as a distinct 2D array in memory:
[R1 . . . RN ], followed by [G ], then [B].
Preferred structure in SIMD matrix operations, spatial filtering, and
scientific computing toolkits.

Memory Alignment Padding:
GPUs often pad 24-bit RGB pixels to 32-bit (RGBX) to align data
with 32-bit memory words, sacrificing spatial efficiency for processing
throughput.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

116 / 533

Higher Bit-Depth Images (10-Bit, 12-Bit, 16-Bit &
HDR) I
Motivation for High Precision:
Eliminates posterization and color banding in smooth gradient
transitions.
Preserves detail in extreme highlights and deep shadow regions during
editing.

Common Higher Bit-Depths:
10-Bit / 12-Bit: 1,024 to 4,096 levels per channel. Used in HDR
video standards (HDR10), digital cinema, and professional video
production.
16-Bit Grayscale/RGB: 65,536 levels per channel (48 bits/pixel for
RGB). Standard for RAW camera captures and medical DICOM scans.

High Dynamic Range (HDR) Formats:
Employs 32-bit floating-point representations per channel (96-bit
RGB) to encode real-world physical luminance values (cd/m2 ).

Editing Headroom:
Extended bit depth provides mathematical precision needed for heavy
contrast adjustments and color grading without quality loss.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

117 / 533

8-Bit Color Images and Color Lookup Tables (CLUT)
I
Indexed Color Concept:
Reduces memory footprint by decoupling pixel values from direct
24-bit RGB representation.
Each pixel stores an 8-bit index (0 to 255) pointing to an entry in a
predefined palette.

Color Lookup Table (CLUT) Architecture:
The CLUT stores up to 256 selected 24-bit RGB color triples
(256 × 3 = 768 bytes).
Mapping mechanism: Pixel Value i → CLUT[i] = (Ri , Gi , Bi ).

Memory Reduction Efficiency:
Requires only 1 byte per pixel regardless of color quality (plus 768
bytes header overhead).
Yields a ∼ 66% savings in memory compared to uncompressed 24-bit
Truecolor.

Applications:
Legacy graphics hardware (VGA), web graphics (GIF format), and
icon assets in embedded UIs.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

118 / 533

Color Quantization and Palette Dithering I
Color Quantization:
The process of reducing a 24-bit Truecolor image (16.7M colors) to an
optimal 8-bit palette (256 colors).
Standard algorithms: Median-Cut, Octree Quantization, and K-Means
Clustering.

Spatial Dithering:
Simulates missing color shades by arranging available palette colors in
spatial patterns.
Effectively reduces visual color banding in continuous tone regions.

Floyd-Steinberg Error Diffusion:
Computes quantization error for a pixel: E = Poriginal − Pquantized .
Distributes error E to unhandled neighboring pixels using spatial
7
3
5
1
coefficients ( 16
, 16
, 16
, 16
).

Trade-off:
Dithering enhances perceived visual quality but introduces
high-frequency spatial noise, which degrades entropy compression.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

119 / 533

Comparative Analysis of Image Data Types I
Summary of Characteristics across Data Types:
Data Type

Bit Depth

Colors / Levels

1-Bit Binary

1 bit

2

8-Bit Grayscale

8 bits

256

8-Bit Indexed

8 bits

256 (from 16.7M)

24-Bit Truecolor

24 bits

16, 777, 216

16-Bit / Channel

48 bits

2.81 × 1014

32-Bit Float

96 bits

Continuous

GTU Course

DI04000051: Multimedia Technologies

Primary Applications
Line art, OCR, binary masks
CV preprocessing,
monochrome
Web GIF, icons,
embedded UI
Digital photography, standard web
RAW camera, medical DICOM
HDR rendering, scientific computing

Lecture Slides

120 / 533

Data Type Conversions and Arithmetic Pitfalls I

Type Casting and Scaling Rules:
Converting float32 [0, 1] to uint8 [0, 255] requires scaling and
rounding: Iuint8 = round(Ifloat × 255).
Direct unscaled integer truncation causes severe visual distortion and
clipping.

Overflow and Underflow:
Standard uint8 arithmetic wraps modulo 256 (200 + 100 = 44).
Image operations require saturated arithmetic:
Iout = min(255, max(0, Iin )).

Accumulated Rounding Errors:
Sequential filtering operations in low-bit-depth integer domains
accumulate severe quantization error.
Recommended Workflow: Cast to float32 → Perform processing
→ Apply saturation clamping → Re-cast to uint8 for display.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

121 / 533

Overview of Digital Image File Formats I

Role of File Formats: Encapsulate pixel data, color space definitions,
compression structures, and metadata into standardized byte streams.
Raster vs. Vector Encoding:
Raster Formats: Store discrete grid array of pixel values (e.g., BMP,
JPEG, PNG, TIFF).
Vector Formats: Store algorithmic draw commands, geometric
primitives, and mathematical paths (e.g., WMF, PS, PDF).

Compression Paradigms:
Lossless: Perfect reconstruction of source data via entropy coding
(e.g., LZW, Deflate, Run-Length Encoding).
Lossy : Discards visually redundant frequency data to maximize
compression ratios (e.g., DCT, Wavelets).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

122 / 533

Uncompressed & Simple Formats: BMP and Netpbm
I
Windows BMP (Bitmap Image File):
Native Microsoft Windows format; stores uncompressed or
RLE-encoded pixel arrays.
Structure: File Header (14 bytes), DIB Header
(BITMAPINFOHEADER), Optional Color Palette, Pixel Data.
Bottom-up scanline ordering by default; supports 1-bit to 32-bit depth.

Netpbm Format Family:
Simple ASCII or binary formats designed for easy programmatic
processing.
PBM (Portable BitMap): 1-bit monochrome images (P1 ASCII, P4
Binary).
PGM (Portable GrayMap): Grayscale images with defined maxval (P2
ASCII, P5 Binary).
PPM (Portable PixMap): 24-bit RGB color images (P3 ASCII, P6
Binary).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

123 / 533

Indexed Lossless Compression: GIF I

Graphics Interchange Format (GIF):
Standardized by CompuServe (GIF87a, GIF89a).
Color Palette: Limited to indexed 8-bit palette (maximum 256 colors
selected from 24-bit RGB space).
Compression: Lempel-Ziv-Welch (LZW) lossless dictionary-based
algorithm.

GIF89a Features:
Single-index transparency (1-bit keying).
Multi-frame animation blocks with inter-frame delay controls and
disposal methods.
Interlacing support (4-pass line fetching for progressive rendering).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

124 / 533

Continuous-Tone Lossy Compression: JPEG I

Joint Photographic Experts Group (JPEG / JFIF):
Standard lossy compression designed for natural photographic imagery.

JPEG Compression Pipeline:
Color Transformation: RGB to YCb Cr color space.
Chroma Subsampling : Reduces spatial resolution of Cb and Cr
channels (e.g., 4:2:0, 4:2:2).
Discrete Cosine Transform (DCT): Converts 8 × 8 pixel blocks into
frequency domain coefficients.
Quantization: High-frequency components divided by quantization
matrix tables (primary source of loss).
Entropy Encoding : Zig-zag scan ordering followed by Run-Length and
Huffman Coding.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

125 / 533

Advanced Lossless Compression: PNG I

Portable Network Graphics (PNG):
Open, patent-free replacement for GIF; optimized for web graphics.

Key Capabilities:
Truecolor support (up to 48-bit RGB) and 16-bit grayscale.
Full 8-bit alpha channel transparency (256 levels of opacity).
Chunk-based data organization (e.g., IHDR, PLTE, IDAT, IEND).

Compression Architecture:
Predictive Filtering : 5 spatial filter types (None, Sub, Up, Average,
Paeth) applied per scanline to minimize entropy.
Deflate Engine: LZ77 sliding-window dictionary compression
combined with Huffman coding.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

126 / 533

Professional & Flexible Archival: TIFF I

Tagged Image File Format (TIFF):
Extensible container standard widely used in printing, publishing, and
medical imaging.

Structural Architecture:
Header points to Image File Directories (IFDs) containing metadata
tags.
Flexible tag system defines color spaces (RGB, CMYK, YCbCr, CIE
L*a*b*), bit depths, and compression codecs.

Codec Flexibility:
Supports uncompressed raw, LZW, Deflate, PackBits RLE, and
CCITT Group 4 fax compression.
Multi-page document storage and pyramid tiled layouts for massive
geospatial data.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

127 / 533

Vector Graphics & Page Description: WMF, PS, and
PDF I

Windows Metafile Format (WMF / EMF):
Stores GDI drawing function calls to render scalable vector primitives
and raster clips.
Enhanced Metafile (EMF) extends WMF with 32-bit record structures
and color management.

PostScript (PS) & Portable Document Format (PDF):
PostScript: Turing-complete programming language for vector
graphics and typography.
PDF : Object-based non-executable representation derived from PS;
embeds vector paths, raster streams, and fonts cleanly.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

128 / 533

Metadata Integration: EXIF I

Exchangeable Image File Format (EXIF):
Standard specifying metadata structures embedded directly within
JPEG, TIFF, and HEIF headers.

Embedded Information Tags:
Camera Settings: Exposure time, f-number, ISO speed, focal length,
metering mode, flash status.
Spatial & Temporal Data: Timestamp, GPS coordinates (latitude,
longitude, altitude).
Image Characteristics: Resolution, color space (sRGB, Adobe RGB),
preview thumbnail.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

129 / 533

Next-Generation Containers: HEIF I

High Efficiency Image File Format (HEIF / HEIC):
Based on the ISO Base Media File Format (ISOBMFF); utilizes HEVC
/ H.265 intra-frame video compression.

Technical Advantages over JPEG:
Provides up to 50% better compression efficiency at equivalent visual
quality.
Supports 10-bit and 12-bit deep color arrays, HDR, wide color gamut
(Rec. 2020).

Container Capabilities:
Multi-item storage (image sequences, burst shots, living photos).
Non-destructive edits (cropping, rotation) stored as metadata
transformations.
Auxiliary data streams (depth maps, alpha transparency, image grids).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

130 / 533

Advanced Image Representation: PTM & Reflectance
Maps I

Polynomial Texture Mapping (PTM):
Computational photography format for Reflectance Transformation
Imaging (RTI).

Mathematical Formulation:
Encodes luminance L at pixel (u, v ) as a second-order biquadratic
polynomial of light vector directions (ui , vi ):
L(u, v ; ui , vi ) = a0 ui2 + a1 vi2 + a2 ui vi + a3 ui + a4 vi + a5

Applications:
Interactive relighting, specular enhancement, and surface normal
extraction for archaeological and forensic inspection.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

131 / 533

Comparative Summary of Image Formats I

Format Characteristics Matrix:
BMP / Netpbm: Uncompressed / simple; primary use in system-level
processing and education.
GIF : Lossless 8-bit indexed; primary use in low-color web animation.
JPEG : Lossy DCT; primary use in web imagery and photography.
PNG : Lossless Deflate; primary use in web UI, transparency, graphics.
TIFF : Multi-codec tag container; primary use in print production and
archiving.
HEIF : Lossy/Lossless HEVC stream; primary use in modern mobile
imaging.
PTM: Polynomial coefficients; primary use in surface reflectance
analysis.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

132 / 533

Overview of Digital Image File Formats I

Fundamentals of Digital Image Storage:
Digital image files store pixel array values, color palettes, compression
metadata, and structural headers.
Main classifications: Raster (Bitmap) graphics and Vector graphics.

Raster vs. Vector Representations:
Raster Formats: Store discrete grid values (e.g., JPEG, PNG, TIFF,
BMP, GIF, HEIF, Netpbm).
Vector/Metafile Formats: Store mathematical primitives such as
lines, curves, and polygons (e.g., WMF, PostScript, PDF).

Compression Paradigms:
Lossless Compression: Preserves exact bitstream upon decompression
(e.g., LZW in GIF/TIFF, DEFLATE in PNG, RLE in BMP).
Lossy Compression: Discards perceptually redundant data to
maximize compression ratios (e.g., DCT in JPEG, HEVC in HEIF).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

133 / 533

GIF: Graphics Interchange Format I
Core Specifications:
Developed by CompuServe in 1987 (GIF87a) and expanded in 1989
(GIF89a).
Limited to a maximum color depth of 8 bits per pixel (indexed color
space with 256 colors maximum).

Compression & Encoding:
Utilizes lossless LZW (Lempel-Ziv-Welch) dictionary-based
compression.
Efficient for synthetic graphics, line art, and simple logos with flat
color areas.

Advanced Features (GIF89a):
Transparency: Single-index boolean palette transparency (no alpha
blending).
Animation: Multi-image frame sequencing with variable frame delay
control blocks.
Interlacing: 4-pass row rendering allowing progressive preview loading
over low-bandwidth channels.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

134 / 533

JPEG: Joint Photographic Experts Group I
Standard Overview:
ISO/IEC 10918-1 standard designed specifically for continuous-tone
photographic images.
Supports 24-bit truecolor space (8 bits per channel in RGB/YCbCr).

Lossy Compression Pipeline:
Color Space Transformation: Converts RGB to YCb Cr
luminance-chrominance space.
Chroma Subsampling: Reduces spatial resolution of chrominance
components (e.g., 4:2:2 or 4:2:0).
Discrete Cosine Transform (DCT): Transforms 8 × 8 pixel blocks
into frequency domain coefficients.
Quantisation: Applies psychovisual quantisation tables to
high-frequency coefficients (primary source of loss).
Entropy Encoding: Employs Zig-zag scanning followed by
Run-Length and Huffman encoding.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

135 / 533

PNG: Portable Network Graphics I
Design Purpose:
Created as a patent-free replacement for GIF under W3C
recommendation (ISO/IEC 15948).
Exclusively lossless raster image format.

Technical Characteristics:
Supports palette-based (1-8 bit), grayscale (1-16 bit), and truecolor
(24-48 bit) images.
Full 8-bit or 16-bit Alpha Channel Transparency supporting 256 or
65536 levels of opacity per pixel.

Filtering and DEFLATE Compression:
Delta Filtering: Applies 5 byte-level prediction filters (None, Sub, Up,
Average, Paeth) per scanline to optimize spatial redundancy.
DEFLATE Algorithm: Compresses filtered scanlines using LZ77 and
Huffman coding.
Adam7 Interlacing: 7-pass two-dimensional interlacing system.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

136 / 533

TIFF: Tagged Image File Format I

Architecture & Container Model:
Maintained by Adobe Systems; widely used in high-end publishing,
medical imaging, and archiving.
Flexible tag-based header architecture organizing data into Image File
Directories (IFDs).

Payload Versatility:
Accommodates uncompressed raw pixels, or compressed payloads
using LZW, DEFLATE/ZIP, CCITT Group 3/4, or JPEG.
Supports multiple color spaces: Monochrome, Grayscale, Palette,
RGB, CMYK, YCbCr, and CIELAB.

Extensibility:
Multi-page document storage within a single file container.
32-bit offset limit (classic TIFF) vs. 64-bit offsets in BigTIFF for files
exceeding 4 GB.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

137 / 533

Windows BMP & Windows WMF I

Windows Device-Independent Bitmap (BMP):
Native raster image format for Microsoft Windows operating systems.
Structure: File Header (BITMAPFILEHEADER), Information Header
(BITMAPINFOHEADER), Color Table, and Pixel Array.
Compression: Typically uncompressed raw pixel values; optional RLE
(Run-Length Encoding) for 4-bit and 8-bit images.
Stores pixel rows bottom-to-top by default.

Windows Metafile (WMF):
16-bit vector and graphics metafile format introduced in Windows 3.0
(evolved to 32-bit EMF).
Stores sequences of Graphics Device Interface (GDI) function calls
(e.g., lines, arcs, polygons, text rendering).
Resolution-independent vector graphics with embedded raster
capability.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

138 / 533

Netpbm Format Suite (PBM, PGM, PPM) I
Philosophical Design:
Minimalist graphics formats designed for maximum portability,
simplicity, and ease of parser implementation.
Widely used as intermediate exchange formats in UNIX image
processing pipelines.

Sub-Format Taxonomy:
PBM (Portable BitMap): 1-bit monochrome images (Black/White).
PGM (Portable GrayMap): Grayscale images (typically 8 to 16 bits
per pixel).
PPM (Portable PixMap): 24-bit RGB color images.

Magic Number Encodings:
Plain ASCII Encodings: P1 (PBM), P2 (PGM), P3 (PPM) –
human-readable text headers and data.
Binary Encodings: P4 (PBM), P5 (PGM), P6 (PPM) – compact
binary payload format.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

139 / 533

EXIF: Exchangeable Image File Format I

Standard Specifications:
Maintained by JEIDA / JEITA for embedded metadata in digital
photography.
Integrated directly into JPEG, TIFF, and HEIF header structures.

Metadata Categories:
Camera Settings: Exposure time, F-number, ISO speed rating, focal
length, metering mode, flash status.
Spatial & Temporal Data: Time/date timestamp, GPS coordinates
(latitude, longitude, altitude), camera orientation.
Device Parameters: Camera make, model, lens profile, serial
numbers, firmware version.

Security & Privacy Implications:
Location metadata leakage in web uploads requiring EXIF stripping
protocols during image editing workflows.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

140 / 533

HEIF: High Efficiency Image File Format I
Modern Container Standard:
Defined by ISO/IEC 23008-12 (MPEG) based on the ISO Base Media
File Format (ISOBMFF).
Container format capable of storing individual images, image
collections, image sequences, and auxiliary data.

Compression Efficiency:
Utilizes HEVC (H.265) video coding intra-frame techniques (or AV1
in AVIF).
Achieves roughly 50% higher compression efficiency than JPEG at
equivalent visual quality.

Advanced Structural Features:
Deep color support (10-bit, 12-bit, 16-bit precision and HDR dynamic
range).
Embedded depth maps, alpha channels, non-destructive editing
operations, and gain maps.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

141 / 533

PostScript (PS) & Portable Document Format
(PDF) I
PostScript (PS):
Dynamic, Turing-complete Page Description Language (PDL)
developed by Adobe Systems.
Stack-based execution model using Reverse Polish Notation (RPN).
Describes rasterization of vector primitives, typography, and
embedded bitmap images.

Encapsulated PostScript (EPS):
Sub-format constrained to single-page vector graphic interchange with
optional low-resolution thumbnail preview.

PDF (Portable Document Format):
Standardized format (ISO 32000) derived from PostScript.
Replaces procedural programming constructs with a declarative object
hierarchy (Catalog, Pages, Streams).
Enables fast linearised page viewing, font embedding, and precise
document layout fidelity.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

142 / 533

Polynomial Texture Maps (PTM) I

Computational Photography & RTI:
Image-based rendering technique introduced by Tom Malzbender (HP
Labs) within Reflectance Transformation Imaging (RTI).

Mathematical Formulation:
Models pixel luminance L as a second-order polynomial function of
projection lighting directions (u, v ):
L(u, v ) = a0 u 2 + a1 v 2 + a2 uv + a3 u + a4 v + a5
Stores six coefficients (a0 , a1 , a2 , a3 , a4 , a5 ) per pixel alongside RGB
color values.

Applications:
Interactive relighting, specular enhancement, and surface normal
extraction for cultural heritage artifact documentation and forensics.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

143 / 533

Comparative Analysis & Selection Strategy I

Format Selection Matrix:
Photographs: JPEG (legacy/web), HEIF/AVIF (modern
web/mobile), TIFF (archival raw processing).
Graphics & Web UI: PNG (lossless + alpha), GIF (legacy simple
animation), WebP/AVIF (modern replacement).
Vector & Print Workflow: WMF/EMF (legacy Windows),
PostScript/PDF (pre-press print publishing).
Specialized Analysis: Netpbm (quick scripting/prototyping),
PTM/RTI (scientific reflectance surface modeling).

Summary Recommendation:
Image editing workflows require matching compression characteristics,
metadata retention (EXIF/IFD), and color space fidelity to target
delivery channels.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

144 / 533

Overview of Digital Image File Formats I

Fundamental Categories:
Raster (Bitmap) Formats: Represent images as a 2D grid of pixels
(color/intensity values). Examples: JPEG, PNG, GIF, TIFF, BMP.
Vector Formats: Represent images using geometric primitives
(points, lines, curves, polygons) defined by mathematical equations.
Examples: WMF, PostScript, PDF.

Compression Paradigms:
Lossless Compression: Preserves exact pixel data without fidelity loss
(e.g., PNG, GIF, LZW-TIFF).
Lossy Compression: Achieves high compression ratios by discarding
perceptually redundant visual data (e.g., JPEG, HEIF).

Key Evaluation Criteria: Spatial resolution, color depth (bits per
pixel), metadata overhead, computational decoding complexity, and
container flexibility.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

145 / 533

GIF: Graphics Interchange Format I

Historical Context & Architecture:
Introduced by CompuServe in 1987 (GIF87a) and enhanced in 1989
(GIF89a).
Indexed color model supporting a maximum palette of 256 colors
chosen from a 24-bit RGB color space (8 bits per pixel).

Compression & Technical Features:
Uses Lempel-Ziv-Welch (LZW) lossless compression algorithm.
GIF89a Extensions: Supports simple boolean 1-bit transparency
(index marked as transparent) and multi-frame animation.
Interlacing: 4-pass interlacing scheme enabling progressive display
rendering over band-limited channels.

Limitations: Unsuitable for continuous-tone photographic images
due to color quantization artifacts.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

146 / 533

JPEG: Joint Photographic Experts Group I
Standard Overview: ISO/IEC 10918-1 standard designed specifically
for continuous-tone photographic imagery.
Encoding Pipeline:
Color Space Transformation: Converts RGB to YCb Cr color space;
applies sub-sampling to chrominance components (e.g., 4:2:0 or
4:2:2).
Discrete Cosine Transform (DCT): Divides each channel into 8 × 8
pixel blocks and converts spatial signals to frequency domain
coefficients.
Quantization: Applies psycho-visual quantization matrix to divide
DCT coefficients; introduces controlled lossiness by rounding
high-frequency values.
Entropy Coding: Uses Zig-zag scanning, Run-Length Encoding
(RLE), and Huffman coding (or arithmetic coding).

Trade-offs: Excellent compression ratios (10:1 to 20:1 with high
quality); suffers from blockiness and ringing artifacts at high
compression rates.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

147 / 533

PNG: Portable Network Graphics I
Standard Motivation: Designed as a patent-free replacement for
GIF standard under W3C recommendations.
Technical Capabilities:
Supports Truecolor (24-bit RGB / 48-bit RGB), Grayscale, and
Indexed-color palettes.
Full Alpha Channel support: Up to 16-bit alpha channel for smooth
variable transparency (anti-aliased blending).

Compression Architecture:
2D Delta Filtering: Applies per-scanline predictor filters (None, Sub,
Up, Average, Paeth) to reduce sample entropy prior to compression.
DEFLATE Algorithm: Combines LZ77 derivative algorithm and
Huffman coding.

Integrity & Chunk Architecture: Extensible chunk-based structure
(e.g., IHDR, PLTE, IDAT, IEND) with CRC-32 checksum validation
per chunk.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

148 / 533

TIFF: Tagged Image File Format I
Purpose & Scope: Flexible, platform-independent raster format
widely used in desktop publishing, medical imaging, and professional
photography.
Architectural Layout:
Starts with a fixed 8-byte Image File Header (IFH).
Contains one or more Image File Directories (IFDs) containing tags
with metadata attributes (width, height, compression type, color
space).

Compression Flexibility:
Supports Uncompressed raw data, CCITT Group 3/4 (facsimile),
LZW, JPEG, and Deflate compression.

Advanced Capabilities: Supports multi-page documents,
floating-point radiometric data, deep color (16-bit/channel), and
BigTIFF extension for files > 4 GB.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

149 / 533

Windows Native Formats: BMP & WMF I

Windows Bitmap (BMP):
Standard native raster format in Microsoft Windows environment.
Structure: File Header (BITMAPFILEHEADER), Information Header
(BITMAPINFOHEADER), optional Color Table, and raw pixel array.
Typically uncompressed (device-independent bitmap), though simple
Run-Length Encoding (RLE4/RLE8) is supported for 4-bit and 8-bit
images.

Windows Metafile (WMF):
16-bit 2D vector graphic format introduced in Windows 3.0
(superseded by 32-bit EMF).
Stores a sequence of GDI (Graphics Device Interface) function calls
(e.g., Rectangle, Polyline, BitBlt).
Enables dynamic scaling without pixelation, but presents security and
cross-platform compatibility challenges.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

150 / 533

Netpbm Formats & Polynomial Texture Maps (PTM)
I
Netpbm Portable Formats:
Family of simple, uncompressed graphic formats designed for
Unix/Linux image processing tools.
PBM (P1/P4): Portable BitMap (1-bit monochrome binary).
PGM (P2/P5): Portable GrayMap (grayscale, typically 8-bit or
16-bit).
PPM (P3/P6): Portable PixMap (24-bit RGB color).
Formats feature human-readable ASCII (P1–P3) or compact binary
(P4–P6) magic numbers.

Polynomial Texture Maps (PTM):
Image-based rendering format representing spatially varying
reflectance capabilities.
Encodes surface reflectance per pixel using a 2D second-order
polynomial function of directional light vector components (u, v ).
Widely used in computational archaeology, surface relighting, and
material characterization.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

151 / 533

Metadata Standards: EXIF Architecture I
Exchangeable Image File Format (EXIF):
Standard specifying image, sound, and ancillary tags used by digital
cameras and imaging systems.
Created by JEIDA/JEITA; embedded typically within JPEG (APP1
marker) and TIFF headers.

Stored Metadata Fields:
Camera Settings: Exposure time, f-number (aperture), ISO speed
ratings, focal length, metering mode, flash status.
Temporal & Spatial Data: Timestamp of image capture, GPS
coordinates (latitude, longitude, altitude).
Hardware Data: Camera manufacturer, model, lens parameters,
firmware version.

Applications & Privacy Concerns: Essential for computational
photography and asset management, but requires scrubbing to
protect user location privacy.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

152 / 533

High Efficiency Image File Format (HEIF) I

Standard Specification: MPEG-H Part 12 (ISO/IEC 23008-12)
container format.
Codec Integration:
Encodes image streams using High Efficiency Video Coding (HEVC /
H.265) intra-frame compression.
Achieves up to 50% greater compression efficiency than JPEG at
equivalent visual quality.

Container Capabilities:
Supports single static images, image collections, and animated burst
sequences.
Multi-layer support: Auxiliary items like depth maps, alpha masks,
and HDR gain maps.
Non-destructive editing operations (rotation, cropping, overlaying)
specified as metadata without re-encoding.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

153 / 533

Page Description Languages: PostScript & PDF I

PostScript (PS):
Turing-complete, stack-based page description language developed by
Adobe (1982).
Interprets vector primitives, text fonts (Type 1/TrueType), and
embedded raster graphics into raster imaging display devices.

Portable Document Format (PDF):
Standardized ISO 32000 file format evolved from PostScript.
Replaces procedural execution with structured object hierarchy
(Catalog, Pages, Streams, Font dictionaries).
Supports vector path definitions, text with character encoding maps,
embedded raster images (JPEG, JPEG2000, JBIG2), and interactive
features.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

154 / 533

Comparative Technical Analysis of Image Formats I

Raster vs. Vector Feature Matrix:
Lossy Photographic: JPEG (DCT), HEIF (HEVC intra) — optimized
for continuous tonal gradations.
Lossless Graphics / Web: PNG (DEFLATE), GIF (LZW) —
optimized for sharp edges, text, and flat colors.
Archival / Professional: TIFF (Tags/Multi-page) —
raw/uncompressed or LZW flexibility.
Page Description / Print: PostScript, PDF —
resolution-independent vector rendering.

Compression & Feature Comparison Summary:
Max Color Depth: GIF (8-bit), JPEG (8-bit standard), PNG (48-bit),
TIFF (64-bit+), HEIF (10/12-bit HDR).
Transparency Support: GIF (1-bit), PNG (8/16-bit Alpha), TIFF
(Alpha channel), HEIF (Auxiliary layer).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

155 / 533

Summary & Format Selection Engineering Criteria I

Selection Matrix for Image Editing Pipelines:
Web Distribution: WebP / HEIF / AVIF for speed; JPEG for
universal fallback; PNG for transparent assets.
Master Editing & Storage: TIFF or native PSD/EXR to avoid
cumulative generational loss during multi-pass editing.
Scientific & Medical Imaging: DICOM / TIFF / Netpbm for precise
uncompressed bit depths and custom metadata fields.
Computational Photography: HEIF + EXIF for multi-frame depth
maps and metadata.

Key Takeaways:
Format selection governs trade-offs between compression efficiency,
fidelity, decoding speed, and metadata support.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

156 / 533

Introduction to Image Editing Tools & Interface I

Image Editing Software Overview: Digital image editing software
(e.g., Photoshop, GIMP, Affinity Photo) provides interactive graphical
environments to manipulate raster graphics.
Core Interface Components:
Canvas Workspace: Central area displaying the active pixel matrix.
Tool Toolbar: Panel containing selection, painting, retouching, and
vector tools.
Option Panel: Dynamic menu displaying parameter settings for the
currently active tool.
Layer/History Panels: Workspace control windows managing stack
ordering, masks, and edit histories.

Raster vs. Vector Editing: Raster tools process individual pixel
color values, whereas vector operations manage
resolution-independent mathematical paths.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

157 / 533

Geometric Operations: Cropping and Scaling I

Image Cropping:
Process of removing outer unwanted areas from a digital image.
Enhances composition, adjusts aspect ratios (e.g., 16:9, 4:3, 1:1), and
improves visual focus.
Utilizes the Rule of Thirds grid overlays for optimal composition
framing.

Image Scaling (Resampling):
Changing physical image dimensions or pixel density.
Downsampling: Decreasing pixel counts; uses spatial averaging to
preserve sharpness.
Upsampling: Increasing pixel counts; requires spatial interpolation
algorithms (Nearest Neighbor, Bilinear, Bicubic).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

158 / 533

Image Straightening and Framing Techniques I

Horizon Straightening:
Corrects perspective skew and tilted horizons resulting from camera
misalignments.
Employs ruler alignment tools or automated perspective grid
correction algorithms.
Requires post-rotation cropping to eliminate blank canvas borders.

Digital Image Framing:
Adding structural borders, vignettes, or visual margins around an
image canvas.
Border Creation: Canvas expansion via pixel padding or stroke layer
effects.
Vignetting: Radial gradient masking to darken image edges and draw
viewer focus inward.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

159 / 533

Image Enhancement: Brightness & Contrast I

Luminance Manipulation:
Brightness: Uniform scalar addition or subtraction across all pixel
RGB intensity values.
Formula: Iout (x, y ) = Iin (x, y ) + β, where β is the brightness offset.

Contrast Adjustment:
Expands or contracts the dynamic range between light and dark pixel
values.
Linear scaling formula: Iout (x, y ) = α · Iin (x, y ) + β, where α > 1
enhances contrast.

Histogram Equalization:
Non-linear technique redistributing pixel intensity frequencies across
the full dynamic scale.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

160 / 533

Correction of Exposure and Saturation Defects I

Correcting Dull (Low Contrast) Images:
Identified by narrow, clustered histograms lacking deep blacks or
bright whites.
Remediation via Level Clipping (stretching black/white sliders) and
Tone Curve adjustment.

Handling Oversaturated Images:
Caused by excessive chromatic intensity, leading to clipped color
channels and lost texture details.
Remediation via HSL (Hue, Saturation, Lightness) controls, reducing
specific channel saturation values.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

161 / 533

Noise Reduction and Smoothing Filters I

Types of Digital Image Noise:
Luminance Noise: Random variation in brightness values, appearing
as graininess.
Chrominance Noise: Random color flecks, typically caused by high
ISO sensitivity or sensor heat.

Noise Reduction Techniques:
Gaussian Filtering: Blurs noise using a weighted 2D spatial kernel,
preserving overall structures.
Median Filtering: Replaces pixel values with local window medians;
highly effective against salt-and-pepper noise.
Bilateral Filtering: Edge-preserving smoothing filter that weights
pixels by spatial distance and radiometric similarity.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

162 / 533

Specialized Retouching: Red-Eye Removal I

Origin of the Red-Eye Effect:
Reflection of camera flash light off the vascularized retina through a
dilated pupil in low light.

Detection and Correction Workflow:
Pupil Localization: Identifies circular regions with high red channel
dominance (R ≫ G , B).
Desaturation & Darkening: Converts red-dominant pixels to neutral
gray/black tones while preserving catchlights.
Feathering: Blends corrected pupil boundaries smoothly into
surrounding iris structures.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

163 / 533

Image Restoration and Damage Repair I

Digital Inpainting Techniques:
Reconstructing lost, degraded, or unwanted parts of an image
seamlessly with surrounding context.

Core Repair Tools:
Clone Stamp Tool: Samples exact pixel coordinates from a target
source region onto damaged zones.
Healing Brush Tool: Blends source textures with target color and
luminance values for smooth transitions.
Content-Aware Fill: Algorithmically synthesizes replacement textures
based on global image features.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

164 / 533

Adding and Styling Text on Images I

Text Engine Fundamentals:
Text is inserted as resolution-independent vector layers rendered over
raster backgrounds.

Typography Parameters:
Font Selection & Kerning: Choosing appropriate typefaces and
adjusting character spacing.
Alignment & Leading: Managing line height and structural text
placement.

Visual Legibility Controls:
Adding drop shadows, outer glows, and stroke outlines to ensure
readability over complex backgrounds.
Applying semi-transparent background overlays or gradient masks
behind text blocks.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

165 / 533

Layer Architecture and Non-Destructive Editing I

Concept of Image Layers:
Stacked transparent sheets containing separate visual elements,
enabling isolated editing.

Layer Types & Properties:
Pixel Layers: Standard raster content layers.
Adjustment Layers: Non-destructive color/contrast modification
layers applied to underlying stacks.
Blending Modes: Mathematical formulas (e.g., Multiply, Screen,
Overlay) determining pixel composite interaction.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

166 / 533

Working on Part of an Image: Selections and Masks I

Selection Tools:
Isolating target pixel coordinates for localized editing without altering
unselected regions.
Marquee & Lasso: Geometric and freehand polygonal boundary
selection tools.
Magic Wand & Quick Selection: Color/luminance tolerance-based
pixel selection algorithms.

Layer Masking:
Non-destructive visibility control using grayscale alpha maps (White =
visible, Black = hidden, Gray = transparent).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

167 / 533

Summary and Best Practices in Image Editing I

Key Takeaways:
Mastery of basic tools, geometrical edits, color enhancement, and
restoration techniques.
Layer-based workflows enable flexible and non-destructive image
manipulation.

Professional Editing Best Practices:
Always retain original raw/uncompressed source files before editing.
Utilize adjustment layers and masks rather than permanent pixel
destructions.
Maintain realistic saturation and sharpness levels to prevent artificial
artifacting.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

168 / 533

Overview of Image Editing Tools & Interface I
Image Editing Concept: The process of altering, enhancing, or
restoring digital images using specialized software.
Core Interface Components:
Canvas/Workspace: Central area displaying the active digital image.
Tool Palette: Contains selection, retouching, vector, and navigation
controls.
Options Bar: Context-sensitive bar displaying settings for the active
tool.
Panels: Dockable modules for managing layers, history, color
channels, and adjustments.

Primary Tool Categories:
Selection Tools: Marquee, Lasso, Magic Wand, Quick Selection.
Transform Tools: Crop, Scale, Rotate, Skew, Perspective.
Retouching Tools: Clone Stamp, Healing Brush, Spot Healing, Patch
tool.
Drawing Tools: Brush, Pencil, Gradient, Eraser.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

169 / 533

Geometric Transformations: Straightening, Cropping,
and Scaling I
Image Straightening:
Adjusts tilted horizons or skewed architectural lines.
Uses ruler tools or auto-detection to rotate the canvas to a reference
angle.
Automatically clips peripheral empty pixels resulting from rotation.

Cropping:
Removes unwanted outer areas to improve subject focus and
composition.
Follows composition guides such as the Rule of Thirds or Golden
Ratio.
Constrains output to standardized aspect ratios (e.g., 1:1, 16:9, 4:3).

Scaling & Resampling:
Scaling: Alters physical dimensions or target print resolution.
Downsampling: Decreases total pixel count, discarding data via
interpolation algorithms.
Upsampling: Increases pixel count by interpolating new pixels
(requires edge-preserving algorithms).
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

170 / 533

Framing Techniques and Image Composition I
Concept of Digital Framing:
Adding structural borders, vignettes, or graphical frames to direct
viewer focus.
Encapsulating image contents for aesthetic presentation or digital
layouts.

Methods of Framing:
Canvas Expansion: Increases canvas size around image bounds to
create solid or styled borders.
Vignetting: Subtle darkening or lightening of peripheral corners to
emphasize central elements.
Overlay Masks: Applying vector paths or layer masks to frame
images within geometric shapes.

Applications:
Digital photography presentation, web publishing, and graphic design
assets.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

171 / 533

Image Enhancement: Brightness and Tonal
Adjustments I
Fundamental Tonal Adjustments:
Brightness: Uniform shift in global luminance across all pixel values.
Contrast: Expands or contracts the dynamic range between shadows
and highlights.

Tonal Mapping Tools:
Levels Histogram: Enables explicit mapping of Black Point, Midtones
(Gamma), and White Point.
Curves Adjustment: Provides non-linear control across tonal ranges
via editable control points.

Preserving Dynamic Range:
Prevents highlight clipping (pure white loss) and shadow clipping
(crushed darks).
Uses histogram analysis to achieve balanced distribution of luminance
values.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

172 / 533

Correcting Dull and Oversaturated Images I

Correcting Dull Images:
Characteristics: Low contrast, narrow histogram distribution, flat
colors.
Correction Workflow:
Expand tonal range using Levels slider to span full dynamic range.
Apply an S-Curve transformation in Curves to boost contrast.
Selectively increase Vibrance and Saturation levels.

Correcting Oversaturated Images:
Characteristics: Harsh chromatic intensity, loss of subtle texture
detail in vivid areas.
Correction Workflow:
Reduce global or selective color saturation via Hue/Saturation controls.
Adjust targeted color channels using HSL (Hue, Saturation,
Luminance) sliders.
Map colors into target output color spaces (e.g., sRGB).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

173 / 533

Noise Reduction Techniques in Digital Images I
Digital Noise Types:
Luminance Noise: Fine, grain-like variations in brightness across
uniform regions.
Chrominance (Color) Noise: Random colored pixels (typically red,
green, or blue speckles).

Causes:
High ISO settings, low-light exposures, sensor heat, or excessive
digital amplification.

Denoising Algorithms:
Spatial Filtering: Median, Gaussian, and Bilateral edge-preserving
filtering.
Frequency Denoising: Separating high-frequency noise from
structural edge information.
AI Noise Reduction: Machine learning models that suppress artifacts
while retaining fine details.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

174 / 533

Specialized Corrections: Removing Red-Eye I

Cause of Red-Eye:
Occurs in flash photography when light reflects off the vascular retina
in low-light environments.

Algorithmic Correction:
Detection: Identifies circular regions with high red-channel intensity
within detected pupil bounds.
Desaturation & Darkening: Decreases red saturation and reduces
luminance values in targeted pixels.
Reflection Preservation: Retains bright specular catchlights to
ensure natural pupil appearance.

Manual Correction:
Isolating the pupil area with a fine selection tool and reducing red
saturation manually.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

175 / 533

Image Repair and Restoration Techniques I
Target Defects:
Scratches, dust spots, tears, creases, and physical aging artifacts on
scanned media.

Restoration Tools:
Clone Stamp Tool: Samples source pixels from an intact area and
paints them onto damaged regions.
Healing Brush Tool: Blends source texture with destination lighting
and shading.
Spot Healing Brush: Automatically computes replacement pixels
from surrounding local context.
Content-Aware Fill: Uses pattern matching to synthesize coherent
replacement structures.

Workflow Principle:
Conduct repair work on dedicated blank layers set to sample
underlying data for non-destructive edits.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

176 / 533

Adding and Formatting Text in Images I

Text Formats in Image Editors:
Vector Text Layers: Preserves crisp rendering and full editability
regardless of zoom level.
Rasterized Text: Converts typography into fixed pixel data for
applying raster filters.

Typographic Controls:
Typeface & Weight: Setting visual hierarchy and readability.
Kerning & Tracking: Adjusting spacing between individual letter
pairs and entire character strings.
Leading: Controlling line height for multi-line text blocks.

Text Effects:
Non-destructive layer styles (drop shadows, strokes, bevels) and text
warping along custom paths.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

177 / 533

Layer Architecture and Non-Destructive Editing I
Layer Concept:
Stacked transparent planes containing independent image elements,
adjustments, or graphic objects.

Layer Types:
Pixel Layers: Standard raster content.
Adjustment Layers: Applies non-destructive color/tone modifications
to all underlying layers.
Vector/Type Layers: Contains editable paths and vector graphics.
Smart Objects: Container layers preserving source data during
transformations and filtering.

Blending Modes & Opacity:
Opacity: Controls transparency level (0%–100%).
Blending Modes: Mathematical algorithms (e.g., Multiply, Screen,
Overlay) governing layer interactions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

178 / 533

Localized Editing: Working on Part of an Image I

Purpose of Localized Adjustments:
Isolates specific regions for color, tone, or retouching work without
altering adjacent areas.

Selection Tools:
Geometric: Rectangular and Elliptical Marquee tools.
Freehand: Lasso, Polygonal Lasso, Magnetic Lasso.
Color/Edge Based: Magic Wand (tolerance-driven), Quick Selection,
AI Subject Selection.

Masking and Softening:
Feathering: Blurs selection boundaries for smooth integration.
Layer Masks: Grayscale masks where white reveals, black conceals,
and gray provides partial transparency.
Quick Mask Mode: Paints selections directly using standard brush
tools.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

179 / 533

Lecture 16: Image Editing – Course Overview &
Learning Objectives I

Unit Focus: Digital Image Editing Concepts, Tools, and Workflows.
Course Module: DI04000051 – Lecture 16.
Key Learning Objectives:
Navigate and customize modern digital image editing workspace
interfaces.
Master fundamental geometric operations: cropping, straightening,
scaling, and framing.
Apply tonal and chromatic corrections: brightness adjustment, dull
image enhancement, and saturation control.
Perform digital restoration: noise reduction, red-eye removal, and
artifact repairing.
Utilize layers, non-destructive workflows, typography, and precise area
selections.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

180 / 533

Image Editing Tool Interface & Environment I
Primary Interface Components:
Menu Bar: Global commands, plugin filters, view preferences, and file
management.
Tool Toolbar: Access point for selection, painting, retouching, and
vector tools.
Options / Properties Bar: Dynamic context-sensitive parameters for
the currently active tool.
Canvas Display Workspace: Main preview viewport with configurable
grids, guides, and rulers.
Panel Dock: Collapsible windows for Layers, Channels, Paths,
History, and Adjustments.

Workspace Customization & Ergonomics:
Configurable keyboard shortcuts for high-efficiency editing pipelines.
Color management setup: sRGB, Adobe RGB (1998), and Display P3
profile matching.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

181 / 533

Categorization of Core Image Editing Tools I

Selection Tools: Marquee (Rectangular/Elliptical), Lasso
(Freehand/Polygonal/Magnetic), Quick Selection, and Magic Wand.
Crop and Transform Tools: Crop, Perspective Crop, Slice,
Artboard, and Ruler tool for straightening.
Retouching and Repair Tools: Spot Healing Brush, Healing Brush,
Patch Tool, Content-Aware Move, and Clone Stamp.
Paint and Draw Tools: Brush, Pencil, Color Replacement, Gradient,
Paint Bucket, and Eraser.
Vector and Type Tools: Type Tool (Horizontal/Vertical), Pen Tool
(Path/Shape creation), and Direct Selection.
Navigation and Utility Tools: Hand Tool, Zoom Tool, Eyedropper
(Color Sampler), and Measure Tool.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

182 / 533

Spatial Transformations: Straightening, Cropping, &
Scaling I
Image Straightening:
Corrects horizon tilt and rotational distortion using reference line
algorithms.
Automated edge fill using Content-Aware technology or controlled
canvas trimming.

Cropping & Composition:
Eliminates unnecessary peripheral elements to refine image
composition.
Adheres to standard aspect ratios (e.g., 16:9, 4:3, 1:1) and
compositional rules (Rule of Thirds, Golden Ratio).

Scaling & Resampling:
Downsampling vs. Upsampling: Interpolation algorithms (Bicubic
Sharper, Bicubic Smoother, Nearest Neighbor).
Distinguishing image dimension alterations (pixels) from physical print
size alterations (DPI/PPI).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

183 / 533

Framing Techniques & Canvas Boundary
Adjustments I
Canvas Expansion:
Adding positive margin borders around the original image dimensions
without altering source pixel density.
Setting explicit background colors, matte values, or transparent
padding.

Digital Framing Methods:
Vignetting: Subtly darkening or lightening image corners to draw
focus inward.
Decorative and Structural Borders: Using layer stroke effects, inner
shadows, and vector masks.

Mockup & Smart Object Framing:
Embedding edited images into real-world device frames or physical
picture frames using perspective transformations.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

184 / 533

Tone & Exposure Enhancements: Brightness &
Contrast I
Brightness and Contrast Mechanics:
Brightness shifts the overall tonal values across the intensity spectrum
linearly.
Contrast expands or compresses the tonal distance between light
highlights and dark shadows.

Histogram-Based Corrections:
Evaluating tonal distribution: Identifying underexposed, overexposed,
and flat dynamic ranges.

Advanced Tonal Adjustments:
Levels Adjustment: Setting Black Point, White Point, and Midtone
Gamma (γ) controls.
Curves Adjustment: Non-linear control over luminance curves for
precise highlight, shadow, and midtone tuning.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

185 / 533

Correcting Dull Images & Oversaturation I

Revitalizing Dull Images:
Cause: Low dynamic range, poor contrast, and flat lighting during
acquisition.
Solution: Stretch input black/white points via Levels, apply S-curves
in Curves, and enhance local clarity/micro-contrast.

Managing Oversaturated Images:
Cause: Unnatural color clipping, loss of fine texture details in vivid
regions.
Distinction: Hue/Saturation (global linear multiplier) vs. Vibrance
(smart saturation protecting muted colors and skin tones).
Targeted Color Desaturation: Selective HSL adjustments to isolate
and attenuate clipping color channels.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

186 / 533

Image Restoration: Denoising & Red-Eye Removal I

Digital Noise Reduction:
Luminance Noise: High ISO grain effect; corrected via spatial
averaging and bilateral filtering.
Chrominance Noise: Random color specks; corrected via color
channel smoothing without destroying edge sharpness.
Balance: Avoiding over-smoothing, which leads to artificial waxy
textures.

Red-Eye Removal Algorithm:
Cause: Flash reflection off the vascular choroid layer behind the retina
in low ambient light.
Mechanism: Automatic pupil detection, targeted desaturation of red
hues, and controlled pupil darkening.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

187 / 533

Advanced Image Repair & Retouching Techniques I

Healing Tools:
Spot Healing Brush: Uses texture synthesis from surrounding pixels
to eliminate small blemishes automatically.
Healing Brush Tool: Blends source texture with destination lighting
and shading.

Patch & Content-Aware Fill:
Replaces large unwanted regions by sampling contextually similar
image regions.

Clone Stamp Tool:
Exact pixel duplication from a defined source sampling point to a
target location.
Essential for reconstructive retouching, seam suppression, and pattern
repetition.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

188 / 533

Typography & Vector Elements in Image Editing I

Adding Text to Images:
Vector-based text layers maintain sharp rendering at any scaling
resolution.
Point Text vs. Paragraph/Area Text containers for headline vs. body
copy.

Typographic Properties:
Font Family Selection, Weight (Light/Regular/Bold), Size, Kerning,
Tracking, and Leading.

Text-to-Image Integration & Style Effects:
Layer Styles: Drop Shadows, Outer Glows, Stroke Outlines, and
Bevel/Emboss.
Legibility Optimizations: Placing text over dark overlays,
semi-transparent matte boxes, or applying Gaussian blur to
background layers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

189 / 533

Layer Architecture & Non-Destructive Editing I

Fundamentals of Layers:
Stacked transparent sheets allowing independent manipulation of
visual elements.
Layer Types: Pixel Layers, Adjustment Layers, Text Layers, Shape
Layers, and Smart Objects.

Layer Operations & Organization:
Opacity and Fill percentages for transparency control.
Blending Modes (e.g., Multiply, Screen, Overlay, Soft Light) for
algorithmic pixel color interactions.
Grouping, Color Coding, and Naming for complex multi-layer
compositions.

Non-Destructive Workflow:
Retaining original image pixels untouched by using adjustment layers
and smart filters.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

190 / 533

Working on Part of an Image: Selections & Layer
Masks I

Isolated Editing via Selections:
Restricting modifications (adjustments, painting, filtering) strictly
within active selection boundaries.
Selection Refinement: Feathering (edge softening),
Expanding/Contracting, and Anti-Aliasing.

Layer Masks (Alpha Channel Masking):
Non-destructive visibility control: Grayscale values determine
transparency (White = Visible, Black = Hidden, Gray = Partial).
Seamless blending of multiple image components using soft brush
masking.

Summary: Mastering tools, color corrections, retouching, and layer
masking enables professional digital image editing workflows.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

191 / 533

Introduction to Image Editing Tools & Interface I
Overview of Image Editing Software: Digital image editing
software provides a virtual studio environment designed to manipulate
pixel data, vector graphic elements, and color profiles.
Core Graphical User Interface (GUI) Components:
Canvas / Workspace Window : The central display region where the
active image is loaded and rendered.
Tool Palette / Toolbar : Contains selection, transformation, drawing,
retouching, and vector construction tools.
Options / Control Bar : Context-sensitive bar that displays active
parameters for the selected tool (e.g., brush size, opacity, tolerance).
Panels / Docks: Modular windows managing layers, history steps,
color swatches, channels, and histogram distributions.

Primary Categories of Editing Tools:
Navigation & Viewing : Zoom tool, Hand/Pan tool, Navigator panel.
Selection Tools: Marquee, Lasso, Magic Wand, Quick Selection, and
Object Selection.
Retouching Tools: Spot Healing, Clone Stamp, Patch tool, Eraser.
Vector & Text Tools: Pen tool, Direct Selection, Type tool, Shape
tools.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

192 / 533

Basic Geometric Manipulations: Straightening,
Cropping, and Scaling I
Image Straightening:
Corrects horizon tilt or vertical misalignment caused by camera
orientation during acquisition.
Mechanism: Calculates rotation angle θ relative to a user-defined
reference line and applies bilinear or bicubic interpolation to rotate
pixel grid.

Cropping:
Process of removing unwanted outer areas from an image to improve
framing, aspect ratio, or focus.
Modifies image dimensions (W × H) while maintaining native pixel
density in remaining region.
Incorporates composition rules such as the Rule of Thirds or Golden
Ratio.

Image Scaling (Resampling):
Upscaling (Interpolation): Increases pixel count using algorithms like
Nearest Neighbor, Bilinear, Bicubic, or AI-driven super-resolution.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

193 / 533

Basic Geometric Manipulations: Straightening,
Cropping, and Scaling II

Downscaling (Decimation): Reduces resolution by filtering
high-frequency details to prevent aliasing (Moiré patterns).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

194 / 533

Framing and Compositional Adjustments I

Framing Techniques:
Digital Borders & Matting : Adding geometric margins or color frames
around the canvas for presentation.
Vignetting Effects: Intentional darkening or lightening of image edges
to draw viewer attention toward the central subject.

Compositional Alignment:
Utilizing rule-of-thirds grid overlays during cropping to align key
interest points.
Perspective Correction: Adjusting 2D perspective planes (Keystoning)
to correct converging parallel lines in architectural photography.

Aspect Ratio Standardization:
Constraining scale and crop ratios to standard target displays (e.g.,
16:9, 4:3, 1:1, 3:2) for digital publishing and print workflows.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

195 / 533

Image Enhancement: Brightness and Contrast
Correction I
Understanding Image Histograms:
Graphical representation of tonal distribution showing pixel counts
across intensity values from 0 (black) to 255 (white).

Brightness Adjustments:
Shifts pixel values linearly across the intensity dynamic range:
g (x, y ) = f (x, y ) + c.
Excessive positive shift causes highlight clipping (pure white
saturation); negative shift leads to shadow crushing.

Contrast Adjustments:
Expands or contracts the tonal span between dark and light regions:
g (x, y ) = a · f (x, y ) + b.
High contrast increases separation between shadows and highlights;
low contrast compresses tones toward mid-gray.

Non-Linear Tonal Curves & Levels:
Levels Adjustment: Controls input/output shadow, highlight, and
midtone gamma (γ) points.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

196 / 533

Image Enhancement: Brightness and Contrast
Correction II

Curves Adjustment: Allows precise point-by-point mapping across the
tonal curve S = T (r ).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

197 / 533

Enhancing Dull and Oversaturated Images I
Correcting Dull / Low-Contrast Images:
Characteristics: Histogram is concentrated in a narrow band (typically
midtones), lacking true blacks or bright whites.
Remediation: Apply Histogram Equalization or stretch input
black/white points using Levels/Curves.
Dehaze Algorithms: Contrast-based enhancement to eliminate
atmospheric haze and regain local contrast.

Managing Oversaturated Images:
Characteristics: Excessive color intensity leading to loss of texture
detail in specific color channels.
Saturation vs. Vibrance:
Saturation: Linear scaling of overall color intensity; easily clips already
vivid colors.
Vibrance: Smart non-linear adjustment that targets muted tones while
protecting skin tones and already saturated colors.

Selective HSL Adjustments: Modifying Hue, Saturation, and
Luminance on individual color channels (e.g., Red, Green, Blue, Cyan,
Magenta, Yellow).
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

198 / 533

Noise Reduction and Image Denoising Techniques I
Types of Digital Image Noise:
Luminance Noise: Random variation in brightness creating a grainy
texture (similar to film grain).
Chrominance (Color) Noise: Random speckles of unnatural color (red,
green, blue artifacts) visible in low-light photos.

Denoising Algorithms & Spatial Filters:
Gaussian Blur Filter : Smooths noise but degrades sharp edge detail.
Median Filtering : Effective for salt-and-pepper noise removal while
preserving major boundaries.
Bilateral Filtering : Smooths planar regions while respecting edge
gradients to retain crisp details.
Wavelet & AI Denoising : Transforms image into frequency domain to
isolate noise components from structural data.

Trade-off in Denoising:
Over-filtering leads to loss of fine texture details and creates an
artificial ”plastic” appearance.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

199 / 533

Red-Eye Removal and Retouching Fundamentals I

Origin of the Red-Eye Phenomenon:
Caused by camera flash reflecting off the vascularized retina through a
dilated pupil in low-ambient lighting.

Red-Eye Removal Algorithms:
Color Detection: Identifies high-concentration red channel pixels
within a circular pupil boundary.
Desaturation & Luminance Reduction: Desaturates the red hue to
neutral gray/black while preserving catchlights (specular highlights).

Basic Portrait Retouching:
Softening skin textures while retaining pores.
Enhancing iris contrast and highlight balance.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

200 / 533

Image Repair: Healing, Patching, and Clone
Stamping I
Clone Stamp Tool:
Copies exact pixel values from a specified source point (xs , ys ) to a
target destination (xd , yd ).
Useful for duplicating structures or manual pixel reconstruction.

Spot Healing & Healing Brush Tools:
Uses algorithmic texture blending by solving Poisson partial
differential equations.
Matches the surrounding region’s lighting, shading, and texture while
replacing target pixels.

Patch Tool & Content-Aware Fill:
Enables selection-based repair of large irregular areas.
Content-Aware Technology : Analyzes surrounding image patterns and
seamlessly synthesizes matching background structures.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

201 / 533

Adding and Styling Text in Digital Images I
Text Engine Fundamentals:
Text is initially defined as vector typography (resolution-independent
outline paths).
Rendered onto the pixel raster canvas via anti-aliasing algorithms to
eliminate jagged stair-stepping edges.

Typography Control Parameters:
Font Family & Weight: Selecting appropriate typefaces (Serif,
Sans-serif, Script, Display).
Size, Leading, & Kerning : Controlling character height, line spacing,
and letter pair spacing.
Alignment & Tracking : Text justification and overall character
spacing density.

Layer Effects & Styling for Text:
Applying non-destructive visual styles such as Drop Shadow, Outer
Glow, Stroke, and Bevel & Emboss to ensure contrast over complex
backgrounds.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

202 / 533

Layer-Based Editing, Blending Modes, and
Non-Destructive Workflows I
The Layer Concept:
Stacks transparent sheets containing isolated visual elements (raster
images, text, shapes, adjustments).
Enables independent editing of image components without
permanently altering underlying data.

Layer Types:
Pixel/Raster Layers: Standard image layers containing color pixel
data.
Adjustment Layers: Non-destructive layers applying dynamic
color/tonal changes to all layers beneath them.
Vector/Smart Object Layers: Preserves original source data for
lossless scaling and filtering.

Blending Modes:
Mathematical formulas determining how pixels interact with
underlying layers (e.g., Multiply for darkening, Screen for lightening,
Overlay for contrast).
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

203 / 533

Working on Selective Regions: Selections, Masks,
and Local Adjustments I

Selection Fundamentals:
Isolates a subset of image pixels so that modifications apply
exclusively within the bounded region.
Feathering : Blurs selection edges to create a soft, smooth transition
zone (anti-aliased edge fading).

Layer Masks:
Grayscale bitmap linked to a layer controlling opacity (White = 100%
visible, Black = 100% hidden, Gray = partial transparency).
Core component of non-destructive editing workflows.

Local Adjustment Techniques:
Combining selections or masks with adjustment layers to perform
targeted dodge, burn, color grading, or sharpening.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

204 / 533

Lecture Overview & Learning Objectives I

Unit Context: Image Editing and Computational Photography.
Lecture Focus: Integration of Artificial Intelligence (AI) plugins into
standard image editing software.
Key Learning Objectives:
Understand the software engineering architecture of host-plugin
interfaces.
Contrast traditional digital image processing with deep-learning-driven
pipelines.
Analyze AI super-resolution techniques and loss functions.
Examine semantic segmentation, trimap generation, and alpha
matting algorithms.
Study deep generative image inpainting and object removal
mechanisms.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

205 / 533

Plugin Architecture in Image Editing Software I
Host Application & Plugin Ecosystem:
Host software (e.g., Adobe Photoshop, GIMP, Affinity Photo) provides
core document management, canvas rendering, and undo/redo stacks.
Plugins act as dynamically linked libraries (.dll, .so, .dylib) or
decoupled out-of-process RPC services.

API Layer & Software Development Kits (SDKs):
Native C/C++ SDKs expose function pointers for direct image buffer
manipulation and filter registration.
High-level wrappers and IPC bridges allow external machine learning
runtimes to interface with host canvas memory.

Host-Plugin Execution Lifecycle:
Discovery and registration during host application startup.
Parameter initialization and hardware capability query (GPU
availability, VRAM allocation).
Processing phase: Tile-based pixel stream execution.
Cleanup and resource deallocation upon completion.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

206 / 533

Host-Plugin Data Transfer & Memory Management I

Memory Buffer Layouts:
Image data represented as multidimensional contiguous arrays
(H × W × C ).
Formats include 8-bit unsigned integer (uint8), 16-bit integer, and
32-bit floating-point (float32) per channel.
Strided memory representations accommodate Region of Interest
(ROI) cropping and padding.

Tiled Processing & Out-of-Process Execution:
Host applications divide high-resolution canvases into non-overlapping
tiles to constrain peak memory consumption.
Modern AI plugins execute inference in isolated worker processes via
Inter-Process Communication (Shared Memory, gRPC, Named Pipes).
Out-of-process execution prevents neural runtime exceptions (e.g.,
CUDA Out-Of-Memory) from crashing the host process.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

207 / 533

Evolution of AI in Digital Image Processing I

Traditional Digital Image Processing (DIP):
Relies on hand-crafted analytical spatial kernels (e.g., Gaussian filter,
Sobel operator, Canny edge detection).
Uses deterministic mathematical transformations (Fourier Transform,
Wavelet Decomposition, Discrete Cosine Transform).
Limitations: Inability to capture semantic context; sensitive to noise
and manual parameter tuning.

Deep Learning-Driven DIP:
Replaces fixed heuristic kernels with learnable convolutional and
transformer layers.
End-to-end data-driven mappings trained on large-scale image
datasets.
Semantic awareness: Networks perform pixel manipulation guided by
high-level scene understanding.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

208 / 533

Deep Learning Integration & Hardware Acceleration I

Machine Learning Runtimes in Plugins:
Embedding lightweight inference engines: ONNX Runtime, LibTorch,
TensorRT, CoreML, DirectML.
Model quantization: Converting FP32 parameters to FP16 or INT8
precision to maximize inference throughput.

Hardware Acceleration Pipeline:
CPU fallback using SIMD vector instructions (AVX-512, ARM
NEON).
GPU/NPU acceleration via CUDA, ROCm, Vulkan Kompute, or Apple
Metal API.
Zero-copy memory sharing between host Graphics API
(OpenGL/DirectX) and ML tensor buffers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

209 / 533

AI-Based Image Upscaling: Super-Resolution
Fundamentals I

Single Image Super-Resolution (SISR) Problem Formulation:
Given a low-resolution image ILR , reconstruct a high-resolution
estimate IˆHR such that IˆHR ≈ IHR .
Degradation model: ILR = (IHR ∗ k) ↓s +n, where k is a blur kernel, ↓s
is downsampling by scale s, and n is additive noise.

Classical Interpolation vs Deep Super-Resolution:
Classical (Bicubic, Lanczos): Smooths high-frequency content,
producing blurry edges and ringing artifacts.
Learning-based: Synthesizes plausible high-frequency textures and fine
lines based on learned natural image priors.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

210 / 533

Advanced Super-Resolution Architectures I
Evolution of SR Model Families:
SRCNN: Early 3-layer CNN operating directly in upscaled feature
space.
ESRGAN: Enhanced Super-Resolution GAN using
Residual-in-Residual Dense Blocks (RRDB) and Relativistic
Discriminators.
SwinIR: Swin Transformer blocks capturing long-range contextual
dependencies for image restoration.

Perceptual vs Reconstruction Loss Functions:
Pixel-wise Losses (MSE, L1): Encourage smooth, over-averaged
outputs with high PSNR but poor visual crispness.
Perceptual Loss: Computes feature space distance using pre-trained
deep networks (e.g., VGG-19 feature layers).
Adversarial Loss: GAN discriminators force generated outputs onto
the natural image manifold.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

211 / 533

Background and Foreground Removal: Semantic
Segmentation I
Problem Statement:
Automatically partition an input image into foreground subjects
(people, products, animals) and background regions.

Deep Architecture Components:
Encoder-Decoder Networks (e.g., U-Net, DeepLabV3+ with Atrous
Spatial Pyramid Pooling).
Feature Extractor Backbones (ResNet, MobileNet, EfficientNet)
pre-trained on large-scale segmentation datasets.

Single-Pass Binary Mask Generation:
Assigns a probability value pi ∈ [0, 1] per pixel indicating foreground
membership.
Output passed through binary thresholding to produce hard selection
masks for host editing tools.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

212 / 533

Alpha Matting and Edge Refinement I

Formulation of Image Matting:
Observed color Ii modeled as a linear combination of foreground Fi
and background Bi :
Ii = αi Fi + (1 − αi )Bi ,

αi ∈ [0, 1]

Ill-posed inverse problem: 7 unknowns per pixel (Fr ,g ,b , Br ,g ,b , α) given
only 3 known color values (Ir ,g ,b ).

Trimap-Guided and Trimap-Free Deep Matting:
Trimap: Image partitioned into Definite Foreground, Definite
Background, and Unknown transition regions (hair, fur, glass).
Modern AI Matting (e.g., MODNet, ViT-Matting): Learns trimap
generation and continuous alpha estimation end-to-end without user
manual trimaps.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

213 / 533

Image Inpainting and Object Removal: Principles I

Core Objective:
Fill missing or masked target regions Ω within an image I using
information from unmasked regions I \ Ω, preserving structural and
texture coherence.

Traditional vs Deep Learning Inpainting:
Traditional (Diffusion-based, PatchMatch): Copies surrounding pixels
or solves PDEs; fails when missing regions require semantic synthesis.
Deep Learning: Infers missing semantic content (e.g., reconstructing a
missing face component or complex background pattern).

Handling Irregular Masks:
Partial Convolutions (PConv): Re-normalizes convolution output
based only on valid unmasked pixels.
Gated Convolutions: Learns dynamic feature gating mechanisms per
channel and spatial location.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

214 / 533

Deep Generative Inpainting & Contextual Attention I

Advanced Architectural Innovations:
Contextual Attention Modules: Borrow feature patches from distant
unmasked image areas to refine local generated details.
Fast Fourier Convolutions (LaMa - Large Mask Inpainting): Uses
frequency domain convolutions to achieve an image-wide receptive
field for large mask completion.

Generative Diffusion for Inpainting:
Latent Diffusion Models (LDMs) conditioned on surrounding pixels
and optional text prompts.
Iterative denoising process in latent space guided by mask boundaries
for seamless edge integration.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

215 / 533

Summary and Future Horizons in AI Image Plugins I

Key Takeaways:
Host-plugin API architecture facilitates low-latency data passing and
hardware-accelerated deep learning execution.
AI Super-Resolution relies on perceptual losses and generative models
to surpass classical bicubic limits.
Semantic segmentation and alpha matting resolve fine transparent
boundaries (hair, glass) in real time.
Image inpainting leverages global receptive fields (FFCs) and diffusion
models for complex object removal.

Future Trends:
On-device real-time NPU execution.
Multimodal diffusion and text-guided neural layer editing.
Non-destructive neural parameter adjustments embedded directly in
host non-linear editing graphs.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

216 / 533

Lecture 19: AI Plugins for Image Editing – Overview I

Unit Context: Image Editing and Digital Processing Workflows
Primary Objective: Understand the computational integration of AI
models into modern image editing software via plugin architectures.
Key Topics Covered:
Architecture of image editing software plugins (IPC, API wrappers,
host interfaces).
Evolution of AI in digital image processing (Traditional heuristics vs.
Deep Learning).
Deep Learning Super-Resolution (SRCNN, ESRGAN, SwinIR).
Automated background/foreground segmentation and alpha matting
(MODNet, DeepLab).
Image inpainting and generative object removal (LaMa, Contextual
Attention, Latent Diffusion).
Hardware acceleration, model quantization, and ethical deployment.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

217 / 533

Plugin Architecture in Image Editing Software I
Host Application Interface:
Host software (e.g., Photoshop, GIMP, Affinity) exposes Application
Programming Interfaces (APIs) for feature extensions.
Legacy plugins relied on synchronous C/C++ native dynamic libraries
(.dll, .so, .plugin).

Inter-Process Communication (IPC):
Heavy deep learning models require isolated runtimes (Python,
PyTorch, C++ ONNX Runtime).
Modern AI plugins operate in separate worker processes
communicating via gRPC, named pipes, or Shared Memory (SHM).

Zero-Copy Memory Buffers:
High-resolution images (8K, 16K) present immense memory copy
overhead between host process and AI worker process.
Shared memory memory-mapped files or DirectX/Vulkan/CUDA
shared handles enable zero-copy tensor transfers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

218 / 533

Host-Plugin Data Pipeline for Deep Learning I

Execution Lifecycle:
Host Stage: User selects region of interest (ROI) or layer; host locks
memory buffer.
IPC Bridge Stage: Host transmits metadata (dimensions, pixel
format, bounding boxes) and SHM pointers to the plugin process.
Plugin Stage: Ingestion, color space normalization (e.g., sRGB to
float32 [0, 1]), tensor construction.
Inference Stage: Execution across compute hardware via ONNX
Runtime, TensorRT, or CoreML.
Writeback Stage: Inverse transformation of output tensor, writing
directly back to host pixel buffer.

Non-Destructive Editing Integration:
Output rendered as smart filters, layer masks, or parametric
adjustment nodes rather than rasterizing destructively.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

219 / 533

Overview of AI in Digital Image Processing I

Evolution of Image Editing Techniques:
First-Generation (Classical Signal Processing): Convolution kernels,
spatial domain transformations, Fourier domain filtering (e.g., bicubic
upscaling, Gaussian blur, Laplacian sharpening).
Second-Generation (Hand-Crafted Features): Edge detectors
(Canny), SIFT/SURF, graph-cut segmentation, and PatchMatch
non-parametric synthesis.
Third-Generation (Deep Learning & Generative AI): Convolutional
Neural Networks (CNNs), Vision Transformers (ViTs), and Latent
Diffusion Models (LDMs).

Shift in Paradigm:
Transitioning from pixel-level algorithmic manipulation to high-level
semantic understanding and perceptual loss optimization.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

220 / 533

Capabilities of AI Plugins: Super-Resolution Models I

Definition: Reconstructing a high-resolution image IHR from a
low-resolution input ILR , solving an ill-posed inverse problem.
Model Architectures:
SRCNN: Early end-to-end CNN learning direct spatial mappings
between low and high resolutions.
ESRGAN: Enhanced Super-Resolution Generative Adversarial
Networks using Residual-in-Residual Dense Blocks (RRDB) and
Relativistic GAN loss.
SwinIR: Restormer and Vision Transformer architectures using
self-attention to capture long-range spatial dependencies.

Artifact Mitigation: Eliminating checkerboard artifacts caused by
transposed convolutions via sub-pixel convolution (PixelShuffle)
layers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

221 / 533

Super-Resolution Loss Functions and Evaluation I

Mathematical Formulation:
Pixel Loss: L1 = ∥IHR − IˆHR ∥1 (Yields high PSNR but produces blurry
textures).
Perceptual Loss: Lpercep = ∥ϕj (IHR ) − ϕj (IˆHR )∥22 , measured using
feature maps of pretrained VGG networks.
Adversarial Loss: LGAN , forcing the generator to synthesize fine
high-frequency details indistinguishable from natural images.

Evaluation Metrics:
Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index
(SSIM).
Perceptual metrics: Learned Perceptual Image Patch Similarity
(LPIPS) and Fréchet Inception Distance (FID).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

222 / 533

Background and Foreground Removal: Segmentation
I

Semantic vs. Instance vs. Panoptic Segmentation:
Pixel-wise classification dividing image content into background
classes and target foreground entities.

Deep Learning Architectures:
U-Net & DeepLabV3+: Encoder-decoder networks using Atrous
Spatial Pyramid Pooling (ASPP) to capture multi-scale context.
Segment Anything Model (SAM): Promptable vision foundation
model enabling real-time zero-shot segmentation masks in editing
plugins.

Plugin Workflow:
User provides a bounding box or point click; SAM generates
high-precision polygonal alpha masks automatically.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

223 / 533

Alpha Matting and Fine Feature Extraction I

The Matting Problem:
Hard binary masks (I = F · α + B · (1 − α) with α ∈ {0, 1}) fail on
semi-transparent structures like hair, fur, and glass.
Alpha Matting solves for continuous transparency values α ∈ [0, 1].

Trimap-Based vs. Trimap-Free Networks:
Classical plugins required user-defined trimaps (Foreground,
Background, Unknown).
Modern AI matting networks (e.g., MODNet, ViTMatte) perform
trimap-free matte generation by decoupling scene semantics from
detail matting.

Integration in Plugins:
Instant automatic edge refinement without manual brush fine-tuning.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

224 / 533

Image Inpainting: Classical vs. Deep Learning I

Goal: Filling missing or removed regions Ω in an image I with
visually plausible background content.
Classical Patch-Based Methods:
PatchMatch: Fast nearest-neighbor field matching that samples
existing textures from unmasked regions.
Limitations: Incapable of hallucinating novel structural components
(e.g., completing a face or complex architectural geometric patterns).

Deep Learning Inpainting:
Partial Convolutions (PConvs): Re-normalizing convolution
operations based exclusively on valid non-masked pixels.
Contextual Attention: Learning long-range correlations to copy
background feature representations into target hole areas.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

225 / 533

Generative Inpainting and Diffusion-Based Removal I

Large Mask Inpainting (LaMa):
Utilizes Fast Fourier Convolutions (FFCs) with global receptive fields
to achieve state-of-the-art broad area background synthesis.

Latent Diffusion Inpainting Models:
Input image x converted to latent code z; mask m concatenated along
latent channel dimensions.
Reverse diffusion process guided by text prompts or unconditional
context predicts noise ϵθ (zt , t, c).
Enables contextual replacement, interactive object replacement, and
complex structure generation.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

226 / 533

Edge Inference, Optimization & Hardware
Acceleration I

Inference Optimization Pipelines:
Model Quantization: FP32 to FP16 / INT8 parameter conversion to
minimize VRAM usage and maximize throughput.
Graph Optimizations: Operator fusion (e.g., Conv + ReLU merging)
and memory allocation optimizations via TensorRT or ONNX
Runtime.

Cross-Platform API Acceleration:
DirectML (Windows), Metal Performance Shaders (macOS), and
CUDA/ROCm (Linux).

Latency vs. Quality Trade-offs:
Real-time preview modes (lightweight CNNs / reduced diffusion steps)
versus final render modes (full-precision diffusion / SwinIR).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

227 / 533

Ethical Considerations and Future Plugin
Architectures I

Provenance and Verification:
Integration of Coalition for Content Provenance and Authenticity
(C2PA) metadata standards into plugin output buffers.
Imperceptible digital watermarking (e.g., Stable Signature, SynthID)
to track synthetic manipulations.

Future Trends:
Hybrid cloud-edge execution: Offloading heavy multi-billion parameter
foundation models while performing low-latency spatial previews
locally.
Multi-modal editing interfaces: Natural language guided layer
manipulation, depth-aware composition, and 3D-aware neural
rendering plugins.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

228 / 533

Lecture 20: Image Editing – Overview & Objectives I
Unit Focus: Modern paradigms in Digital Image Editing and
automated processing.
Topic Scope: AI Plugins for Image Editing, Plugin Architecture,
Deep Learning Integration, and Practical Capabilities.
Core Learning Objectives:
Understand the software plugin architecture enabling third-party AI
integration.
Trace the transition from deterministic image processing to AI-driven
techniques.
Examine AI super-resolution algorithms (ESRGAN, SwinIR) and
upscaling mechanics.
Analyze segmentation and alpha matting models for
background/foreground extraction.
Explore generative inpainting frameworks (GANs, Diffusion Models)
for object removal.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

229 / 533

Introduction to Plugin Architecture in Image Editing
Software I
Definition of Plugin Architecture:
A modular software design allowing external dynamic link libraries
(DLLs/Shared Objects) to extend host application functionality
without recompiling core source code.

Host-Plugin Data Flow Pipeline:
Host Application: Manages document state, UI rendering, undo
history stack, and memory buffers.
API / SDK Interfacing: Exposes raw pixel data pointers, color space
profiles (RGB, LAB, CMYK), selection masks, and layer metadata.
Plugin Process: Receives buffer, executes transformation algorithm,
returns modified pixel buffer to host.

Host-Plugin Communication Mechanisms:
Direct Shared Memory / In-Process DLL Calls (legacy C/C++ APIs
like Adobe Photoshop Filter SDK).
Inter-Process Communication (IPC) & REST APIs (modern isolated
sandboxed plugins using WebAssembly, Python bindings, or gRPC).
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

230 / 533

AI-Specific Plugin Infrastructure & Acceleration I
Challenges of Traditional Plugins for AI Models:
High compute requirements, large neural network weights, and strict
low-latency execution constraints.

Modern AI Plugin Stack:
Inference Engines: ONNX Runtime, TensorRT, CoreML, OpenVINO
embedded directly inside plugin binaries.
Hardware Abstraction Layer: Direct compute access to GPU/NPU
via CUDA, Metal, or Vulkan APIs.
Asynchronous Threading: Offloading heavy inference calls off the
main UI rendering thread to keep host UI responsive.

Deployment Models:
Local Execution: On-device inference ensuring user privacy and
offline functionality.
Cloud-Assisted Inference: API calls to remote server clusters for
high-parameter models (e.g., Stable Diffusion, Adobe Firefly).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

231 / 533

Overview of AI in Digital Image Processing I

Paradigm Shift in Image Editing:
Traditional DSP: Hand-crafted spatial filters, convolution kernels
(Sobel, Gaussian), frequency domain transforms (FFT, Wavelets),
explicit mathematical heuristic rules.
AI-Driven Processing: Data-driven feature learning using
Convolutional Neural Networks (CNNs) and Vision Transformers
(ViTs).

Key Advantages of AI Methods:
High-level semantic context awareness (understanding objects, faces,
textures, lighting).
Non-linear, complex mapping capabilities for inverse problems
(deblurring, denoising, super-resolution).
Automated execution replacing tedious manual masking, cloning, and
color grading operations.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

232 / 533

Deep Learning Frameworks for Image Editing I
Encoder-Decoder Frameworks (U-Net):
Symmetrical architecture with contracting encoder path (feature
extraction) and expanding decoder path (reconstruction).
Skip connections retain high-resolution spatial details directly from
encoder to decoder layers.

Generative Adversarial Networks (GANs):
Generator network produces synthetic image modifications;
Discriminator evaluates perceptual authenticity.
Adversarial loss ensures sharp, realistic texture generation over
blurriness caused by pure L1 /L2 losses.

Diffusion & Attention-Based Models:
Latent Diffusion Models (LDM) guide image synthesis via iterative
denoising in latent feature space.
Self-Attention mechanisms capture global context across distant pixel
regions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

233 / 533

AI-Based Image Upscaling (Super-Resolution) I

Problem Formulation:
Recover high-resolution image IHR from low-resolution image ILR
where ILR = (IHR ∗ k) ↓s +n.
Severe ill-posed inverse problem: single low-res pixel maps to multiple
plausible high-res details.

Traditional vs. AI Upscaling:
Bicubic/Lanczos Interpolation: Smoothes pixels via spatial weighted
averaging; lacks ability to synthesize non-existent sharp edges.
Single Image Super-Resolution (SISR) Models: Synthesize plausible
high-frequency details (hair strands, fabric textures, architectural
lines).

Representative Super-Resolution Pipelines:
SRCNN (Super-Resolution CNN), Real-ESRGAN, SwinIR
(Transformer-based SR).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

234 / 533

Super-Resolution Architectures & Loss Functions I
Super-Resolution Model Design Principles:
Residual-in-Residual Dense Blocks (RRDB) to stabilize deep network
training.
Pixel-Shuffle / Sub-Pixel Convolutional layers for efficient upsampling
without checkerboard artifacts.

Multi-Component Loss Function Formulation:
Ltotal = αLpixel + βLperceptual + γLgan
Pixel Loss (L1 / L2 ): Measures pixel-wise mean absolute error;
ensures structural alignment.
Perceptual Loss (LVGG ): Computes distance between deep feature
maps from pre-trained VGG networks.
Adversarial Loss (LGAN ): Pushes generated output distribution
toward natural image distribution.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

235 / 533

AI-Driven Background & Foreground Removal I

Overview of Image Matting & Segmentation:
Isolating subject matter from complex background scenes
automatically.
Traditional methods (Chroma keying, Magic Wand, GrabCut) fail on
semi-transparent boundaries, fine hair, and complex lighting.

Mathematical Model of Alpha Matting:
Ii = αi Fi + (1 − αi )Bi ,

αi ∈ [0, 1]

Ii : Observed RGB color at pixel i.
Fi , Bi : Unknown foreground and background colors.
αi : Alpha opacity value (αi = 1 → full foreground, αi = 0 → full
background).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

236 / 533

Deep Learning Models for Background Extraction I

Semantic Segmentation Backbone Models:
U-2-Net (Nested U-structure for Salient Object Detection).
Segment Anything Model (SAM) by Meta: Zero-shot promptable
segmentation for arbitrary objects.
MODNet (Real-time Trimap-free Motion-based Matting).

Trimap-Free AI Matting Pipeline:
Stage 1: Coarse Mask Generation: High-level semantic localization
of primary subject.
Stage 2: Boundary Refinement: High-resolution detail prediction
(fine hair strands, translucent glass).
Stage 3: Alpha Map Blending: Output continuous 8-bit or 16-bit
alpha channel mask directly into host software stack.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

237 / 533

Image Inpainting & Object Removal I

Concept of Digital Image Inpainting:
Reconstructing missing, damaged, or undesirable target regions in an
image seamlessly.

Traditional vs. Deep Learning Inpainting:
Exemplar-Based Inpainting (PatchMatch): Copies surrounding pixel
patches into target region; fails on unique non-repetitive structures
and high-level semantics.
Deep Learning-Based Inpainting: Hallucinates visually plausible
content based on learned prior probability distributions of real-world
objects.

Key Challenges:
Maintaining global structural consistency and local texture fidelity.
Handling irregular masks and complex background continuity.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

238 / 533

Advanced Inpainting Architectures: GANs &
Diffusion I

Partial Convolution & Gated Convolution:
Masks invalid (missing) pixels during spatial convolution steps to
prevent color bleeding artifacts.

Fast High-Resolution Inpainting (LaMa):
Leverages Fast Fourier Convolutions (FFCs) with receptive fields
spanning the entire image for global structure restoration.

Text-Guided Diffusion Inpainting (Stable Diffusion Inpainting):
Conditioned on spatial mask and optional textual prompt (e.g.,
“replace with clean wall”).
Uses latent space cross-attention to synthesize complex,
context-aware replacements seamlessly.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

239 / 533

Summary, Future Trends, & Ethical Considerations I

Summary of Key Capabilities:
AI plugins transform host editors into intelligent creative suites via
super-resolution, automatic matting, and generative object removal.

Emerging Technical Trends:
Real-time neural execution via localized NPU hardware acceleration.
Multimodal, text-and-gesture guided dynamic image manipulation
plugins.
Non-destructive generative layering in standard software pipelines.

Ethical & Practical Considerations:
Deepfake risks, image forensics/watermarking (C2PA standard),
copyright in training datasets.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

240 / 533

Introduction to Video & Basic Concepts I
Definition of Video:
A sequence of static image frames displayed in rapid succession to
create the visual perception of continuous motion.
Based on physiological and psychological optical phenomena:
Persistence of Vision and the Phi Phenomenon.
Combines spatial data (individual pixel arrays) with temporal data
(changes occurring over time).

Core Mechanism of Motion Perception:
The human visual system processes rapid frame changes above a
critical flicker fusion threshold (≈ 16–24 frames per second).
When individual frames are rendered faster than visual decay, discrete
images blend into seamless motion.

Fundamental Dimensions of Video Signals:
Spatial Dimension: Horizontal and vertical resolution measured in
total pixels (W × H).
Temporal Dimension: Temporal sampling frequency measured in
frames per second (fps).
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

241 / 533

Introduction to Video & Basic Concepts II

Amplitude/Color Dimension: Bit depth and color space defining
amplitude precision per channel.
Data Bandwidth: Required transmission or storage capacity
measured in bit rate (kbps/Mbps).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

242 / 533

How Video Works: Spatial and Temporal Dynamics I
Spatial vs. Temporal Information:
Spatial Information: Details, textures, geometry, and color within an
individual static frame.
Temporal Information: Motion vectors, illumination changes, and
transitions across consecutive frames.

Synthesizing Continuous Motion:
Stroboscopic Effect: Rapid substitution of stationary images to
simulate continuous velocity.
Motion Blur: Natural optical integration of moving objects within
sensor exposure time, preventing motion judder.

End-to-End Video Processing Pipeline:
Capture: Lens focuses light onto CCD/CMOS optical sensors to
generate electrical charges.
Spatial-Temporal Sampling: Discretisation of spatial image plane
into pixels and time into frames.
Quantisation: Analog-to-digital amplitude conversion into binary
values.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

243 / 533

How Video Works: Spatial and Temporal Dynamics II

Encoding & Compression: Elimination of spatial and temporal
statistical redundancies.
Transport & Storage: Transmission over network channels or storage
on physical media.
Decoding & Rendering: Reconstruction and display on flat-panel
matrices.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

244 / 533

Frame Rate (FPS) & Temporal Resolution I
Definition of Frame Rate:
Measured in Frames Per Second (fps) or Hertz (Hz).
Defines the temporal sampling frequency of a video stream.

Standard Frame Rate Specifications:
24 fps: Traditional cinema standard; produces cinematic motion blur
and aesthetic texture.
25 fps / 50 fps: PAL and SECAM broadcast standards (derived from
50 Hz European AC grid).
29.97 fps / 59.94 fps: NTSC broadcast standards (modified from 30
fps to eliminate color subcarrier interference).
60 fps / 120 fps+: High Frame Rate (HFR) used in sports, gaming,
and virtual reality to minimize motion blur and input latency.

Flicker Fusion Threshold:
Frequency at which an intermittent light source appears completely
steady to the human eye (≈ 50–60 Hz).
Rates below this threshold induce visible flickering, causing visual
fatigue and eye strain.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

245 / 533

Display Resolution & Spatial Detail I
Definition of Spatial Resolution:
Total count of discrete pixels constituting each horizontal line and
vertical column (W × H).
Directly determines visual sharpness, visual acuity demands, and
image detail.

Standard Resolution Profiles:
Standard Definition (SD): 720 × 480 (NTSC 480i/p) or 720 × 576
(PAL 576i/p).
High Definition (HD / 720p): 1280 × 720 pixels (≈ 0.92
megapixels).
Full High Definition (FHD / 1080p): 1920 × 1080 pixels (≈ 2.07
megapixels).
Ultra High Definition (4K UHD): 3840 × 2160 pixels (≈ 8.29
megapixels).
Cinema 4K (DCI 4K): 4096 × 2160 pixels.
8K UHD: 7680 × 4320 pixels (≈ 33.18 megapixels).

Pixel Density & Viewing Metrics:
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

246 / 533

Display Resolution & Spatial Detail II

Pixels Per Inch (PPI): Physical concentration of pixels across display
dimensions.
Angular Acuity: Human eye resolves down to approximately 1
arcminute of visual angle.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

247 / 533

Aspect Ratio & Display Standards I
Definition of Aspect Ratio:
Proportional relationship between the width and height of an image
frame or display screen (W : H).

Standard Aspect Ratios:
4:3 (1.33:1): Legacy television standard (NTSC/PAL) and classic
computer monitors.
16:9 (1.78:1): Modern universal widescreen standard for HDTV, web
video, and modern monitors.
21:9 / 2.39:1: Anamorphic theatrical cinema format for widescreen
feature films.
1:1 & 9:16: Square and vertical formats popular in mobile platforms
and digital applications.

Aspect Ratio Matching Techniques:
Letterboxing: Black horizontal bands at top/bottom to display wide
content on narrower screens.
Pillarboxing: Black vertical bands on left/right to display 4:3 content
on 16:9 screens.
Cropping / Scaling: Non-uniform fitting or center-cropping to fill
active screen space.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

248 / 533

Bit Rate & Data Bandwidth I
Definition of Bit Rate:
Quantity of data processed per unit time in a video stream, expressed
in kbps or Mbps.

Uncompressed Bit Rate Formula:
Bit Rate = Width × Height × Frame Rate × Color Depth
Example: Uncompressed 1080p24 (24-bit RGB):
1920 × 1080 × 24 × 24 ≈ 1.19 Gbps

Bit Rate Control Strategies:
Constant Bit Rate (CBR): Fixed data allocation over time; suitable
for streaming over constrained network channels.
Variable Bit Rate (VBR): Dynamic data allocation; allocates higher
bits to complex motion scenes and lower bits to static scenes,
optimizing quality and file size.

Impact on Quality & Capacity:
Higher bit rate reduces compression artifacts (blocking, ringing).
Storage Requirement = Bit Rate × Duration.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

249 / 533

Video Codecs vs. Container Formats I
Video Codec (Coder-Decoder):
Algorithm that compresses raw video for storage/transmission and
decompresses it for display.
Employs lossy spatial (intra-frame) and temporal (inter-frame)
compression methods.
Common Codecs:
H.264 / AVC: Industry standard for compatibility.
H.265 / HEVC: High efficiency codec offering ≈ 50% bandwidth
savings over H.264.
AV1: Royalty-free open-source video encoding format.
VP9: Open video coding standard developed by Google.

Container Format (Wrapper):
File structure packaging synchronized video tracks, audio streams,
subtitles, and metadata.
Common Containers: MP4 (.mp4), Matroska (.mkv), QuickTime
(.mov), WebM (.webm), AVI (.avi).
Key Distinction: A container holds streams; a codec determines
stream compression.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

250 / 533

Analog Video Fundamentals & Signal Types I
Principles of Analog Video:
Encodes light intensity and color values as continuous analog voltage
signals over time.
Raster Scanning: Electron beam sweeps sequentially line-by-line
across cathode ray tube (CRT) displays.
Synchronization: Horizontal Sync (HSYNC) and Vertical Sync
(VSYNC) pulses align beam trajectory.

Analog Signal Standards:
Composite Video (CVBS): Luminance (Y ), chrominance (C ), and
sync signals combined on a single conductor; vulnerable to dot crawl
and color bleeding.
S-Video (Separated Video): Separates Luminance (Y ) and
Chrominance (C ) onto distinct physical pins, reducing crosstalk.
Component Video (YPbPr ): Separates signal into luminance (Y )
and two color difference signals (Pb = B − Y , Pr = R − Y ) for
maximum analog signal fidelity.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

251 / 533

Scanning Standards: Interlaced vs. Progressive I
Interlaced Scanning (e.g., 1080i, 480i):
Divides each complete frame into two fields: Field 1 (odd lines) and
Field 2 (even lines).
Fields are captured and displayed sequentially at twice the frame
refresh rate.
Advantage: Cuts temporal bandwidth requirements by 50% without
increasing visible flicker.
Disadvantage: Introduces interlace combing artifacts during rapid
lateral motion.

Progressive Scanning (e.g., 1080p, 4K):
Scans and draws all horizontal lines sequentially in a single pass top to
bottom.
Advantage: Eliminates combing artifacts; essential for modern LCD,
OLED, and digital displays.
Disadvantage: Requires twice the data bandwidth compared to
interlaced scanning at identical frame rates.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

252 / 533

Digital Video & Chroma Subsampling I
Digital Video Architecture:
Analog video is digitized via Analog-to-Digital Converters (ADC)
using spatial, temporal, and amplitude sampling.
Uses YCbCr color representation separating Luminance (Y ) from
Chrominance (Cb, Cr ).

Chroma Subsampling Schemes (J : a : b):
Exploits human visual system’s lower spatial sensitivity to color
variations than to luminance.
4:4:4: Uncompressed color; full luma and chroma resolution for each
pixel.
4:2:2: 50% horizontal chroma reduction; standard for professional
broadcast production.
4:2:0: 50% horizontal and vertical chroma reduction; standard for
consumer distribution (MP4, Blu-ray, streaming), reducing color data
by 75%.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

253 / 533

Comparative Analysis: Analog vs. Digital Video I
Signal Transmission & Fidelity:
Analog Video: Continuous voltage waveform; subject to signal loss,
electrical noise, generational degradation, and phase distortion.
Digital Video: Discrete binary data stream (0s and 1s); immune to
generational loss, supporting error correction and encryption.

Storage, Access, & Editing:
Analog: Magnetic tapes (VHS, Betacam); linear sequential access,
physical tape wear.
Digital: SSDs, flash storage, optical media, cloud storage; non-linear
editing (NLE), instant random access, bit-exact copying.

Architectural Comparison Matrix:
Parameter
Signal Structure
Storage Media
Generational Loss
Compression
Television Formats

GTU Course

Analog Video
Continuous electrical waveform
Magnetic Tape / LaserDisc
High (Degrades with each copy)
Minimal (Analog filtering)
NTSC, PAL, SECAM

Digital Video
Discrete binary stream (0s and 1s)
SSD, Hard Drive, Optical, Cloud
Zero (Bit-exact reproduction)
Advanced Codecs (H.264, HEVC, AV1)
ATSC, DVB, ISDB

DI04000051: Multimedia Technologies

Lecture Slides

254 / 533

Introduction to Video & How Video Works I

Definition of Video: A sequence of static images (frames) displayed
rapidly in succession to create the illusion of continuous motion.
Persistence of Vision: Human visual system retains an image for
approximately 1/16th to 1/10th of a second.
Phi Phenomenon & Flicker Fusion Threshold:
Phi Phenomenon: Brain perceives continuous movement between
adjacent flashing images.
Flicker Fusion: Minimum rate (≈ 24 FPS) where discrete light flashes
merge into seamless light.

Core Components of Video Signal:
Spatial Information: Individual image structure (pixels/lines, color,
brightness).
Temporal Information: Changes occurring across sequential frames
over time.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

255 / 533

Frame Rate and Scanning Techniques I
Frame Rate (Frames Per Second - FPS):
24 FPS: Standard cinematic film rate creating traditional motion blur
aesthetics.
25 / 30 FPS: Standard broadcast television rates (PAL/NTSC
systems).
60+ FPS: High Frame Rate (HFR) used in gaming, live sports, and
slow-motion video capture.

Progressive Scanning (p):
Scans every line of the frame sequentially from top to bottom (e.g.,
1080p).
Eliminates comb artifacts; optimal for modern digital displays (LCD,
OLED).

Interlaced Scanning (i):
Divides frame into two fields: odd lines (field 1) and even lines (field
2) scanned alternately.
Halves transmission bandwidth (e.g., 1080i), but introduces interlace
tearing artifacts during high motion.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

256 / 533

Analog Video Fundamentals & Standards I
Analog Video Concept: Video represented as a continuous electrical
waveform corresponding to luminance and chrominance over time.
Legacy Broadcast Standards:
NTSC (National Television System Committee): 525 lines, 29.97
FPS, 60 Hz interlaced, standard in North America & Japan.
PAL (Phase Alternating Line): 625 lines, 25 FPS, 50 Hz interlaced,
standard in Europe & Asia.
SECAM (Sécam): 625 lines, 25 FPS, 50 Hz interlaced, standard in
France & Eastern Europe.

Signal Transmission Formats:
Composite Video: Luminance (Y ) and Chrominance (C ) combined
onto a single channel/cable.
S-Video: Separate Y and C channels to reduce cross-color visual
artifacts.
Component Video (YPbPr ): Luminance and color-difference signals
split across three dedicated physical lines.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

257 / 533

Digital Video & Color Space Representation I
Digital Video Transition: Analog signals sampled spatially
(resolution) and temporally (FPS), then quantized into discrete
binary levels.
Luminance and Chrominance (YCbCr ):
Y : Luminance (brightness component, to which the human eye is
most sensitive).
Cb: Blue-difference chrominance (B − Y ).
Cr : Red-difference chrominance (R − Y ).

Chroma Subsampling Schemes:
4:4:4: Uncompressed color; full resolution for luminance and both
chrominance channels.
4:2:2: Halves horizontal chroma resolution; common in professional
acquisition.
4:2:0: Halves horizontal and vertical chroma resolution; standard for
consumer distribution (H.264, Web streaming).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

258 / 533

Video Resolution and Aspect Ratio I

Spatial Resolution: Total count of discrete pixels composing each
frame (Width × Height).
Standard Definition (SD): 720 × 480 (NTSC) / 720 × 576 (PAL).
Full High Definition (FHD): 1920 × 1080 pixels.
Ultra High Definition (4K UHD): 3840 × 2160 pixels.
8K UHD: 7680 × 4320 pixels.

Aspect Ratio: Proportional relationship between frame width and
frame height (W : H).
4 : 3: Traditional standard-definition television and CRT display
format.
16 : 9: Universal standard widescreen ratio for HDTV, desktop
monitors, and online video.
21 : 9 (2.39 : 1): Anamorphic cinematic ultrawide format.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

259 / 533

Bit Rate and Data Rate Dynamics I
Bit Rate Definition: Total amount of digital data processed per unit
of time, expressed in bits per second (bps, Mbps, Gbps).
Uncompressed Bit Rate Formula:
Bit Rate = Width × Height × Color Depth × Frame Rate
Example Calculation (1080p60, 24-bit color depth):
1920 × 1080 × 24 × 60 ≈ 2.98 Gbps (Uncompressed)
Bit Rate Control Modes:
Constant Bit Rate (CBR): Fixed data rate throughout stream;
predictable bandwidth but inefficient for storage.
Variable Bit Rate (VBR): Allocates more bits to complex action
scenes and fewer bits to static content.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

260 / 533

Video Compression Principles & Codecs I
Need for Compression: Raw digital video requires prohibitive
storage and transmission bandwidth, necessitating compression ratios
up to 100 : 1 or higher.
Codec (Coder-Decoder): Software or hardware implementation that
encodes raw video into compressed binary and decodes it for playback.
Spatial Compression (Intra-frame):
Reduces redundant image data within a single isolated frame.
Utilizes Discrete Cosine Transform (DCT) or Wavelet algorithms
(similar to JPEG).

Temporal Compression (Inter-frame):
Eliminates redundancies between consecutive frames using motion
estimation and compensation.
Uses Group of Pictures (GOP): I-frames (Intra), P-frames (Predicted),
B-frames (Bi-directional).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

261 / 533

Codecs vs. Container Formats I

Architectural Distinction:
Codec: The algorithm responsible for compressing and decompressing
the video/audio raw streams (e.g., H.264, AV1).
Container Format: The wrapper file structure that holds video/audio
streams, subtitles, and metadata together.

Industry Standard Video Codecs:
H.264 / AVC: Industry standard codec with universal hardware
acceleration support.
H.265 / HEVC: 50% higher compression efficiency than H.264;
baseline for 4K/HDR content.
VP9 & AV1: Open-source, royalty-free high-efficiency web codecs.

Common Video Containers:
MP4 (.mp4), MKV (.mkv), WebM (.webm), MOV (.mov), AVI
(.avi).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

262 / 533

Analog vs. Digital Video: Architectural Comparison I
Signal Representation:
Analog : Continuous electrical voltages; susceptible to noise,
attenuation, and generation loss.
Digital: Discrete binary values (0s and 1s); immune to noise
degradation and permits loss-free copying.

Storage & Editing Workflow:
Analog : Sequential magnetic tape media (VHS, Betacam); editing is
linear, time-consuming, and destructive.
Digital: Random-access storage (SSDs, HDDs); enables instant
Non-Linear Editing (NLE).

Transmission Characteristics:
Analog : Degrades gracefully with continuous quality decay over
distance.
Digital: Maintains perfect signal until reaching the cliff effect (abrupt
total signal breakdown).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

263 / 533

Modern Video Delivery & Adaptive Streaming I

Adaptive Bitrate Streaming (ABR):
Video source encoded into an encoding ladder of multiple resolutions
and bitrates.
Segmented into short chunks (2–10 seconds in length).
Client player dynamically switches chunk quality based on real-time
network throughput and CPU load.

Dominant Streaming Protocols:
HLS (HTTP Live Streaming): Apple-developed standard using
TS/fMP4 media fragments over HTTP.
DASH (Dynamic Adaptive Streaming over HTTP): International ISO
open standard protocol.

Emerging Video Technologies:
High Dynamic Range (HDR10, Dolby Vision), Wide Color Gamut
(Rec. 2020), and AI-driven perceptual video coding.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

264 / 533

Summary & Key Parameters Overview I

Core Terminology Synthesis:
Codec: Algorithm defining compression efficiency and system
computational load.
Resolution: Spatial pixel grid dimensions (W × H).
Bit Rate: Transmission data volume per second, directly governing
visual quality vs. file size.
Frame Rate: Temporal fluidity measured in Frames Per Second
(FPS).
Aspect Ratio: Dimensional width-to-height ratio (W : H) of the
visual frame.

Fundamental Takeaway:
Video engineering centers on optimizing multidimensional trade-offs
between spatial detail, temporal motion smoothness, available
transmission bandwidth, and computational complexity.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

265 / 533

Introduction to Video Concepts I

What is Video?
Video is a sequence of static visual images (frames) displayed in rapid
succession to create the optical illusion of continuous motion,
synchronized with continuous audio signals.
Multimodal Medium: Combines temporal visual information with
spatial structure and acoustic synchronization.
Core Dimensions:
Temporal Dimension: Frame rate and temporal sampling.
Spatial Dimension: Resolution, aspect ratio, and color depth.
Data Dimension: Bit rate, compression ratio, and storage footprint.

Transition Dynamics: Historical movement from continuous analog
broadcast signals to discrete, quantized digital bitstreams.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

266 / 533

How Video Works: Visual Perception & Frame Rate I
Biological Basis of Motion Perception
Human vision relies on two key physiological phenomena to perceive
continuous motion from static image sequences:
Persistence of Vision: The human eye retains an image on the
1
1
retina for approximately 10
to 20
of a second after visual stimulus
ceases.
Phi Phenomenon: The brain infers continuous movement between
closely spaced adjacent visual stimuli.
Frame Rate (FPS - Frames Per Second): The frequency at which
consecutive images are captured or displayed.
24 FPS: Standard cinematic frame rate (gives traditional film
aesthetic).
25 / 30 FPS: Standard broadcast television rates (PAL/SECAM at
25 FPS, NTSC at 29.97/30 FPS).
60+ FPS: High Frame Rate (HFR) used for sports, video gaming,
and smooth motion depiction.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

267 / 533

How Video Works: Visual Perception & Frame Rate II

Flicker Fusion Threshold: The minimum presentation rate (typically
48–60 Hz) required to eliminate visible brightness perception flicker.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

268 / 533

Video Resolution and Pixel Density I
Definition of Resolution
Video resolution defines the spatial detail of an image frame, quantified by
the total number of pixels across horizontal and vertical axes.
Common Resolution Standards:
Standard Definition (SD): 720 × 480 (NTSC) or 720 × 576 (PAL).
High Definition (HD / 720p): 1280 × 720 pixels.
Full High Definition (FHD / 1080p): 1920 × 1080 pixels (≈ 2.07
megapixels).
Ultra High Definition (4K UHD): 3840 × 2160 pixels (≈ 8.29
megapixels).
8K UHD: 7680 × 4320 pixels (≈ 33.18 megapixels).

Progressive vs. Interlaced Scanning:
Interlaced (e.g., 1080i): Transmits odd lines in field 1, even lines in
field 2; halves bandwidth but may introduce combing artifacts.
Progressive (e.g., 1080p): Draws all horizontal scan lines
sequentially in a single pass; superior sharp motion quality.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

269 / 533

Aspect Ratio and Display Compatibility I
Understanding Aspect Ratio
The proportional relationship between the width and height of a video
image, expressed as a ratio W : H.
Standard Ratios:
4:3 (1.33:1): Traditional legacy broadcast standard (CRT televisions).
16:9 (1.78:1): Universal modern widescreen format for HDTV, web
streaming, and displays.
21:9 (2.35:1 / 2.39:1): Anamorphic theatrical cinematic aspect ratio.
9:16 (0.56:1): Vertical video format optimized for mobile platforms.

Aspect Ratio Matching Techniques:
Letterboxing: Black bars at top and bottom when fitting wide
content on tall screens.
Pillarboxing: Black bars at left and right sides when fitting 4:3
content on 16:9 displays.
Cropping / Pan & Scan: Truncating sides to fill screen space, losing
visual content.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

270 / 533

Bit Rate and Data Rate Dynamics I
Definition of Bit Rate
Bit rate represents the volume of digital data processed, transmitted, or
recorded per unit of time, typically measured in Megabits per second
(Mbps).
Formula for Uncompressed Raw Bit Rate:
Bit Rate = Width × Height × Frame Rate × Color Depth
Example (Raw 1080p30 at 24-bit color):
1920 × 1080 × 30 × 24 ≈ 1.49 Gbps (Gigabits per second)
Bit Rate Control Modes:
CBR (Constant Bit Rate): Uniform data rate throughout; ideal for
live broadcasting despite variable scene complexity.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

271 / 533

Bit Rate and Data Rate Dynamics II

VBR (Variable Bit Rate): Allocates more bits to complex motion
scenes and fewer to static scenes; optimizes overall quality per
megabyte.

Impact: Higher bit rate increases visual fidelity and reduces
compression artifacts, requiring greater storage capacity and network
bandwidth.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

272 / 533

Video Codecs and Compression Principles I
What is a Codec?
A Codec (COder-DECoder) is a software algorithm or hardware engine
that compresses raw uncompressed video for storage/transmission and
decompresses it for playback.
Spatial Compression (Intra-frame):
Reduces redundant information within an individual single frame
(similar to JPEG compression).
Utilizes Discrete Cosine Transform (DCT) or Discrete Wavelet
Transform (DWT).

Temporal Compression (Inter-frame):
Exploits redundancy across consecutive time frames using motion
estimation and compensation.
Uses Group of Pictures (GOP) structures: I-frames (Intra), P-frames
(Predicted), and B-frames (Bi-directional predicted).

Prominent Codec Standards:
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

273 / 533

Video Codecs and Compression Principles II

H.264 / AVC: Industry standard for web, broadcast, and mobile
streaming.
H.265 / HEVC: Delivers ≈ 50% better compression efficiency than
H.264 for 4K/8K.
AV1 & VP9: Open-source, royalty-free codecs for high-efficiency
internet distribution.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

274 / 533

Codecs vs. Container Formats I
Distinction Between Container and Codec
A container is a wrapper format holding multimedia streams, while a
codec dictates how those streams are encoded and compressed.
Container Formats (File Extensions):
Functions as a box encapsulating video streams, audio tracks,
subtitles, and metadata.
Examples: MP4 (.mp4), Matroska (.mkv), QuickTime (.mov), Audio
Video Interleave (.avi), WebM (.webm).

Video Codec Streams Inside Containers:
An MP4 container can enclose H.264, H.265, or AV1 video streams
alongside AAC or MP3 audio.

Metadata Subsystem: Containers manage timestamps,
synchronization tags, chapter markers, and color space flags.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

275 / 533

Analog Video Fundamentals I
Principles of Analog Video
Analog video represents spatial brightness and color intensities as
continuously varying electrical voltage levels over continuous time domains.
Raster Scan Mechanics:
Electron beam sweeps across CRT phosphor screen from top-left to
bottom-right.
Horizontal synchronization pulses guide beam return (retrace) across
lines.
Vertical synchronization pulses reset beam to top after completing a
field/frame.

Luminance and Chrominance Breakdown:
Y Signal (Luminance): Carries grayscale brightness values.
C Signal (Chrominance): Carries color information modulated onto
subcarriers (hue and saturation).

Signal Degradation: Susceptible to noise, attenuation, generation
loss, ghosts, and radio frequency interference.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

276 / 533

Legacy Analog Television Standards I
Global Broadcast Systems
Historical broadcast television relied on three primary analog color
encoding systems across continents:
NTSC (National Television System Committee):
Adopted in North America and Japan.
525 scan lines at 29.97 / 30 FPS (60 Hz field rate).
Sensitive to phase errors, leading to color shift artifacts (”Never Twice
the Same Color”).

PAL (Phase Alternating Line):
Adopted in Europe, Asia, and Africa.
625 scan lines at 25 FPS (50 Hz field rate).
Inverts phase of color signal on alternating lines to auto-correct phase
errors.

SECAM (Séquentiel Couleur à Mémoire):
Adopted in France, Eastern Europe, and Russia.
625 scan lines at 25 FPS; uses frequency modulation (FM) for chroma
signals.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

277 / 533

Digital Video and Sampling Principles I
Digitization Process
Converting continuous analog video signals into discrete digital bitstreams
requires Spatial, Temporal, and Color Quantization.
Analog-to-Digital Conversion (ADC):
Spatial Sampling: Scanning image into discrete grid array (pixels).
Quantization: Mapping continuous brightness/color levels to discrete
digital code values (e.g., 8-bit = 256 levels per channel, 10-bit = 1024
levels).

Color Spaces & Chroma Subsampling:
Human vision is more sensitive to luminance (Y ) than chrominance
(Cb, Cr ).
4:4:4: Uncompressed color; full bandwidth for luminance and chroma.
4:2:2: Halves horizontal chroma resolution; standard for broadcast
production.
4:2:0: Quarters chroma resolution; standard for consumer distribution
(JPEG, MP4, Blu-ray).
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

278 / 533

Comparative Analysis: Analog vs. Digital Video I
Key Architectural Differences
Comparison of signal integrity, storage efficiency, and transmission
performance.
Signal Integrity:
Analog : Accumulates progressive noise and generation loss during
copying and transmission.
Digital: Perfect regeneration via error detection and forward error
correction (FEC) algorithms.

Storage and Compression:
Analog : Stored on magnetic tape or vinyl disc; rigid non-compressible
format.
Digital: Stored on solid-state drives, hard disks; highly compressible
via modern algorithms.

Random Access and Editing:
Analog : Linear tape seeking; destructive physical editing.
Digital: Non-linear frame access; non-destructive digital editing.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

279 / 533

Summary and Key Concepts I

Core Review
Recap of essential fundamentals in video engineering and animation
pre-requisites.
Video relies on rapid static image displays leveraging persistence of
vision.
Key metrics: Resolution (spatial detail), Frame Rate (temporal
fluidity), Bit Rate (data density), and Aspect Ratio (geometric
proportion).
Codecs compress digital stream data (inter/intra frame), packaged
within wrapper containers.
Digital video overcomes analog vulnerabilities through sampling,
quantization, error correction, and high compression efficiency.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

280 / 533

Introduction to Non-Linear Video Editing (NLE) I

Non-Linear Editing (NLE): Modern digital video editing paradigm
allowing non-destructive access and editing of audio and video frames
at any timeline position.
Core Components of NLE Interface:
Media Pool / Project Bin: Folder structure storing imported media
assets and metadata.
Source Monitor: Display area for viewing and marking raw footage
clips prior to timeline insertion.
Timeline / Sequence: Multi-track workspace where video and audio
clips are arranged chronologically.
Program Monitor: Preview window displaying the active frame
rendered by the timeline playhead.

SMPTE Timecode System: Frame-accurate timing reference
represented as HH:MM:SS:FF (Hours:Minutes:Seconds:Frames).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

281 / 533

Clip Management & Timeline Insertion I

Media Ingestion & Organization: Organizing raw video, audio, and
graphic assets into structured bins.
Selecting Clip Segments:
In-Point (I): Sets the starting boundary of the desired video segment.
Out-Point (O): Sets the ending boundary of the desired video
segment.

Timeline Insertion Operations:
Overwrite Edit: Inserts selected media onto target track, replacing
existing content over the edit duration.
Insert Edit (Ripple): Places clip at playhead position, shifting
downstream media further right on the timeline.
Append Edit: Automatically places the selected media clip at the
absolute end of the timeline sequence.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

282 / 533

Splitting Clips & Sub-clip Management I

Split Clip Operation (Razor / Blade Tool):
Divides a continuous clip segment into two independent sub-clips at
the active playhead coordinate.
Maintains original source file integrity through non-destructive editing
reference pointers.

Exporting Sub-clips:
Isolating specific split sections into standalone sub-clips within project
bins.
Allows individual metadata tagging, sub-clip rendering, and
independent archive management.

Practical Applications: Removing unwanted footage, isolating
highlights, and separating distinct audio/video events.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

283 / 533

Trimming and Slicing Techniques I

Basic Handle Trimming: Dragging the start (head) or end (tail)
edit handles of a clip on the timeline.
Advanced NLE Trimming Modes:
Ripple Edit: Adjusts a clip’s length and automatically ripples
downstream content to close or create timeline gaps.
Roll Edit: Moves the edit point between two adjacent clips,
shortening one while lengthening the other without altering overall
sequence length.
Slip Edit: Changes the in/out source frames of a clip while preserving
its fixed position and duration on the timeline.
Slide Edit: Shifts a clip position along the timeline, adjusting
surrounding clip durations to accommodate the movement.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

284 / 533

Clip Transformation & Motion Control I

Spatial Transformation Parameters:
Position (X , Y ): Translates clip layer relative to frame center
coordinates.
Scale (%): Alters dimensions of video media (uniform vs.
non-uniform scaling).
Rotation (◦ ): Pivots visual frame around the specified transformation
point.
Anchor Point (X0 , Y0 ): Establishes center point for scaling and
rotation operations.

Opacity & Cropping: Controls layer transparency blending and
frame edge concealment.
Dynamic Motion Pathing: Utilizing temporal keyframes to animate
position, scale, and rotation over time.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

285 / 533

Video Effects & Visual Enhancement I

Core Categories of Video Effects:
Color Correction & Grading: Primary color wheels, curves, Look-Up
Tables (LUTs), and color space management.
Spatial Filters: Gaussian blur, directional blur, sharpen, and noise
reduction filters.
Keying & Compositing: Chroma keying (green/blue screen removal),
luma keying, and track matte transparency.
Distortion & Geometry: Lens distortion correction, perspective
adjustments, and warping.

Transitions: Cross dissolves, wipes, motion pushes, and dip-to-black
applied at edit points.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

286 / 533

Audio Effects & Sound Processing I

Audio Track Attributes: Sample rates (e.g., 48 kHz, 44.1 kHz), bit
depth (16-bit, 24-bit), and channel layouts (Mono, Stereo, 5.1).
Essential Digital Audio Effects:
Equalization (EQ): Frequency spectrum adjustment (high-pass,
low-pass, parametric bands).
Dynamic Processing: Compression, limiting, and noise gating to
balance loudness range.
Noise Restoration: Spectral de-noising, hum removal, and de-essing
algorithms.
Acoustic Spatialization: Reverb, delay, and pan controls across
channel configurations.

Audio Automation: Keyframing volume envelopes, channel panning,
and audio ducking behind dialogue.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

287 / 533

Titling & Motion Graphics I

Text Overlay Types: Static text banners, lower-third graphics,
crawling news tickers, and rolling end credits.
Typography & Screen Aesthetics:
Font family selection, contrast hierarchy, tracking, kerning, and line
spacing.
Television Safe Zones: Action-safe boundary (inner 93%) and
Title-safe boundary (inner 80%).

Styling & Animation:
Stroke outlines, drop shadows, background fills, and glassmorphism
panels.
Animating title entrance/exit using keyframed opacity, position, and
scale.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

288 / 533

Multi-Track Compositing & Keyframing I

Track Layering Hierarchy:
Video tracks stack vertically (V1, V2, V3); higher track layers
obscure lower tracks.
Audio tracks stack vertically (A1, A2, A3); sound signals blend
additively during playback.

Blending Modes: Mathematical pixel operations (Multiply, Screen,
Overlay, Add) for complex compositing.
Keyframe Interpolation Curves:
Linear Interpolation: Fixed rate of parameter change between
keyframes.
Bezier / Temporal Curves: Ease-In and Ease-Out velocity curves for
continuous movement.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

289 / 533

Exporting Clips & Intermediate Rendering I

Intermediate Production Codecs:
High-bitrate master formats (Apple ProRes, Avid DNxHD/DNxHR,
GoPro CineForm).
Intra-frame Compression: Encodes every frame individually to
preserve editing performance and image quality.

Partial Sequence Exporting:
Setting timeline work area markers (In/Out) for targeted segment
rendering.
Batch exporting individual split clips vs. rendering full sequence
timelines.

Proxy Workflow Architecture: Generating low-resolution proxy
media for high-speed timeline editing prior to final master relinking.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

290 / 533

Video Exporting Modes & Output Formats I

Codecs vs. Container Formats:
Video Codec: Compression/decompression algorithm (e.g.,
H.264/AVC, H.265/HEVC, AV1, VP9).
Container File: File format wrapper holding encoded tracks and
metadata (e.g., .mp4, .mov, .mkv).

Output Encoding Parameters:
Spatial Resolution (1080p, 4K UHD), Frame Rate (24 fps, 30 fps,
60 fps).
Bitrate Strategies: Constant Bitrate (CBR), Variable Bitrate (VBR
1-Pass / 2-Pass).

Target Destination Presets: Configuring encoding parameters for
Web Streaming, Digital Cinema, and Broadcast standards.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

291 / 533

Lecture Overview: Video Editing Core Concepts I

Unit Context: DI04000051 – Video and Animation.
Lecture Focus: Post-production workflow, non-linear editing (NLE)
mechanisms, and clip processing.
Key Topics Covered:
Clip operations: importing, splitting, slicing, and exporting sub-clips.
Timeline assembly and multi-track spatial transformations.
Video and audio signal processing effects.
Motion titling and dynamic visual typography.
Export parameters: codecs, containers, resolutions, and delivery
profiles.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

292 / 533

Timeline Management & Clip Assembly I

Non-Linear Editing (NLE) Architecture:
Non-destructive editing model preserving original source media asset
files.
Sequence creation with standardized frame rate (e.g., 24, 30, 60 FPS)
and aspect ratios (e.g., 16:9, 9:16).

Adding Clips to the Timeline:
Target track routing (V1–Vn for video, A1–An for audio).
Three-point and four-point editing (In-point, Out-point, Target
position).
Insert edit vs. Overwrite edit modes.

Track Layering Rules:
Higher video tracks (V2, V3) obscure lower tracks (V1) based on
alpha transparency and blending.
Audio tracks sum logarithmically across all active channels.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

293 / 533

Clip Manipulation: Splitting & Slicing I

Split Clip (Razor Tool / Cut):
Divides a single continuous clip event into independent sub-events at
playhead location.
Mathematical representation: Csource (t) → C1 (t0 , tcut ) + C2 (tcut , t1 ).

Slicing & Segment Extraction:
Isolating optimal takes and discarding unwanted frame sequences.
Extracting sub-clips for independent asset management and external
processing.

Exporting Sub-Clips:
Generating standalone intermediate clips (e.g., ProRes, DNxHR) for
visual effects (VFX) pipelines.
Maintaining timecode continuity across isolated clip assets.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

294 / 533

Trimming Techniques & Edit Types I
Basic Trimming:
Adjusting In-Point (Head) and Out-Point (Tail) without moving
neighboring clips.

Advanced Trim Modes:
Ripple Edit: Modifies clip duration and automatically shifts
downstream clips to prevent gaps.
Roll Edit: Adjusts edit point between two adjacent clips
simultaneously without altering sequence duration.
Slip Edit: Changes clip content In/Out points while keeping clip
duration and timeline position fixed.
Slide Edit: Shifts clip position on timeline while automatically
adjusting adjacent clips’ In/Out points.

L-Cuts & J-Cuts (Split Edits):
Lead or lag audio relative to video cut points for fluid narrative
transitions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

295 / 533

Spatial Transformations & Keyframing I

Geometric Transformation Parameters:
Position (X , Y , Z ): Spatial translation within 2D/3D composite
canvas.
Scale (Width, Height): Dynamic zoom factor maintaining or
breaking uniform aspect ratio.
Rotation: Angular orientation around specified anchor point.
Crop & Anchor Point: Defining visible bounding box and center of
spatial rotation/scaling.

Keyframe Interpolation:
Linear Interpolation: Constant rate of change between keyframes.
Bezier / Ease In & Out: Smooth acceleration curves governed by
cubic splines.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

296 / 533

Video Effects & Compositing I

Color Grading & Correction:
Primary adjustments: exposure, contrast, white balance, saturation.
Secondary adjustments: keying specific hue/chroma ranges for
selective color manipulation.
Look-Up Tables (LUTs): standard mathematical color transformation
matrices.

Spatial & Temporal Filters:
Blur, sharpen, noise reduction, motion blur, and optical flow retiming.

Keying & Alpha Compositing:
Chroma Keying (Green/Blue Screen separation via color difference
matte).
Blend Modes: Add, Multiply, Overlay, Screen mathematically
operating on RGB vectors.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

297 / 533

Audio Effects & Sound Design I

Audio Signal Processing Pipeline:
Track level vs. Clip level audio modification.

Essential Audio Effects:
Equalization (EQ): Frequency domain shaping (High-pass, Low-pass,
Parametric EQ).
Dynamic Range Compression: Attenuates peaks to normalize
dynamic range and improve clarity.
Noise Reduction: Spectral subtraction to eliminate stationary
background noise.
Reverb & Delay: Spatial room acoustics simulation.

Transitions & Mixing:
Exponential vs. Constant Power audio crossfades to avoid loudness
dips.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

298 / 533

Titling & Typography in Video I

Classification of Video Titles:
Lower Thirds: Subject identification and context overlay.
Main Title Sequences: Brand identity and dynamic motion graphics.
Subtitles & Captions: Open captions (burned-in) vs. Closed captions
(sidecar files like SRT).

Typography Best Practices for Screens:
Legibility: High contrast against video background (using drop
shadows or translucent backing cards).
Safe Zones: Title Safe area (80% threshold) to prevent edge clipping
on display panels.
Font selection: Sans-serif fonts (e.g., Inter, Montserrat) for clean
digital display.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

299 / 533

Video Export Parameters: Codecs & Containers I

Video Codecs (Compression/Decompression):
Distribution Codecs: H.264 (AVC), H.265 (HEVC), AV1 (high
compression efficiency).
Editing/Intermediary Codecs: Apple ProRes, Avid DNxHR, GoPro
CineForm (intra-frame compression).

Container Formats (Muxing):
Wrappers housing video streams, audio streams, and metadata (.mp4,
.mov, .mkv, .avi).

Compression Paradigms:
Intra-frame (Spatial): Compresses each frame independently (ideal
for editing).
Inter-frame (Temporal): Compresses across IPB frame groups
(GOP) for low file size.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

300 / 533

Multi-Mode Exporting Profiles I

Target Delivery Specifications:
Web/Streaming (YouTube, Vimeo): H.264/H.265, AAC audio,
VBR 2-pass, 15–50 Mbps bitrate.
Broadcast/Television: MXF / ProRes 422 HQ, uncompressed PCM
audio, Rec.709 color space.
Archival/Mastering: Uncompressed or mathematically lossless
ProRes 4444 XQ / DNxHR HQX.

Key Render Settings:
Bitrate Control: CBR (Constant), VBR 1-Pass, VBR 2-Pass.
Color Primaries & Dynamic Range: Rec.709 (SDR) vs. Rec.2020 /
PQ / HLG (HDR).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

301 / 533

Professional Editing Workflow Summary I

Standard Post-Production Pipeline:
1
2
3
4
5
6

Media Ingestion & Organization (Bin structures, proxies generation).
Assembly & Rough Cut (Trimming, slicing, timeline layout).
Fine Cut & Split Edits (J-Cuts, L-Cuts, ripple/roll edits).
Visual Effects & Motion Graphics (Transformations, keying, titling).
Color Grading & Audio Mixing (LUTs, EQ, compression).
Master Export & Multi-Mode Encoding.

Key Takeaway: Efficient NLE workflows balance creative storytelling
with precise technical signal encoding.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

302 / 533

Introduction to Video Editing & NLE I

Non-Linear Editing (NLE): Modern digital video editing paradigm
allowing non-destructive, random access to media assets on a
timeline.
Core Objectives of Video Editing:
Assembling raw visual and audio footage into a coherent narrative.
Correcting visual imperfections and enhancing audio quality.
Adding artistic effects, transitions, and text elements.

Basic Unit of Editing:
Clip: A segmented portion of media (video, audio, image) placed on
the project timeline.
Timeline: The multi-track spatial-temporal workspace where clips are
sequenced.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

303 / 533

Media Management & Timeline Integration I

Importing Media Assets:
Media pool/Project bin management.
File format verification (MP4, MOV, AVI, WAV, AAC).

Adding Clips to the Timeline:
Drag-and-Drop: Moving clips directly from media bins onto targeted
tracks.
Insert Edit: Inserting a clip into the timeline, shifting existing clips
forward in time.
Overwrite Edit: Placing a clip on the timeline by overwriting existing
content at the playhead position.

Track Management:
Multi-track video (V1 , V2 , . . . ) for layering visual elements.
Multi-track audio (A1 , A2 , . . . ) for voiceover, sound effects, and
background music.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

304 / 533

Splitting Clips & Sub-clip Export I

Splitting Clips (Razor/Blade Tool):
Dividing a continuous clip into two or more independent segments at
the playhead location.
Essential for removing unwanted sections, inserting B-roll, or applying
localized effects.
Keyboard shortcuts: Ctrl+K or C.

Creating and Exporting Sub-clips:
Sub-clip Definition: A reference clip representing a specific portion of
a master media file.
Workflow: Setting In (I ) and Out (O) points on source monitor →
Exporting sub-clip to project bin.
Benefits: Streamlines asset management in large-scale projects
without duplicating master media files.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

305 / 533

Trimming & Slicing Techniques I

Basic Trimming:
Adjusting the head (start point) or tail (end point) of a clip on the
timeline.

Advanced Precision Editing Modes:
Ripple Edit: Trims a clip and automatically shifts adjacent clips to
close gaps or accommodate duration changes.
Roll Edit: Adjusts the edit point between two adjacent clips
simultaneously without altering total timeline duration.
Slip Edit: Changes the In and Out points of a clip while maintaining
its position and duration on the timeline.
Slide Edit: Shifts a clip along the timeline while adjusting the
duration of neighboring clips to maintain overall length.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

306 / 533

Spatial Transformations of Video Clips I

Transform Properties:
Position (X , Y ): Translating the clip along horizontal and vertical
axes within frame coordinates.
Scale (Zoom): Resizing video resolution percentage; supports
uniform and non-uniform scaling.
Rotation: Rotating the frame around a specified pivot point (in
degrees).
Anchor Point: Setting the center reference point for scale and
rotation operations.
Opacity: Controlling the transparency level (0% to 100%).

Keyframing Transformations:
Interpolating transform values across frames to create dynamic motion
graphics (pan, zoom, tilt).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

307 / 533

Video Effects & Visual Enhancement I

Categories of Video Effects:
Color Correction & Grading: Adjusting exposure, contrast, color
temperature, and LUTs (Look-Up Tables).
Stylistic Filters: Blur, sharpen, glow, stylize, distortion, and grain
application.
Compositing & Keying: Chroma Keying (Green/Blue Screen removal
via Ultra Key or Difference Matte).
Video Transitions: Cross Dissolve, Wipe, Slide, Cut, Zoom, and Push
effects between clips.

Effect Controls & Adjustment Layers:
Applying non-destructive effects globally across multiple clips using
Adjustment Layers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

308 / 533

Audio Effects & Sound Editing I

Audio Track Adjustments:
Volume gain normalization and track faders (dB).
Audio Panning (L/R stereo balance).

Essential Audio Effects:
Noise Reduction / Denoise: Removing background hums and
ambient static noise.
Equalization (EQ): Parametric EQ for frequency shaping (bass boost,
vocal clarity).
Dynamics / Compressor: Balancing audio dynamic range and
preventing clipping distortion.
Reverb & Delay: Simulating spatial acoustics.

Audio Ducking & Transitions:
Automatically lowering background music when voiceover is present.
Constant Power and Constant Gain audio transitions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

309 / 533

Adding Titles & Motion Graphics I

Types of Video Titles:
Static Titles: Fixed headers, labels, and informational callouts.
Lower Thirds: Graphic overlay located in the lower region of the
frame displaying names and designations.
Rolling & Crawling Text: Vertical credits scroll (end credits) or
horizontal text crawl (news ticker).

Title Customization Parameters:
Font typography, size, leading, tracking, kerning, alignment.
Fill color, gradients, stroke outlines, drop shadows, and background
shapes.

Motion Graphic Templates (MOGRTs):
Reusable animated vector graphics for professional lower thirds and
title openers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

310 / 533

Exporting Clips & Sub-clips I

Single Clip Export Workflow:
Marking timeline In (I ) and Out (O) points to isolate a specific clip or
section.
Exporting individual highlights without rendering the entire timeline.

Batch Exporting:
Queueing multiple sequences or sub-clips to Media Encoder / render
queue.
Simultaneous background rendering of multiple video assets.

Mastering vs. Proxy Export:
Master File: Uncompressed or low-compression archive file (e.g.,
ProRes 422, DNxHR).
Proxy File: Low-resolution lightweight file for offline editing or rapid
review.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

311 / 533

Export Video Modes & Encoding Settings I

Containers vs. Codecs:
Containers (Formats): .mp4, .mov, .mkv, .avi (file wrappers).
Codecs (Compression): H.264 (AVC), H.265 (HEVC), ProRes, AV1,
VP9.

Bitrate & Encoding Options:
CBR (Constant Bit Rate): Uniform data rate; predictable file size.
VBR (Variable Bit Rate 1-Pass / 2-Pass): Dynamic allocation
based on visual complexity.

Export Modes by Platform:
Web Streaming: H.264, MP4, AAC audio, matching platform
resolution (1080p, 4K).
Broadcast: High bitrate MXF / ProRes with multi-channel audio
tracks.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

312 / 533

Summary of Video Editing Workflow I

End-to-End NLE Pipeline:
Media Ingestion → Assembly Cut → Fine Cut (Trimming/Slicing) →
Transforms & Effects → Audio Mix → Titling → Export.

Best Practices:
Maintain organized project directory structure and bins.
Utilize keyboard shortcuts for trimming and razor tools to improve
editing efficiency.
Always verify audio peak levels (keep below 0 dB to prevent clipping,
target −6 dB to −12 dB).
Match export parameters precisely to final distribution channel
requirements.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

313 / 533

Lecture Overview: AI Features in Video Editing I
Course Code: DI04000051
Unit: Video and Animation
Topic: AI Features in Video Editing Tools (Motion Tracking, Object
Detection, Video Stabilization)
Lecture Objectives:
Understand the transition from manual video processing to automated
AI pipelines.
Examine theoretical algorithms underlying computer vision in video
editing.
Explore motion tracking, deep optical flow, and multi-object tracking
(MOT).
Study AI-driven object detection, automatic rotoscoping, and
temporal segmentation.
Analyze video stabilization via motion model estimation and deep
feature alignment.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

314 / 533

1. Evolution of AI in Modern Video Editing I
Traditional Video Editing Workflows:
Frame-by-frame manual keyframing and rotoscoping.
High computational cost and laborious human effort for pixel isolation.
Sensitivity to lighting changes, motion blur, and sudden occlusions.

AI-Powered Paradigms:
Neural network inference integrated directly into Non-Linear Editors
(NLEs).
Context-aware temporal consistency across adjacent video frames.
Automated detection of complex features, edge boundaries, and
camera motion.

Core AI Capabilities in NLEs:
Object tracking and automated mask propagation.
Neural scene cut detection and smart reframing.
Feature-guided video stabilization and camera smoothing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

315 / 533

2. Fundamentals of Motion Tracking I
Definition and Core Concept:
Estimating the trajectory of target pixels or objects over successive
video frames.
Mathematically modeled as finding displacement vectors (dx, dy ) over
temporal frame index t.

Traditional Point Tracking Techniques:
Lucas-Kanade Optical Flow: Assumes brightness constancy and
local motion smoothness.
Kanade-Lucas-Tomasi (KLT) Tracker: Extracts corner feature
points (Shi-Tomasi) and tracks them gradient-wise.
Mean-Shift / CamShift: Color-histogram matching over spatial
probability distributions.

Limitations of Classical Tracking:
Fails under extreme motion blur, scale changes, or dynamic
illumination variations.
Susceptible to severe drift during target occlusions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

316 / 533

3. Deep Learning for Motion Tracking I

Deep Optical Flow Architectures:
FlowNet & PIVNet: End-to-end Convolutional Networks predicting
dense pixel-wise flow fields.
RAFT (Recurrent All-Pairs Field Transforms): Computes all-pairs
correlation volumes with GRU-based iterative updates.

Multi-Object Tracking (MOT) Frameworks:
Tracking-by-Detection: Detects objects independently per frame,
then links bounding boxes.
Kalman Filter + Hungarian Algorithm (SORT / DeepSORT):
Predicts trajectory state and matches using deep appearance
embeddings.

Point-Tracking Transformers:
CoTracker & PIPs: Tracks long-range persistent points across
multi-frame temporal windows using self-attention mechanisms.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

317 / 533

4. Object Detection in Video Editing I
Core Principles of Video Object Detection:
Identifying class categories and spatial locations of entities across
video streams.
Extending single-frame object detectors (YOLO, Faster R-CNN) with
temporal feature aggregation.

Key Capabilities Enabled in NLEs:
Smart Cutout & Subject Isolation: Automatic separation of
foreground actors from background.
Face and Pose Detection: Facial landmark tracking for automated
skin retouching and privacy blurring.
Content-Aware Reframing: Dynamic aspect ratio adjustment (e.g.,
16:9 to 9:16) keeping primary subjects centered.

Temporal Consistency Challenges:
Preventing bounding box flicker and classification jitter between
consecutive frames.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

318 / 533

5. AI Masking and Rotoscoping I
Automated Video Object Segmentation (VOS):
Propagating user-selected mask annotations across hundreds of
frames automatically.
Replaces traditional manual spline curve tweaking.

Segment Anything Model in Video (SAM / SAM 2):
Promptable segmentation using bounding boxes, points, or text
queries.
Memory-bank mechanism stores spatial and temporal embeddings for
persistent tracking.

Applications in Post-Production:
Selective color grading and localized visual effects (VFX).
Object removal and AI-driven temporal video inpainting.
Background replacement without green screens (Chroma Key-less
matte generation).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

319 / 533

6. Video Stabilization Fundamentals I
Objective of Video Stabilization:
Removing high-frequency unwanted camera shakes while preserving
low-frequency intentional panning/zooming.

Traditional Stabilization Pipeline:
Feature Extraction: Detect interest points (SIFT, SURF, ORB) in
frame t and t + 1.
Motion Model Estimation: Compute transformation matrices
(Affine, Homography, Perspective) using RANSAC.
Path Smoothing: Filter trajectory parameters via low-pass Gaussian
smoothing or moving average filters.
Warping and Cropping: Apply inverse transform to frames, followed
by adaptive edge cropping.

Limitations of 2D/3D Traditional Methods:
Perspective distortion and parallax artifacts in complex 3D scenes.
Rolling shutter wobble caused by CMOS sensor scanning delays.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

320 / 533

7. Deep Learning for Video Stabilization I
Advanced Motion Representation:
Moving beyond simple global 2D homographies to dense per-pixel
motion field representations.

Mesh Warp & Sub-Feature Alignment:
Grid-based mesh transformations (e.g., Warp Stabilizer) driven by
spatial feature constraints.
Neural mesh optimization reduces local distortion and preserves
straight lines.

Deep Learning-Based Stabilizers:
Dense Flow Estimators: Predict smooth dense optical flow fields
directly via deep CNNs.
End-to-End Generative Stabilization: Synthesis of unstabilized
frames into smooth frames using temporal attention networks.

Rolling Shutter Correction:
Neural networks model scanline velocity vectors to rectify geometric
image bending.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

321 / 533

8. Integration of AI Features in Commercial NLEs I

Adobe Premiere Pro / After Effects:
Sensei Engine: Auto Reframe, Scene Edit Detection, Roto Brush 3
(neural propagation).

DaVinci Resolve (Studio):
Neural Engine: Magic Mask (person/object isolation), Smart
Reframe, Surface Tracking (warped textures).

Cloud & Generative AI Platforms (RunwayML, CapCut):
Instant text-prompted object tracking, background removal, and
multi-modal motion synthesis.

Blender & Open-Source Tools:
Open-source tracking nodes utilizing OpenCV and ONNX runtime
integration for real-time inference.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

322 / 533

9. Performance Metrics & Computational
Engineering I
Quantitative Evaluation Metrics:
Tracking Precision / Recall: Multiple Object Tracking Accuracy
(MOTA) and IDF1 score.
Stabilization Quality Metrics: Inter-frame fidelity, cropping ratio,
and distortion score.
Segmentation Accuracy: Mean Intersection over Union (mIoU) and
boundary F-measure.

Real-Time Hardware Acceleration:
Tensor Core utilization (FP16 / INT8 quantization) for sub-30ms
inference per frame.
Dedicated NPU/GPU pipelines avoiding CPU-GPU memory transfer
bottlenecks.

Temporal Memory Management:
Sliding window attention mechanisms to constrain VRAM
consumption in 4K/8K video processing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

323 / 533

10. Summary and Future Directions I

Summary of Key Takeaways:
AI motion tracking provides reliable, occlusion-aware point and object
trajectories.
Deep object detection and VOS transform manual rotoscoping into
interactive, prompt-based masking.
AI-driven video stabilization solves complex non-rigid motion and
rolling shutter distortions.

Emerging Trends in AI Video Editing:
NeRFs and 3D Gaussian Splatting: 3D-aware camera stabilization
and novel view synthesis.
Generative Video Inpainting: Contextual reconstruction of occluded
regions during tracking.
Real-Time Edge AI: Embedded AI inference directly within camera
hardware during acquisition.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

324 / 533

Introduction to AI in Video Editing I
Evolution of Video Post-Production: Transition from traditional
frame-by-frame manual editing to automated machine intelligence.
Core Pillars of AI Integration:
Spatial Analysis: Feature extraction, object boundary estimation, and
pixel segmentation.
Temporal Dynamics: Frame-to-frame correspondence, velocity vector
field computation, and trajectory synthesis.

Key Capabilities Covered in Unit:
Automated Motion Tracking (Point, Feature, and Visual Object
Tracking).
Real-Time Object Detection and Automated Masking/Rotoscoping.
Video Stabilization and Camera Shake Mitigation.

Computational Paradigm: Moving from heuristic signal processing
to deep neural network representation learning.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

325 / 533

Theoretical Foundations: Visual Feature Extraction I

Feature Detection and Matching:
Corner Detection: Harris Corner
Detector

P
Ix2 Ix Iy
(M =
w (x, y )
).
Ix Iy
Iy2
Scale-Invariant Feature Transform (SIFT) and Oriented FAST and
Rotated BRIEF (ORB).

Spatio-Temporal Video Representation:
Video tensor representation: V ∈ RH×W ×C ×T .
Spatial dimensions (H × W ), color channels (C ), time sequence (T ).

Challenges in Temporal Signal Processing:
Occlusions, rapid motion blur, illumination variation, and non-rigid
deformation.
Computational burden of processing high-resolution temporal
sequences (4K/8K at 60 fps).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

326 / 533

Motion Tracking I: Optical Flow & KLT Tracker I

Optical Flow Constraint Equation:
Assumes Brightness Constancy: I (x, y , t) = I (x + dx, y + dy , t + dt).
First-order Taylor

 Expansion: Ix u + Iy v + It = 0, where
dy
,
.
(u, v ) = dx
dt dt

Lucas-Kanade Feature Tracking:
Assumes local spatial consistency in asmall
 window Ω.
u
Solves overdetermined linear system:
= (AT A)−1 AT b.
v

Kanade-Lucas-Tomasi (KLT) Algorithm:
Selects good features to track based on eigenvalues of the gradient
matrix.
Pyramidal Lucas-Kanade for handling large pixel displacements across
frames.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

327 / 533

Motion Tracking II: Deep Learning Paradigms I

Siamese Neural Networks (SiamFC, SiamRPN):
Formulates tracking as a cross-correlation metric learning task:
f (z, x) = φ(z) ∗ φ(x).
Target template z cross-correlated with search region x in latent
feature space.

Transformer-Based Tracking (TransT, OSTrack):
Replaces traditional cross-correlation with self-attention and
cross-attention mechanisms.
Captures global spatial dependencies and complex temporal context.

Deep Optical Flow Architectures:
FlowNet, PIV-LiteFlowNet, and RAFT (Recurrent All-Pairs Field
Transforms).
High-level feature correlation volumes combined with convex
upsampling.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

328 / 533

Object Detection in Video: Fundamentals I
Single-Frame vs. Multi-Frame Detection:
Single-frame: Independent detection per frame ignoring temporal
coherence.
Video Object Detection (VOD): Aggregating temporal features across
adjacent frames to improve detection stability.

Core Architectures:
Two-Stage Detectors: Faster R-CNN (Region Proposal Network +
RoI Align).
One-Stage Detectors: YOLO (You Only Look Once) series and Single
Shot MultiBox Detector (SSD).

Metrics for Video Object Detection:
|B ∩B |

Intersection over Union (IoU = |Bpp ∪Bgg | ).
Mean Average Precision (mAP) evaluated across spatial locations and
time steps.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

329 / 533

Spatio-Temporal Detection & Tracking-by-Detection I

Tracking-by-Detection Framework:
Detect objects independently per frame → Associate bounding boxes
across frames.
Data association via Hungarian Algorithm and Kalman Filtering (e.g.,
SORT, DeepSORT).

Deep Association Features:
Re-Identification (Re-ID) embeddings for maintaining object identity
through long occlusions.
Dynamic appearance modeling using online update modules.

Temporal Feature Aggregation:
Deformable Convolutional Networks (DCN) guided by optical flow for
feature alignment (e.g., FGFA).
Memory-augmented video object detectors.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

330 / 533

AI-Driven Rotoscoping & Video Segmentation I

Video Object Segmentation (VOS):
Semi-supervised VOS: User provides mask on the first frame; model
propagates mask temporally.
Unsupervised VOS: Automatic primary foreground object discovery.

Segment Anything Model for Video (SAM 2):
Streaming architecture with memory bank storing spatial features and
object pointers across time.
Promptable video segmentation supporting points, boxes, and masks.

Impact on Video Production:
Replaces labor-intensive manual vector path masking (rotoscoping).
Enables real-time background removal without physical chroma
keying.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

331 / 533

Video Stabilization: Classical Principles I

Camera Motion Model:
2D Models: Rigid, Affine (x′ = Ax + b), Homography.
3D Models: Epipolar geometry, Structure-from-Motion (SfM), depth
map estimation.

Pipeline of Video Stabilization:
1. Feature Point Tracking (e.g., KLT across frame sequence).
2. Motion Trajectory Estimation: Filtering original camera paths.
3. Trajectory Smoothing: Gaussian filtering, low-pass filtering,
L1 -optimal path smoothing.
4. Frame Warping and Cropping: Compensating for border artifacts.

Evaluation Metrics: Inter-frame transformation fidelity, cropping
ratio, and distortion metric.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

332 / 533

AI & Deep Learning for Video Stabilization I

Deep Learning-Based Camera Motion Smoothing:
Learning smooth transformations directly from unstable raw frame
sequences using CNNs/RNNs.
Implicit motion estimation via optical flow fields without explicit
feature correspondence matching.

Inpainting and Synthesis for Boundary Completion:
Classical stabilization suffers from resolution loss due to heavy
cropping.
Generative Adversarial Networks (GANs) and Diffusion Models
synthesize missing border regions.

3D Video Stabilization via Depth-Aware Neural Rendering:
NeRF (Neural Radiance Fields) and 3D Gaussian Splatting for novel
view synthesis during camera stabilization.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

333 / 533

Commercial Implementations & Industrial Tools I
Adobe Sensei Integration:
Content-Aware Fill for Video (Optical flow-guided temporal
background replacement).
Auto Reframe (Intelligent subject tracking for multi-aspect-ratio
exports).

DaVinci Resolve Neural Engine:
Magic Mask (Object and person isolation using deep segmentation
networks).
Speed Warp (AI optical flow for high-quality slow-motion
interpolation).
Perspective & 3D Camera Tracker.

Cloud & Next-Gen AI Platforms:
Runway ML (Green Screen, Inpainting, Motion Brush).
Topaz Video AI (AI-based stabilization, frame interpolation, and
upscaling).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

334 / 533

Challenges, Trade-Offs & Future Directions I

Computational Constraints vs. Real-Time Processing:
High-fidelity models (SAM 2, RAFT) require significant GPU VRAM
and latency trade-offs.
Hardware acceleration via TensorRT, Apple Neural Engine, and NPU
offloading.

Artifact Mitigation:
Motion Blur Discrepancies: Stabilized camera motion with
mismatched motion blur vectors.
Rolling Shutter Distortion: Dynamic CMOS sensor distortion under
high-frequency movement.

Future Paradigms:
End-to-End Generative Video Editing (Text-guided spatio-temporal
video manipulation).
Multi-Modal AI agents conducting end-to-end editorial decisions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

335 / 533

Introduction & The Power of Motion I

Definition of Animation: The technique of creating the illusion of
movement and temporal change by rapidly displaying a sequential
series of static images.
The Power of Motion in Multimedia:
Attention Grabber: Dynamic visuals naturally capture and direct
human visual focus faster than static elements.
Visual Explanation: Clarifies complex temporal and spatial
relationships (e.g., fluid dynamics, mechanical assemblies).
Emotional Engagement: Imparts personality, realism, and aesthetic
appeal to user interfaces and storytelling.

Cognitive Impact: Enhances concept retention, improves visual
feedback, and provides spatial context during UI transitions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

336 / 533

Animation Fundamentals & Perception Mechanics I
Biological Basis of Visual Motion:
Persistence of Vision: The short-term retention of an image on the
human retina (∼ 1/16 s), allowing discrete frames to blend together.
Phi Phenomenon: The brain’s perceptual illusion of smooth,
continuous motion when viewing adjacent visual stimuli in rapid
succession.

Temporal Parameters:
Frame Rate (FPS): Frames per second determining motion fluidity.
Industry Standards: Cinema (24 fps), Broadcast TV (25/29.97 fps),
High-FPS Video/Gaming (60+ fps).
Animating on Ones/Twos: Drawing a distinct image every frame
(24 fps) vs. every two frames (12 fps).

Core Concepts:
Keyframes: Primary frames defining the start, end, and major
extreme poses of an action.
In-betweening (Tweening): Generating intermediate frames between
keyframes to create fluid motion.
Onion Skinning: Displaying translucent overlays of preceding and
succeeding frames for precise spatial alignment.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

337 / 533

12 Principles of Animation: Deconstruction &
Weight I

1. Squash and Stretch: Expresses weight, gravity, and flexibility by
deforming an object’s shape while maintaining its overall volume.
2. Anticipation: Prepares the audience for a primary action using
reverse counter-movements (e.g., bending knees before jumping).
3. Staging: Directs the viewer’s attention to emphasize key story
elements through clear silhouettes and framing.
4. Straight Ahead Action vs. Pose to Pose:
Straight Ahead: Animating frame-by-frame chronologically; ideal for
continuous, unpredictable fluid dynamics (fire, water).
Pose to Pose: Drawing main keyframes first and filling in-betweens
later; best for structured, proportional character movements.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

338 / 533

12 Principles of Animation: Dynamics & Timing I

5. Follow Through and Overlapping Action:
Follow Through: Loosely attached parts (hair, clothing) continue
moving after the main body stops.
Overlapping Action: Different body parts move at varying speeds
and timing offsets.

6. Slow In and Slow Out (Ease In / Ease Out): Simulates
natural physics by placing more frames near the start and end of a
movement to represent acceleration and deceleration.
7. Arcs: Organic motion generally follows natural curved paths
rather than mechanical straight lines.
8. Secondary Action: Subordinate movements (e.g., swinging arms
while walking) that enrich the main action without distracting from it.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

339 / 533

12 Principles of Animation: Presentation & Polish I

9. Timing: The total number of frames allocated to an action;
defines physical weight, speed, and emotional tone.
10. Exaggeration: Pushing movement, features, and expressions
beyond literal reality to heighten readability and dramatic impact.
11. Solid Drawing: Maintaining 3D volume, balance, perspective,
and weight in 2D artistic execution.
12. Appeal: Designing characters with charismatic visual presence,
distinct personality, and clear visual hierarchy.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

340 / 533

Traditional & Physical Animation Techniques I

Cel (Traditional Hand-Drawn) Animation:
Sequential artwork drawn manually on transparent celluloid sheets
overlaying static background paintings.
High labor intensity, requiring manual ink-and-paint processes and
multiplane camera setups.

Stop Motion Animation:
Physical objects physically manipulated and photographed
frame-by-frame.
Sub-variants: Claymation (malleable clay figures), Puppet Animation
(articulated armatures), Cutout (flat paper shapes), Pixilation (real
human actors captured frame-by-frame).

Rotoscoping: Tracing over live-action video footage frame-by-frame
to achieve realistic human motion data.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

341 / 533

Computer-Assisted 2D Animation I

Vector vs. Raster Animation:
Vector: Resolution-independent, calculated using mathematical paths
(Bézier curves), lightweight file size, enables smooth automated
interpolation.
Raster: Pixel-based, allows rich artistic texturing and detailed brush
strokes, but yields larger file sizes and scaling artifacts.

2D Character Rigging:
Building hierarchical skeletal bone structures attached to vector
meshes.
Utilizes Kinematics (Forward & Inverse Kinematics) to animate
characters without redrawing frames.

Computerized Tweening: Algorithmic interpolation of position,
scale, rotation, color, and opacity keyframe parameters.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

342 / 533

3D Computer Animation Pipeline I
1. Modeling & Rigging:
Constructing 3D geometry (polygonal meshes or NURBS surfaces).
Creating skeletal armatures, inverse kinematics (IK) handles, and
binding skin weights.

2. Animation Mechanics:
Setting translation, rotation, and scaling keyframes along X , Y , Z
axes.
Utilizing Graph Editors to fine-tune non-linear motion curves via
Bézier handles.

3. Dynamics & Simulation:
Physics-based algorithmic generation for cloth, hair, particle effects,
fluid dynamics, and rigid collisions.

4. Lighting, Texturing & Rendering: Rasterizing or ray-tracing 3D
spatial scenes into 2D output image sequences.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

343 / 533

Animation File Formats & Delivery Media I
GIF (Graphics Interchange Format):
Legacy 8-bit format (max 256 colors), LZW compression, simple
binary transparency, high file overhead for complex frames.

APNG (Animated Portable Network Graphics):
Backward-compatible extension of PNG; supports 24-bit true color
and 8-bit alpha-channel transparency.

Animated WebP:
Modern web format supporting lossy/lossless compression, 24-bit
color, and alpha channel; provides significantly smaller file size than
GIF.

Lottie (JSON-based Animation):
Vector animation data exported as JSON code; rendered
programmatically via JavaScript (SVG/Canvas), achieving resolution
independence and tiny footprints.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

344 / 533

Video vs. Animated Graphics: Technical Comparison
I
Video Codecs (MP4/H.264, WebM/VP9, AV1):
Rely on spatial and temporal inter-frame compression (I, P, B frames)
and motion compensation.
Optimized for photorealistic content at continuous high frame rates;
generally lack native alpha transparency in baseline profiles.

Animated Graphic Formats (APNG, WebP, GIF):
Frame-by-frame container structure; simple to render, native loop
controls, support alpha transparency, but scale poorly for long
photographic content.

Format Selection Criteria:
UI Micro-interactions: Lottie / SVG / CSS.
Transparent Graphical Loops: Animated WebP / APNG.
Full-Motion Video Clips: MP4 (H.264/H.265) / WebM.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

345 / 533

Interactive & Web-Based Animation Technologies I

CSS Animations & Transitions:
Declarative keyframe rules defined directly in stylesheets;
GPU-accelerated by modern browser engines for 60 fps performance.

SVG Animation (SMIL & Scripted):
Animating vector DOM elements programmatically via JavaScript or
inline CSS attributes.

HTML5 Canvas & WebGL:
Immediate-mode scriptable rendering context for complex particle
systems, 2D gaming loops, and hardware-accelerated 3D scenes.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

346 / 533

Summary & Comparative Overview I

Core Takeaways:
Animation leverages perceptual phenomena
(Persistence of Vision + Phi Phenomenon) to simulate motion.
Disney’s 12 principles remain fundamental across 2D, 3D, and UI
animation domains.
Selecting the appropriate animation technique and delivery format
balances visual quality, interactivity, and bandwidth.

Quick Reference Selection:
Vector UI / Dynamic Icons: Lottie / CSS3 / SVG.
Short Graphical Loops: WebP / APNG.
Photorealistic Motion: H.264 / AV1 Video Containers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

347 / 533

Introduction & The Power of Motion I
Definition of Animation: The rapid display of a sequence of 2D or
3D artwork or images to create an illusion of movement.
The Power of Motion:
Captures human visual attention more effectively than static visual
elements.
Expresses complex temporal changes, storytelling, and mechanical
operations.
Enhances user experience (UX) through intuitive feedback,
micro-interactions, and visual continuity.

Applications Across Domains:
Entertainment: Feature films, visual effects (VFX), and interactive
video games.
Education & Simulation: Scientific visualization, medical training, and
flight simulators.
User Interfaces: Mobile transitions, web micro-animations, and system
status indicators.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

348 / 533

Animation Fundamentals & Biological Basis I

Biological Foundation:
Persistence of Vision: The physiological phenomenon where the
human eye retains an image for a fraction of a second after its
removal.
Phi Phenomenon: The psychological optical illusion of perceiving
continuous motion between separate stationary images displayed in
rapid succession.

Core Concepts:
Frames: Individual static images forming a complete animation
sequence.
Frame Rate (FPS): Frames Per Second dictates the temporal
smoothness of visual perception.
Cinematic Film Standard: 24 FPS.
Broadcast Television (NTSC / PAL): 29.97 / 25 FPS.
Digital Video & Gaming: 30, 60, or 120 FPS.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

349 / 533

Key Concepts: Keyframing & Interpolation I

Keyframes:
Critical frames defining the starting, ending, or pivotal points of a
smooth transition.
Historically created by lead animators (master artists).

In-betweening (Tweening):
Generating intermediate frames between keyframes to create fluid
motion.
Traditionally rendered by assistant animators; now computed
automatically by graphics software.

Mathematical Interpolation:
Linear Interpolation: Constant velocity between keyframes;
movement appears rigid or robotic.
Bézier / Spline Curves: Smooth acceleration and deceleration
mimicking real-world physics.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

350 / 533

12 Principles of Animation (Part 1) I
1. Squash and Stretch:
Gives weight, flexibility, and mass to drawn objects. Volume must
remain consistent.

2. Anticipation:
Prepares the audience for a major action (e.g., bending knees prior to
a jump).

3. Staging:
Directs the audience focus and clarifies the primary focal idea within a
scene layout.

4. Straight Ahead Action & Pose to Pose:
Straight Ahead: Animating frame-by-frame sequentially; ideal for
fluid, unpredictable motion.
Pose to Pose: Drawing key poses first, then filling intervals; ideal for
structured scenes.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

351 / 533

12 Principles of Animation (Part 2) I

5. Follow Through and Overlapping Action:
Follow Through: Unattached or trailing parts continue moving after
the main body stops.
Overlapping Action: Distinct parts of an object move at varying rates.

6. Slow In and Slow Out (Ease In / Ease Out):
Gradual movement startup and deceleration to reflect real-world
inertia and momentum.

7. Arcs:
Natural mechanical motion follows curved trajectories rather than
linear paths.

8. Secondary Action:
Subordinate gestures that enrich and support the primary action (e.g.,
whistling while walking).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

352 / 533

12 Principles of Animation (Part 3) I

9. Timing:
The total number of frames assigned to an action, establishing speed,
weight, and emotion.

10. Exaggeration:
Pushing physical movements beyond strict reality to heighten
theatrical and artistic impact.

11. Solid Drawing:
Applying 3D principles of spatial depth, volume, balance, and shadow
to 2D artwork.

12. Appeal:
Infusing characters with charisma, clear visual design, and emotional
connection.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

353 / 533

Traditional vs. Computer Animation I

Traditional Animation (Cel Animation):
Hand-drawn on transparent celluloid sheets (cels) and photographed
over background art.
High labor intensity, requiring thousands of physical drawings per
minute of screen time.

Computer-Assisted 2D Animation:
Digital software replaces physical paper, providing automated
ink-and-paint, layering, and tweening.

Computer-Generated Animation (3D CGI):
Virtual 3D space creation involving polygon mesh modeling, skeletal
rigging, and lighting simulation.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

354 / 533

2D Animation Techniques & Morphing I

Vector-Based Animation:
Uses mathematical expressions for points, lines, and curves instead of
fixed pixel arrays.
Resolution-independent, lightweight, and scalable for dynamic user
interfaces.

Cut-Out Animation:
Characters constructed from segmented 2D body parts attached to a
skeletal hierarchy.

Morphing & Shape Tweening:
Smoothly transforming one source image or vector shape into a
completely different target shape.
Involves mapping source control points to target control points across
intermediate frames.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

355 / 533

3D Animation Pipeline I

1. Modeling: Constructing 3D surface geometry meshes using
polygons or NURBS curves.
2. Texturing & Shading: Applying color maps, surface relief, and
optical reflection properties.
3. Rigging & Skinning:
Building an internal hierarchical skeleton (bones and joints).
Binding the outer 3D mesh surface to the bone joints (skinning).

4. Keyframing & Motion Capture:
Manual joint positioning or capturing physical motion data from
actors using sensor suits.

5. Simulation & Rendering:
Computing dynamic physics (cloth, hair, fluids) and ray-traced
lighting into final 2D frames.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

356 / 533

Animation File Formats (Raster & Vector) I

Graphics Interchange Format (GIF):
Legacy raster animation format; capped at 256 colors per frame;
supports binary transparency.

APNG (Animated PNG):
Extension of PNG; supports full 24-bit truecolor images and 8-bit
alpha channel transparency.

Lottie / JSON:
JSON-based vector format exported from Adobe After Effects;
extremely small payload size.

SVG Animation (SMIL / CSS / JS):
Scalable Vector Graphics manipulated directly via web CSS keyframes
or JavaScript.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

357 / 533

Video-Based Formats & Format Comparison I

Video Containers for Animation (MP4 / WebM):
Utilize advanced inter-frame temporal compression algorithms (H.264,
H.265, VP9, AV1).
Far higher compression efficiency than raster GIF for complex
photographic or long animations.

Format Selection Criteria:
File Size Efficiency: Lottie / SVG < MP4 / WebM < APNG < GIF.
Interactivity: SVG / Lottie allow real-time DOM scripting;
pre-rendered video does not.
Visual Fidelity: APNG and Lottie maintain crisp alpha transparency
and vector scalability.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

358 / 533

Summary & Key Takeaways I

Core Principles Summary:
Animation rests on visual perception phenomena: Persistence of
Vision and Phi Phenomenon.
The 12 Principles of Animation govern fluid, natural, and emotionally
expressive movement across all mediums.
Keyframing combined with digital tweening automates labor-intensive
frame generation.
Choice of file format depends on trade-offs between raster resolution,
alpha channels, vector scalability, and file footprint.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

359 / 533

Introduction: The Power of Motion in Multimedia I

Definition of Animation: The process of creating the illusion of
motion and change by rapidly displaying a sequence of static images.
Cognitive Impact of Motion:
Directs viewer attention to key user interface elements.
Enhances storytelling, emotional engagement, and temporal
awareness.
Communicates state changes, progress, and spatial relationships.

Applications in Digital Media:
UI/UX micro-interactions and visual feedback.
Educational visualizations and technical simulations.
Entertainment, game engines, and digital marketing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

360 / 533

Animation Fundamentals: Vision & Frame Rates I

Biological Basis:
Persistence of Vision: Retinal memory retains an image for
approximately 1/16th of a second after visual stimulus ceases.
Phi Phenomenon: The brain perceives continuous motion between
rapidly alternating static images.

Frame Rates (Frames Per Second - FPS):
12 FPS (”On Twos”): Minimum threshold for acceptable motion
illusion; common in traditional 2D animation.
24 FPS: Standard cinematic rate for film and animation.
30 FPS: Broadcast television standard (NTSC/digital).
60+ FPS: Real-time graphics, high-end gaming, and smooth UI
transitions.

Flicker Fusion Threshold: Frequency at which an intermittent light
source appears continuous to the human eye (≈ 50–60 Hz).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

361 / 533

12 Principles of Animation: Part 1 I

1. Squash and Stretch:
Imparts weight, flexibility, and volume to animated objects.
Critical constraint: Total volume must remain constant during
distortion.

2. Anticipation:
Prepares the audience for a major action (e.g., bending knees before
jumping).

3. Staging:
Directs audience attention and clarifies the main element of a scene.

4. Straight Ahead Action vs. Pose to Pose:
Straight Ahead: Frame-by-frame creation for dynamic/unpredictable
motion.
Pose to Pose: Drawing key frames first, then filling inbetween frames.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

362 / 533

12 Principles of Animation: Part 2 I
5. Follow Through and Overlapping Action:
Follow Through: Unattached body parts continue moving after a
character stops.
Overlapping Action: Different parts of the body move at different
rates.

6. Slow In and Slow Out (Ease In / Ease Out):
Accelerating into poses and decelerating out to simulate physical
inertia.

7. Arcs:
Natural actions follow curved trajectories rather than straight linear
paths.

8. Secondary Action:
Adds dimension to the main action (e.g., swinging arms while
walking).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

363 / 533

12 Principles of Animation: Part 3 I

9. Timing:
Number of frames allocated to an action; establishes physical laws
(weight, speed) and emotion.

10. Exaggeration:
Accentuating reality in visual form to heighten dramatic or comedic
effect.

11. Solid Drawing:
Respecting 3D space, volume, weight, shadow, and anatomy in 2D
forms.

12. Appeal:
Creating charismatic design, engaging personality, and visual interest
for characters.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

364 / 533

Traditional vs. Physical Animation Techniques I

Traditional Cel Animation:
Hand-drawn individual frames on transparent celluloid sheets.
Layering background, middleground, and foreground elements.

Stop Motion Animation:
Physical manipulation of real-world objects photographed
frame-by-frame.
Claymation: Malleable plasticine models over wire armatures.
Cut-out Animation: Flat materials arranged on planes.
Puppetry / Object Animation: Rigid or articulated physical figurines.

Rotoscoping:
Tracing live-action footage frame-by-frame for realistic movement.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

365 / 533

Computer-Assisted & Digital Animation Workflow I
Keyframing:
Animators define critical start and end positions (Keyframes).

Inbetweening (Tweening):
Automated or manual creation of intermediate frames between
keyframes.
Mathematical interpolation curves: Linear, Ease-In, Ease-Out, Bezier.

Rigging & Kinematics:
Forward Kinematics (FK): Rotating parent joint affects all child
joints.
Inverse Kinematics (IK): Positioning end-effector automatically
calculates joint angles.

Morphing:
Smooth transformation of one shape/image into another via vector
mapping.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

366 / 533

2D Vector vs. 3D Computer Graphics Pipelines I

2D Vector Animation:
Resolution-independent mathematical paths (curves, fills, strokes).
Low storage footprint, highly dynamic, ideal for web/mobile UI.

3D Computer Graphics Pipeline:
Modeling: Constructing polygonal meshes in 3D coordinate space.
Texturing & Shading: Surface optical property definition.
Animation / Motion: Skeleton binding (skinning) and pose control.
Lighting & Rendering: Ray-tracing/rasterization to 2D image
sequence.

Motion Capture (MoCap):
Tracking physical actor movements with sensors to drive digital rigs.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

367 / 533

Animated File Formats: Raster & Vector I

GIF (Graphics Interchange Format):
8-bit color palette limit (256 colors), lossless LZW compression.
Simple frame delay metadata; legacy web standard, high file size.

APNG (Animated Portable Network Graphics):
24-bit color support + 8-bit alpha transparency.
Backward compatible with static PNG decoders.

WebP (Animated):
Modern format supporting lossy/lossless compression with alpha
channel.
Significantly smaller file sizes compared to GIF/APNG.

Lottie / JSON Vector Format:
JSON-based animation data exported from vector tools.
Ultra-lightweight, infinitely scalable, client-rendered via JavaScript.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

368 / 533

Video-Based Animation & Codecs I

HTML5 / Web Video Containers:
MP4 (H.264 / HEVC): High compression efficiency, universal
playback.
WebM (VP9 / AV1): Open-source, high performance for modern
web browsers.

ProRes 4444 / Animation Codecs:
Uncompressed/near-lossless formats with alpha channel for video
compositing.

Sprite Sheets:
Grid of sequential animation frames merged into a single image asset.
Rapid GPU texture mapping in game engines and web Canvas
rendering.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

369 / 533

Technical Trade-offs & Optimization Strategies I

Bandwidth vs. Fidelity:
Vector (Lottie/SVG) for UI components; Video (MP4/WebM) for rich
scenes.

Performance Optimization:
Offloading rendering to GPU acceleration (CSS transforms, WebGL).
Reducing keyframe density and optimizing vector path node counts.

Accessibility & UX Guidelines:
Honoring prefers-reduced-motion media queries in web
environments.
Preventing vestibular trigger events by limiting persistent
flickering/flashing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

370 / 533

Summary & Key Takeaways I

Power of Motion: Animation enhances visual hierarchy,
engagement, and UI state communication.
Foundational Laws: Disney’s 12 principles bridge physical reality
and creative storytelling.
Evolution of Techniques: Transition from hand-drawn cels to vector
tweening, 3D rigs, and MoCap.
Modern Delivery Formats: Choosing the right format (Lottie,
WebP, MP4, Sprite Sheets) balances performance and visual quality.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

371 / 533

Introduction to Dynamic Motion & Physical Realism I

Overview
Animation relies on conveying physical believability through motion
dynamics rather than strict adherence to real-world physics calculations.
Believability vs. Accuracy: Audience perception demands
convincing mechanics, weight, and momentum over exact numerical
simulation.
Fundamental Test Cases:
Rolling ball (continuous translation with rotational synchronization).
Bouncing ball (gravity, acceleration, deformation, and energy decay).
Scene composition (combining multiple dynamic elements seamlessly).

Core Objective: Mastering timing, spacing, arcs, and volume
preservation to build believable animated sequences.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

372 / 533

Physics Principles in Computer Animation I
Newtonian Mechanics for Animators
Understanding physical forces provides a solid foundation for keyframing
realistic weight and momentum.
2

Gravity & Freefall: Constant downward acceleration (g ≈ 9.8 m/s )
resulting in non-linear spacing between frames.
Inertia & Momentum: Objects resist changes in state of motion;
heavier objects require more force to accelerate or stop.
Friction & Surface Resistance: Drag forces reduce forward
momentum, causing sliding objects to transition into rolling or
stopping.
Deformation Dynamics: Impact forces generate stress, resulting in
temporary elastic strain (squash) followed by kinetic rebound
(stretch).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

373 / 533

Mechanics of a Rolling Ball: Rotation vs. Translation
I
Synchronizing Arc Length and Angular Velocity
A ball rolling without slipping must strictly synchronize linear displacement
with rotational angle.
No-Slip Condition Equation:
s =R ·θ

⇒

v =R ·ω

where s is linear distance, R is radius, θ is rotation angle in radians, v
is linear velocity, and ω is angular velocity.
Keyframe Calculation:
If the ball travels distance ∆x, it must rotate by ∆θ = ∆x
radians
R
∆x
(360◦ × 2πR
).

Visual Slippage Pitfall: Incorrect rotation rates cause the object to
appear to slide or spin on ice.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

374 / 533

Keyframing & Spacing for Rolling Motion I

Easing and Motion Curves
Controlling ease-in and ease-out handles ensures natural momentum
accumulation and friction decay.
Acceleration Phase (Ease-In):
Denser keyframes initially, gradually widening frame spacing as
velocity increases.
Rotational curve must match the translation graph identically.

Deceleration Phase (Ease-Out):
Exponential decay in translation spacing caused by rolling friction.
Rotational speed decays proportionally to prevent post-stop spinning.

Surface Incline Adjustments: Slopes alter linear acceleration
(⃗a = g sin θ), requiring parabolic keyframe curves.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

375 / 533

Mechanics of a Bouncing Ball: Squash and Stretch I
Deformation and Volume Preservation
Squash and stretch conveys weight, material rigidity, and impact force
without altering perceived volume.
Pre-Impact Stretch: As velocity reaches maximum just before
contact, elongation occurs along the vector of motion.
Point of Contact & Squash: Kinetic energy transforms into
deformation energy, flattening the sphere horizontally while preserving
volume.
Constant Volume Constraint:
V =

4
πrx ry rz = Constant
3

=⇒

∆Y · ∆X 2 = 1

Material Properties: Hard steel balls experience minimal
deformation; rubber balls showcase pronounced squash/stretch.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

376 / 533

Timing & Spacing in Bouncing Dynamics I

Parabolic Trajectories and Frame Spacing
Gravity dictates parabolic flight arcs with distinct frame density variations.
Apex (Peak of Jump):
Vertical velocity becomes zero (vy = 0).
Spacing is tightly bunched (”hang time”), accentuating weightlessness
at the peak.

Descent and Ascent:
Acceleration increases frame spacing downward (d ∝ t 2 ).
Fastest movement occurs immediately before and after impact.

Horizontal Velocity (vx ): Remains relatively constant across
bounces unless friction or wind drag act on the projectile.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

377 / 533

Energy Dissipation & Coefficient of Restitution I

Simulating Energy Loss across Successive Bounces
Real-world collisions dissipate kinetic energy into heat and sound, lowering
subsequent bounce heights.
Coefficient of Restitution (e):
r
vafter
hn+1
e=
=
vbefore
hn

(0 < e < 1)

Height Decay Pattern:
Each bounce height hn+1 = e 2 · hn .
Time interval between impacts decreases proportionally: Tn+1 = e · Tn .

Final Settlement: Frequency of micro-bounces increases rapidly
until motion dampens completely into a roll or stop.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

378 / 533

Designing Animated Scenes: Staging & Composition
I

Directing Viewer Focus and Narrative Flow
Staging ensures that every motion, pose, and background element serves
the scene’s primary narrative objective.
Rule of Thirds & Focal Points: Placing key motion paths along
compositional intersections guides line-of-sight naturally.
Anticipation & Readability:
Preceding major actions with counter-movement (e.g., pulling back
before rolling forward).
Ensuring silhouettes are clear and uncluttered by secondary action.

Screen Direction Consistency: Maintaining standard vector
direction across cuts prevents spatial disorientation.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

379 / 533

Foreground, Midground, & Background Layering I

Creating Depth and Parallax in 2D/3D Spaces
Multi-plane visual staging builds immersive, three-dimensional
environmental scale.
Background Layer: Distant environment moving at minimal velocity
relative to camera (parallax scrolling).
Midground Layer: Primary action plane housing the main characters
and key interactions (e.g., rolling/bouncing ball).
Foreground Layer: Fast-moving framing elements (foliage,
structures) that enhance speed perception.
Atmospheric Perspective: Desaturating background layers and
lowering contrast deepens visual distance.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

380 / 533

Environmental Interaction & Secondary Motion I

Integrating Objects with Dynamic Backgrounds
An animated scene feels alive when primary motion triggers reactions
throughout the surrounding scene.
Environmental Impacts:
A bouncing ball impacting dirt leaves dust particles, ripples water, or
bends grass.
Heavy collisions cause subtle camera shakes or surface deformations.

Follow-Through & Overlapping Action:
Attached or trailing elements (tails, ribbons, shadows) lag behind
primary force vectors.

Shadow Consistency: Ground shadows dynamically scale, soften,
and align with environmental light sources.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

381 / 533

Workflow Pipeline: From Storyboard to Render I
Systematic Production Stages for Scene Creation
A structured pipeline prevents compounding errors in timing, layout, and
composition.
1. Storyboard & Animatic: Rough sketch sequences set to timed
audio tracks to establish pace and camera cuts.
2. Layout & Blocking: Establishing key poses, extreme positions,
and spatial paths for major elements.
3. Inbetweening & Splining: Refining motion curves and
interpolation handles for fluid frame transitions.
4. Secondary Animation & FX: Adding particle systems,
squash/stretch tweaks, and cloth/fluid responses.
5. Lighting, Rendering, & Compositing: Final shading, depth of
field, motion blur integration, and layer blending.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

382 / 533

Summary & Key Takeaways I

Core Principles of Effective Animation
Mastering fundamental exercise tasks unlocks the ability to build complex,
believable visual stories.
Rolling Motion: Perfect synchronization between linear translation
(s) and rotational angle (θ).
Bouncing Motion: Application of gravity acceleration, parabolic
arcs, squash/stretch volume preservation, and energy decay (e).
Scene Assembly: Strategic staging, parallax layering, depth cues,
and reactive environmental secondary motion.
Ultimate Goal: Believability over strict simulation, driven by
masterfully controlled timing, spacing, and staging.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

383 / 533

Principles of Effective Animation I
Fundamental Animation Principles: Practical application of
classical principles to create believable motion and physical realism.
Squash and Stretch: Gives rigidity or elasticity to an object;
preserves overall volume while indicating force, impact, and material
composition.
Timing and Spacing:
Timing : The total duration or number of frames allocated for an
action.
Spacing : The displacement of an object between consecutive frames,
defining acceleration and velocity.

Ease-In and Ease-Out: Simulates real-world inertia where objects
gradually accelerate from rest and decelerate before stopping.
Arcs: Natural physical movement follows curved trajectories rather
than linear paths due to gravity and angular momentum.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

384 / 533

Physics of Motion in Animation I

Kinematics and Force Simulation: Translating physical laws
(Newtonian mechanics) into visual timing and frame distribution.
Mass and Inertia: Higher mass requires more frames to accelerate
(ease-out) and decelerate (ease-in), establishing visual weight.
Gravitational Acceleration: Constant downward acceleration
2
(g = 9.8 m/s ) dictates quadratic spacing on vertical movement.
Friction and Resistance: Air resistance and surface friction reduce
velocity over time, altering horizontal frame spacing.
Believability vs. Realism: Exaggerating physical principles enhances
viewer perception without breaking physical plausibility.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

385 / 533

Mechanics of a Rolling Ball I

Dual Motion Dynamics: A rolling ball combines linear translation
along a surface with rotational motion around its center.
No-Slip Condition: Pure rolling requires exact synchronization
between linear displacement (s) and angular rotation (θ):
Linear distance covered per full revolution: s = 2πr .
Angular displacement equation: θ = sr (in radians).

Visual Indicators of Rotation: Smooth uniform spheres require
surface details (stripes, shading, highlights, or textures) to show
rotational movement.
Surface Contact Point: The point of contact with the ground has
zero instantaneous velocity relative to the surface.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

386 / 533

Animating the Rolling Ball I

Step 1: Establishing the Path and Distance: Define the linear
trajectory and calculate total distance across the frame layout.
Step 2: Keyframing Linear Position:
Position keyframes at start, intermediate acceleration points, and rest.
Apply velocity curves (ease-in/ease-out) to match applied impulse or
friction.

Step 3: Synchronizing Rotation: Match rotation keyframe values
directly to linear displacement to prevent visual slipping or skidding.
Step 4: Surface Contact Realism: Account for slight compression
or squashing on softer surfaces (e.g., grass or sand).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

387 / 533

Mechanics of a Bouncing Ball I

Parabolic Trajectory: Combining constant horizontal motion with
vertical gravitational motion yields a symmetrical parabolic path.
Energy Loss and Coefficient of Restitution (e):
Kinetic energy converts to elastic deformation energy upon collision.
Post-impact peak height decreases exponentially: hn+1 = e 2 hn .

Deformation at Impact: Maximum deformation occurs at full
ground contact; kinetic energy is temporarily stored as elastic
compression.
Velocity Vector Inversion: Vertical velocity drops rapidly to zero at
maximum compression before reversing direction upward.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

388 / 533

Keyframing and Timing the Bouncing Ball I

Spacing Breakdown along the Arc:
Apex (Top of Arc): Slow motion (dense frame spacing) due to low
vertical velocity.
Descent/Ascent: Fast motion (wide frame spacing) near ground level.

Squash and Stretch Implementation:
In-Flight: Stretch along the path of movement as velocity peaks
immediately before/after impact.
Impact Frame: Flatten/squash horizontally while preserving total
volume (V ∝ r 2 h = constant).
Apex Frame: Return to perfect spherical shape.

Impact Sharpness: Use sharp direction changes (clamped tangents)
at collision frames rather than smooth bezier curves.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

389 / 533

Material Properties and Weight Dynamics I

Heavy Objects (e.g., Bowling Ball / Lead Sphere):
High mass, minimal elastic deformation (negligible squash/stretch).
Fast descent, minimal rebound height, rapid energy dissipation upon
impact.

Elastic/Light Objects (e.g., Rubber Ball / Tennis Ball):
High elasticity, prominent squash on impact, high rebound ratio.
Extended bouncing sequence with gradual height decay.

Special Dynamics (e.g., Water Balloon / Ping Pong Ball):
Ping pong: Rapid cadence, low inertia, highly reactive rebound.
Water balloon: Extreme volume deformation, internal fluid drag,
delayed shape recovery.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

390 / 533

Designing an Animated Scene: Staging and
Composition I

Staging Principles: Presenting an idea clearly so that the visual
emphasis is unambiguous to the audience.
Spatial Composition and Depth:
Foreground: Establishes scale, framing elements, and immediate
context.
Midground: Primary stage for active character or object motion.
Background: Environment, distant horizon, and atmosphere.

Line of Action: An imaginary line guiding the viewer’s eye through
main visual movement vectors across the frame.
Focal Point Control: Contrast, framing, motion paths, and lighting
focus viewer attention on key active elements.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

391 / 533

Secondary Action and Overlapping Motion I

Secondary Action: Subordinate motion that supports and enhances
the main action without distracting from it.
Follow-Through and Overlapping Action:
Follow-Through: Unattached or flexible parts continue moving after
the main body stops.
Overlapping Action: Different body parts or environmental elements
move at different rates.

Anticipation: Preparing the audience for an action (e.g., crouching
down before a high bounce or leap).
Practical Application: A rolling ball colliding with dynamic scene
elements (e.g., striking dominoes or pushing flexible foliage).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

392 / 533

Environmental Dynamics and Parallax I

Multiplane Parallax Scrolling: Moving background layers at varying
speeds relative to the camera to simulate 3D depth.
Foreground objects move faster across the screen than background
elements.

Dynamic Lighting and Shadows:
Ground contact shadows scale and deform with object height and
compression.
Light shifts accentuate rotational detail and spatial altitude during
bounces.

Environmental Interaction: Dust puffs, ground compression marks,
splash ripples, and motion blur streaks enhance scene realism.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

393 / 533

Production Workflow for Scene Creation I

Phase 1: Storyboarding & Concept Design: Rough visual
breakdown of beats, camera angles, and action timing.
Phase 2: Layout & Blocking:
Setting up environment geometry, camera positions, and safe margins.
Creating extreme keyframes for primary objects (rolling/bouncing
sequences).

Phase 3: In-Betweening & Splining: Generating breakdown
frames, refining velocity curves, and perfecting rotational
synchronization.
Phase 4: Polish & Render: Adding secondary elements,
environmental FX, lighting, shading, and compositing final frames.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

394 / 533

Summary and Practical Guidelines I

Core Takeaways:
Mastering basic exercises (rolling ball, bouncing ball) provides
foundation for complex character and physics animation.
Physics calculations (gravity, rotation matching, volume preservation)
guide spacing and timing choices.

Practical Lab Task:
Construct a multi-stage animated scene: A ball rolls down an incline,
bounces over an obstacle, and comes to rest.
Apply proper squash, stretch, rotation sync (θ = s/r ),
ease-in/ease-out curves, and parallax background layers.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

395 / 533

Lecture Overview & Learning Objectives I
Unit Focus: Multimedia Information Sharing in modern digital
ecosystems.
Lecture Scope: Analysis of online social media sharing
infrastructures, protocols, and content paradigms.
Key Learning Objectives:
Define and differentiate User-Generated Content (UGC) and
User-Generated Media Content (UGMC).
Examine structural models and graph representations of Online Social
Networks (OSNs).
Evaluate multimedia transcoding, delivery architectures (CDNs, Edge
Caching), and scalability bottlenecks.
Explore algorithmic recommendation systems, content distribution
feeds, and social graphs.
Address security, copyright enforcement (fingerprinting, DRM), and
privacy preservation techniques.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

396 / 533

Taxonomy of Social Media Services I
Classification Framework:
User-Generated Content (UGC) Platforms: Focus on creation,
aggregation, and dissemination of user-authored media (e.g., text,
images, video).
Online Social Networking (OSN) Services: Focus on explicit
relationship building, profile identity, and social graph propagation.
User-Generated Media Sharing Platforms: Specialized UGC
platforms optimized for high-bandwidth multimedia (video streams,
high-resolution imagery, podcasts).

Core System Attributes:
Asymmetric vs. Symmetric Connections: Directed edges (followers)
vs. undirected edges (friends).
Interaction Paradigms: Push-based newsfeeds, pull-based
search/discovery, and algorithmic stream filtering.
Data Heterogeneity: Multimodal content requiring real-time
indexing, metadata extraction, and adaptive streaming.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

397 / 533

User-Generated Content (UGC): Foundations I
Definition & Core Characteristics:
Media content created directly by end-users rather than professional
publishers or traditional broadcasters.
High velocity, massive volume, and high variability in production
quality and format.

The UGC Value Chain:
Authoring: Mobile capture, client-side filtering, lightweight editing.
Ingestion: Multi-protocol upload (HTTP POST, WebSocket,
chunked streaming).
Processing: Automated moderation, feature extraction, transcoding
into multiple bitrates.
Distribution: Global edge delivery via content distribution networks.

Challenges in UGC Management:
Scale: Millions of concurrent uploads per minute.
Metadata Sparsity: Sparse or noisy user labels requiring deep learning
automated tagging.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

398 / 533

UGC Ingestion, Processing & Storage Pipeline I
Ingestion Tier:
Resumable chunked uploads over HTTP/2 or HTTP/3 to mitigate
transient network drops.
Edge upload acceleration utilizing nearby Point of Presence (PoP)
servers.

Processing Tier:
Asynchronous message queues (e.g., Kafka, RabbitMQ) decoupling
upload from heavy processing.
Distributed video/image transcoding worker clusters generating
Adaptive Bitrate (ABR) profiles (e.g., H.264/AVC, H.265/HEVC,
AV1, VP9).

Storage Architecture:
Blob Storage: Object stores (e.g., AWS S3, Ceph) for raw and
encoded media files.
Metadata Database: Distributed NoSQL/SQL databases for
metadata, annotations, and indexing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

399 / 533

Online Social Networks (OSN): Structural Models I

Social Graph Formalism:
Modeled as a graph G = (V , E ), where V represents users (nodes)
and E represents social relationships (edges).
Directed graphs represent asymmetric relationships (e.g., Twitter/X
followers).
Undirected graphs represent mutual friendships (e.g., Facebook
connections).

Graph Topology Properties:
Small-World Phenomenon: High clustering coefficient with short
average path length (six degrees of separation).
Power-Law Degree Distribution: Scale-free network dynamics where
node degree distribution follows P(k) ∼ k −γ .
Community Structure: Densely connected subgraphs forming distinct
interest or social groups.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

400 / 533

Graph-Based Modeling of OSN Dynamics I

Information Diffusion Models:
Independent Cascade Model (ICM): Probabilistic activation of
neighboring nodes upon content sharing.
Linear Threshold Model (LTM): Node activation occurs when
cumulative influence of active neighbors exceeds a threshold.

Centrality Metrics for Influence Analysis:
Degree Centrality: Measures immediate reach of a node
(CD (v ) = deg(v )).
Betweenness Centrality:PQuantifies node role as a bridge over
shortest paths (CB (v ) = s̸=v ̸=t σσst st(v ) ).
PageRank / Eigenvector Centrality: Evaluates recursive influence
based on incoming link quality.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

401 / 533

User-Generated Media Content Sharing Platforms I

Characteristics of Media-Centric Sharing:
High bandwidth utilization requiring efficient multimedia compression.
Latency sensitivity: Low startup time and zero buffering expectations
for short-form/long-form video.

Short-Form vs. Long-Form Media Delivery:
Short-Form Video (e.g., TikTok, Instagram Reels): High-frequency
swipe interactions, aggressive prefetching, micro-chunk caching.
Long-Form Video (e.g., YouTube): Dynamic adaptive streaming
(DASH/HLS), tiered caching, variable segment durations.

Multimodal Media Integration:
Simultaneous processing of visual streams, audio tracks,
closed-caption text, and user comments.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

402 / 533

Media Delivery Infrastructure: CDNs & Edge
Caching I
Content Delivery Networks (CDNs):
Geographically distributed edge nodes caching popular UGC to reduce
origin server load and latency.
Anycast IP routing for directing client requests to the nearest edge
server.

Caching Strategies for UGC:
Zipfian Popularity Distribution: A small percentage of viral media
accounts for the majority of requests (P(r ) ∝ 1/r α ).
Long-Tail Content Challenge: Caching strategies must handle
millions of obscure, low-view videos without cache pollution.

Hybrid CDN-P2P Architectures:
Peer-assisted content distribution to offload edge bandwidth during
live streaming events.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

403 / 533

Algorithmic Feed Filtering & Content
Recommendation I
Feed Architecture:
Fan-out on Write: Pushing content to all follower timelines upon
creation (suitable for low-degree publishers).
Fan-out on Read: Fetching content from followees upon user login
(suitable for high-degree/celebrity accounts).

Recommendation Algorithms:
Collaborative Filtering: Matrix factorization and graph neural
networks (GNNs) modeling user-item interactions.
Content-Based Filtering: Deep feature embedding extraction from
media (CNNs/Transformers) matched to user interest vectors.

Engagement Optimization:
Multi-objective ranking balancing click-through rate (CTR), watch
time, shares, and comments.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

404 / 533

Security, Privacy & Copyright Protection I

Copyright Identification & Enforcement:
Acoustic and visual fingerprinting (e.g., Content ID) for automated
copyright detection.
Perceptual hashing algorithms (e.g., pHash, PhotoDNA) to flag illicit
or copied media.

Privacy Preservation:
Access control lists (ACLs), relationship-based permission models.
Anonymization and differential privacy in social graph mining and
data analytics.

Trust and Safety:
Automated deepfake detection, fake news mitigation, and deep
multimodal content moderation.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

405 / 533

Summary & Future Research Directions I

Summary of Core Concepts:
Social media sharing relies on robust UGC ingestion, scalable OSN
graph representations, and optimized delivery networks.
User-generated media content sharing requires sophisticated adaptive
bitrate streaming, caching, and algorithmic feeds.

Emerging Trends & Future Directions:
Generative AI UGC: AI-assisted and AI-generated media shifting
content creation dynamics.
Decentralized Social Networks: Protocols (e.g., ActivityPub, AT
Protocol) decentralizing graph ownership.
Immersive Media Sharing: 3D volumetric video, AR/VR social
environments, and spatial computing infrastructure.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

406 / 533

Introduction to Multimedia Information Sharing I
Evolution of Digital Media Sharing:
Shift from traditional broadcast media (one-to-many) to interactive
digital ecosystems (many-to-many).
Proliferation of high-speed mobile networks (4G/5G) enabling
ubiquitous media uploading and streaming.

Core Pillars of Social Media Infrastructure:
User-Generated Content (UGC): Media assets created, published, and
consumed directly by end users.
Online Social Networking (OSN): Structural frameworks modeling
user identities, relationships, and interactions.
User-Generated Media Content Sharing: Distributed streaming and
storage platforms optimized for rich media formats.

Key Technical Challenges:
Scalable storage and bandwidth provisioning.
Real-time processing, transcoding, and adaptive bit-rate streaming.
Metadata management and efficient graph queries for social feeds.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

407 / 533

User-Generated Content (UGC) Fundamentals I
Definition and Characteristics:
Content created outside of professional routines and commercial
production environments.
High degree of heterogeneity in media quality, resolution, format, and
temporal length.
Asymmetric creation-consumption patterns (long-tail distribution of
popularity).

Taxonomy of UGC Media Types:
Micro-blogs and text-rich posts accompanied by embedded
images/videos.
Short-form video reels and live video broadcasts.
Audio clips, podcasts, and interactive spatial media.

Lifecycle of UGC Assets:
Creation → Ingestion → Processing/Transcoding → Storage →
Distribution → Consumption.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

408 / 533

Content Creation and Ingestion Architectures I

Client-Side Pre-processing:
On-device compression, resizing, and feature extraction (e.g., edge AI
filters).
Chunked upload protocols to handle network volatility and resume
failed transfers.

Ingestion Gateway Layer:
Load balancing and edge access points optimizing network latency.
Immediate metadata validation, sanitization, and virus scanning.

Asynchronous Transcoding Pipelines:
Queue-driven worker nodes converting raw uploads into multiple
bitrate profiles (e.g., H.264, H.265, AV1).
Dynamic segment generation (DASH/HLS) tailored for adaptive
bitrate (ABR) streaming.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

409 / 533

Online Social Networking (OSN) Foundations I
Structural Elements of OSNs:
User Profile Management: Unique identification, metadata, and
preference modeling.
Relationship Networks: Directed graphs (e.g., follower models) vs.
undirected graphs (e.g., friend models).

Feed Generation Strategies:
Push Model (Fan-out on write): Writes activity to all followers’
timelines; fast read, high write overhead for celebrity nodes.
Pull Model (Fan-out on read): Aggregates activity from followees
upon user login; low write overhead, high read latency.
Hybrid Feed Architecture: Dynamic routing based on user node
degree (follower count).

Interactions and Feedback Loops:
Explicit signals (likes, shares, comments) vs. implicit signals (watch
time, scroll velocity).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

410 / 533

Social Graph Modeling and Topology I
Graph Theory Formulations:
Social Network modeled as G = (V , E ), where V represents user
nodes and E represents social connections.
Adjacency matrix representations and sparse matrix storage schemes.

Topological Properties of OSNs:
Small-World Property: Low average path length between any two
arbitrary nodes (L ∝ log |V |).
High Clustering Coefficient: High likelihood that neighbors of a node
are connected.
Power-Law Degree Distribution: Scale-free network dynamics
(P(k) ∼ k −γ ).

Graph Data Management:
Distributed graph databases (e.g., Neo4j, Apache Giraph, custom
in-memory stores).
Graph partitioning and caching techniques for sub-graph traversal.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

411 / 533

User-Generated Media Content Sharing Systems I

Architecture of Video/Image Platforms:
Representative Services: YouTube, Instagram, TikTok, Twitch.
Hybrid cloud-edge infrastructure balancing storage efficiency and
low-latency delivery.

Key Subsystems:
Storage Subsystem: Multi-tier storage (Hot storage for trending
content, Warm/Cold storage for archive).
Indexing Subsystem: High-throughput indexing engines supporting
multi-modal search queries.
Analytics Subsystem: Real-time telemetry processing for usage metrics
and event tracking.

Challenges in Large-Scale Media Sharing:
Explosive storage growth (109 + uploads daily).
Managing high flash-crowd events and viral content spikes.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

412 / 533

Media Delivery and CDN Integration I
Content Delivery Network (CDN) Role:
Cache caching hierarchies: Edge Nodes → Regional CDNs → Origin
Storage.
Reducing Backbone Traffic: Serving 80%+ of requests directly from
local PoPs (Points of Presence).

Caching Strategies for UGC:
Traditional caching (LRU, LFU) struggles due to heavy-tailed UGC
request distributions.
Popularity prediction models driven by social signals (retweets, shares,
acceleration rate).

Adaptive Streaming Infrastructure:
Dynamic Bitrate Switching via HTTP Live Streaming (HLS) and
Dynamic Adaptive Streaming over HTTP (DASH).
Quality of Experience (QoE) optimization minimizing stall ratio and
buffering delays.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

413 / 533

Recommendation Systems and Content Discovery I
Role of Recommendation Engines:
Filtering massive media catalogs to deliver personalized user feeds.
Maximizing user engagement metrics (click-through rate, dwell time,
session duration).

Algorithmic Paradigms:
Collaborative Filtering: Matrix factorization and deep neural
collaborative filtering based on user-item interaction matrices.
Content-Based Filtering: Extracting visual, acoustic, and text
embeddings via Deep Neural Networks (DNNs).
Graph-Based Recommendation: Random walks (e.g., Pixie) and
Graph Neural Networks (GNNs) over social graph topologies.

System Constraints:
Two-stage recommendation pipeline: Candidate Generation
(Retrieval) followed by Heavy Ranking.
Low latency constraints (< 100 ms response time).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

414 / 533

Security, Privacy, and Content Moderation I

Content Moderation and Compliance:
Automated filtering pipelines using multimodal classification
algorithms (computer vision, NLP).
Perceptual hashing (e.g., PhotoDNA, PDQ) for detecting
illegal/copyrighted material.

Privacy Concerns in OSN/UGC:
Granular Access Control Lists (ACLs) for social posts.
Privacy-preserving media sharing: Differential privacy, secure
multi-party computation, and metadata stripping (EXIF data
removal).

Security Threats:
Sybil attacks, botnets, spam campaigns, and synthetic deepfake media
proliferation.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

415 / 533

Performance Metrics and System Scalability I
System-Level Metrics:
Throughput and IOPS (Input/Output Operations Per Second) for
media ingestion.
Storage Efficiency: Deduplication ratios, compression efficiency, cold
data tiering percentage.
Latency: P99 tail latency for feed generation and video playback
startup.

User Experience (QoE) Metrics:
Video Rebuffer Frequency and Mean Time Between Rebuffers
(MTBR).
Initial Video Startup Delay (VSD).
Visual Quality Score (e.g., VMAF - Video Multi-Method Assessment
Fusion).

Scalability Design Patterns:
Horizontal sharding of databases and microservices architecture.
Decoupling compute and storage layers for independent elasticity.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

416 / 533

Summary and Emerging Trends I

Summary of Core Concepts:
UGC, OSN, and media content sharing form the technical backbone
of modern interactive social platforms.
Effective platforms integrate graph modeling, high-throughput media
ingestion, CDN caching, and AI-driven recommendations.

Emerging Paradigm Shifts:
Generative AI as a Co-Creator: AI-assisted and fully AI-generated
media content (AIGC).
Immersive Media Sharing: Volumetric video, 360-degree spatial media,
and Metaverse/AR/VR social integration.
Decentralized Social Networks: Federated protocols (e.g.,
ActivityPub, AT Protocol) shifting control from monolithic silos to
open ecosystems.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

417 / 533

Case Study Overview: YouTube Ecosystem I

Dominance in Multimedia Sharing: YouTube represents the
premier global User-Generated Content (UGC) platform, processing
massive concurrent ingestion and playback demands.
Core System Pillars:
Data formats, adaptive encoding, and rich metadata structures.
Content characteristics and statistical distributions of video dynamics.
Graph-theoretic properties (small-world phenomena) governing video
networks.
Economic and operational infrastructure from the creator and partner
perspective.

Lecture Objectives: Analyze the architectural mechanisms, graph
dynamics, and business engine underlying large-scale multimedia
information sharing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

418 / 533

YouTube Video Formats and Data Architecture I

Codec Evolution:
Legacy formats: Sorenson Spark and H.263 encapsulated in Flash
Video (FLV) containers.
Modern Era: H.264/AVC, VP9, and open-source AV1 for
high-efficiency multi-resolution compression.

Adaptive Bitrate Streaming (ABR):
Implementation of Dynamic Adaptive Streaming over HTTP (DASH).
Segmentation of media streams into discrete chunk segments (2–10
seconds).
Dynamic resolution switching across tiers (144p up to 4K/8K) based
on real-time network throughput.

Container Demuxing: Separation of video and audio streams at
edge servers to minimize transmission overhead and client buffer lag.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

419 / 533

Metadata Structures and Content Annotation I

Explicit Primary Metadata:
Textual attributes: Title, text description, tags, category, duration,
upload timestamp.
Spatial and temporal specifications: Resolution, framerate, aspect
ratio, audio channel configuration.

Implicit and Behavioral Metadata:
Engagement signals: View counts, watch time distribution, likes,
dislikes, and comment threads.
Automated indexing: Automatic Speech Recognition (ASR) closed
captions and visual frame/thumbnail embeddings.

Role in Information Retrieval: Metadata drives multi-modal search
indexing, candidate selection, and recommendation graph
construction.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

420 / 533

Statistical Characteristics of YouTube Content I

UGC vs. Traditional Video-on-Demand (VoD):
Traditional VoD (e.g., Netflix): Long-duration, high-bitrate,
professionally produced studio content.
UGC (YouTube): Extreme variance in video duration, technical
quality, encoding efficiency, and topic tail.

Heavy-Tailed Popularity Distributions:
Video popularity follows Zipf-like power-law distributions: A tiny
fraction of videos capture the vast majority of global views.
File duration and upload frequency exhibit long-tail distribution
characteristics.

Temporal Decay Dynamics:
Early view accumulation dictates long-term viral potential.
News/trending items suffer rapid exponential decay, whereas
evergreen educational videos maintain steady traffic.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

421 / 533

Storage, Caching, and CDN Delivery I

Massive Ingestion Scale: High upload throughput requires
distributed ingestion pipelines and asynchronous background encoding
queues.
Multi-Tiered Edge Caching:
Hot content: Cached at ISP-level edge nodes (Google Global Cache –
GGC) to minimize backbone latency.
Warm/Cold content: Stored in centralized Google data centers and
fetched on demand.

Bandwidth Optimization:
Buffer prefetching minimizes startup playback delay.
Adaptive rate throttling prevents wasted bandwidth consumption on
early-abandoned video sessions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

422 / 533

Small-World Networks in YouTube Videos I

Graph Formulation:
Nodes (V ): Individual videos or channels.
Directed Edges (E ): Recommendation links, user co-visitation
patterns, video responses, or shared tag sets.

Small-World Properties:
High Clustering Coefficient (C ): Local clusters of topically related
videos form dense, clique-like communities.
Short Average Path Length (L): Any video in the network can be
reached from another within a small number of recommendation hops
(L ≈ log |V |).

Scale-Free Degree Distribution: Node degree distribution follows
power-law behavior P(k) ∼ k −γ , featuring prominent hub nodes.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

423 / 533

Implications of Small-World Topology I

Information Diffusion and Navigability:
Enables rapid content discovery and viral propagation across disparate
graph clusters.
Recommendation engines navigate the small-world graph to balance
exploitation (local clusters) and exploration (long-range shortcuts).

Algorithmic and Structural Vulnerabilities:
Filter Bubbles: High clustering can constrain users inside highly
isolated, self-referential subgraphs.
Centrality Skew: Super-hubs attract disproportionate traffic,
reinforcing preferential attachment (rich-get-richer dynamics).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

424 / 533

YouTube from a Partner’s View: Monetization I

YouTube Partner Program (YPP):
Eligibility thresholds based on cumulative public watch hours and
subscriber counts.
Structured revenue-sharing framework between platform operators and
independent creators.

Key Economic Metrics:
CPM (Cost Per Mille): Advertiser cost per 1,000 ad impressions.
RPM (Revenue Per Mille): Net partner revenue earned per 1,000
total video views.

Dynamic Ad Auctions: Real-Time Bidding (RTB) engines match ad
spots (pre-roll, mid-roll, display) with viewer demographic and
behavioral profiles.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

425 / 533

Content ID and Automated Copyright Management I

Content ID Architecture: Automated audio and visual
fingerprinting database containing tens of millions of reference files.
Matching Mechanism: Uploaded media streams are fingerprinted
and compared against reference databases during ingestion.
Automated Rights Holder Policy Actions:
Block: Restrict global or geo-specific video availability.
Monetize: Redirect advertising revenue from the uploader to the
verified rights owner.
Track: Monitor viewer metrics without interrupting public playback.

Operational Scale: Processes billions of reference checks daily with
ultra-low false-positive rates.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

426 / 533

Creator Optimization and Recommendation
Objectives I

Creator Key Performance Indicators (KPIs):
Click-Through Rate (CTR) optimization via custom thumbnail design
and title framing.
Average Percentage Viewed (APV) and audience retention curve
stabilization.

Platform Recommendation Evolution:
Shifted from optimizing raw view counts to maximizing Total Watch
Time and Session Duration.
Deep learning recommendation architectures prioritize long-term user
satisfaction over transient clickbait.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

427 / 533

Comparative Infrastructure Matrix I

Dimension
Ingest & Catalog
Caching Strategy
Graph Topology
Monetization
Rights Enforcement

GTU Course

YouTube (UGC Model)
Massive ingest, variable quality, long-tail
catalog
Multi-tiered edge/datacenter dynamic
caching
Small-world social and recommendation
network
Ad-driven (AVOD), YPP revenue share,
memberships
Automated Content ID fingerprinting at
upload

Enterprise VoD (e.g., Netflix)
Controlled ingest, curated studio quality,
bounded catalog
Aggressive edge pre-positioning of popular catalog
Item-similarity and collaborative filtering
matrix
Subscription-driven (SVOD), flat licensing fees
Manual contractual rights licensing enforcement

DI04000051: Multimedia Technologies

Lecture Slides

428 / 533

Summary and Research Directions I
Summary of Core Pillars:
Synergistic integration of adaptive video codecs, dynamic metadata,
and CDN caching.
UGC dynamics characterized by heavy-tailed Zipf popularity and
short-term temporal decay.
Small-world graph topologies supporting efficient recommendation
traversal.
Robust partner ecosystem anchored by Content ID and Real-Time Ad
Bidding.

Emerging Research Frontiers:
Multi-modal AI representation learning for real-time video
understanding.
Mitigating filter bubbles and centrality bias in small-world
recommendation networks.
Scalable copyright authentication in the era of generative AI
multimedia.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

429 / 533

Introduction to YouTube as a Multimedia Sharing
Platform I
Context in Multimedia Information Sharing:
YouTube represents the preeminent global case study in large-scale
multimedia storage, indexing, and distribution.
Serves billions of users with massive heterogeneous content uploads
per minute.

Core System Requirements:
Scalable storage for multi-resolution video files.
Low-latency global content delivery under varying network conditions.
Intelligent discovery algorithms to match user intent with relevant
content.

Analytical Perspectives:
Technical: Video encodings, structural metadata, adaptive streaming
protocols.
Graph Topology: Small-world networks, recommendation graphs,
scale-free dynamics.
Partner & Ecosystem: Monetization systems, Content ID, audience
analytics.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

430 / 533

Video Format and Technical Metadata Structure I
Container Formats & Video Codecs:
Codecs: H.264/AVC, VP9, and AV1 enable high compression
efficiency across hardware platforms.
Audio Codecs: AAC and Opus provide flexible bitrate options.
Containers: MP4 and WebM support chunked streaming and fast
metadata parsing.

Structural Metadata:
Hardware encoding attributes: Bitrate, frame rate (24/30/60 fps),
aspect ratio, and resolution (144p to 4K/8K).
Container metadata headers (e.g., MP4 moov atom) positioned at the
start for immediate playback initiation.

Semantic & Contextual Metadata:
User-provided: Title, description text, tags, category selection,
language tags.
System-generated: Playlists, automatic speech recognition (ASR)
transcripts, time-stamped chapters, thumbnail previews.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

431 / 533

Empirical Characteristics of YouTube Content I

Duration & File Size Distributions:
Bimodal/Heavy-tailed distribution: Short-form clips (Shorts) alongside
long-form videos (lectures, podcasts, live streams).
Asymmetric file sizes requiring adaptive storage allocation.

Popularity Dynamics & Skewness:
Highly skewed access patterns following Pareto distributions (80/20
rule).
A tiny fraction of viral videos generate the vast majority of total views
and bandwidth demands.

Temporal Lifespan Patterns:
News/Trending content: Exponential early view growth followed by
rapid decay.
Educational/Evergreen content: Steady, long-tail viewing trajectory
over years.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

432 / 533

Small-World Network Topology in YouTube Videos I

Graph Formulation of Video Ecosystem:
Graph representation G = (V , E ) where nodes V represent videos and
directed edges E denote recommendation links or co-viewing paths.

Small-World Properties (Watts-Strogatz Model):
High Clustering Coefficient (C ): Videos within specific topical
niches form tightly interconnected clusters.
Short Average Path Length (L): Users can traverse between
arbitrary, disparate video domains in remarkably few clicks.

Scale-Free Degree Distribution (Barabási-Albert Model):
In-degree distribution follows a power law: P(k) ∝ k −γ .
Central ”hub” videos (highly popular or foundational content) connect
distinct sub-topical clusters.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

433 / 533

Graph Metrics & Recommendation Network
Dynamics I
Co-Viewing Graphs & Random Walks:
Transition probabilities calculated based on user session histories and
video co-occurrence.
PageRank and random walk algorithms discover relevant video
trajectories across the small-world graph.

Community Detection & Topic Clustering:
Modularity optimization and spectral clustering identify dense
semantic communities.
Cluster boundaries prevent recommendation collapse into isolated
echo chambers.

Network Expansion & Diversification:
Recommendation engines inject long-range links into candidate
generation graphs to facilitate serendipitous discovery.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

434 / 533

Delivery Infrastructure & Adaptive Bitrate Streaming
I

Ingestion and Transcoding Pipeline:
Parallelized encoding cluster converts uploaded master video files into
multiple resolution-bitrate variants simultaneously.

Dynamic Adaptive Streaming over HTTP (DASH):
Video files segment into small chunks (2–10 seconds).
Client player dynamically selects chunk quality based on buffer state,
throughput estimation, and device capabilities.

Hierarchical Content Delivery Network (CDN):
Hot edge caches at ISP points of presence store popular video chunks
for low latency.
Centralized cold storage handles long-tail requests efficiently.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

435 / 533

YouTube from a Partner’s View: Ecosystem Roles I

The Partner Landscape:
Independent creators, digital media publishers, production studios,
Multi-Channel Networks (MCNs).

Partner Program Architecture:
Qualification criteria based on cumulative watch hours, subscriber
counts, and safety compliance.
Contractual framework for revenue distribution and intellectual
property governance.

Platform Value Proposition:
Zero storage/bandwidth infrastructure costs for creators.
Global audience reach combined with sophisticated monetization
mechanisms.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

436 / 533

Monetization & Automated Content Identification I

Content ID Architecture:
Digital audio and visual fingerprinting system matching uploaded files
against a reference database.
Automated real-time detection of copyright-protected material.

Rights Holder Policy Enforcement:
Block: Restrict viewing access in specified geographic territories.
Monetize: Run advertisements on claimed content and route revenue
to rights holders.
Track: Monitor video viewership statistics without altering playback
availability.

Monetization Streams:
Ad auctions (CPM/CPC), YouTube Premium subscription revenue
sharing, channel memberships, and super chats.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

437 / 533

Partner Analytics & Audience Insights I

Key Performance Indicators (KPIs):
Impression Click-Through Rate (CTR): Measure of thumbnail and
title effectiveness.
Audience Retention Curves: Moment-by-moment viewer drop-off
tracking to identify engagement peaks and troughs.

Algorithmic Optimization Metrics:
Average View Duration (AVD) and Total Watch Time heavily
influence algorithmic promotion.

Traffic Source Decomposition:
Breakdown of views by Browse features, Suggested videos, YouTube
Search, and External embedded players.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

438 / 533

Metadata Indexing & Multimodal Search
Architecture I
Multimodal Feature Extraction:
Textual Features: Titles, descriptions, tags, and auto-generated ASR
transcripts.
Visual/Audio Embeddings: Neural network embeddings extracted
from video keyframes and acoustic tracks.

Hybrid Search Indexing:
Inverted indexes for BM25 keyword matching combined with dense
vector indexes for Approximate Nearest Neighbor (ANN) semantic
search.

Multi-Stage Ranking Pipeline:
Candidate Generation (Retrieval): Filtering millions of videos down to
hundreds based on query similarity.
Ranking (Scoring): Deep neural network re-ranking using user
context, engagement history, and video freshness.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

439 / 533

Scalability Challenges & Strategic Directions I

Infrastructure Bottlenecks:
Managing exabyte-scale storage growth alongside petabit-scale egress
bandwidth requirements.
High compute overhead for real-time AI transcoding,
recommendations, and moderation.

Content Safety & Integrity:
Detecting synthetic/manipulated media (Deepfakes) and automated
policy enforcement at upload scale.

Future Directions in Video Sharing:
Ultra-low latency streaming, interactive video formats, spatial/3D
media, and generative AI content pipelines.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

440 / 533

Overview of Cloud Computing Paradigms I
NIST Definition: Cloud computing is a model for enabling
ubiquitous, convenient, on-demand network access to a shared pool
of configurable computing resources (e.g., networks, servers, storage,
applications, and services).
Essential Characteristics:
On-demand self-service without requiring human interaction with
service providers.
Broad network access accessible through standard platforms (mobile
phones, tablets, laptops).
Resource pooling using multi-tenant models with location
independence.
Rapid elasticity for rapid scaling up or down according to demand.
Measured service for automated resource usage monitoring and
metering.

Service Delivery Models:
Infrastructure as a Service (IaaS): Provides raw compute power,
storage, and networking (e.g., AWS EC2, Google Compute Engine).
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

441 / 533

Overview of Cloud Computing Paradigms II

Platform as a Service (PaaS): Offers execution environments and
development frameworks (e.g., AWS Elastic Beanstalk, Google App
Engine).
Software as a Service (SaaS): Delivers complete end-user software
applications over the internet (e.g., Google Workspace, Microsoft
365).

Deployment Models: Public, Private, Hybrid, and Community
clouds, each balancing control, security, and cost.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

442 / 533

Architectural Principles of Cloud Infrastructure I
Scalable Distributed Systems:
Horizontal scaling (scale-out) vs. vertical scaling (scale-up).
Auto-scaling groups dynamically adjusting provisioned instances based
on metric thresholds (CPU, memory, bandwidth).

Distributed Storage Architecture:
Block Storage: High-performance volume storage attached to
instances for transactional data.
Object Storage: Flat namespace storage optimized for unstructured
media data (e.g., AWS S3, Google Cloud Storage).

Virtualization and Containerization:
Hypervisor-based virtualization (KVM, Xen) providing strict isolation.
Container technology (Docker, Kubernetes) enabling lightweight,
high-density deployment of microservices.

Multi-Tenancy and Isolation: Secure logical partitioning of
hardware resources across distinct client organizations.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

443 / 533

Introduction to Multimedia Cloud Computing (MCC)
I
Definition: Multimedia Cloud Computing (MCC) applies cloud
resources to store, process, transcode, stream, and analyze
heterogeneous media content (video, audio, 3D graphics, AR/VR).
Key Motivating Factors:
Explosive growth of high-definition (HD), 4K, 8K, and spatial video
formats requiring massive storage and bandwidth.
Heterogeneity of consumer playback devices (smartphones, TVs, web
browsers) demanding adaptive formats.
Computationally intensive tasks like deep learning video analytics and
multi-bitrate encoding.

Technical Challenges:
Strict latency requirements for real-time interactive media vs. high
throughput for non-interactive streaming.
Quality of Service (QoS) and Quality of Experience (QoE)
maintenance over unpredictable IP networks.
Efficient resource scheduling for GPU-accelerated video processing
clusters.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

444 / 533

Cloud Storage and Transcoding Architectures I

Massive-Scale Storage Management:
Hierarchical storage management: Tiering from high-speed hot
storage (SSDs) to cost-effective cold archiving (tape/Glacier).
Redundancy mechanisms: Replication across availability zones and
erasure coding to guarantee 99.999999999% durability.

Cloud Video Transcoding Pipelines:
Split-and-Merge Paradigm: Dividing large master video files into small
temporal chunks processed in parallel across worker nodes.
Parallel Transcoding Workflows: Codec conversion (H.264/AVC,
H.265/HEVC, AV1, VP9) across multiple resolutions and bitrates
simultaneously.

GPU Acceleration: Leveraging CUDA and NVENC hardware
encoders inside cloud instances to accelerate real-time rendering and
compression.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

445 / 533

Cloud-Assisted Media Sharing Systems I
System Architecture:
Ingestion Gateway: Ingests user-generated content (UGC) or
professional streams via protocols such as RTMP, SRT, or WebRTC.
Cloud Processing Core: Manages metadata extraction, automated
video indexing, thumbnail generation, and multi-bitrate packaging.
Delivery Engine: Serves manifest files (.m3u8, .mpd) and media
segments over HTTP.

Adaptive Bitrate Streaming (ABR) Orchestration:
Generation of HLS (HTTP Live Streaming) and DASH (Dynamic
Adaptive Streaming over HTTP) presentation manifests.
Dynamic segment creation responding to real-time client buffer
reports and bandwidth estimations.

Social & Collaborative Sharing: Cloud backend handles social
graphs, user authorization, comment feeds, and real-time interactive
features.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

446 / 533

Cloud-CDN Synergy in Media Distribution I

Hybrid Cloud-CDN Topologies:
Cloud Origin: Acts as the centralized master repository and dynamic
transcoding farm.
Content Delivery Network (CDN): Geographically distributed edge
nodes caching popular media segments near end-users.

Origin-Edge Synchronization:
Push vs. Pull Caching: Proactive edge pre-fetching of trending
content vs. reactive pull on cache miss.
Cache Invalidation: Automated purge strategies and short-lived
Time-To-Live (TTL) policies for dynamic live streams.

Edge Computing (MEC Integration):
Running lightweight serverless functions at edge nodes for per-user
manifest manipulation and dynamic ad insertion.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

447 / 533

QoS and QoE Optimization Strategies I

Distinction Between Metrics:
QoS (System-centric): Throughput, packet loss rate, jitter,
propagation delay, and server response latency.
QoE (User-centric): Initial buffering time (startup delay), stall
frequency, stall duration, and video perceptual quality (VMAF,
PSNR).

Resource Allocation and Load Balancing:
Global Server Load Balancing (GSLB) directing clients to the optimal
cloud region or CDN edge.
Dynamic bandwidth allocation and queue management in virtual
switches.

Service Level Agreements (SLAs): Enforcing strict uptime,
throughput, and error rate guarantees for premium media streaming
clients.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

448 / 533

Content Protection and DRM in Multimedia Clouds I

Security Architecture for Cloud Media:
Encryption at Rest: AES-256 encryption of raw and processed media
assets stored in object storage.
Encryption in Transit: TLS/HTTPS protocol enforcement across
ingestion, internal processing, and edge distribution networks.

Digital Rights Management (DRM) Integration:
Cloud-based Key Management Systems (KMS) issuing content keys
dynamically upon authentication.
Multi-DRM Ecosystem: Simultaneous support for Google Widevine,
Apple FairPlay, and Microsoft PlayReady via Common Encryption
(CENC).

Dynamic Watermarking: Inserting imperceptible, user-specific
forensic watermarks during cloud rendering to track unauthorized
redistribution.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

449 / 533

Case Study: Netflix Cloud Migration and
Microservices I
Background & Migration Motivation:
Transitioned from monolithic data centers to AWS (Amazon Web
Services) cloud infrastructure following a major database failure in
2008.
Completed full migration in 2016 to achieve massive global elasticity
and reliability.

Microservices Architecture:
Decomposed monolithic applications into hundreds of decoupled,
independently scalable microservices.
API Gateway routing, dynamic service discovery (Eureka), and
client-side load balancing (Ribbon).

Resilience Engineering (Chaos Engineering):
Developed Simian Army (including Chaos Monkey) to systematically
introduce random instance failures in production, guaranteeing
continuous fault tolerance.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

450 / 533

Case Study: Netflix Video Processing and Open
Connect CDN I
Advanced Video Encoding Pipeline:
Per-Title and Per-Shot Encoding: Custom optimization of bitrate
ladders tailored specifically to visual complexity per title/shot, saving
up to 20–50% bandwidth.
Multi-codec packaging serving AV1, HEVC, and VP9 alongside legacy
H.264.

Open Connect CDN Architecture:
Custom-designed appliance hardware deployed directly inside Internet
Service Provider (ISP) networks worldwide.
Overnight pre-positioning: Predictive algorithmic caching of catalog
assets into Open Connect Appliances during off-peak network hours.

Operational Impact: Completely offloads core cloud infrastructure
during video playback, serving over 100 million concurrent streams
efficiently.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

451 / 533

Future Horizons in Cloud-Based Multimedia Systems
I

Edge-Cloud Convergence:
Ultra-low latency processing using 5G Multi-access Edge Computing
(MEC) for interactive live events and cloud gaming.

AI-Driven Multimedia Cloud Operations:
Machine learning models predicting network congestion and
dynamically tuning ABR algorithms.
Automated real-time video summarization, indexing, and generative
AI content synthesis in the cloud.

Serverless Media Workflows:
Event-driven micro-functions (AWS Lambda, Cloud Functions)
executing image re-encoding, metadata parsing, and notification
pushes dynamically without dedicated server management.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

452 / 533

Lecture Overview: Cloud Computing for Multimedia I

Unit Context: Multimedia Information Sharing in modern
distributed environments.
Core Focus: Role of Cloud Computing in storing, processing,
transcoding, and distributing high-volume multimedia content.
Key Topics Covered:
Fundamentals of Cloud Computing (IaaS, PaaS, SaaS, Deployment
Models).
Multimedia Cloud Computing (MCC) architecture, QoS, and QoE
requirements.
Cloud-assisted media sharing paradigms, CDNs, and adaptive bitrate
streaming (ABR).
Comprehensive Case Study: Netflix global streaming architecture.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

453 / 533

Cloud Computing Overview: Service & Deployment
Models I
Definition: On-demand availability of computer system resources,
especially data storage and compute power, without direct active
management by the user.
Service Models:
Infrastructure as a Service (IaaS): Raw compute (VMs), storage,
and networking (e.g., AWS EC2, Google Compute Engine).
Platform as a Service (PaaS): Execution environments, database
management, and development frameworks (e.g., AWS Elastic
Beanstalk, Heroku).
Software as a Service (SaaS): On-demand software applications
accessible via web/API (e.g., Google Workspace, Office 365).

Deployment Models: Public Cloud, Private Cloud, Community
Cloud, and Hybrid Cloud architectures.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

454 / 533

Characteristics & Advantages of Cloud Computing I

Essential Characteristics:
On-Demand Self-Service: Automated provisioning without human
interaction with service providers.
Broad Network Access: Capabilities available over network
mechanisms supporting diverse thin/thick client platforms.
Resource Pooling: Multi-tenant model dynamically assigning
physical/virtual resources.
Rapid Elasticity: Capabilities elastically provisioned and released to
scale rapidly outward and inward.
Measured Service: Resource usage monitored, controlled, and
reported transparently (Pay-as-you-go).

Multimedia Relevance: Elasticity enables handling peak traffic
during major live media events.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

455 / 533

Multimedia Cloud Computing (MCC): Fundamentals
& Architecture I

Definition: MCC integrates cloud computing into multimedia domain
to perform distributed storage, parallel processing, media rendering,
and adaptive delivery.
Key Components of MCC Architecture:
Multimedia Cloud Storage: Distributed object storage for massive
video files (e.g., AWS S3).
Cloud Transcoding Engine: Distributed GPU/CPU clusters running
parallel video encoding (e.g., FFmpeg-based microservices).
Media Delivery Platform: Intelligent streaming servers supporting
HTTP Dynamic Streaming (HDS), HLS, and MPEG-DASH.

Decoupled Processing: Separating ingestion, processing, storage,
and streaming layers for scalable performance.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

456 / 533

MCC Technical Challenges: QoS, QoE, and Latency I

Quality of Service (QoS): Network-centric metrics such as
throughput, packet loss rate, delay, and jitter.
Quality of Experience (QoE): User-centric subjective quality,
measuring startup delay, buffering ratio, video stall frequency, and
visual quality switching.
Core Technical Challenges:
Bandwidth Volatility: Fluctuating wireless and access network
capacities.
Compute Heterogeneity: Variable processing times across cloud
virtual machines during parallel encoding.
Real-Time Latency: Strict low-latency requirements for interactive
live streaming (e.g., WebRTC, Low-Latency HLS).
Storage Overhead: High cost of storing redundant multi-bitrate
video representations.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

457 / 533

Cloud-Assisted Media Sharing Systems I

Paradigm Shift: Traditional Client-Server and Peer-to-Peer (P2P)
systems transitioned to Cloud-Assisted Media Sharing (CAMS).
Hybrid Cloud-P2P Architectures:
Cloud acts as a reliable seed and tracker node, providing guaranteed
baseline QoS.
Edge peers offload bandwidth overhead during high popularity bursts.

Cloud-Edge Collaboration:
Mobile Edge Computing (MEC) caches popular video segments closer
to end users.
Cloud centralized control plane manages global caching strategies and
traffic steering.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

458 / 533

Adaptive Bitrate Streaming & Transcoding Pipelines I

Adaptive Bitrate Streaming (ABR):
Source video encoded into multiple resolutions and bitrate levels
(Encoding Ladder).
Video partitioned into short temporal chunks (typically 2–6 seconds).
Client-side ABR algorithm dynamically requests optimal
representation based on buffer state and throughput estimates.

Distributed Transcoding Pipeline:
Video chunking: Master file split into independent temporal GOPs
(Group of Pictures).
Parallel encoding across worker nodes in cloud serverless/container
clusters.
Manifest generation: Dynamically building MPD (MPEG-DASH) or
M3U8 (HLS) playlists.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

459 / 533

Content Delivery Networks (CDNs) & Edge
Integration I

Role of CDNs in Media Sharing: Geographically distributed
network of proxy servers and data centers providing high availability
and performance.
Key Mechanisms:
Edge Caching: Stores video segments near end-users to reduce
latency and backhaul network traffic.
Request Routing: DNS-based or HTTP 302 redirection to direct
clients to the optimal Point of Presence (PoP).
Origin Shielding: Intermediate caching layer preventing origin server
overload during viral traffic spikes.

Multi-CDN Strategies: Dynamic real-time CDN switching based on
client performance telemetry.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

460 / 533

Case Study: Netflix Infrastructure Overview I

Architectural Split:
Control Plane (AWS Cloud): Runs user authentication,
recommendations, search, metadata management, billing, and
transcoding.
Data Plane (Open Connect CDN): Custom-built CDN designed
specifically for video delivery.

Microservices Ecosystem:
Hundreds of independent microservices deployed across AWS EC2
instances.
Fully automated deployment, scaling, and service discovery.

Data Storage Architecture: Polyglot persistence using AWS S3 for
media assets, Cassandra for user state, and DynamoDB for key-value
stores.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

461 / 533

Case Study: Netflix Open Connect CDN & Encoding
I

Open Connect Appliances (OCAs):
Custom hardware servers deployed directly inside Internet Service
Provider (ISP) networks globally.
Over 95% of Netflix traffic is served directly via IXPs or ISP local
caches.

Per-Title and Per-Shot Encoding:
Per-Title Encoding: Tailors the bitrate ladder based on complexity of
video content (e.g., animation vs. high-action sports).
Per-Shot Encoding: Divides video into individual shots, optimizing
QP (Quantization Parameter) and bitrate per scene.

Modern Codec Adoption: AV1 and HEVC adoption reduces
bandwidth consumption by up to 30% compared to AVC/H.264.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

462 / 533

Case Study: Netflix Resilience & Chaos Engineering I

High Availability Design: Multi-region AWS deployment with
automatic failover capabilities.
Chaos Engineering Principles: Proactively injecting failures into
production systems to test system resilience.
Simian Army Toolset:
Chaos Monkey: Randomly terminates virtual machine instances in
production during business hours.
Chaos Gorilla: Simulates an entire AWS Availability Zone (AZ)
outage.
Chaos Kong: Simulates an entire AWS Region failure, triggering
automated multi-region traffic redirection.

Impact: Zero extended outages despite continuous hardware,
network, and cloud platform failures.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

463 / 533

Future Directions in Multimedia Cloud Computing I

AI-Driven Video Encoding: Neural video compression (e.g., Deep
Video Compression) offering superior rate-distortion performance.
Edge-Cloud Synergy: Shifting AI inference (e.g., super-resolution,
video object tracking) to edge devices and local base stations.
Immersive Media Streaming: Cloud rendering for Cloud VR/AR,
volumetric video, and 360-degree interactive video.
Green Cloud Computing: Carbon-aware multimedia processing and
energy-efficient cloud transcoding workflows.
Summary: Cloud computing provides the essential elasticity,
reliability, and computational power required for modern global
multimedia distribution systems.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

464 / 533

Lecture Overview & Reality-Virtuality Continuum I
Unit Context: Augmented Reality (AR) and Virtual Reality (VR)
systems within spatial computing.
Lecture Objectives:
Define and differentiate Augmented Reality (AR), Virtual Reality
(VR), and Mixed Reality (MR).
Analyze the Milgram-Kishino Reality-Virtuality Continuum.
Deconstruct the complete functional workflow of an AR pipeline.
Explore core hardware and software components: sensory input,
tracking, world generation, and spatial emission.

Milgram-Kishino Continuum (1994):
Classifies environments along a continuous spectrum between Real
Environment and Virtual Environment.
Real Environment: Pure physical reality without computer-generated
augmentation.
Augmented Reality (AR): Predominantly real world enhanced with
digital information overlay.
Augmented Virtuality (AV): Predominantly synthetic world
incorporating real-world physical elements.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

465 / 533

Lecture Overview & Reality-Virtuality Continuum II

Virtual Reality (VR): Entirely computer-generated, fully immersive
synthetic environment.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

466 / 533

Defining Reality Spectrum: AR, VR, MR, and XR I
Virtual Reality (VR):
Replaces physical perception entirely with synthetic visual, auditory,
and haptics cues.
User is sensorily isolated from physical surroundings via
Head-Mounted Displays (HMDs).

Augmented Reality (AR):
Superimposes computer-generated graphics, text, or 3D models onto
the user’s real-world view.
Maintains physical environment primacy while providing context-aware
annotations.

Mixed Reality (MR):
Advanced form of spatial computing where physical and virtual
objects co-exist and interact in real time.
Requires occlusion handling, depth anchoring, and dynamic spatial
collisions.

Extended Reality (XR):
Umbrella term encapsulating AR, VR, MR, and all future immersive
computing technologies.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

467 / 533

Virtual Reality (VR): Concepts and Architecture I
Key Characteristics:
Immersion: Egocentric perspective with high field-of-view (FoV)
visual displays.
Presence: Psychological state of feeling physically located inside the
virtual environment.
Interactivity: Real-time response to user motion and spatial inputs.

Hardware Architecture:
Display Systems: Dual OLED/LCD micro-displays, stereoscopic
optical lenses (Fresnel or Pancake optics).
Head Tracking: 6 Degrees of Freedom (6DoF) tracking capturing
orientation (pitch, roll, yaw) and translation (x, y , z).
Input Devices: Handheld controllers, spatial glove interfaces, and
full-body haptic suits.

Rendering Demands:
Ultra-low motion-to-photon latency (< 20 ms) to prevent visually
induced motion sickness (cybersickness).
High frame rates (90–120 Hz) and high per-eye pixel resolutions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

468 / 533

Augmented Reality (AR): Fundamentals &
Characteristics I

Azuma’s Formal Definition of AR (1997):
Combines real and virtual objects in a physical environment.
Operates interactively in real time.
Registers (aligns) real and virtual objects with each other in 3D space.

Core Functional Goals:
Context-aware spatial information enhancement.
Seamless visual blending without disrupting natural spatial perception.
Precise geometric registration to maintain digital object anchoring.

Key Application Domains:
Industrial maintenance and assembly guidance.
Medical surgery visualization and navigation.
Navigation, remote collaboration, and tactical defense systems.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

469 / 533

Mixed Reality (MR): Hybrid Physical-Digital
Interaction I
Foundational Pillars of MR:
Environment Understanding: Real-time surface detection, spatial
mesh construction, and semantic labeling.
Human Understanding: Gaze tracking, hand joint skeleton tracking,
spatial voice commands.
Spatial Anchoring: Persistent coordinate frames tied to physical
geographic/spatial positions.

Bi-Directional Interaction:
Digital content responds to real-world physics (e.g., bouncing off
physical tables).
Physical light conditions dynamically illuminate digital synthetic
models (environment lighting estimation).

Occlusion & Depth Integrity:
Physical objects correctly hide digital content positioned behind them
(depth sorting).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

470 / 533

Comparative Matrix: AR vs. VR vs. MR I
Environment Context:
VR: 100% synthetic environment; completely occludes real world.
AR: Physical environment primary; synthetic overlays attached to
display plane or anchor.
MR: Merged physical and synthetic environment; real and digital
co-exist interactively.

Display & Tracking Technologies:
VR: Opaque Head-Mounted Displays (HMDs) with 3DoF/6DoF
tracking.
AR: Optical See-Through (OST) / Video See-Through (VST)
displays with feature tracking.
MR: Advanced OST/VST displays with real-time 3D spatial mapping
and depth sensing.

Interaction Modalities:
VR: Motion controllers, joysticks, spatial interaction triggers.
AR: Touchscreens, heads-up display (HUD) controls, voice commands.
MR: Direct natural hand gesture manipulation, eye-gaze tracking,
physics interactions.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

471 / 533

End-to-End Workflow of an Augmented Reality
System I
AR Operational Pipeline Overview:
Stage 1: Sensory Data Collection: Gathering multimodal inputs
from physical environment.
Stage 2: Localization & Alignment: Estimating 6DoF camera pose
and mapping physical space.
Stage 3: World Generation: Synthesizing 3D graphics, lighting, and
physics simulation.
Stage 4: Emission & Display: Projecting final composite frames
onto optical displays.

Feedback Loop Dynamics:
Closed-loop execution executed at high refresh rates (60–120 Hz) to
maintain spatial stability.
Continuous pose updating minimizes visual drift and registration
errors.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

472 / 533

AR Pipeline Step 1: Sensory Data Collection I

Sensor Array Modalities:
Optical RGB Sensors: Capture ambient light intensity and visual
feature points.
Depth Sensors (ToF / Structured Light): Measure per-pixel
distance metrics for surface detection.
Inertial Measurement Units (IMUs): High-frequency (1000 Hz)
accelerometers and gyroscopes for motion acceleration.
LiDAR / Infrared Scanners: High-precision spatial range scanning
and point-cloud generation.

Data Preprocessing & Synchronization:
Noise reduction filtering (Kalman filtering, Complementary filters).
Hardware timestamp alignment across optical cameras and IMU
sensor streams.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

473 / 533

AR Pipeline Step 2: Localization and Alignment I

Pose Estimation:
Determining sensor position (x, y , z) and orientation (θ, ϕ, ψ) relative
to world origin.

Tracking Approaches:
Marker-Based Tracking: Detects known planar targets (QR codes,
fiducial markers like ArUco).
Markerless Tracking: Relies on natural feature extraction (ORB,
SIFT, FAST keypoints).

Visual-Inertial SLAM (VI-SLAM):
Combines visual feature tracking with IMU dead reckoning to build a
map while updating pose.
Bundle Adjustment (BA) minimizes reprojection errors across
historical keyframes.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

474 / 533

AR Pipeline Step 3: World Generation and Rendering
I

3D Scene Reconstruction:
Constructing geometric spatial meshes from depth point clouds.

Photorealistic Integration:
Light Estimation: Sampling real-world high dynamic range (HDR)
light probes to render realistic shadows and specular highlights on
virtual models.
Occlusion Processing: Utilizing depth buffers to mask virtual
graphics behind real physical structures.

Physics Simulation:
Rigid body dynamics, gravity simulation, and collision detection
against real mesh boundaries.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

475 / 533

AR Pipeline Step 4: Spatial Emission and Display
Systems I

Display Paradigms:
Optical See-Through (OST): Uses beam splitters or diffractive
waveguides. Real world viewed directly; light from micro-display
recombined visually.
Video See-Through (VST): Real world captured via cameras,
merged digitally with 3D graphics, rendered on opaque screens.
Spatial Projection AR: Direct projection of digital light onto physical
surfaces.

Spatial Audio Emission:
Head-Related Transfer Function (HRTF) audio rendering for 3D
positional audio.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

476 / 533

Technical Challenges and Future Trends in AR/VR I

Technical Bottlenecks:
Vergence-Accommodation Conflict (VAC): Visual fatigue caused by
fixed display focal planes vs natural eye convergence.
Motion-to-Photon Latency: Delay between physical head rotation
and display update (< 10 ms requirement).
Form Factor & Power Constraints: Thermal dissipation, weight,
battery capacity in lightweight glasses.

Emerging Horizons:
Waveguide optics innovations (diffractive, reflective, holographic).
Neural rendering techniques (NeRFs and 3D Gaussian Splatting) for
ultra-realistic spatial capture.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

477 / 533

Introduction to Extended Reality (XR) I

Extended Reality (XR) is an umbrella term encompassing Virtual
Reality (VR), Augmented Reality (AR), and Mixed Reality (MR).
Blends the physical and digital worlds to create immersive
human-computer interaction paradigms.
Driven by advancements in computer vision, spatial computing,
sensor fusion, and display hardware.
Applications span medical simulation, industrial maintenance,
education, navigation, and entertainment.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

478 / 533

Defining Virtual Reality (VR) I

Core Concept: Immerses the user in a fully synthetic,
computer-generated 3D environment, completely isolating them from
the physical world.
Key Characteristics:
High degree of visual, auditory, and haptic immersion.
Full occlusion of real-world optical path using Head-Mounted Displays
(HMDs) or CAVE systems.
6-Degrees-of-Freedom (6DoF) tracking of head and hand movements.

Primary Objective: Provide a sense of presence where the user
perceives themselves to be inside the virtual world.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

479 / 533

Defining Augmented Reality (AR) I

Core Concept: Superimposes computer-generated digital
information (graphics, text, 3D models) onto the user’s view of the
real world in real time.
Key Characteristics:
Combines real and virtual objects in a real environment.
Operates interactively in real time.
Aligns and registers virtual elements precisely with 3D physical objects.

Primary Objective: Enhance human perception by adding
contextual information to the physical surroundings.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

480 / 533

Defining Mixed Reality (MR) and Reality-Virtuality
Continuum I

Milgram’s Reality-Virtuality Continuum:
Spans from pure Real Environment to pure Virtual Environment.
Intermediate spectrum is defined as Mixed Reality (MR).

Augmented Virtuality (AV): Real-world physical objects are merged
into a predominantly virtual environment.
Mixed Reality (MR):
Merges real and digital worlds where physical and virtual objects
co-exist and interact dynamically in real time.
Occlusion, lighting coherence, and spatial anchoring enable seamless
interaction.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

481 / 533

Comparative Analysis: VR vs. AR vs. MR I

Environment Context:
VR: Entirely synthetic environment; zero physical world interaction.
AR: Physical environment primary; synthetic overlays secondary.
MR: Integrated physical and synthetic environment with bidirectional
interaction.

Optical Transmission:
VR: Video passthrough or opaque display panels.
AR: Optical see-through or low-latency video see-through.
MR: Advanced optical waveguides with spatial mapping sensors.

Spatial Awareness: VR requires boundary limits, while AR and MR
require spatial mapping and depth perception.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

482 / 533

End-to-End Workflow of Augmented Reality I

Data Acquisition: Capture physical environment metrics via
multi-modal sensors.
Localization and Alignment: Compute camera pose and register
spatial coordinate systems.
World Generation: Build 3D spatial mesh, reconstruct geometry,
and perform lighting estimation.
Synthesis and Rendering: Render virtual objects with accurate
perspective, scale, and occlusion.
Emission: Project rendered frames onto display hardware for human
visual perception.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

483 / 533

AR Component 1: Sensory Data Collection I

Optical Sensors: RGB cameras, infrared (IR) sensors, and stereo
camera pairs for visual input.
Depth Sensors: Time-of-Flight (ToF), Structured Light, and LiDAR
sensors for real-time spatial depth sensing.
Inertial Measurement Units (IMUs):
High-frequency accelerometers and gyroscopes.
Measures linear acceleration and angular velocity for motion tracking.

Environmental Sensors: Ambient light sensors and microphones for
context-aware augmentation.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

484 / 533

AR Component 2: Localization and Alignment I

Camera Pose Estimation: Determining the 6DoF position (x, y , z)
and orientation (θ, ϕ, ψ) of the AR device relative to the environment.
Tracking Paradigms:
Marker-based Tracking: Uses fiducial markers (e.g., QR codes,
ArUco tags) with known geometry.
Markerless Tracking: Uses natural features (edges, corners, surface
textures).

SLAM (Simultaneous Localization and Mapping):
Visual-Inertial SLAM (VI-SLAM) combines IMU data with feature
tracking.
Enables drift correction and accurate coordinate registration without
prior maps.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

485 / 533

AR Component 3: World Generation I

Scene Understanding: Plane detection (floors, walls, tables),
semantic segmentation, and object classification.
3D Mesh Reconstruction: Generating volumetric spatial maps to
enable physical collisions and physics simulation for virtual items.
Occlusion Management: Resolving foreground-background
relationships so real objects obscure virtual objects when in front.
Photometric Alignment: Estimating real-world directional lighting,
shadows, and reflections for realistic rendering.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

486 / 533

AR Component 4: Emission (Display Systems and
Optics) I

Optical See-Through (OST):
Uses beam splitters or holographic waveguides.
Preserves direct, zero-latency physical vision while overlaying light
paths.

Video See-Through (VST):
Captures real world via external cameras, composites digital elements,
and displays on opaque screens.
Allows complete pixel-level control and perfect occlusion, but
introduces latency.

Spatial Projection (SAR): Projecting light directly onto physical
surfaces in the environment.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

487 / 533

Technical Challenges and Future Directions I

Vergence-Accommodation Conflict (VAC): Visual fatigue caused
by mismatch between focal depth and display distance.
Latency and Motion-to-Photon Time: Sub-20ms threshold
required to eliminate motion sickness and lag.
Form Factor and Thermal Management: Power efficiency, battery
life, and compact optical design in smart glasses.
Future Trends: AI-driven spatial understanding, light field displays,
micro-LED displays, and cloud-assisted edge rendering.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

488 / 533

Lecture Overview & Learning Objectives I

Lecture Unit: Augmented Reality and Virtual Reality
Lecture Topic: Virtual Reality Workflow and Core Components
Primary Objectives:
Analyze the end-to-end operational workflow of Virtual Reality (VR)
systems.
Deconstruct the four foundational components of VR: Virtual World,
Immersion, Sensory Feedback, and Interactivity.
Understand hardware and software architectures supporting real-time
synthetic environments.
Evaluate key performance metrics including Motion-to-Photon latency
and spatial tracking.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

489 / 533

Fundamentals of Virtual Reality (VR) I

Definition of VR: A computer-generated, three-dimensional
synthetic environment that surrounds the user and responds in real
time to human actions.
Reality-Virtuality Continuum:
Situated at the extreme right of Milgram’s continuum (completely
digital environment).
Distinct from Augmented Reality (AR) which overlays digital assets
onto physical surroundings.

Core System Objectives:
Replace physical sensory inputs with high-fidelity digital signals.
Provide continuous, low-latency interaction loops.
Create convincing perceptual illusions of spatial presence.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

490 / 533

High-Level VR System Workflow I

1. User Tracking & Input Capture:
Continuous tracking of head, hands, and body posture via
3DoF/6DoF sensors.
Capture of controller inputs, hand gestures, and gaze vectors.

2. Simulation Engine Processing:
Physics engine updates object positions, collisions, and state changes.
Scene graph update based on world logic and user interactions.

3. Multi-Modal Rendering Pipeline:
Visual rendering of left and right eye buffers (Stereoscopic view).
Spatial audio rendering via Head-Related Transfer Functions (HRTF).
Haptic wave-form generation for controllers and haptic gloves.

4. Sensory Output Display:
Displaying rendered frames on HMD micro-displays or CAVE screens.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

491 / 533

Component 1: The Virtual World I

Concept: The digital content, spatial rules, and objects that
populate the synthetic environment.
Key Elements of Virtual Worlds:
3D Assets & Geometry: Polygonal meshes, procedural terrain,
textures, and lighting maps.
Behavioral Rules: Physical laws (gravity, mass, velocity) and dynamic
object behaviors.
Scene Graph Structure: Hierarchical spatial data organization
representing spatial relationships.

Spatial Coordinate Systems:
Continuous conversion between Object Space, World Space,
Camera/Eye Space, and Clip Space.
Real-time matrix transformations for dynamic camera positioning.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

492 / 533

Component 2: Immersion – Psychological &
Technical Aspects I

Defining Immersion vs. Presence:
Immersion: Objective, quantifiable measure of system technology
(FOV, resolution, tracking accuracy).
Presence: Subjective psychological state of ”being there” experienced
by the user.

Determinants of Immersion:
Field of View (FOV): Human horizontal FOV ≈ 200◦ ; modern HMDs
target 110◦ − 180◦ .
Display Quality: High pixel density (PPD), minimal screen-door
effect, high dynamic range.
Degrees of Freedom (DoF): 3DoF (orientation only) vs. 6DoF
(orientation + translation).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

493 / 533

Visual & Auditory Immersion Mechanisms I

Stereoscopic Depth Cues:
Binocular Parallax: Displaying distinct images to left and right eyes
to generate depth perception.
Accommodation & Convergence: Matching focal depth and eye
convergence to prevent vergence-accommodation conflict.

Spatial Audio Immersion:
Binaural Audio: Simulating sound arrival time differences (ITD) and
level differences (ILD).
HRTF Filtering: Applying acoustic filters modeled on human ear
geometry for 3D sound placement.

Physical Sensory Isolation: Blocking ambient physical light and
sound to eliminate real-world distractors.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

494 / 533

Component 3: Sensory Feedback Systems I

Multimodal Feedback Integration: Synchronizing visual, auditory,
and tactile signals to create coherent realism.
Haptic Feedback Modalities:
Tactile Feedback: Vibrations, surface textures, and temperature
simulation via eccentric rotating mass (ERM) motors or linear
resonant actuators (LRA).
Kinesthetic Feedback: Force feedback resisting user movement,
simulating weight and physical boundaries using exoskeletons.

Vestibular & Somatosensory Feedback:
Motion platforms for flight/driving simulation to stimulate inner-ear
movement perception.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

495 / 533

Component 4: Interactivity & User Engagement I

Definition of Interactivity: The extent to which users can modify
the form and content of the virtual environment in real time.
Core Interaction Metaphors:
Selection & Manipulation: Ray-casting, direct virtual hand gripping,
and distance grabbing.
Navigation & Locomotion: Continuous joystick movement,
room-scale physical walking, and teleportation.
System Control: Diegetic spatial menus, hand-gesture triggers, and
voice commands.

Natural User Interfaces (NUI): Leveraging intuitive real-world
movements to minimize cognitive load.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

496 / 533

Hardware & Software Architecture I

Hardware Layer:
Sensors & Tracking: Inertial Measurement Units (IMUs), infrared
cameras, LiDAR, optical inside-out tracking.
Displays & Optics: OLED/Micro-LED panels, pancake and Fresnel
lens optics.

Software Layer:
VR Engines: Unity, Unreal Engine, WebXR standards.
Runtime Middleware: OpenXR API, SteamVR, Oculus SDK
managing spatial anchor management and compositing.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

497 / 533

Performance Metrics & Technical Challenges I

Motion-to-Photon (M2P) Latency:
Time elapsed between physical head movement and updated pixel
display on HMD.
Target threshold: < 20 ms to prevent visual lag and vestibular
disorientation.

Cyber-sickness / Simulator Sickness:
Caused by sensory conflict between visual movement cues and
vestibular stillness.
Mitigation: High refresh rates (90–120 Hz), asynchronous
timewarp/reprojection, fixed reference frames.

Rendering Optimization: Foveated rendering using eye tracking to
concentrate GPU compute on central vision.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

498 / 533

Summary & Future Frontiers in VR Workflow I

Synthesis of Core Components:
A successful VR system harmonizes the Virtual World, Immersion,
Sensory Feedback, and Interactivity into a unified feedback loop.

Emerging Technological Frontiers:
Photorealistic real-time neural rendering (NeRFs, Gaussian Splatting).
Advanced microfluidic haptic gloves for granular tactile sensations.
Direct Brain-Computer Interfaces (BCI) for intentional control and
neural sensory injection.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

499 / 533

Lecture Overview & Learning Objectives I

Course Unit: Augmented Reality and Virtual Reality
Topic Focus: Virtual Reality Workflow and its Core Components.
Key Learning Objectives:
Comprehend the end-to-end operational workflow of modern VR
systems.
Analyze the 4 foundational components: Virtual World, Immersion,
Sensory Feedback, and Interactivity.
Understand hardware and software loop interactions, focusing on
motion-to-photon latency.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

500 / 533

High-Level Virtual Reality System Workflow I
Sense-Process-Render Cycle: VR operates on a tight continuous
closed loop.
Stage 1: User Sensing & Input Extraction
Tracking position, orientation (6DoF), gestures, eye movements, and
controller inputs.

Stage 2: Simulation Engine Computation
State updating, collision detection, object dynamics, and scene graph
evaluation.

Stage 3: Rendering & Multi-Modal Synthesis
Stereo graphics rendering, spatial audio processing, and haptic signal
generation.

Stage 4: Output Display & Perception
HMD visual presentation, binaural audio output, and tactile force
feedback.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

501 / 533

Core Component 1: The Virtual World I

Definition: An artificial 3D mathematical content space maintained
by the VR engine.
Geometry & Spatial Modeling:
Composed of polygonal meshes, textures, lighting maps, and spatial
anchors.

Behavioral Rules & Dynamic States:
Scripted logic governing object behavior, physics parameters (gravity,
mass, friction), and environment rules.

Persistence & Telepresence:
Maintains spatial consistency and object state regardless of user gaze
direction.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

502 / 533

Core Component 2: Immersion I

Definition: Objective, quantifiable degree to which a VR system
delivers sensory stimuli masking the real world.
Physical (Technological) Immersion:
High field-of-view (FOV), high refresh rates, spatial audio, and
low-latency tracking.

Psychological Immersion (Presence):
The subjective mental state of “being there” inside the virtual
environment.

Sensory Isolation:
Minimizing real-world distractions through ergonomic hardware sealing
and acoustic occlusion.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

503 / 533

Core Component 3: Sensory Feedback I

Visual Feedback:
Stereoscopic rendering providing binocular depth cues (disparity,
convergence, parallax).

Auditory Feedback:
3D spatial audio utilizing Head-Related Transfer Functions (HRTF)
for localized sound.

Haptic & Tactile Feedback:
Force-feedback actuators, vibrotactile arrays, and thermal or resistive
skin stimulation.

Vestibular & Proprioceptive Feedback:
Motion platforms and galvanic vestibular stimulation (GVS) aligning
balance with visual cues.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

504 / 533

Core Component 4: Interactivity I

Real-Time Responsiveness:
Immediate dynamic updates to the virtual world based on user action.

6 Degrees of Freedom (6DoF):
Translational movements (X , Y , Z ) and rotational movements (Yaw,
Pitch, Roll).

Natural User Interfaces (NUI):
Hand-tracking, eye gaze selection, speech recognition, and
bi-directional object manipulation.

Direct Manipulation:
Physical-like interaction with virtual objects obeying real-world physics
constraints.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

505 / 533

VR Workflow Deep Dive: Sensor Fusion & Tracking I

Tracking Technologies:
Inertial Measurement Units (IMUs: Accelerometer, Gyroscope,
Magnetometer).
Optical tracking: Inside-out (onboard cameras) vs. Outside-in
(external base stations).

Sensor Fusion Algorithms:
Extended Kalman Filters (EKF) combining high-rate IMU data with
low-drift visual data.

Pose Estimation Pipeline:
Raw Sensor Input → Calibration → Feature Tracking → Pose Vector
Output.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

506 / 533

VR Workflow Deep Dive: Physics & Graphics
Pipeline I

Simulation State Update:
Broad-phase and narrow-phase collision detection, rigid body
dynamics update.

Stereoscopic Graphics Rendering:
Dual projection matrices rendering independent viewpoints for Left
Eye and Right Eye.

Late-Stage Reprojection:
Asynchronous Timewarp (ATW) adjusting rendered frames using
updated IMU orientation.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

507 / 533

VR Workflow Deep Dive: Output Loop & Haptic
Coupling I

Visual Display Refresh Loop:
High refresh rates (90 Hz – 120 Hz) with low-persistence displays to
eliminate motion blur.

Spatial Audio Pipeline:
Real-time HRTF convolution based on current listener pose and sound
source coordinates.

Closed-Loop Haptic Feedback:
Real-time telemetry sent to controller haptic motors synced to impact
timestamps.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

508 / 533

Performance Metrics & Human Factors I

Motion-to-Photon (M2P) Latency:
Delay between physical movement and display update. Must remain
below 20 ms.

Visual-Induced Motion Sickness (VIMS):
Caused by sensory conflict between visual movement cues and
vestibular balance perception.

Vergence-Accommodation Conflict (VAC):
Discrepancy between optical focal distance of HMD screens and
stereoscopic convergence distance.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

509 / 533

Summary of VR Workflow and Components I

System Integration Summary:
VR is an interdependent closed-loop system coupling input tracking,
simulation, and rendering.

Core Component Interplay:
The Virtual World defines content state.
Immersion establishes sensory isolation.
Sensory Feedback delivers visual, acoustic, and haptic cues.
Interactivity enables user agency and closed-loop control.

Key Takeaway: Minimizing M2P latency and resolving sensory
conflicts are critical to effective VR systems.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

510 / 533

Taxonomy and Spectrum of AR/VR Applications I
Reality-Virtuality Continuum (Milgram & Kishino):
Real Environment → Augmented Reality (AR) → Augmented
Virtuality (AV) → Virtual Reality (VR).
Mixed Reality (MR) spans the spectrum between purely physical and
purely synthetic environments.

Core Application Pillars:
Augmented Reality: Superimposes computer-generated perceptual
information onto real-world view (e.g., heads-up displays, spatial
overlays).
Virtual Reality: Completely replaces the real environment with a fully
synthetic 3D digital environment via Head-Mounted Displays (HMDs).

Domain Categorization:
Industrial, Medical, Defense, AEC
(Architecture/Engineering/Construction), Pedagogy, and
Commercial/Retail.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

511 / 533

Industrial Manufacturing and Assembly I

Computer-Aided Assembly & Inspection:
AR overlays animated 3D CAD step-by-step instructions directly onto
physical machinery.
Reduces error rates in complex wiring harnesses (e.g., aerospace
manufacturing) by up to 90%.

Digital Twin Integration:
Real-time telemetry data (IoT sensor feeds) visualised over physical
equipment using AR.
Enables predictive maintenance, thermal mapping, and stress
diagnostics without dismantling hardware.

Remote Industrial Expertise:
Field technicians share live spatial video feeds with offsite specialists
who annotate the physical space in real time.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

512 / 533

Healthcare and Surgical Interventions I
Image-Guided Surgery (IGS):
AR fuses pre-operative CT/MRI scans directly onto the patient’s
anatomical surface during live operations.
Enhances precision in neurosurgery, orthopedics, and oncology by
visualising sub-surface structures.

Medical Training and Simulation:
VR allows surgical trainees to perform complex procedures in zero-risk
virtual operating rooms.
Haptic feedback force-rendering simulates realistic tissue resistance,
bone drilling, and suturing.

Therapy and Rehabilitation:
VR exposure therapy (VRET) for treating PTSD, phobias, and anxiety
disorders.
Motor-function neurorehabilitation after stroke using gamified VR
movement exercises.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

513 / 533

Defense, Aerospace, and Flight Simulation I

Heads-Up Displays (HUD) and Helmet-Mounted Displays
(HMD):
Jet fighter helmet systems (e.g., F-35 Gen III HMD) project critical
flight instrumentation, target tracking, and night vision onto pilot
line-of-sight.

Flight and Combat Simulation:
High-fidelity VR cockpits with 6-DOF (Degrees of Freedom) motion
platforms for full flight domain training.
Cost-effective flight hour replacement while enabling dangerous
failure-mode emergency simulations.

Tactical Battlefield AR:
Synthetic Training Environment (STE) overlays friendly/enemy
positions, navigation waypoints, and terrain maps on soldier eyewear.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

514 / 533

Education, STEM, and Academic Research I

Interactive STEM Visualization:
Spatial interaction with complex 3D structures: molecular orbitals,
crystallographic lattices, dynamic physical fields
(electrostatic/gravitational).

Virtual Laboratories and Fieldwork:
VR labs enable high-hazard chemistry experiments, high-cost physics
apparatus access, and distant geological field surveys.

Pedagogical Impact:
Transition from passive 2D reading to active experiential learning.
Demonstrated improvements in spatial reasoning, long-term concept
retention, and student engagement.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

515 / 533

Architecture, Engineering, and Construction (AEC) I

Building Information Modeling (BIM) Visualization:
AR overlays 3D BIM blueprints onto physical construction sites for
spatial clash detection (e.g., HVAC duct conflicts with structural
beams).

Virtual Architectural Walkthroughs:
VR enables clients and architects to experience 1:1 scale immersive
spatial walk-throughs prior to ground breaking.
Rapid prototyping of spatial ergonomics, lighting conditions, and
material aesthetics.

Urban Planning and Infrastructure:
Mixed Reality visualization of large-scale civil engineering projects in
existing urban landscapes.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

516 / 533

Retail, E-Commerce, and Marketing I

Spatial Product Placement (V-Commerce):
Markerless AR (SLAM-based) allows consumers to project furniture
and home goods into physical living spaces at correct scale.

Virtual Try-On (VTO) Systems:
Real-time facial tracking and body pose estimation for virtual apparel,
eyewear, cosmetics, and footwear preview.
Reduces e-commerce return rates significantly by establishing accurate
size and aesthetic expectations.

Experiential Marketing Campaigns:
Interactive location-based AR promotions and brand engagements
augmenting print media and store locations.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

517 / 533

Entertainment, Gaming, and Digital Media I

Immersive Gaming and Virtual Worlds:
Full 6-DOF room-scale tracking combined with spatial audio engines
for complete player immersion.
Massively multiplayer virtual environments offering user-generated
content creation and virtual economies.

Location-Based Entertainment (LBE):
Specialized physical arenas with haptic vests, physical walls matched
to VR geometry, and environmental effects (wind, heat).

Cinematic Immersive Storytelling:
360-degree volumetric video and real-time rendered virtual production
(e.g., LED wall virtual sets driven by spatial tracking).

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

518 / 533

Telepresence, Remote Collaboration, and Spatial
Work I

Holographic Telepresence:
Real-time 3D volumetric capture of remote participants rendered as
life-sized avatars in shared spatial coordinates.

Spatial Virtual Desktops:
Unlimited virtual screen real estate replacing physical monitor arrays,
supporting multi-window spatial productivity.

Collaborative CAD and Design Review:
Geographically dispersed engineering teams interacting simultaneously
with the same 3D spatial model in real time.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

519 / 533

HCI Paradigms and Key Technical Challenges I

Human-Computer Interaction (HCI) Interfaces:
Natural user interfaces: multimodal input combining eye tracking,
hand gesture recognition, and voice commands.

Performance Bottlenecks & Hardware Limitations:
Motion-to-Photon Latency: Must be < 20 ms to prevent
vestibulo-ocular reflex (VOR) mismatch and motion sickness.
Field of View (FOV) constraints, display resolution (pixels per degree,
PPD), and optical vergence-accommodation conflict (VAC).

Spatial Mapping & Tracking Requirements:
High-precision Simultaneous Localization and Mapping (SLAM) in
dynamic, unstructured lighting conditions.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

520 / 533

Future Horizons: Spatial Computing and AI
Convergence I

Convergence with Artificial Intelligence:
Generative AI for instant 3D asset/world synthesis; multimodal AI
agents acting as context-aware spatial assistants.

WebXR and Open Ecosystems:
Standardized browser-based spatial experiences removing friction from
app installation.

Advanced Hardware Form Factors:
Lightweight smart glasses using waveguide optics, micro-OLED
displays, and low-power spatial co-processors.
Direct neural control interfaces (EMG/BCI) for seamless spatial input.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

521 / 533

Introduction to AR/VR Applications & Industry
Scope I

Taxonomy of Spatial Computing Applications
Augmented Reality (AR): Superimposes computer-generated digital
content onto the user’s real-world environment, enhancing situational
awareness.
Virtual Reality (VR): Completely immerses the user in a synthetic
3D virtual environment, isolating them from physical surroundings.
Mixed Reality (MR): Merges real and virtual worlds where physical
and digital objects co-exist and interact in real-time.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

522 / 533

Introduction to AR/VR Applications & Industry
Scope II

Core Technological Drivers
High-precision spatial mapping and Simultaneous Localization and
Mapping (SLAM).
Ultra-low latency motion-to-photon rendering (< 20 ms).
Edge computing and 5G/6G wireless streaming for lightweight
head-mounted displays.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

523 / 533

Healthcare & Medical Training Applications I
Surgical Navigation & Image-Guided Intervention
AR overlay of patient 3D anatomical models (derived from CT/MRI
scans) directly onto the surgical site.
Enhanced precision in minimally invasive surgeries, orthopedic
procedures, and neurosurgery.
Real-time projection of vital signs and tumor margins into surgeon
optical view fields.

Medical Simulation & Rehabilitation
VR-based surgical simulators equipped with force-feedback (haptic)
devices for risk-free surgical practice.
Physical neuro-rehabilitation using gamified VR to foster neural
plasticity in stroke recovery.
Virtual exposure therapy (VRET) for treating post-traumatic stress
disorder (PTSD), phobias, and anxiety.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

524 / 533

Industrial Design, Manufacturing & Maintenance I
Digital Twins & CAD Visualization
Immersive 1:1 scale visualization of complex Computer-Aided Design
(CAD) assemblies.
Collaborative spatial design reviews across globally distributed
engineering teams.
Real-time simulation of structural dynamics and fluid dynamics within
VR space.

Assembly & Remote Maintenance
Hands-free AR smart glasses displaying step-by-step assembly
workflows and torque specifications.
Remote expert assistance: offsite specialists annotate real-time field
video feeds.
Reduction in human operational errors, assembly cycle times, and
training overheads.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

525 / 533

Education, E-Learning & Skill Acquisition I
Pedagogical Transformation
Shift from passive learning to active, constructivist learning through
spatial immersion.
Enhanced spatial cognition and retention through interactive 3D
visualizations.

Domain Applications in Academia
STEM Education: Interactive manipulation of subatomic particles,
molecular structures, and astrophysical phenomena.
High-Risk Vocational Training: Hazardous chemical handling,
high-voltage electrical grid repair, and emergency response drills.
History & Geography: Immersive historical reenactments and
virtual field trips to inaccessible planetary terrain.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

526 / 533

Architecture, Urban Planning & Real Estate I
Architectural Walkthroughs & BIM
Direct integration with Building Information Modeling (BIM) software
platforms.
Real-time lighting, material finish, and spatial acoustics evaluation
before physical construction.
Client walkthroughs enabling early detection of structural and
aesthetic design flaws.

Urban Planning & Real Estate
AR overlay of proposed city infrastructure onto existing urban
landscapes for zoning review.
Environmental impact simulation: shadow analysis, wind flow
dynamics, and traffic patterns.
Virtual staging and remote property tours in residential and
commercial real estate.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

527 / 533

Military, Aerospace & Simulation Training I
Flight & Tactical Simulators
High-fidelity VR flight simulators reducing training costs and
eliminating operational flight risks.
Synthetic Training Environments (STE) enabling multi-branch
combat training scenarios.
G-force and environment simulation combined with physiological
tracking.

Tactical AR Systems
Integrated Visual Augmentation Systems (IVAS) providing night
vision, thermal imaging, and squad tracking.
Real-time heads-up display (HUD) target acquisition and navigational
overlay in degraded environments.
Maintenance procedures on defense aircraft and armored vehicles via
AR overlay guides.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

528 / 533

Retail, E-Commerce & Virtual Try-Ons I
Spatial Commerce & Product Visualization
True-to-scale AR placement of furniture, appliances, and home decor
in user living spaces.
Interactive 3D product customization (colors, materials, options) prior
to purchasing.
Significant reduction in product return rates and higher customer
conversion efficiency.

Virtual Try-On Technologies
Markerless facial tracking for virtual cosmetics, eyewear, and jewelry
try-ons.
Body tracking and pose estimation for real-time virtual apparel fitting
rooms.
In-store AR navigation and personalized location-aware promotional
offers.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

529 / 533

Entertainment, Gaming & Interactive Media I
Virtual Reality Gaming
6DoF (Six Degrees of Freedom) tracking enabling physical movement
within digital gameplay environments.
Real-time graphics rendering engines (Unreal Engine, Unity)
optimized for dual-eye stereo output.
Haptic suits, gloves, and omnidirectional treadmills for enhanced
sensory immersion.

Media Production & Volumetric Cinema
Location-Based Entertainment (LBE) venues offering shared
physical-virtual experiences.
Volumetric video capture enabling viewer-controlled camera
perspectives in live performances.
Augmented reality live events, stage projections, and interactive
broadcasting.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

530 / 533

Cultural Heritage, Tourism & Digital Museums I
Digital Preservation & Photogrammetry
High-resolution photogrammetric 3D modeling of endangered
monuments, artifacts, and heritage sites.
Virtual archives offering worldwide access to delicate or restricted
archaeological items.

Augmented Heritage Tours
On-site AR reconstruction of historical ruins to visualize ancient
structures in original glory.
Interactive spatial storytelling and historic character avatars acting as
tour guides.
Virtual eco-tourism enabling exploration of sensitive ecosystems
without physical footprint.

GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

531 / 533

Human-Computer Interaction & Collaboration in
AR/VR I
Spatial Workspaces & Telepresence
Infinite virtual desktop setups superseding physical monitor display
limitations.
Photorealistic 3D avatar telepresence fostering natural non-verbal
communication (gaze, gestures).
Multi-user real-time spatial whiteboarding, CAD collaboration, and
dynamic data exploration.

Next-Generation Multimodal Interfaces
Eye-tracking driven foveated rendering and gaze-based UI selection.
Computer-vision markerless hand tracking and micro-gesture
recognition.
Spatial audio engines simulating directional sound propagation,
reflections, and occlusion.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

532 / 533

Technical Challenges, Ethics & Future Horizons I
Engineering & Human Factors Challenges
Display & Optics: Vergence-Accommodation Conflict (VAC), field
of view (FOV) limitations, resolution.
Compute & Power: Thermal throttling, battery capacity vs.
form-factor weight trade-offs.
Motion Sickness: Latency jitter and vestibular-ocular mismatch
mitigation.

Ethical & Strategic Considerations
Spatial data privacy, biometric tracking safeguards (eye/gaze
tracking, facial telemetry).
Digital addiction, social isolation, and physical environment safety
hazards.
Convergence with Generative AI for automated 3D scene generation
and intelligent virtual agents.
GTU Course

DI04000051: Multimedia Technologies

Lecture Slides

533 / 533