Everything You Need to Know about Video Compression

This is the most comprehensive guide on MP4 H264/AAC encoding on YouTube! Start from uncompressed frames and finish with a playable MP4 file 1% of the original size. The H264/AAC MP4 file, also known as AVC/AAC, is currently used by the majority of all videos on the internet. However, almost nobody knows exactly how these files work. In this video, we will uncover the exact processes used by real encoders to encode an uncompressed video into an MP4 file. Old version: https://youtu.be/90rKiHf3mNI ******** PLEASE CHECK THESE OUT!!! ******** H264 (AVC) Specs: https://www.itu.int/rec/T-REC-H.264-202108-S/en AAC Specs: https://ossrs.io/lts/zh-cn/assets/files/ISO_IEC_13818-7-AAC-2004-67b015c6ddfc9a4af83665738477124a.pdf MP4 container format documentation: https://developer.apple.com/documentation/quicktime-file-format Online MP4 parser: https://onlinemp4parser.com/ *Open source encoders:* H264: https://github.com/cisco/openh264/tree/master/codec/encoder/core AAC: https://github.com/knik0/faac/tree/master/libfaac *Side notes:* At 24:45, because the chroma coded block pattern is already defined in the mb_type, it doesn't need to be encoded. The luma coded block pattern is only encoded if the 3rd suffix bit is 0. This only applies for intra_16x16, not in P slices. Also, only one intra prediction mode is stored for both chroma blocks, stored in intra_chroma_pred_mode. 0:00 Intro 1:27 Uncompressed video and audio format *VIDEO COMPRESSION (I slice)* 2:05 Slices, Macroblocks, IDR frame 2:21 Color Space Conversion: RGB → YCrCb 2:53 4:2:0 Chroma Subsampling 4:05 Intra Prediction 6:09 Residuals 7:21 Frequency Transform (Luma) 10:36 Frequency Transform (Chroma) 11:18 Quantization 13:08 Zigzag scan 14:03 Video compression summary *BIT ENCODING PROCESSES* 14:19 CABAC (Context Adaptive Binary Arithmetic Coding) 15:10 CABAC Initialization 17:33 CABAC Encoding process (Important!!!!) 20:11 CABAC Termination symbol (end_of_slice_flag = 1, ctxidx=276) 21:24 CABAC bypass mode 22:01 Unary, Signed exp-golomb, Unsigned exp-golomb *I-SLICE ENCODING* 23:08 Macroblock type 24:45 (Only for Intra_4x4) Table of prediction modes 25:46 Intra chroma prediction mode 26:02 CBP Luma (See side notes in description) 26:46 CBP Chroma 27:32 QP Delta 27:48 Luma residuals 30:37 Chroma DC (Blue then red) 31:18 Chroma AC (Blue then red) *Slice headers/info* 32:02 Slice types (I slice, P slice, B slice, IDR frame) 33:21 SPS and PPS (Sequence/Picture parameter sets) 34:48 NAL Unit header and slice header 38:02 RSBP Trailing bits (Put after CABAC termination symbol) 38:15 NAL Unit 4 byte size prefix *VIDEO COMPRESSION (P slice)* 38:26 P slice header walkthrough 38:48 Luma inter prediction 40:12 Hexagon search (Inter prediction) 41:02 Half-pel (Half-pixel) refinement 41:48 6-tap interpolation (Half-pixel search) 43:04 Quarter-pixel refinement 43:56 Inter prediction heuristics 44:56 Chroma inter prediction 45:20 Inter prediction residuals *P-SLICE ENCODING* 45:38 Macroblock skip flag (Has nothing changed?) 46:04 Macroblock type (P slices) 46:33 Sub macroblock types 46:44 Reference frame index 47:11 Motion vectors (All Xs then all Ys) 49:03 CBP luma, CBP chroma, QP_delta, residuals... 49:23 Slice type assignments *PART 2: AUDIO* 49:42 Uncompressed audio *Audio compression* 50:14 Split into audio frames 51:01 Frequency transform (Modified discrete cosine transform) 52:30 (Informative) How to find the pitch of a frequency coefficient 52:45 Audio quantization 53:02 Split frames into 49 bands 53:50 Determine quantization weights, then quantize *Audio encoding* 55:33 ↓Read the AAC specs in description for more info!↓ 55:56 Audio frame info 57:08 Scale factor data 57:53 ↓Read the AAC specs in description for more info!↓ 58:03 Spectral data (Frequency coefficients) 1:00:10 Section data (Band codebooks) *MP4 Container* 1:00:55 The MP4 file 1:01:50 ftyp 1:02:15 mdat 1:02:49 moov 1:04:41 stbl 1:04:55 avcC (Put SPS and PPS here) 1:06:53 Audio track 1:08:08 ↓Check the description for more info!↓ 1:08:37 Recap 1:09:05 Decoding an MP4 file *H264 glitches I found* Missing the initial IDR frame: https://www.youtube.com/watch?v=QlMVNMo7DM0 Sound Effect by freesound_community from Pixabay #multimedia #science #computers #computerscience #computerengineering #algorithm #file #fileformat #mp4 #mp3 #h264 #compression #jpeg #imagecompression #videocompression #aac #audio #film #movie #motionpicture #effectking #localstarbinit #comedyeffect #vikramsarkar #tiktokeffects #moneylove #catloverlife #ciddubbing #flyxoedits #effects #processing #foryou #recommended #technical #bit #bytes #coding #encoding #encoder #codec #decoder #itu #software #megabytes #fft #dct #tech #math #residuals #luma #chroma #rgb #pixels #picture #png #jpg #mov #avi