An Introduction to FFmpeg Interacting with the computer • command-line interface (CLI) (GUI) Reto Kromer • AV Preservation by reto.ch •

Open-Source Tools and Resources for Audio-Visual Archives Elías Querejeta Zine Eskola Donostia (San Sebastián), Spain 19, 21, 26 and 28 May 2020

1 2

Components

Tools Libraries • • libavutil • ffplay • The FFmpeg Family • ffprobe • libavformat • libavdevice • libavfilter • libswscale • libswresample

3 4 Audio-Visual Exemple Data Transformations

QuickTime demultiplex WAVE demultiplex: libavformat ProRes 422 HQ pcm_s24le decode decode: libavcodec yuv422p10le filter filter: libavfilter pcm_s16le AAC encode yuv420p encode: libavcodec H.264 multiplex multiplex: libavformat MP4

5 6

File Transformations Metadata Extraction ffmpeg (CLI) (GUI, CLI) ➔ ffmpeg.org ➔ mediaarea.net/MediaInfo ffprobe (CLI) FFmpeg Cookbook for Archivists ➔ ffmpeg.org ➔ avpres.net/FFmpeg/ ffmprovisr ➔ amiaopensource.github.io/ffmprovisr/

7 8 Media Player A GUI for FFmpeg

VLC (GUI) FFCommand Engine (GUI) ➔ www..org/vlc/ ➔ github.com/ColorlabMD/FFCommand_Engine (CLI) ➔ mpv.io version 0.6 for macOS via Homebrew: ffplay (CLI) brew tap avpres/formulae ➔ ffmpeg.org brew install --HEAD ffcommand-engine

9 10

avantages disavantages

TIFF data easier DPX to process bigger files OpenEXR

JPEG 2000 smaller files data complexer FFV1 to process

11 12 Single Images and Streams RAWcooked RAWcooked (CLI) • encoding into (.mkv) using FFV1 ➔ mediaarea.net/RAWcooked video and FLAC audio codec • all metadata preserved • decoding with bit-by-bit reversibility possibility to embed sidecar files, for • example MD5, LUT, XML, PDF • compatibility with media players

13 14

Exercises

15 16 Set the Working Space Generate an Image File

Linux/Mac: ffmpeg cd ~/Desktop -lavfi mandelbrot -t 10 Windows: -:v rawvideo cd Desktop -pix_fmt uyvy422 mandelbrot.avi

17 18

Play the Image File Generate a Sound File ffplay ffmpeg mandelbrot.avi -lavfi sine=frequency=440 -t 10 la.

19 20 262 Hz mandela.avi -c:vcopy -i ffmpeg Merge ImageandSound

-i 277 Hz

la.wav mandelbrot.avi 294 Hz 311 Hz

330 Hz

349 Hz 23 21 370 Hz 392 Hz 415 Hz 440 Hz 466 Hz 494 Hz

523 Hz ffplay mandela.avi ffplay la.wav Play theSoundFile Play theAVFile 24 22 Extract the Metadata Container ffprobe ffprobe mandela.avi -show_format mandela.avi

25 26

Codec Container and Codec ffprobe ffprobe -show_streams -show_format mandela.avi -show_streams mandela.avi

27 28 Format the Output Save the Metadata ffprobe ffprobe -show_format -show_format -show_streams -show_streams -print_format -print_format json mandela.avi mandela.avi > mandela.txt

29 30

Find Help ffmpeg -h ffmpeg - ffmpeg -decoders ffmpeg -h decoder=aac Transformations ffmpeg -encoders ffmpeg -h encoder=libx264 ffmpeg -filters ffmpeg -pix_fmts

31 32 Modify the Container MD5 Checksums (1) ffmpeg ffmpeg -i mandelbrot.avi -i mandelbrot.avi -c copy -f framemd5 mandelbrot.mov mandelbrot_avi_framemd5.txt

33 34

MD5 Checksums (2) Compare the Checksums ffmpeg /Mac: -i mandelbrot.mov diff -s -f framemd5 mandelbrot_avi_framemd5.txt mandelbrot_mov_framemd5.txt mandelbrot_mov_framemd5.txt

Windows: fc mandelbrot_avi_framemd5.txt mandelbrot_mov_framemd5.txt

35 36 Play Single Images

ffplay -loop 0 Master to Access DUFAY_TIFF/Dufay_%06d.tif

37 38

File Transformations • from the master file to a mezzanine file • from the master file to an access file • from the mezzanine file to an access file

39 40 Video Exemple Is the Quality the Same?

master file ➔ mezzanine file ➔ access file QuickTime demultiplex master file ➔ access file ProRes 422 HQ decode yuv422p10le filter yuv420p H.264 encode multiplex MP4

41 42

Master –> Mezzanine ProRes 422 and ProRes 4444 ffmpeg • QuickTime (.mov) -f image2 -framerate 24 Matroska (.mkv) -i DUFAY_TIFF/Dufay_%06d.tif • MXF = (.mxl) -filter:v • "scale=1440:1080:flags=lanczos, pad=1920:1080:240:0" -c:v prores_ks -profile:v 3 Dufay_ProRes.mkv

43 44 Master –> Access Mezzanine –> Access ffmpeg ffmpeg -f image2 -framerate 24 -i Dufay_ProRes.mkv -i DUFAY_TIFF/Dufay_%06d.tif -pix_fmt yuv420p -filter:v -c:v libx264 -preset veryslow -crf 30 "scale=1440:1080:flags=lanczos, Dufay_2_H264.mp4 pad=1920:1080:240:0" -pix_fmt yuv420p -c:v libx264 -preset veryslow -crf 30 Dufay_1_H264.mp4

45 46

Quality control Split screen

• difference file ffmpeg split screen -i Dufay_1_H264.mp4 • -i Dufay_2_H264.mp4 -filter_complex "[0]crop=iw/2:ih:0:0[left]; [1]crop=iw/2:ih:iw/2:0[right]; [left][right]hstack" Dufay_H264_split.mp4

47 48 Difference file AV Preservation by reto.ch ffmpeg zone industrielle Le Trési 3 -i Dufay_1_H264.mp4 1028 Préverenges -i Dufay_2_H264.mp4 Switzerland -filter_complex "[1]format=yuva444p, Web: reto.ch lut=c3=128, Twitter: @retoch negate[1_with_alpha]; Email: [email protected] [0][1_with_alpha]overlay" Dufay_H264_delta.mp4

49 50