An Introduction to FFmpeg Interacting with the computer • command-line interface (CLI) graphical user interface (GUI) Reto Kromer • AV Preservation by reto.ch • Open-Source Tools and Resources for Audio-Visual Archives Elías Querejeta Zine Eskola Donostia (San Sebastián), Spain 19, 21, 26 and 28 May 2020 1 2 Components Tools Libraries • ffmpeg • libavutil • ffplay • libavcodec The FFmpeg Family • ffprobe • libavformat • libavdevice • libavfilter • libswscale • libswresample 3 4 Audio-Visual Exemple Data Transformations QuickTime demultiplex WAVE demultiplex: libavformat ProRes 422 HQ pcm_s24le decode decode: libavcodec yuv422p10le filter filter: libavfilter pcm_s16le AAC encode yuv420p encode: libavcodec H.264 multiplex multiplex: libavformat MP4 5 6 File Transformations Metadata Extraction ffmpeg (CLI) MediaInfo (GUI, CLI) ➔ ffmpeg.org ➔ mediaarea.net/MediaInfo ffprobe (CLI) FFmpeg Cookbook for Archivists ➔ ffmpeg.org ➔ avpres.net/FFmpeg/ ffmprovisr ➔ amiaopensource.github.io/ffmprovisr/ 7 8 Media Player A GUI for FFmpeg VLC (GUI) FFCommand Engine (GUI) ➔ www.videolan.org/vlc/ ➔ github.com/ColorlabMD/FFCommand_Engine mpv (CLI) ➔ mpv.io version 0.6 for macOS via Homebrew: ffplay (CLI) brew tap avpres/formulae ➔ ffmpeg.org brew install --HEAD ffcommand-engine 9 10 avantages disavantages TIFF data easier DPX to process bigger files OpenEXR JPEG 2000 smaller files data complexer FFV1 to process 11 12 Single Images and Streams RAWcooked RAWcooked (CLI) • encoding into Matroska (.mkv) using FFV1 ➔ mediaarea.net/RAWcooked video codec and FLAC audio codec • all metadata preserved • decoding with bit-by-bit reversibility possibility to embed sidecar files, for • example MD5, LUT, XML, PDF • compatibility with media players 13 14 Exercises Basics 15 16 Set the Working Space Generate an Image File Linux/Mac: ffmpeg cd ~/Desktop -lavfi mandelbrot -t 10 Windows: -c:v rawvideo cd Desktop -pix_fmt uyvy422 mandelbrot.avi 17 18 Play the Image File Generate a Sound File ffplay ffmpeg mandelbrot.avi -lavfi sine=frequency=440 -t 10 la.wav 19 20 Play the Sound File ffplay la.wav 311 Hz 415 Hz 370 Hz 277 Hz 466 Hz 330 Hz 523 Hz 349 Hz 392 Hz 440 Hz 494 Hz 294 Hz 262 Hz 21 22 Merge Image and Sound Play the AV File ffmpeg ffplay -i mandelbrot.avi mandela.avi -i la.wav -c:v copy mandela.avi 23 24 Extract the Metadata Container ffprobe ffprobe mandela.avi -show_format mandela.avi 25 26 Codec Container and Codec ffprobe ffprobe -show_streams -show_format mandela.avi -show_streams mandela.avi 27 28 Format the Output Save the Metadata ffprobe ffprobe -show_format -show_format -show_streams -show_streams -print_format json -print_format json mandela.avi mandela.avi > mandela.txt 29 30 Find Help ffmpeg -h ffmpeg -codecs ffmpeg -decoders ffmpeg -h decoder=aac Transformations ffmpeg -encoders ffmpeg -h encoder=libx264 ffmpeg -filters ffmpeg -pix_fmts 31 32 Modify the Container MD5 Checksums (1) ffmpeg ffmpeg -i mandelbrot.avi -i mandelbrot.avi -c copy -f framemd5 mandelbrot.mov mandelbrot_avi_framemd5.txt 33 34 MD5 Checksums (2) Compare the Checksums ffmpeg Linux/Mac: -i mandelbrot.mov diff -s -f framemd5 mandelbrot_avi_framemd5.txt mandelbrot_mov_framemd5.txt mandelbrot_mov_framemd5.txt Windows: fc mandelbrot_avi_framemd5.txt mandelbrot_mov_framemd5.txt 35 36 Play Single Images ffplay -loop 0 Master to Access DUFAY_TIFF/Dufay_%06d.tif 37 38 File Transformations • from the master file to a mezzanine file • from the master file to an access file • from the mezzanine file to an access file 39 40 Video Exemple Is the Quality the Same? master file ➔ mezzanine file ➔ access file QuickTime demultiplex master file ➔ access file ProRes 422 HQ decode yuv422p10le filter yuv420p H.264 encode multiplex MP4 41 42 Master –> Mezzanine ProRes 422 and ProRes 4444 ffmpeg • QuickTime (.mov) -f image2 -framerate 24 Matroska (.mkv) -i DUFAY_TIFF/Dufay_%06d.tif • MXF = Material eXchange Format (.mxl) -filter:v • "scale=1440:1080:flags=lanczos, pad=1920:1080:240:0" -c:v prores_ks -profile:v 3 Dufay_ProRes.mkv 43 44 Master –> Access Mezzanine –> Access ffmpeg ffmpeg -f image2 -framerate 24 -i Dufay_ProRes.mkv -i DUFAY_TIFF/Dufay_%06d.tif -pix_fmt yuv420p -filter:v -c:v libx264 -preset veryslow -crf 30 "scale=1440:1080:flags=lanczos, Dufay_2_H264.mp4 pad=1920:1080:240:0" -pix_fmt yuv420p -c:v libx264 -preset veryslow -crf 30 Dufay_1_H264.mp4 45 46 Quality control Split screen • difference file ffmpeg split screen -i Dufay_1_H264.mp4 • -i Dufay_2_H264.mp4 -filter_complex "[0]crop=iw/2:ih:0:0[left]; [1]crop=iw/2:ih:iw/2:0[right]; [left][right]hstack" Dufay_H264_split.mp4 47 48 Difference file AV Preservation by reto.ch ffmpeg zone industrielle Le Trési 3 -i Dufay_1_H264.mp4 1028 Préverenges -i Dufay_2_H264.mp4 Switzerland -filter_complex "[1]format=yuva444p, Web: reto.ch lut=c3=128, Twitter: @retoch negate[1_with_alpha]; Email: [email protected] [0][1_with_alpha]overlay" Dufay_H264_delta.mp4 49 50.