An Introduction to FFmpeg Interacting with the computer • command-line interface (CLI) graphical user interface (GUI) Reto Kromer • AV Preservation by reto.ch •
Open-Source Tools and Resources for Audio-Visual Archives Elías Querejeta Zine Eskola Donostia (San Sebastián), Spain 19, 21, 26 and 28 May 2020
1 2
Components
Tools Libraries • ffmpeg • libavutil • ffplay • libavcodec The FFmpeg Family • ffprobe • libavformat • libavdevice • libavfilter • libswscale • libswresample
3 4 Audio-Visual Exemple Data Transformations
QuickTime demultiplex WAVE demultiplex: libavformat ProRes 422 HQ pcm_s24le decode decode: libavcodec yuv422p10le filter filter: libavfilter pcm_s16le AAC encode yuv420p encode: libavcodec H.264 multiplex multiplex: libavformat MP4
5 6
File Transformations Metadata Extraction ffmpeg (CLI) MediaInfo (GUI, CLI) ➔ ffmpeg.org ➔ mediaarea.net/MediaInfo ffprobe (CLI) FFmpeg Cookbook for Archivists ➔ ffmpeg.org ➔ avpres.net/FFmpeg/ ffmprovisr ➔ amiaopensource.github.io/ffmprovisr/
7 8 Media Player A GUI for FFmpeg
VLC (GUI) FFCommand Engine (GUI) ➔ www.videolan.org/vlc/ ➔ github.com/ColorlabMD/FFCommand_Engine mpv (CLI) ➔ mpv.io version 0.6 for macOS via Homebrew: ffplay (CLI) brew tap avpres/formulae ➔ ffmpeg.org brew install --HEAD ffcommand-engine
9 10
avantages disavantages
TIFF data easier DPX to process bigger files OpenEXR
JPEG 2000 smaller files data complexer FFV1 to process
11 12 Single Images and Streams RAWcooked RAWcooked (CLI) • encoding into Matroska (.mkv) using FFV1 ➔ mediaarea.net/RAWcooked video codec and FLAC audio codec • all metadata preserved • decoding with bit-by-bit reversibility possibility to embed sidecar files, for • example MD5, LUT, XML, PDF • compatibility with media players
13 14
Exercises Basics
15 16 Set the Working Space Generate an Image File
Linux/Mac: ffmpeg cd ~/Desktop -lavfi mandelbrot -t 10 Windows: -c:v rawvideo cd Desktop -pix_fmt uyvy422 mandelbrot.avi
17 18
Play the Image File Generate a Sound File ffplay ffmpeg mandelbrot.avi -lavfi sine=frequency=440 -t 10 la.wav
19 20 262 Hz mandela.avi -c:vcopy -i ffmpeg Merge ImageandSound
-i 277 Hz
la.wav mandelbrot.avi 294 Hz 311 Hz
330 Hz
349 Hz 23 21 370 Hz 392 Hz 415 Hz 440 Hz 466 Hz 494 Hz
523 Hz ffplay mandela.avi ffplay la.wav Play theSoundFile Play theAVFile 24 22 Extract the Metadata Container ffprobe ffprobe mandela.avi -show_format mandela.avi
25 26
Codec Container and Codec ffprobe ffprobe -show_streams -show_format mandela.avi -show_streams mandela.avi
27 28 Format the Output Save the Metadata ffprobe ffprobe -show_format -show_format -show_streams -show_streams -print_format json -print_format json mandela.avi mandela.avi > mandela.txt
29 30
Find Help ffmpeg -h ffmpeg -codecs ffmpeg -decoders ffmpeg -h decoder=aac Transformations ffmpeg -encoders ffmpeg -h encoder=libx264 ffmpeg -filters ffmpeg -pix_fmts
31 32 Modify the Container MD5 Checksums (1) ffmpeg ffmpeg -i mandelbrot.avi -i mandelbrot.avi -c copy -f framemd5 mandelbrot.mov mandelbrot_avi_framemd5.txt
33 34
MD5 Checksums (2) Compare the Checksums ffmpeg Linux/Mac: -i mandelbrot.mov diff -s -f framemd5 mandelbrot_avi_framemd5.txt mandelbrot_mov_framemd5.txt mandelbrot_mov_framemd5.txt
Windows: fc mandelbrot_avi_framemd5.txt mandelbrot_mov_framemd5.txt
35 36 Play Single Images
ffplay -loop 0 Master to Access DUFAY_TIFF/Dufay_%06d.tif
37 38
File Transformations • from the master file to a mezzanine file • from the master file to an access file • from the mezzanine file to an access file
39 40 Video Exemple Is the Quality the Same?
master file ➔ mezzanine file ➔ access file QuickTime demultiplex master file ➔ access file ProRes 422 HQ decode yuv422p10le filter yuv420p H.264 encode multiplex MP4
41 42
Master –> Mezzanine ProRes 422 and ProRes 4444 ffmpeg • QuickTime (.mov) -f image2 -framerate 24 Matroska (.mkv) -i DUFAY_TIFF/Dufay_%06d.tif • MXF = Material eXchange Format (.mxl) -filter:v • "scale=1440:1080:flags=lanczos, pad=1920:1080:240:0" -c:v prores_ks -profile:v 3 Dufay_ProRes.mkv
43 44 Master –> Access Mezzanine –> Access ffmpeg ffmpeg -f image2 -framerate 24 -i Dufay_ProRes.mkv -i DUFAY_TIFF/Dufay_%06d.tif -pix_fmt yuv420p -filter:v -c:v libx264 -preset veryslow -crf 30 "scale=1440:1080:flags=lanczos, Dufay_2_H264.mp4 pad=1920:1080:240:0" -pix_fmt yuv420p -c:v libx264 -preset veryslow -crf 30 Dufay_1_H264.mp4
45 46
Quality control Split screen
• difference file ffmpeg split screen -i Dufay_1_H264.mp4 • -i Dufay_2_H264.mp4 -filter_complex "[0]crop=iw/2:ih:0:0[left]; [1]crop=iw/2:ih:iw/2:0[right]; [left][right]hstack" Dufay_H264_split.mp4
47 48 Difference file AV Preservation by reto.ch ffmpeg zone industrielle Le Trési 3 -i Dufay_1_H264.mp4 1028 Préverenges -i Dufay_2_H264.mp4 Switzerland -filter_complex "[1]format=yuva444p, Web: reto.ch lut=c3=128, Twitter: @retoch negate[1_with_alpha]; Email: [email protected] [0][1_with_alpha]overlay" Dufay_H264_delta.mp4
49 50