None of My Pixel Is Your Business: Active Watermarking Cancellation Against Video Streaming Service Authors

None of My Pixel is Your Business: Active Watermarking Cancellation Against Video Streaming Service Authors

• Wang Kang • Alibaba Group • TUNA • Hui Yi-Qun • Tsinghua University, Master Candidate • TUNA

(FamilyName⎵GivenName) Outline

• Background • (2/3)Static Experiments • Problem & Market Overview • (3/3)Realtime Processing • Visible Watermark • FFmpeg & halide • Invisible Watermark (Forensic Watermark) • libtmblock • (0/3)Theoretical Estimation • Experiment Results • Principles • Discussion • Method #1: Pre-distort • Related & Future Work • Method #2: Post-process • With AI • Lossless Recovery Range • Our Goal • (1/3)Logo Extraction • Drop Visible Watermark • Principles • Use DRM, or like a real man. • Code Landscape: Video Streaming Services in China

http://v.163.com/ https://www.pixnet.net/ http://music.163.com/ http://www.pptv.com/ http://www.56.com/ http://v.iqilu.com/ http://www.acfun.tv/ http://v.qq.com/ http://tieba.baidu.com/ http://live.qq.com/ http://www.baomihua.com/ http://video.sina.com.cn/ http://www.bilibili.com/ http://video.weibo.com/ http://www.dilidili.com/ http://tv.sohu.com/ http://www.douban.com/ http://www.tudou.com/ http://www.douyutv.com/ http://www.xiami.com/ http://www.panda.tv/ http://www.isuntv.com/ http://v.ifeng.com/ http://www.yinyuetai.com/ http://www.fun.tv/ http://www.youku.com/ http://www.iqiyi.com/ http://www.zhanqi.tv/lives http://www.joy.cn/ http://www.cntv.cn/ http://www.ku6.com/ http://huaban.com/ http://www.kugou.com/ http://tvcast.naver.com/ http://www.kuwo.cn/ http://www.mgtv.com/ http://www.le.com/ http://www.huomao.com/ http://www.lizhi.fm/ http://www.quanmin.tv/ http://www.miaopai.com/ http://www.365yg.com/ See Also: http://www.miomio.tv/ https://www.ixigua.com/ https://github.com/soimort/you-get/blob/develop/README.md#supported-sites https://www.douyin.com/ https://www.kuaishou.com/ # youtube-dl --list-extractors Watermark Technology Overview

Forensic/Invisible Watermark Transform-domain Wavelet-based Fourier-based Compressed-domain Bitstream Spatial-domain LSB Blue Channel (simple but effective) Embedding and Extracting method Additive Multiplicative Fw:Fw:Fw:Fw:Fw:Fw: Watermark

• Wild west of copyright • A highly competitive market • Forwarded and reposted.. .. • Just don’t care.. UX A Historical Tradition? by Emperor 乾隆 (1735-1795) 神: Oh God Full Life Circle of Video

Browser

UGC

Playback Mobile Endpoint VDN, Video Distribution Network

PGC Desktop Full Life Circle of Video

Playback UGC PGC VDN, Video Distribution Network Endpoint

Browser Mobile Desktop

Pre Post Distorted Process of Watermark Watermark Wish List

• A library that collects all logos • high-precision, with alpha channel, size, position • A FFmpeg filter that is able to do: • pre-distortion with specified logo before uploading/streaming • post-process with specified logo • A browser plugin/mobile app that can: • perform post-process against watermarked video (0/3) Theoretical Estimation to get the probability of recovery Definition of Alpha Compositing

(ROutput, GOutput, BOutput) = (1 − α)(RInput, GInput, BInput) + α(RLogo, GLogo, BLogo)

https://en.wikipedia.org/wiki/Alpha_compositing#Composing_alpha_blending_with_gamma_correction Proposed Method #1: Pre-distortion

1 α P′ = P − X 1 − α 1 − α

Where:

P: (R,G,B) of Original Pixel ⍺: the ⍺ channel of Extracted Logo X: (R,G,B) of Extracted Logo P’: (R,G,B) of Predistorted Pixel Lossless Limitation: Proof: (When watermarked later…) P, P′, X ∈ [0,255] (1 − α)P′ + αX {α ∈ [0,1) Given and X, the original pixel must be between: = P − αX + αX = P ⍺ P ∈ [αX, αX + 255(1 − α)] Visualizing ⍺=0 .. 1

Logo Pixel

P ∈ [αX, αX + 255(1 − α)]

Original Pixel ⍺=0=0.1=0.2=0.3=0.4=0.5=0.6=0.7=0.8=0.9=0.99

Recoverable Color Area ⍺=0 .. 1

⍺=0 ⍺=0.1 ⍺=0.2 ⍺=0.3 ⍺=0.4 ⍺=0.5

⍺=0.6 ⍺=0.7 ⍺=0.8 ⍺=0.9 ⍺=0.99 Recoverable Color Area Proposed Method #2: Post-process

RGBOutput = (1 − α)RGBInput + αRGBLogo Where:

P0: (R,G,B) of Original Pixel P: (R,G,B) of Watermarked Pixel ⍺: the ⍺ channel of Extracted Logo X: (R,G,B) of Extracted Logo P = (1 − α)P0 + αX P’: (R,G,B) of Recovered Pixel

Lossless Limitation: 1 α P′ = P = P − X P ∈ [0,255] 0 1 − α 1 − α

Quantitative loss: int -> float -> int (compensated by human brain) Engineering Challenges

• Extraction of Transparent • Gamma Correction Watermark • Realtime Performance • Theoretical Recoverable Range • Framework • Position Fine-Tuning • Live Streaming Protocol • Shadow and Edge Handling • Video Codec • Color Space Management • sRGB / Linear RGB (1/3) Logo Extraction with high-precision metadata if lucky enough.. ⍺ Channel Recovery of Logo (Watermark)

Logo Logo

Y1 = (1 − α)X1 + αL Y2 = (1 − α)X2 + αL X1 = (0,0,0) X2 = (255,255,255) *With enough patience.. just watch and wait for a pure-color scene

ΔX α = 1 − ΔY Logo ΔX ⋅ Y − ΔY ⋅ X L = 1 1 ΔX − ΔY (232,232,232,⍺=0.88)

(0,0,0, ⍺=0.4) Logo Extracting Automation

As a content provider, you can just inject two {black, white} frames.. two of any pure color scenes are fine a simple automation script should do well. Or if you really get sufficient funds, deploy an advertisement. Other Methods for Extracting Pixel Estimation with AI Long Exposure Algorithm Time-varying part averaged (2/3) Static Experiments Effective Development Approach

• Challenge: Image/video processing is • not so easy to develop • hard to debug • Write Python prototype scripts Halide • tuning: algorithm, media resources, color, position • life is short, use Python tmblock libtmblock • https://github.com/tmblock/testcase • Write C++ code with Halide • to improve performance FFmpeg • to integrate with FFmpeg for realtime processing • compare with Python version as test case Halide: a language for image processing and computational photography http://halide-lang.org/

Halide is a new programming language designed to make it easier to write high-performance image processing code on modern machines. Its current front end is embedded in C++. Compiler targets include x86/SSE, ARM v7/NEON, CUDA, and OpenCL.

TensorFlow MIT CSAIL GNURadio 2013 Apache mxnet: Halide IR(Intermediate Representation) dmlc/HalideIR

enough performance by native scheduler enough space for future optimization Pre-distorted Cancelled Result: Pre-distorted then Watermarked

Method #2: Post

(3/3) Realtime Processing with FFmpeg

Project TMBlock System Architecture TMBlock

After a few days’ coding and building…

We released the project TMBlock

# Build tmblock $ cmake -DHALIDE_DISTRIB_DIR=path/to/halide -DHALIDE_DISTRIB_USE_STATIC_LIBRARY=TRUE ..

# Build ffmpeg with tmblock $ ./configure --enable-tmblock --extra-cflags="-I$HOME/Develop/TMBlock/libtmblock/include/" —extra-ldflags="-L$HOME/Develop/TMBlock/libtmblock/ build/" $ make -j16

# If you are using ArchLinux: https://aur.archlinux.org/packages/ffmpeg-tmblock-git/ # yaourt -S ffmpeg-tmblock-git

# Run.. $ LD_LIBRARY_PATH=$HOME/Develop/TMBlock/libtmblock/build/ ./ffmpeg_g -i input.mp4 -i logo.png -filter_complex "[0:v][1:v] tmblock=x=100:y=100:func=pre [out]" -map "[out]" -t 10 -pix_fmt yuv420p —y output.mp4 #^^^ supported func: pre / post / embed Pre-distorted (Anti-watermarked) Watermarked

Pre-distorted—>Watermark Watermarked->Post-processed Pre-distorted (Anti-watermarked) Watermarked

Pre-distorted—>Watermark Watermarked->Post-processed Why? Recoverable Range

• White: (255,255,255) ⍺=0.5 • Lossless Recoverable Range: • (127-255, 127-255, 127-255)

Range of 100% Recovery 90% Recovery *Human Eyes Recoverable Area

ProTip

TM Record under sky. Demo: Bilibili.tv

bilibili.tv(NYSE: BILI): One of biggest video providers in China Still picture watermark on top-left ○ A little trouble: Blurred edge Solving PNG Codec Issue Watermark cancelled completely with method #1: pre-distortion lucky zone.

Pre-distorted Media

Problem: Difference between PNG parsing libraries for edge blurring Other ways…

• Transform • to transform one watermark into another provider's.

• Code Rate Jitter • filling with high-frequency components • force video codec to reduce the code rate near watermark. • eg: World Cup

• Frame Squeezing • sacrificing some resolution by squeezing screen • restoring with a user-defined javascript or iFrame to bypass watermark • on your blog Interesting Work

• CSS is So Overpowered It Can Deanonymize Facebook Users

Some of the recent additions to the Cascading Style Sheets (CSS) web standard are so powerful that a security researcher has abused them to deanonymize visitors to a demo site and reveal their Facebook usernames, avatars, and if they liked a particular web page of Facebook. Information leaked via this attack could aid some advertisers link IP addresses or advertising profiles to real- life persons, posing a serious threat to a user's online privacy. The leak isn't specific to Facebook but affects all sites which allow their content to be embedded on other web pages via iframes.

Interesting new research surface. Countermeasure (for the evil side)

• Edge Blurring (232,232,232,⍺=0.88) Recovery Range: [204, 255] • Combine transparent part with non-transparent part • Combine multiple ⍺ to make recovery range unavailable • to provide a better user experience • Position Randomize • Breathing Style (0,0,0, ⍺=0.4) • Dynamic Watermark Recovery Range: [0, 153] • Anti-Anti-Watermark

Conclusion & Future Work

• What we have done so far: • Theoretical estimation and experiment results of • pre-process method • post-process method • Realtime processing framework • Debug testbed • Running code can be found at https://github.com/tmblock/ • Future Work • Auto detect and extract from arbitrary video sequence • With AI: • Enhanced image recovery with GAN • Recovery of loss caused by opaque watermarks • Integrated the algorithm into USB Camera with FPGA • On the fly Hopefully..

• This work will be able to push video content providers to stop using ugly and annoying visible watermark. • User experience matters. • To protect copyright, • please deploy DRM or invisible watermark • or be a man • CC or Open Source/Open Access This is a hack

of the people, by the people, and for the people.

The watermark shall perish from the earth. Reference

• https://blog.huiyiqun.me/2016/10/28/livestream-your-desktop-with-nginx-rtmp-module.html • https://github.com/tmblock • https://halide-lang.org/ • https://www.nmm-hd.org/newbbs/viewtopic.php?t=649 Thank you.

Q&A