Signal Processing for Pen and Touch Sensors

May 31, 2021 Masayuki Miyamoto Wacom

1. Pen and Touch User Interface 2. EMR (Electro-Magnetic Resonance) Sensing Technology 3. Capacitive Sensing Technology - Principle - SNR Enhancement - Noise Immunity - Sensor Requirement - Passive Pen - Active Pen 4. Latest Technical Challenges

Games

Pen Tablet Products for Creative Users

Lalaport, Mitsui Fudosan Retail Management, Japan

Citibank, Korea Wacom Clipboard Signature Tablets Wacom Ink Layer Language

- No Battery in the pen

https://tablet.wacom.co.jp/what/news-img/W8002basis.pdf

[An Example]

https://tablet.wacom.co.jp/what/news-img/W8002basis.pdf

Substrate https://www.bareconductive.com/blogs/blog/how-do-the- touch-boards-capacitive-sensors-work

Sensing Control Circuit Sensing Control Circuit

1. Charge Integration (Charge to Voltage Conversion)

Vout = - Cm/Cint(Vdrive – Vref) Cm

Vdrive

2. Frequency Response

Rtx Cm Rrx vi vo = G(jω)vi

Ctx Crx

C 11 C 21 C31 CM1

3 C 12 C 22 C 32 C M2

C C 1j C 2j C 3j Mj 2

C 1N C 2N C 3N C MN Relative SNR Relative

0 10 100 Sequential Drive Number of Drive Channels Time M SNR decreases as the number of channel increases

C11 C21 C31 CM1 M

C12 C22 C32 CM2 퐒퐣 = Sj1, ⋯ , SjL = ෍ Cij퐕퐢 i=1

C1j C2j C3j CMj Signal Reconstruction

C1N C2N C3N CMN M 퐒퐣, 퐕퐥 = σi=1 Cij퐕퐢, 퐕퐥 M = σi=1 Cij 퐕퐢, 퐕퐥 M = σi=1 Cij Lδil = C L Parallel Drive lj 퐕ퟏ 퐕풊 퐕푴 Time L if 퐕퐢, 퐕퐥 = σk=1 VikVlk = Lδil M

Parallel Drive 3

2 Relative SNR Relative 1

Sequential Drive 0 10 100 Number of Drive Channels

(Assumption) Sensor’s channel resistance is 0.

Sequential Drive Parallel Drive C(fF) C(fF)

S32 S32 S28 S28 S24 S24 S20 S20 S16 S16 S12 S12 D0 D0 D2 D4 S8 D2 D4 S8 D6 D8 D6 D8 D10 D12 D10 D12 D14 D16 S4 D14 D16 S4 D18 D20 D18 D20 D22 D24 D22 D24 D26 S0 D26 S0 D28 D30 D28 D30 D32 D34 D32 D34

#(Drive & Sense Cycle) = 348 = 87 x 4 #(Drive & Sense Cycle) = 254 = 127 x 2

#(Drive Channel) = 87

Hadamard Matrix: Mutually Orthogonal

DC saturation of the AFE vs Gain Periodicity vs Noise Immunity

https://en.wikipedia.org/wiki/Hadamard_matrix

Confidential. All Rights Reserved. Copyright © 2021 Wacom Co., Ltd. Parallel drive: M-Sequence Shifted MLS (Maximum Length Sequence)s: Pseudo Orthogonal -1 1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 M-Sequence (L = 31 = 26-1) 1 -1 -1 -1 1 -1 35 MLS has Pseudo Randomness -1 -1 1 1 -1 -1 30 - Less DC component -1 1 -1 1 -1 1 ・・・ 25 -> No DC offset issue 1 1 -1 -1 1 1 -> Higher Gain -1 -1 1 20 L 1 -1 -1 퐕퐢, 퐕퐥 = ෍ VikVlk -> Higher SNR 1 1 -1 15 1 1 1 k=1 1 1 1 = L if i = k, - Pseudo Randomness 1 1 1 10 = −1 otherwise. -1 1 1 -> Spread spectrum -1 -1 1 Correlation Auto 5 -1 -1 -1 -> Better Noise Immunity 1 -1 -1 0 1 1 -1 -1 1 1 1 -1 1 -5 1 1 -1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 1 1 -1 1 1 1 -1 1 [Example Sequence] 0000100101100111110001101110101 -1 1 -1 1 -1 1 generated from x 5 + x 2 + 1 (primitive polynomial)

Touch Sensor # of Channel: 88×50 Channel pitch: 5.0mm

CH1：200mV/div k-th sense channel

Air Gap: 1.5mm LCD: 19.5” FHD

Horizontal：20us/div

CH1：200mV/div CH2：200mV/div

CH1-CH2：20mV/div

Horizontal：20us/div

- Common Mode Noise Cancellation - Higher SNR thanks to Higher Gain

IEEE J. Solid-State Circuits, vol. 50, no. 1, pp. 335–343, Jan. 2015.

phase-1 phase-2

cha nne l sense channel 2i Inn cha nne l sense channel 2i+1 CVC i CVC: Charge to Voltage Converter cha nne l sense channel 2i+2 Inp

i=0, …, [N/2]-1.

IEEE J. Solid-State Circuits, vol. 50, no. 1, pp. 335–343, Jan. 2015.

68kHz(Fundamental)

136kHz(2nd)

204kHz(3rd) 272kHz(4th) AC Adaptor

Battery 200kHz

48kHz(Fundamental)

96kHz(2nd)

192kHz(4th)

144kHz(3rd)

Light: ON

Light: OFF 200kHz

Confidential. All Rights Reserved. Copyright © 2021 Wacom Co., Ltd. Sensor Material Requirement - RC Time constant: ITO’s time constant is too large to realize large format sensors over 30-inch - Light Transmittance - Visibility: Color, Moire, etc. - Bezel Area

Time Constant Comparison A Random Mesh Pattern Sheet Time Constant Metal Width Resistance (Normalized) Copper Metal Mesh 0.003Ω/sq. 7um 1 (Thickness : 7um) Silver Paste 0.2Ω/sq. 6um 78 (Thickness : 9um) ITO on Glass 20Ω/sq. - 260 on Film 150Ω/sq. - 1950

Sensor pitch = 5mm

200μm 200μm

Tip Deformation

Conductive fabrics

Active Pen sends out electrical signal to touch sensor. The signal can be modulated with pen’s information: the button, pressure, color, ID, etc.

Touch controller calculates stylus (x, y) coordinates from the received signal and demodulates pen’s information.

TC synchronizes to the pen Not easy to realize Simultaneous Multiple Pens Operation.

y-coordinate

x-coordinate TC

Confidential. All Rights Reserved. Copyright © 2021 Wacom Co., Ltd. Two Way Active Pen System Each pen synchronizes to TC through Uplink Beacon from TC. Easy to realize Simultaneous Multiple Pens operation.

1. Uplink: for Synchronization

Beacon TC

2. Downlink: Multiple Stylus Detection

Confidential. All Rights Reserved. Copyright © 2021 Wacom Co., Ltd. Comparison of Capacitive Pen Technology Passive One Way Active Two Way Active Button No Yes Yes Hover NG OK OK Multiple Styluses / w OK / No No / No OK / OK Different Properties Dead Region* Yes No No In-cell Panel Yes No Yes

[Dead Region] It is impossible to detect passive pen signal placed very close to a “palm”, since the passive pen signal is small and buried in the palm signal due to its fringing capacitance.

US9977519B2

Protocol 1 Protocol 2 Protocol 3

Architecture PROS CONS Approach 1 Pen enables all the protocol Free competition Complex Implementation Approach 2 Use a unified protocol Conceptually Simple Restricted Competition

240Hz Multiple Active/Passive Pens with 41dB/32dB SNR for 0.5mm Diameter, 85nm CMOS(1P6M) IEEE Int. Solid-State Circuits Conf. (ISSCC) Dig. Tech. Papers, pp. 120–122, Feb. 2015.

In-cell Sensors for LCD Small Bezel –> Less Design Constraints Simple Structure –> Simple Manufacturing Process / Simple Supply Chain –> Low cost / Less Lead Time

Conventional Hybrid In-cell In-cell Cover Glass RX RX RX RX Film Cover Glass TX RX RX RX RX Cover Glass TX RX RX RX RX Display Display Display

Display Touch Display Touch TDDI Driver Controller Driver Controller

- Mutual Sensing - Mutual Sensing - Self Sensing - Two Layer Out-cell Sensor - Hybrid In-cell Sensor - In-cell Sensor - Two ICs - Two ICs - One IC: TDDI*

*Touch and Display Driver Integration

Segmented Vcom Sensor - No Additional Layer for Touch - Sharing between Display and Touch Processing

TDDI: Touch and Display Driver Integration

Substrate https://www.bareconductive.com/blogs/blog/how-do-the- touch-boards-capacitive-sensors-work

Display drive V blank Long H blank Touch sensing Display Driver & Touch/Pen … Controller Uplink Uplink t

Pen synchronizes to Display with Uplink Pen Downlink t

1 frame

CONS: Less time available for Touch/Pen processing PROS: No Display Noise in Touch/Pen period

2. Design Constraints - Huge Parasitic Capacitance - Stronger Nosie Injection

3. Signal Processing - Total Architecture: Sensor structure, Panel Drive, Sensing, etc. - Digitally Enhanced Analog Performance: Dynamic Range, SNR, etc.

Harmony “Wa” between computers & human beings Wacom Co., Ltd.

2-510-1, Toyonodai, Kazo-shi, Head Office Saitama, Japan Date of Founding July 12, 1983 Paid-in Capital JPY 4.2 bn. (as of March 31, 2020) President & CEO Nobutaka Ide Revenue JPY 108.5 bn. (FY 03/2021) Employees 1,012 (incl. temporary staff) (as of March 31, 2020) Stock Market Tokyo Stock Exchange 1st Section (6727) Business Line ・Brand products (creative pen tablet, etc.) ・Technology solution (digital pen sensor system, etc.)