Proceedings on Privacy Enhancing Technologies 2021; ..:1–23

Iñigo Querejeta-Azurmendi*, Panagiotis Papadopoulos, Matteo Varvello, Antonio Nappa, Jiexin Zhang, and Benjamin Livshits ZKSENSE: A Friction-less Privacy-Preserving Human Attestation Mechanism for Mobile Devices

Abstract: Recent studies show that 20.4% of the inter- 1. Introduction net traffic originates from automated agents. To iden- Automated software agents that interact with content tify and block such ill-intentioned traffic, mechanisms in a human-like way, are becoming more prevalent and that verify the humanness of the user are widely de- pernicious in the recent years. Web scraping, com- ployed, with CAPTCHAs being the most popular. Tra- petitive data mining, account hijacking, spam and ad ditional CAPTCHAs require extra user effort (e.g., solv- fraud are attacks that such agents launch by mimick- ing mathematical puzzles), which can severely down- ing human actions at large scale. According to a recent grade the end-user’s experience, especially on mobile, study [1], 20.4% of the 2019 internet traffic was fraudu- and provide sporadic humanness verification of ques- lent, associated with user, albeit not human, activity. tionable accuracy. More recent solutions like Google’s In the ad market specifically, such type of fraudulent reCAPTCHA v3, leverage user data, thus raising sig- traffic costs companies between $6.5-$19 billions in the nificant privacy concerns. To address these issues, U.S. alone, and it is estimated that this will grow to $50 we present zkSENSE: the first zero-knowledge proof- billion by 2025 [2]. When it comes to the ever-growing based humanness attestation system for mobile devices. mobile traffic, a recent study [3] (using data spanning 17 zkSENSE moves the human attestation to the edge: billion transactions) observes 189 million attacks origi- onto the user’s very own device, where humanness of nated specifically from mobile devices; this is an increase the user is assessed in a privacy-preserving and seamless of 12% compared to the previous six months. manner. zkSENSE achieves this by classifying motion The “Completely Automated Public Turing tests sensor outputs of the mobile device, based on a model to tell Computers and Humans Apart” (or just trained by using both publicly available sensor data and CAPTCHA) is the current state-of-the-art mechanism data collected from a small group of volunteers. To en- to assess the humanness of a user. CAPTCHAs are sure the integrity of the process, the classification result widely deployed across the internet to identify and block is enclosed in a zero-knowledge proof of humanness that fraudulent non-human traffic. The major downsides can be safely shared with a remote server. We implement of current CAPTCHA solutions (e.g., Securimage [4], zkSENSE as an Android service to demonstrate its ef- hCaptcha [5]) include: (i) questionable accuracy: fectiveness and practicality. In our evaluation, we show various past works demonstrate how CAPTCHAs can that zkSENSE successfully verifies the humanness of a be solved within milliseconds [6–10], (ii) added fric- user across a variety of attacking scenarios and demon- tion: additional user actions are required (e.g., image, strate 92% accuracy. On a two years old Samsung S9, audio, math, or textual challenges) that significantly im- zkSENSE’s attestation takes around 3 seconds (when poverish the user experience [11], especially on mobile visual CAPTCHAs need 9.8 seconds) and consumes a devices, (iii) discrimination: poor implementations negligible amount of battery. often block access to content [12], especially to visual- arXiv:1911.07649v4 [cs.CR] 7 Sep 2021 impaired users [13] (iv) serious privacy implica- Keywords: Human Attestation, Privacy Preserving Bot tions: to reduce friction, Google’s reCAPTCHAv3 [14] Detection, Frictionless verification of humanness replaces proof-of-work challenges with extensive user DOI foobar tracking. Google’s servers attest user’s humanness by collecting and validating behavioral data [15] (i.e., typ- ing patterns, mouse clicks, stored cookies, installed plu- *Corresponding Author: Iñigo Querejeta-Azurmendi: gins), thus raising significant privacy concerns [16–18]. Universidad Carlos III Madrid / ITFI, CSIC. Part of the work The goal of this paper is to build a humanness at- performed while working at Brave Software. testation alternative that will put an end to the false Panagiotis Papadopoulos: Telefónica Research dilemma: humanness attestation at the cost of user ex- Matteo Varvello: Bell Labs Antonio Nappa: University of California, Berkeley Jiexin Zhang: University of Cambridge Benjamin Livshits: Brave Software/Imperial College Proceedings on Privacy Enhancing Technologies 2021; ..:2–23 perience or at the cost of user privacy? By leveraging tegrity of the classification result reported to a re- the device’s motion sensors we demonstrate that it is mote server. possible to build a humanness attestation mechanism 3. We implement an Android SDK and a demo app2 on the edge that preserves the privacy of the users and to showcase the detection accuracy of zkSENSE. runs seamlessly on the background thus requiring zero zkSENSE is invisible to the user and capable of ver- interaction from the mobile user. ifying their humanness with accuracy higher than re- The key intuition behind our approach is that when- lated proposals [19] (92%). Performance evaluation ever a (human) user interacts with a mobile device, results of our prototype show that the entire attesta- the force applied during the touch event generates mo- tion operation takes less than 3 seconds (compared tion. This motion can be captured by the device’s sen- to 9 sec [21] required by Android reCAPTCHA [22] sors (e.g., accelerometer and gyroscope) and used to and consumes less than 5 mAh of power. uniquely distinguish real users from automated agents. Further, this detection can run on the user device, with- 2. Goals and Threat Model out requiring secure execution environment (unlike re- In this section, we describe (i) the basic design principles lated proposals [19, 20]), but more importantly with- that a humanness verification mechanism must follow, out sharing private information with any server (unlike and (ii) how existing mechanisms work and compare state-of-the-art [14]), apart from a proof that guarantees with zkSENSE. Specifically, a successful human attes- the integrity of the attestation result. Secure execution tation mechanism must: environments are rapidly evolving and might soon be- A) Be friction-less: Most existing mechanisms re- come a crucial tool for human attestation mechanisms, quire the user to solve mathematical quizzes or im- however, we have not yet seen wide adoption by low- age/audio challenges [4, 5], thus severely hampering the end devices. We realize this vision with zkSENSE, a user experience. According to studies [21, 23]: (i) hu- friction-less and privacy-preserving mechanism for hu- mans only agree on what the CAPTCHA says 71% of manness attestation that aims to replace CAPTCHA in the time, (ii) visual CAPTCHAs take 9.8 seconds (on mobile devices. This paper makes the following contri- average) to complete, (iii) audio CAPTCHAs take 28.4 butions: seconds (and 50% of the Audio CAPTCHA users quit). 1. We design a human attestation system (zkSENSE) The profound degradation of the user experience forces that is both friction-less and privacy preserving. service providers to perform user attestations only spo- zkSENSE leverages mobile motion sensors to verify radically [24–26] in an attempt to save their declining that user actions (e.g., type/touch events) on a mo- conversion rates. Related research works [27–29] reduce bile device are triggered by an actual human. Such user friction by requiring, for example, the user to tilt classification takes place by studying the output of their phone during humanness verification. In contrast, the mobile device’s motion sensors during the par- zkSENSE is completely friction-less: humanness is at- ticular user action. To set the ground truth, we use tested via device micro-movements that happen during publicly available sensor traces and we instrument natural user actions like typing, and screen touching. an actual Android browser app to capture both user B) Be privacy-preserving: To mitigate the above, clicks and sensor traces from a small set of real users. mechanisms like reCAPTCHA v3 [14] (i) track the user Our approach is tested under various scenarios: (i) while browsing a webpage, (ii) send raw tracking data when device is resting (on a table), (ii) when there is to third party attestation servers, where (iii) a “risk artificial movement from device’s vibration, or (iii) score” representative of humanness is computed and from an external swinging cradle. then (iv) shared with the webmasters. Of course, this 2. We develop a sub-linear inner product zero- pervasive behavioral tracking raises significant privacy knowledge proof, which we use to build (and open- concerns [16, 17]. Similarly, there are sensor-based ap- source1) zkSVM: a zero-knowledge based library for proaches [19, 27, 28] that transmit raw mobile sensor enclosing results of an SVM (Support-Vector Ma- data to remote attestation servers; something that as chine) classifier into zero-knowledge proofs. zkSVM reported, may reveal keystrokes, gender, age, or be used leverages arithmetic properties of commitment func- to fingerprint users [30–36]. Contrary to that, zkSENSE tions and prover-effective proofs to ensure the in-

1 zkSVM source code: https://github.com/zkSENSE/zkSVM 2 Demo video: https://youtu.be/U-tZKrGb8L0 Proceedings on Privacy Enhancing Technologies 2021; ..:3–23 decouples the humanness attestation procedure from tion attacks to increase their app ranking [41–43], or the server, and moves it to the edge. By performing the (ii) directly: an attacker registers for reward schemes in entire attestation on the user’s very own device, no sen- mobile apps (e.g., Brave’s Ad Rewards [44]) via multiple sitive data but the classification result (enclosed in zero- accounts and claim rewards. knowledge proofs to ensure integrity) leaves the user’s As for CAPTCHA systems, click farms are not in- mobile device. cluded in this threat model, since they rely on malicious C) Be broadly accessible: The majority of but human activity. Additionally, we assume that the CAPTCHA solutions are not accessible to all attacker does not control the device’s OS (rooted de- users [13, 37]. According to a survey [38], CAPTCHAs vice). Such a powerful attacker has the capabilities to are the main source of difficulty for visually impaired launch much more severe attacks (e.g., install spyware, users. Meanwhile, human attestation mechanisms steal passwords or sensitive user data, modify firmware, designed for visually impaired people (like audio- tamper with sensors) [45, 46]. based reCAPTCHA) have been exploited to bypass Server-Auditor: We assume a set of servers that act CAPTCHAs by providing automatic responses [39, 40]. as auditors, simply receiving and verifying the human- zkSENSE’s seamless integration with a user’s natural ness classification results the users report. Contrary to interaction with a device does not introduce any reCAPTCHAv3, such servers are not required to be additional barrier to people affected by any form of trusted by the users. Users can report a server that de- disability. We must clarify however, that zkSENSE is nies issuing tokens to an attested user. not tested to serve people that use voice commands User: Throughout the rest of this paper, when we men- instead of screen touching (and thus do not cause tion user activity, we mean screen interaction, which micro-movements during scroll, clicks, etc.). Extending results (when interpreted by the mobile OS) in click- our classifier to work with the buttons of the devices ing, key typing, or scrolling. We further assume mobile (i.e., lock/unlock, volume up/down) that visually devices equipped with a set of sensors that includes a impaired people use is part of our future work. gyroscope and an accelerometer.

2.1 Threat Model 3. Building Blocks Formal security properties: In Appendix B we in- In this section, we briefly introduce the concepts of zero- troduce formal notions of privacy and verifiability, and knowledge proofs and commitment functions: the basic prove that zkSVM provides them. Informally, these no- pillars of our privacy preserving construction. tions ensure that the user data is kept private from an adversary, and that a proof convinces a verifier that the submitted result can only be originated by applying the 3.1 Preliminaries Let be a cyclic group with prime order p gener- SVM model to the committed vector. zkSENSE does G ated by generator g. Let h be another generator of the not provide any guarantees on the origin of the input group where the discrete logarithm of h with base data itself. Hence, as with other widely deployed hu- G g is not known. Furthermore, let g , . . . , g , h , . . . , h man attestation mechanisms (i.e., reCAPTCHAv3) an 1 n 1 n denote 2n distinct group elements where their rela- adversary capable of forging human activity is capable tive discrete logarithms are unknown. We denote the of bypassing the mechanism. Our scheme provides re- integers modulo p as and write a ∈ S to de- play prevention of proofs, but not of the data used to Zp R note that a is chosen uniformly at random from a set generate such proofs, making it possible to perform sev- S. Although any prime order group where the Deci- eral attestations with a single vector. sional Diffie–Hellman (DDH) assumption holds may be Attacker: We assume the same attacker model as used, in our implementation we used the ristretto255 in CAPTCHA systems: an attacker who is capable of group [47] over Curve25519 [48]. automating user actions while accessing an online ser- Notation: We use bold letters to denote vectors. In vice. The goal of such attacker is to imitate a legitimate particular, g ∈ n is defined by (g , . . . , g ) with g ∈ , user and launch attacks like content scraping, ad view- G 1 n i G and a ∈ n by (a , . . . , a ) with a ∈ . We use ing/clicking, API abuse, spam sending, DDoS perform- Zp 1 n i Zp normal exponentiation notation to denote the multi- ing, etc., for monetary gain. This monetary gain can be exponentiation of two vectors, with ga = Qn gai . achieved either (i) indirectly: e.g., app developers pay i=1 i Moreover, two vector multiplications (or additions) rep- attackers to perform Black Hat App Store Optimiza- resents the entry wise multiplication (or addition). Proceedings on Privacy Enhancing Technologies 2021; ..:4–23

3.2 Zero-Knowledge Proofs 3.3 Homomorphic Commitment Functions In [49] the technique of (interactive) zero-knowledge In this paper, we use additive homomorphic commit- Proof (ZKP) is presented to enable one party (prover) ment functions, a cryptographic primitive that allows to convince another (verifier) about the validity of a a party to commit to a given value, providing the hid- certain statement. The proof must provide the following ing property (the value itself is hidden) and the binding informal properties (we formalise them in Appendix A): property (a commitment cannot be opened with a dif- – Completeness: If the prover and verifier follow the ferent value than the original one). We present our work protocol, the latter always validates. based on the Pedersen Commitment Scheme [61]. The – Knowledge soundness: The verifier only accepts the Pedersen Commitment scheme takes as input message proof if the statement being proven holds. m ∈ Zp and a hiding value r ∈R Zp and commits to it – Honest-verifier zero-knowledge: The prove discloses as follows: m r no other information other than the fact that the Commit(m, r) = g · h . statement is true. Note the additive homomorphic property, where: Blum et al. [50] introduced non-interactive zero- knowledge Proofs (NIZKPs), which enable the prover Commit(m1, r1) · Commit(m2, r2) = to prove the validity of a statement without interacting m1+m2 r1+r2 g · h = Commit(m1 + m2, r1 + r2). with the verifier. NIZKPs enjoy a growing popularity and adoption in the blockchain era, used across vari- Furthermore, in this paper we use a generalization of ous applications mostly for decentralization, verifiabil- the Pedersen Commitment [62], that instead takes a n ity and accountability [51–54]. NIZKPs can be extended message in m ∈ Zp and a hiding value r ∈R Zp and to Signature Proofs of Knowledge (SPK) [55], in which commits to it as follows: the prover signs a message while proving the statement. n m r Y mi r We adopt the Camenisch-Stadler notation [55] to denote Commit(m, r) = g h = gi · h . such proofs and write, for example: i=1 SPK{(x): A = gx ∧ B = Ax}(w) Note that such a commitment function has the same additive property as above, where addition in the mes- to denote the non-interactive signature proof of knowl- sage committed happens entry wise. Finally, a com- edge, over message w, that the prover knows the dis- mitment function has an opening verification function, crete log of A and B with bases g and A respectively, where, given the opening and blinding factor, deter- and that the discrete log is equal in both cases. Values mines whether these correspond to a given commit- in the parenthesis are private (called the witness), while ment, C. More specifically: all other values in the proof are public. We omit the ( notation of the signed message throughout the whole > if C = gm · hr Open(C, m, r, g, h) = paper, but we assume that all SPKs sign a challenge ⊥ else sent by the server every time it requests human attes- tation. This prevents replay attacks of zero knowledge We leverage zero-knowledge proofs over Pedersen proofs. We represent proofs with Π, and define two func- commitments. First, we use a proof of knowledge of tions: the generation function, Π.Gen(), and the verifi- the opening of a commitment [62], ΠOp, and a proof cation function, Π.Verif(). The inputs are, for the case of equality, ΠEq. Particularly: of the generation, the public values and the witness (in m r the example above, A, B and x). On the other hand, ΠOp := SPK {(m, r): C = g · h } . the verification function only takes the public values as and input (in the example above, A and B). Blockchain has created a high interest in building ΠEq := SPK {(m, r1, r2): proofs with minimal size and verifier computation [56– C = gm · hr1 ∧ C = hm · hr2 } , 60], but with the cost of increasing the prover’s time. In 1 2 Section 7., we show how proving execution of the SVM respectively. We also construct a ZKP that allows us to model based on [56] (one of the most widely used such provably exchange any element of the committed vector proofs) is unbearable for mobile devices. by a zero. To this end we provably extract the particular Proceedings on Privacy Enhancing Technologies 2021; ..:5–23

value we want to replace with its corresponding gener- P: It computes blinding vectors, sL, sR for each vector ator, and divide it from the initial commitment. More of the inner product and commits to it: n particularly: sL, sR ∈R Zp (1) ρ ∈R Zp (2) n n m r mj Π0 := SPK (m ∈ Zp , r): C = g h ∧ E = g ∧ ρ s s j S = h g L h R ∈ G (3) m m o C/E = gm1 ··· g j−1 · g j+1 ··· gmn hr . 1 j−1 j+1 n Now the prover defines two linear vector polynomi- n Next, we use the proof that a committed value is the als, l(X), r(X) ∈ Zp [X], and a quadratic polynomial square of another committed value presented in [63]: as follows: n l(X) = a + sL · X ∈ Zp [X] (4) Π := SPK {(m , m , r , r ∈ ): Sq 1 2 1 2 Zp n r(X) = b + sR · X ∈ [X] (5) m1 r1 m2 r2 2 Zp C1 = g · h ∧ C2 = g · h ∧ m1 = m2 . t(X) = hl(X), r(X)i = (6) To prove that a number is within a range, we leverage 2 n t0 + t1 · X + t2 · X ∈ Zp [X] Bulletproofs [59]: By creating like that the polynomials, it allows for  m r l ΠGE := SPK (m, r ∈ Zp): C = g h ∧ m ∈ [0, 2 ] an evaluation of the polynomial at a given point without disclosing any information about the vec- for some positive integer l. We combine these two proofs, tors a or b. The prover needs to convince the verifier ΠSq and ΠGE, to prove the correct computation of the that the constant term of t(X) equals c. To do so, floor of the square root of a committed value. Let C1 = m r m r +1 m +1 r the prover commits to the remaining coefficients of g 1 · h 1 , C2 = g 2 · h 2 and C = C1 · g = g 1 · h 1 . √1 the polynomial We want to prove that m1 = b m2c. It suffices to prove the following two statements: τ1, τ2 ∈R Zp (7) (i) The square of the committed value in C is smaller 1 ti τi Ti = g h ∈ G, i = {1, 2} (8) or equal than C2, and (ii) The square of the committed value in C+1 is greater 1 P → V: S, T1,T2 than C2. ∗ V: C ∈R Zp We denote this proof by: V → P: C Πsqrt := SPK {(m1, m2, r1, r2 ∈ Zp): P: It computes the response using the challenge re- √ m1 r1 m2 r2 ceived C1 = g · h ∧ C2 = g · h ∧ m1 = b m2c} . n l = l(C) = a + sL · C ∈ Zp (9) n 4. Sub-linear Inner Product Zero- r = r(C) = b + sR · C ∈ Zp (10) ˆ Knowledge Proof t = hl, ri ∈ Zp (11) 2 In this section, we present the Sub-linear Inner Prod- τC = τ2 ·C + τ1 ·C + γ (12) uct Zero-Knowledge Proof (IP-ZKP). Our construction µ = α + ρ · C ∈ Zp (13) is based on the non zero-knowledge version presented in [59]. For their use cases, the zero-knowledge property P → V: τC, µ, t,ˆ r, l. is not required, and hence the lack of an explicit defini- V: The verifier needs to check that r, l are correct and tion and study of a zero-knowledge proof. In this section the inner product relation holds with respect to tˆ. we make it explicit, and in Appendix A we prove that it To this end it performs the following checks: provides completeness, knowledge soundness and special 2 tˆ τC ? C C honest-verifier zero-knowledge properties. In our con- g h = V · T1 · T2 (14) α a b C struction the prover has a commitment, A = h g h , P = A · S ∈ G (15) of two vectors a, b with blinding factor α, and a second ? P = hµ · gl · hr (16) commitment V = gchγ , of a value c, with blinding fac- ? tor γ. The prover convinces the verifier that ha, bi = c tˆ= hl, ri (17) holds. The prover and verifier interact as follows (can be made non-interactive with the Fiat-Shamir Heuris- If all checks validate, then this means that the state- tic [64]): ment is true with very high probability. Proceedings on Privacy Enhancing Technologies 2021; ..:6–23

Fig. 1. Output of gyroscope and accelerometer motion sensors during human and automatically triggered click (red). The maximum linear acceleration movement is up to 8.5× greater and the angular rotational velocity is up to 4.9× greater in case of a human trig- gered click.

Fig. 2. Motion sensors output during automatically triggered click with artificial device movement (red): (i) during device vibration (on the left) and (ii) when device is docked on a swing (on the right).

To make this proof logarithmic, we use the same trick a human (left plots) and an automated agent (right as in the original paper. Instead of sending the vectors plots). During the click events (highlighted in red) the r, l to prove the inner product relation, we leverage IP- accelerometer (top plots) senses a max rate of change ZKP over the blinded vectors recursively. We prove the of 0.6 in case of human and 0.07 for an automated agent, following Theorem in Appendix A: i.e., 8.5× greater maximum linear acceleration move- ment. Similarly, the gyroscope (bottom plots) senses Theorem 1. The inner product proof presented in Pro- a max rate of change of 0.024 in case of human click tocol 1 has perfect completeness, perfect special honest and 0.0049 when there is automation, i.e., 4.9× greater verifier zero-knowledge, and knowledge soundness. maximum angular rotational velocity. Figure 2 shows a snapshot of the same sensors’ out- put in the case of automated clicks coupled with two ar- 5. ZKSENSE Overview tificial device movements: vibration and swing. With vi- The key intuition behind zkSENSE is that whenever bration (left plots), we see that the motion generated is a (human) user interacts with the mobile’s display, the comparable with the case of the human click depicted in force applied during the touch event generates motion. Figure 1. We see that the accelerometer senses the same This motion is captured by the embedded IMU (Iner- force with the case of the human click but for a longer tial Measurement Unit) sensors (e.g., accelerometer and time – this verifies the observations of [19]. The gyro- gyroscope). By contrast, when there is automated user scope though senses greater angular rotational velocity activity (e.g., simulated touches) there is no external and for longer time than in the case of a human’s click. force exerted by fingers, and thus there is no noticeable With swing (right plots), the gyroscope senses similar change in the output of the above sensors. angular rotational velocity as with the case of the hu- Figure 1 shows a snapshot of the output produced man click, while the accelerometer senses greater linear by IMU sensors during click events performed both by acceleration movement (up to 3.8×) for a long period. Proceedings on Privacy Enhancing Technologies 2021; ..:7–23

Fig. 3. High-level overview of the zkSENSE architecture. An integrated ML-based classifier studies the patterns of sensor outputs right before, during, and shortly after a click event. To avoid leaking sensitive sensor output outside the device, the classification ap- pears on the user side and the client has to prove the integrity of the reported result to the server.

5.1 System Overview Data Data amount Building upon the above observations, zkSENSE uses Volunteering Users 10 users an ML-based classifier to study the pattern of sensor Duration of collection 22 days outputs before, during, and shortly after a click event Android Devices tested Google Pixel 3, Realme X2 Pro, (see Figure 3). Based on this information, the model Samsung Galaxy S9/S8/S6, Honor 9, decides about whether the action was triggered by a Huawei Mate 20 Lite, OnePlus 6 human or not. Human events collected 7,736 clicks Artificial events collected 25,921 clicks Human Attestation on the Edge: Privacy- preserving evaluation of ML models has become increas- Table 1. Summary of the collected dataset. ingly important especially in the post GDPR era. One approach consists in encrypting the data on the client, mented the open source browser Brave for Android to and runs ML model on such encrypted data at the capture click events (and their corresponding motion server. This can be achieved via Fully Homomorphic En- sensor traces) performed during browsing3. Then, we cryption (FHE) [65–67]: clients encrypt their data with recruited 10 volunteers4 who used our instrumented their own keys and send the ciphertext to the server browser for 22 consecutive days for their daily brows- to evaluate an ML model. Next, the server sends the ing5. The device models used are: Google Pixel 3, Sam- outcome of the homomorphic computation back to the sung Galaxy S9, S8 and S6, OnePlus 6, Realme X2 Pro, user who would provably decrypt it and send back to Huawei Mate 20 Lite and Honor 9. Volunteers were well- the server. It is easy to anticipate that the overhead informed about the purpose of this study and gave us to perform such multi-step operation for each client is consent to collect and analyse the motion sensor traces unbearable for services with millions of clients. generated during their screen touch events. We urged In zkSENSE, we pre-train a model on a server and volunteers to use their phone as normal. we move the classifier to the edge by running it on To generate artificial user traffic, we used adb [68] the user side and only report the result to an attes- to automate software clicks on 4 of the volunteering de- tation server responsible for auditing the humanness of vices. To test different attack scenarios, during the au- the user. This way, zkSENSE ensures that private sen- tomation, we generate software clicks with the device sor data never leaves the user’s device. In Figure 3, we being in 4 different states: present the high level overview of our approach. As we can see, an attestation starts with a click (screen touch). The motion sensor outputs generated during this event 3 We chose a real app to make sure that the volunteers will are used as input to the zkSVM Prover module, which continue using it for a longer period contrary to a possible in- runs a trained model to classify if the action was con- strumented toy app. ducted by a human or not. 4 For a production ready system a larger pool of real users will need to contribute data, but for our study we found it difficult to 5.2 Classification of Humanness recruit users who would contribute their clicks for a long time. 5 During data collection the instrumented browser was running Data Collection: To collect the necessary ground on the users’ devices so we could not control their background truth to train the various tested models, we instru- running services. The data collected included only the raw sensor data during a user click. Proceedings on Privacy Enhancing Technologies 2021; ..:8–23

Classifier F1 (weighted) Recall As shown in Table 2, the four tested classifiers (i.e., SVM, decision tree, random forest, and neural network SVM 0.92 0.95 with ReLU kernel) have similar performance in terms Decision Tree (9 Layers) 0.93 0.95 Random Forest (8 Trees, 10 Layers) 0.93 0.95 of recall (i.e., 0.95 recall). Although, Decision Tree and KNN 0.92 0.93 Random Forest perform slightly better in terms of ac- BotNeural Detection Network (Linear Kernel) 0.86 0.95 curacy, zkSENSE utilizes SVM for compatibility pur- Neural Network (ReLU Kernel) 0.91 0.96 poses as stated in more detail in Section 5.3. Hence, our We divide each click event into two segments: Table 2. Accuracy of the various tested classifiers. zkSENSE’s accuracy in assessing the humanness of a user is 92%. Pressing Down Releasing Up 5.3 Privacy-Preserving and Provable ML To preserve privacy, human attestation in zkSENSE is performed in the user’s device and only the result of the attestation is shared with the server. To ensure that the

6server can verify the integrity of the transmitted result, Fig. 4. The period of a click event starts 50ms before the begin- the user includes a commitment of the sensors, together ning of the action and ends 250ms after the end of it. with a proof that the result corresponds to the model evaluated over the committed values. 1. while resting on a platform (desk/stand) We build the two ML evaluation proving com- 2. while being carried around in pocket ponents of zkSENSE: (i) zkSVM Prover and (ii) 3. while being placed on a swing motion device zkSVM Verifier. The zkSVM Prover checks on the 4. while device is vibrating (triggered by adb) client whether a user is a human based on a model we As summarized in Table 1, by the end of the data collec- pre-trained (Section 5.2), and generates a proof to en- tion, we had 7,736 human-generated clicks and 25,921 sure its proper execution. The zkSVM Verifier, on the artificially generated clicks. server’s side, checks that the proof is correctly gener- Feature Selection: During data collection, accelerom- ated. If the verification is successful, the server will know eter and gyroscope sensors were sampled at 250Hz. For that (a) the ML-based humanness detection model clas- each click event, we not only consider the device motion sifies the user as human or non-human based on the during the touch, but also the device motion right before committed sensor outputs, and that (b) the used model and shortly after the touch. In particular, we consider is the genuine one, without though learning the value of that the period starts 50ms before the finger touches sensor outputs. the screen and finishes 250ms after it. Then, we split SVM enclosed in Zero-Knowledge Proofs: Sec- each period into two segments: (i) before releasing fin- tion 5.2 shows that the different classifiers tested achieve ger and (ii) after releasing finger as depicted in Figure 4. similar accuracy. While decision trees, random forests, For each axis (x, y, z) in accelerometer and gyroscope, or neural networks provide slightly higher F1 accuracy we calculate the average and standard deviation of its than SVM (see Table 2), in zkSENSE we choose SVM outputs in each segment. In addition, we calculate the as the underlying model due to its simplicity at evalua- consecutive difference of sensor outputs in each segment tion time and its suitability with zero-knowledge proofs. and use the average and standard deviation of these dif- Contrarily, neural networks need to perform non-linear ferences as features. operations, while decision trees require several range Classification Accuracy: Using the above features, proofs, which are expensive operations to prove in zero- we test several ML classifiers via 10-fold cross valida- knowledge. As mentioned in Section 5.2, the SVM model we trained uses as features the average (µ) and standard tion. Table 2 presents the weighted F1 score and recall of the different classifiers we tested. We choose weighted deviation (σ) of sensor outputs, together with the aver- age and standard deviation of the consecutive difference F1 score as an evaluation index because our dataset is unbalanced. In this context, recall means the propor- vector. On top of that, before applying the SVM model, tion of correctly identified artificial clicks over all artifi- the extracted features need to be normalised. The goal cial clicks. In other words, recall indicates the ability to of normalisation is to change data values to a common capture artificial clicks. scale, without distorting differences in the ranges of val- Proceedings on Privacy Enhancing Technologies 2021; ..:9–23 ues. Then, trained SVM weights are assigned to each Proof generation: The proof generation is divided normalised feature to calculate the SVM score. in five protocols. First the prover computes the dif- Suppose for each feature fi, the normalisation mean, ference vectors and proves correctness. Next it com- normalisation scale, and SVM weight and intercept are putes the average of all vectors, followed by a computa- Mi, Si, wi and c respectively. Then, the SVM score s tion of the standard deviation. Finally, it evaluates the can be calculated with the equation: normalising linear computations over the results, and 1 sends the opening of the result to the verifier together s = . (18) PN (fi−Mi) with the proofs of correctness. The prover’s secret is −(c+ S wi) e i=1 i + 1 n v ∈ Zp , r ∈ Zp such that Since only the value of fi is secret, we only need to N v r prove the computation of P f wi . Given only in- SH = g · h . i=1 i Si teger values can be processed in the underlying group Procedure 2. (Consecutive difference) In this step the arithmetic of Pedersen commitments, we instead prove j k PN wi d prover’s goal is to compute the difference of consecu- fi 10 and effectively use the parameter d to i=1 Si tive values in the input vector, while keeping it hidden. preserve d-digits after the decimal points of wi . Si Mainly, we want a provable value of Building zkSVM: Without loss of generality, we as- d vd rd sume that the number of sensor inputs is n for every SH = g h , sensor. The protocol is divided in three phases. First, with vd = [v1 − v2, v2 − v3,..., vn−1 − vn, 0]. The in- the setup phase, Setup(λ), where the server generates tuition here is first to get a commitment of the iterated the model and their corresponding weights. Secondly, values of the sensor vector, then leverage the homomor- the proving phase, where the prover fetches the SVM re- phic property to subtract this commitment with SH and lated data, computes the difference vector, and proceeds finally provably replace the value in position n by zero. to provably compute the average and standard deviation To compute the iterated value, the prover first iterates of these values. It then applies the corresponding linear the base generators, to get combinations to the hidden features, and opens the re- sult to send it to the verifier. Finally, the verification giter = [gn, g1, . . . , gn−1]. phase, where the verifier checks that all computations Note that this step can be performed by the verifier as were correctly performed. Then, it checks whether the the generators are public. It then commits the sensor scores do correspond to a human or a bot. vector with this base Setup phase: The setup involves the server, where iter v riter it generates the cryptographic material, and trains the SH = giter · h , SVM model. The details on how this model is generated with riter ∈R Zp, and generates a proof of equality, falls out of the scope of zkSVM, the only requirement iter is that it follows the specifications described at the beg- ΠEq = ΠEq.Gen(g, giter, h, SH ,SH ; v, r, riter). gining of this section. Once these values are computed, Note that they are published to allow provers to fetch them and iter v2 vn v1 riter generate the proofs. SH = g1 ··· gn−1 · gn · h , so now we can simply subtract the two commitments to Procedure 1. (Setup) On input the security parame- get ter, λ, the zkSVM server runs Setup(λ) to generate the SVM and cryptographic parameters. Precisely, it trains iter v1−v2 v2−v3 SH = SH /SH = g1 g2 ··· the SVM model and generates the normalisation mean, vn−1−vn vn−v1 r−riter gn−1 gn · h . normalisation scale, and SVM weight and intercept, Mi, Si, wi and c respectively. It also defines the size of the Finally, the prover replaces the value in the exponent of input vectors, n, which defines how long a touch is con- gn by a zero, to get the final commitment: sidered and the measurement frequency. Next it selects v1−v2 v2−v3 vn−1−vn 0 rd Diff = g1 g2 ··· gn−1 gn · h , (19) the group, G with generators g, h and prime order p. It proceeds by computing two vectors, g, h, of generators and generates a proof of correctness, that act as bases for the Pedersen Vector Commitments. Π = Π .Gen(g, h, S , Diff; v , r − r , r ). (20) Note that the corresponding discrete log of these bases 0 0 H d iter diff  iter  must remain unknown. It stores ∆ = Diff, SH , ΠEq, Π0 . Proceedings on Privacy Enhancing Technologies 2021; ..:10–23

Procedure 3. (Sum of vectors) The prover now com- Note that this step is again reproducible by the verifier, putes µ˜ = N · µ, mainly, the sum of all values. To prov- and hence no proof is required. Next it commits the ably compute this, we leverage IP-ZKP between the ini- average using these products as a base, to get tial commitment, SH , and a Pedersen commitment with 1 µ˜ µ˜ rG µ˜ µ˜ rG base h of the one vector, h , to prove that a third com- G = gΠ · h = g1 ··· gn · h , mitment, Avg = Comm(˜µ, r ), commits to the sum of the µ and committed values in SH , µ˜ µ˜ rH µ˜ µ˜ rH H = hΠ · h = h1 ··· hn · h , hv, 1i =µ. ˜ with rG, rH ∈R Zp. It proves equality between the open- ing of Avg, Gµ˜ and Hµ˜ using Π , and stores the two The user proves correctness of the commitment, Eq proofs, µ 1 ΠIP = ΠIP .Gen(g, h, g, h, SH · h , Avg; v, r). G µ˜ ΠEq = ΠEq.Gen(gΠ, g, h, Avg, G ; µ, rµ, rG), It stores both values M = Avg, Πµ . It repeats the IP and same steps as above with the commitment of the con- secutive difference vector, resulting in a commitment of H µ˜ Π = ΠEq.Gen(hΠ, g, h, Avg, H ; µ, rµ, rH ), 0 µ 0 Eq the average, Avg , and a proof of correctness, ΠIP . It 0  0 µ 0 stores both values M = Avg , ΠIP . one for each base. Finally, the prover commits to v with randomness rS with h as bases, Procedure 4. (Standard deviation) To calculate a fac- v rS tor of the standard deviation, σ, we first compute the HS = h h , variance, σ2. Recall that and proves equality of opening with respect to SH , get- N 1 X ting σ2 = (v − µ)2, N i i=1 S ΠEq = ΠEq.Gen(g, h, h, SH ,HS; v, r, rS). or written differently This allows the prover to leverage relation (21) to prov- 1 σ2 = hv − µ, v − µi, ably compute a commitment of a factor of the variance N using the proof presented in Section 4. To this end, it where µ is a vector with µ in all its positions. For this we computes need the average, but only have a provable commitment N µ˜ N µ˜ of the sum, µ˜. Hence, instead of computing the variance, AS = SH /G · HS /H = we compute N 3 · σ2 by leveraging the inner product gN·v−µ˜ · hN·v−µ˜ · hN·r−rG+N·rS −rH proof and the arithmetic properties of the commitment function. The intuition is the following: if we multiply and the commitment of the factor of the variance, each entry of v by N, we can get the following relation. 3 2 V ar = Comm(N · σ , rV ).

hN · v − µ˜,N · v − µ˜i = It generates a proof of correctness hN · v − N · µ,N · v − N · µi = σ2 hN · (v − µ),N · (v − µ)i = ΠIP = ΠIP .Gen(AS, V ar, g, h, g, h; N 2hv − µ, v − µi = N 3 · σ2. (21) v, µ,˜ r, rG, rH , rS, rV ).

However, we only have SH , and a provable commitment Finally, the prover needs to compute the square of µ˜ (not of µ˜). Moreover, we need a commitment of root of the variance. To this end, it commits to 3 2 v and µ˜ under both bases (g and h). To this end, the the floor√ of the square root of N · σ , Std = prover computes the following steps. First, it computes Commit(b N 3 · σ2c, r√). Then, the prover leverages the the commitment of µ˜ with both bases. To this end, the square root proof introduced in Section 3.3, and gener- prover first computes a product of all the bases, ates

n n σ 2 √ Y Y Πsqrt = Πsqrt.Gen(V ar, Std, g, h; σ , σ, rV , r ). gΠ = gi and hΠ = hi. i=1 i=1 Proceedings on Privacy Enhancing Technologies 2021; ..:11–23

This results in a provable commitment of a factor of the Procedure 7. (Verifying consecutive difference) The floor of the standard deviation, Std. The prover stores, verifier begins by iterating the base of generators:

 µ˜ µ˜ G H S giter = [gn, g1, . . . , gn−1]. Λ = G ,H ,HS, ΠEq, ΠEq, ΠEq, σ2 σ i and then verifies the proof of opening equality, V ar, ΠIP , Std, Πsqrt . ? Π .Verif(g, g , h, S ,Siter) = >. It repeats the same steps above with the consecutive Eq iter H H difference vector (and average), resulting in Next it computes the subtraction commitment to get

iter 0  µ˜ 0 µ˜ 0 0 G 0 H 0 SH = SH /S . Λ = G ,H ,HS , ΠEq , ΠEq , H S 0 0 σ2 0 0 σ 0i ΠEq , V ar , ΠIP , Std , Πsqrt . Finally, it verifies that the proof ? Π .Verif(g, h, S , Diff) = >. Procedure 5. (Computing SVM score) Provably com- 0 H puting SVM score (Eq. 18) reduced to provably com- j k Procedure 8. (Verifying sum of vectors) The verifier PN wi d puting fi 10 where fi are the features. How- i=1 Si checks the inner product proof ever, note that we do not have the features themselves, µ 1 ? but a factor of them. Hence, with this scheme, we need Π .Verify(g, h, g, h, SH · h , Avg) = >. j k IP PN wi d to compute instead r = fi 10 where Ni i=1 Ni·Si It repeats this step for the consecutive difference com- 3/2 equals N if fi is an average, and N if it is a standard mitment and proof. deviation (note that we have different factors of each). Again, wi is public. Let Comm = Commit(f , r ) be the Procedure 9. (Verifying standard deviation) The ver- Ni·Si i i i commitment of feature fi with blinding factor ri. The ifier first computes a product of all the bases, prover computes the following: n n Y Y gΠ = gi and hΠ = hi. N  wi d i=1 i=1 Y N ·S 10 Res = Comm i i = i Next it verifies the proofs of equality of commitments i=1 N      using these bases Y wi d wi d Commit fi · 10 , ri · 10 = G µ˜ ? Ni · Si Ni · Si Π . (g , g, h, Avg, G ) = >, i=1 Eq Verify Π N   ! X wi d and Commit fi 10 , rR , (22) H µ˜ ? Ni · Si Π .Verify(hΠ, g, h, Avg, H ) = >. i=1 Eq Next, the verifier checks that HS commits to the input N j k where r = P r wi 10d , is the blinding factor R i=1 i Ni·Si vector S ? known to the prover. Once these operations have been ΠEq.Verify(g, h, h, SH ,HS) = >. performed, the prover stores the opening of the com- j k Now the verifier needs to generate the commitments un- PN wi d mitment, Score = fi 10 , and the blinding i=1 Ni·Si der which the inner product proof of the variance will factor rR. verify against. To this end it computes

µ˜ µ˜ Procedure 6. (Sending values) The prover sends to L = SH /G and R = HS/H (23) the verifier the following tuple and uses them to verify the inner product proof  0 0  SH , ∆, M, M , Λ, Λ , Score, rR σ2 ? ΠIP .Verify(L · R, V ar, g, h, g, h) = >. Proof Verification: The verifier then performs all re- Finally, the verifier checks the correctness of the factor spective linear combinations commitments, and verifies of the standard deviation commitment the zero-knowledge proofs. If any proof fails or the eval- σ ? uation of the model over Score fails, the verifier denies Πsqrt.Verify(V ar, Std, g, h) = >. the request. Else, it accepts it. More precisely, the veri- It repeats these steps for the consecutive difference com- fier follows the following steps: mitments. Proceedings on Privacy Enhancing Technologies 2021; ..:12–23

Procedure 10. (Computing SVM score) Finally, the the Rust implementation of Bellman’s [72] Groth16 ZK- verifier computes the same linear combinations as the SNARK [56]. This construction has optimal proof size prover, and verification time. However, this comes by trading off

N  wi d Y N ·S 10 prover’s computational complexity and the requirement Res0 = i i , (24) Commi of a trusted setup. i=1 and checks the validity of the received opening, 6.2 ZKSENSE Prototype 0 ? We implemented a prototype SDK of zkSENSE for Open(Res , Score, rR, g, h) = >. Android (∼1,000 lines of Java). The SDK monitors It uses this value to compute the final score as described Android’s accelerometer and gyroscope during a touch in Equation (18). If any of the checks fail or the score event and, by applying a pre-trained model, it deter- determines the user is a bot, it returns ⊥, otherwise it mines if it was performed by a human or not. For demon- returns >. stration purposes, we created a demo app with a user interface that shows the output of the detection model By extending the inner product proof presented in [59] and we provide publicly a video6 that demonstrates its to a zero-knowledge proof and leveraging the arith- functionality. In this demo, we test multiple scenarios metic properties of Pedersen commitments, we present to showcase the accuracy of our system: zkSVM, a privacy-preserving SVM evaluation model. 1. The device is resting on a steady platform and: (a) A human is performing clicks. 6. System Implementation (b) Clicks are simulated. To assess the feasibility and effectiveness of our 2. The device is docked on a swing motion device that approach we developed (i) an open source library produce artificial movement. of zkSVM, and (ii) a prototype Android SDK of 3. The device is held in one hand and: zkSENSE. (a) A human is performing clicks. (b) Clicks are simulated. 6.1 Enclosing SVM Result in a ZKP After reading the output of the accelerometer and The zkSVM library: To prove the integrity of gyroscope sensors, the zkSENSE SDK applies, on the the classification results, we developed an open-sourced background, our pre-trained model and classifies the ori- Rust library that implements the logic presented in Pro- gin of the touch-screen event (i.e., performed by a hu- tocol 2, on enclosing SVM results in zero-knowledge man or not). For the generation of the ZKPs and the proofs. To this end, we additionally implement the Ped- model’s evaluation, we leverage the library we imple- ersen Commitment ZKPs as described in Section 3.3. mented and described previously, which we call from For the proofs ΠSq, ΠEq, Π0 and ΠOp, we base our im- the mobile device using the sensor data. The pre-trained plementation in the work presented in [62]. We use the model, is generated on a server of ours. Apart from gen- range proof presented in [59] and implemented in [69]. erating and distributing the trained model, the server Finally, for ΠIP , we implement the zero-knowledge also acts as the external auditor that verifies the valid- proof presented in Section 4. All the above proofs are im- ity of the transmitted attestation results. plemented using the ristretto255 prime order group over Curve25519 by leveraging the curve25519-dalek [48] li- brary. All our proofs leverage the Fiat-Shamir heuris- 7. Performance Evaluation tic to make the ZKPs non-interactive, which main- In this section, we set out to explore the performance of tains all security properties under the Random Oracle humanness attestation in zkSENSE. More specifically, model [70]. To integrate this library with our detection we benchmark our Android prototype with respect to engine, we use the Android NDK development kit. the duration of its main operations: (i) humanness clas- sification, (ii) Pedersen commitment computation, and General-purpose ZK-SNARK: To compare the (iii) zero-knowledge proof construction. Next, we eval- performance of zkSVM, we implement the SVM exe- uate general resource utilization metrics: (a) CPU, (b) cution using the ZoKrates general-purpose ZK-SNARK toolbox [71]. ZoKrates works as a high-level abstraction for the encoding of the computation to be proved into a ZK-SNARK. ZoKrates constructs the ZKP by using 6 zkSENSE demo: https://youtu.be/U-tZKrGb8L0 Proceedings on Privacy Enhancing Technologies 2021; ..:13–23

Fig. 7. Energy consumption per opera- Fig. 5. Execution time per operation. On Fig. 6. CPU utilization per operation. On tion on a low-end hardware. Energy con- commodity hardware (S9) humanness de- commodity hardware (S9), the ZKP gener- sumption for detection and committing tection and commitments are extremely ation is the most expensive operation, with operation is negligible, when for the ZKP fast (i.e., about 0.3), when the ZKP gener- a median CPU consumption of about 15% computation zkSENSE consumes about 5 ation lasts about 2.9 seconds. on commodity hardware. mAh or 0.2% of the device’s battery. memory, and (c) battery consumption. Our tests cover J3 is 2,600 mAh). The low-end device (J3) is part of the three key operations of a humanness attestation in Batterylab [73, 74], a distributed platform for battery zkSENSE, and a comparative baseline: measurements. It follows that fine grained battery mea- surements (via a Monsoon High Voltage Power Moni- 1. The baseline, where we run our demo application tor [75] directly connected to the device’s battery) are which uses zkSENSE service (see Section 6.) and available for this device. Automation of the above oper- several artificial clicks are generated. ations is realized via adb run over WiFi to avoid noise 2. The detection operation, where sensors input is in the power measurements caused by USB powering. collected and humanness classification realized on Execution time: Figure 5 shows the average duration zkSENSE. 3. The commitment operation where the Pedersen (standard deviation as error-bars) of each zkSENSE’s commitment computation is taking place. operation, per device, when considering zkSVM. Re- 4. The ZKP operation where the proof of correct at- gardless of the device, humanness classification and testation is constructed. commitments are extremely fast, i.e., about 0.3 and 0.6 seconds even on the less powerful J3. The ZKP genera- We test and compare the different implementations tion is instead more challenging, lasting about 2.9 and described in Section 6.: (i) the general-purpose ZK- 39.2 seconds on the S9 and J3, respectively. SNARK and (ii) our zkSVM. Note that for ZK-SNARK To improve visibility, the figure omits results from we ignore the time elapsed while computing the trusted the general purpose ZK-SNARK solution. In this case, setup, as this cannot be computed by the client. We run we measure commitment operations comprised between each stage for an hour and we ensure the same number 24 and 190 seconds, and ZKP generation comprised of artificial clicks by using as an interval the duration between 175 and 600 seconds, depending on the de- of the longest operation (ZKP) as empirically measured vice. This suggests one order of magnitude speedup of on each device under test (Figure 5). zkSVM versus the more generic ZK-SNARK solution. Setup: We leverage a testbed composed of two Android Given the significant difference between zkSVM and devices representative of a mid-end (Samsung Galaxy ZK-SNARK, in the following we mostly focus on results S9, model 2018) and a low-end (Samsung Galaxy J3, obtained via zkSVM. model 2016) device to inspect what is the worst per- For the verification time, the general purpose ZK- formance a user can get on a cheap (around 90 USD) SNARK (808 nanoseconds) outperforms zkSVM (177 device. The S9 mounts an octa-core processor (a Quad- milliseconds). This is expected as the Groth16 approach Core Mongoose M3 at 2.7GHz and a Quad-Core ARM used by ZoKrates has a big prover overhead and a Cortex-A55 at 1.8Ghz), while the J3 is equipped with trusted setup in exchange of minimal communication a quad-core ARM Cortex A53 at 1.2 Ghz. The S9 also and verification time overhead. However, in zkSENSE’s has twice as much memory (4 GB when J3 has 2 GB) scenario, these verification times can be handled by the and a larger battery (3,000 mAh when the battery of server, and instead, zkSVM makes the prover times rea- Proceedings on Privacy Enhancing Technologies 2021; ..:14–23 sonable (as shown, the entire attestation takes a bit less and accelerometer, and for each sensor, we use three than 3 seconds) and removes the need for trusted setup. axis data. Moreover, we split each period into two seg- CPU and memory utilization: Figure 6 shows CPU ments as explained in Section 5.2. This results in a usage per operation and device. Since no significant dif- total of 12 input vectors. For every input vector, the ference was observed between baseline and detection proof consists of 14 KB (83 compressed points and 354 operation, we improve the figure visibility by report- scalars). In our library, we implemented the trivial con- ing only one boxplot representative of both operations. struction, where we build 12 of such proofs in parallel, The figure shows minimal CPU utilization associated resulting in 167 KB. However, there are ways to reduce with humanness classification and commitment opera- this overhead, for which we provide the estimates. The tions. The counter-intuitive higher CPU usage at the S9 opening and equality proofs we used (introduced in Sec- is due to the fact that this is a personal device with po- tion 3.3) could be improved, by implementing them us- tential background activities from other apps. Even on ing the techniques presented in [60]. This would reduce the less powerful J3, committing only consumes about the complexity from linear to logarithmic, resulting in 12% of CPU (median value across devices) with peaks x3 improvement, with a size of the full proof of 56 KB. up to 45% on the J3. The ZKP generation is the most Finally, we could use the batching techniques for the expensive operation, showing a median CPU consump- range proofs, as presented in [59]. This would further tion of about 15% and 30%, on respectively the S9 and reduce the size of the proof to 45 KB. J3. Overall, the CPU analysis suggests minimal impact of zkSENSE’s operation and feasibility even on entry- 7.1 ZKSENSE Vs. reCAPTCHA level devices like the J3. As a next step, we compare the performance In our tests, we also collected memory usage of of zkSENSE with the state-of-the-art privacy- zkSENSE’s via procstats. Detailed results are omit- preserving humanness attestation mechanism (i.e., vi- ted since zkSENSE’s memory consumption is negligi- sual CAPTCHA). To do so, we developed an Android ble, i.e., less than 20MB regardless of device and oper- app which embeds reCAPTCHA for Android [22]. The ation. In comparison, the ZK-SNARK solution requires app is minimal7 to ensure its performance evaluation up to 1GB of memory due to the data generated dur- covers only the CAPTCHA aspect rather than any ex- ing the trusted setup. This is quite limiting in presence tra components. For the same reason, we opted for An- of low-end devices which might not have that amount droid reCAPTCHA rather than setting up a webpage of free memory, requiring swapping and thus a further with a CAPTCHA to solve. This alternative approach increase in execution time. would require an Android browser for testing and the Battery consumption: Figure 7 reports the battery performance evaluation would be tainted by the extra discharge (in mAh) associated with zkSENSE’s key op- cost of running a full browser. We did not evaluate Pri- erations ( detection, commitment, ZKP creation) for the vacy Pass [77] for two reasons: 1) it currently requires J3 – given the S9 is a personal device, we were unable a full browser along with an add-on, 2) it lacks support to wire its battery with the power meter. As expected on mobile devices. Note that we do not expect critical from the previous results, the battery overhead imposed performance difference between Privacy Pass and An- by zkSENSE’s detection and committing operation is droid reCAPTCHA since they use a very similar strat- negligible. Further, ZKP computation only consumes egy. Their difference instead lies in how invasive they about 5 mAh or 0.2% of the J3’s battery (2,600 mAh). are, both in how frequently they require user input and Even assuming one zkSENSE’s humanness verification from a sensor data collection standpoint. every hour of device usage, this would amount to under Using the above application we setup the follow- 1% of battery discharge for the average user (3h15min ing experiment. We enable remote access to the S9 on average, with only 20% of users using the device more device via the browser using Android screen mirror- than 4h30min [76]). ing [78] coupled with noVNC [79]. Then we asked 10 Bandwidth consumption: Finally, we compute what volunteers to visit the device from their browsers and is the bandwidth consumption required by the user to solve CAPTCHAs as needed by the app. The app was send a proof. The proof consists of the commitments coded such that users can continuously request for a new of the difference vector, the average and standard de- viation, for each of the input vectors the user submits. We use data from two sensors, namely the gyroscope 7 Source code: https://github.com/svarvel/CaptchaTest Proceedings on Privacy Enhancing Technologies 2021; ..:15–23

zkSENSE reCAPTCHA an initial window of 10packets (1500B MTU). Given our Automatic/Manual proof has a size of 160KB, the content delivery requires a worst case of: 1 RTT (for TCP) + 1 RTT (for TLS, in Overall time 3 1.4/8.9 sec CPU utilization 15% 3/15% case of unknown server) + 3 RTT for TCP to transfer Memory utilization 20 MB 20 MB the data – assuming slow start (doubling of cwnd), aka Replay protection No Yes 15K (10 MTU sized packets) + 30K + 60K + 60K (1/2 Consumed bandwidth 167 KB 16 KB of the last cwnd available). This sums up to about 3/4 Table 3. Performance of zkSENSE vs. Android reCAPTCHA. RTTs, which we could further reduce assuming a larger initial cwnd, or using QUIC – thus bringing the duration CAPTCHA to solve. Note that Android reCAPTCHA, down to a maximum of 2RTTs. Assuming a CDN runs as reCAPTCHA v3, leverages client side behavior to such a service, as Google does, this would thus cost be- minimize friction, i.e., whether to ask or not a user to tween 20ms (with optimization) up to 100ms. Assuming solve a visual CAPTCHA like “click on all images con- a very bad client connection, e.g., on mobile with RTT taining a boat”. It follows that often users do not need of 150ms, then this would cost an extra 300 to 750ms. to solve any visual CAPTCHA. We label automatic all the samples we collected where our volunteers did not have to solve a CAPTCHA. We instead label manual all 7.2 Summary the samples where human interaction was needed. To in- Our experiments show that the general purpose ZK- crease the chance of showing an actual CAPTCHA, we SNARK is not a viable solution for mobile-based ZKP created 15 CAPTCHAs that the app rotates on. computation (600 sec and 2% battery drain on a low- Over one week, our volunteers have requested end device). By designing our own model (i.e., zkSVM), about 500 CAPTCHAs with a 70/30 split: 350 au- we reduce ZKP’s execution time by 10×, achieving a tomatic and 150 manual. Table 3 directly compares duration of a bit less than 3 sec and 0.2% battery drain. This execution time is comparable with today’s visual zkSENSE with Android reCAPTCHA with respect to: execution time, CPU and memory utilization, and band- CAPTCHA solving time, 9.8 sec on average [21]), thus width consumption. The median was reported for each making zkSENSE a serious competitor to state-of-the- metric, further differentiating between automatic and art mechanisms for humanness attestation. manual for Android reCAPTCHA. The table shows that zkSENSE adds about 1.6 seconds to the time required 8. Related Work by Android reCAPTCHA when no user interaction is Assessing Humanness: To prevent automated pro- needed. zkSENSE instead saves a whole 6 seconds to grams from abusing online services, the widely adopted the (median) user by never requiring any interaction. solution is to deploy a CAPTCHA system. However, Even considering the fastest user in our experiment (5.4 text-based CAPTCHA schemes have been proven to seconds), zkSENSE is about 2x faster – and 10x faster be insecure as machines achieved 99.8% success rate than the slowest user (28.5 seconds). This is possible in identifying distorted text [8, 80, 81]. Audio-based because zkSENSE removes the need of user interac- CAPTCHAs have also been used to assist visually im- tion, at the cost of a higher risk for replay attacks. paired people, but they are difficult to solve, with over With respect to CPU and memory utilization, Table 3 half of users failed during their first attempt [82]. There- shows that the two mechanisms are quite similar and fore, CAPTCHA service providers started to test image- both very lightweight. Bandwidth-wise, reCAPTCHA based CAPTCHA schemes, which require users to select outperforms zkSENSE (16 versus 160KB), which com- images that match given description [83]. Nevertheless, munication overhead is still minimal and bearable even in [7, 84] authors demonstrated that more than 70% of by devices with very little connectivity. image-based Google and Facebook CAPTCHAs can be Table 3 currently reports on "overall time" which in- efficiently solved using deep learning. cludes both the computation time and the time required In [85], authors designed a multi-level data fusion to report to the server. While for Google we have no con- algorithm, which combines scores from individual clicks trol on the server endpoint – located within 10ms in our to generate a robust human evidence. These CAPTCHA experiments – in our experiments the server runs in the systems require users to perform additional tasks wors- same LAN with 1-2 ms delay (negligible). We currently ening user experience, especially on mobile devices [86]. use HTTP (POST) + TLS1.3 to return the proof. For ReCAPTCHA v2 uses a risk analysis engine to avoid TCP, we use an unmodified kernel running Cubic with interrupting users unnecessarily [87]. This engine col- Proceedings on Privacy Enhancing Technologies 2021; ..:16–23 lects and analyses relevant data during click events. Re- attacks. To the best of our knowledge, there are only CAPTCHA v3 no longer requires users to click but in- 2 papers aiming to provide provable machine learning stead it studies user interactions within a webpage and local evaluation without a trusted execution environ- gives a score that represents the likelihood that a user is ment. The first one [96] tries to solve a similar prob- a human [88]. Although these CAPTCHA schemes are lem, where personalization of a user device is done by invisible to users, a plethora of sensitive data, includ- evaluating a model locally on the user’s machine. This ing cookies, browser plugins, and JavaScript objects, is work uses Bayesian classification, for which they need collected [89] that could be used to fingerprint the user. from 100-300 feature words. The generation of correct With the proliferation of smartphones, various ap- model evaluation for such range of feature words ranges proaches leverage the variety of available sensors. Most from 30 to 80 seconds. Moreover, this study uses stan- of them require users to perform additional motion dard techniques for constructing zero-knowledge proofs, tasks. In [29], authors showed that waving gestures which give a big overhead to the verifier. For our par- could be used to attest the intention of users. In [27], ticular use case (where the verifier may need to handle authors designed a bot detection system that asks users several requests simultaneously), such an overhead for to tilt their device according to the description to prove the verifier is not acceptable. The second one [97] pro- they are human. In [28], authors presented a movement- poses a solution where after the evaluation of Random based CAPTCHA scheme that requires users to perform Forest and Hidden Markov models, the user generates certain gestures (e.g., hammering and fishing) using a zero-knowledge proof of correct evaluation. However, their device. In [90], authors exploited touch screen data this paper misses an evaluation study or availability of during screen unlocking to authenticate users. In [91], the code, which makes a study of the scalability of their authors suggested a brightness-based bot prevention approach inaccessible. mechanism that generates a sequence of circles with different brightness when typing a PIN; users will in- 9. Conclusion put misleading lie digits in circles with low brightness. Service providers need a reliable way to attest whether In [92], authors proposed a behavioural-based authen- a client is human or not and thus prevent user-side au- tication scheme, which uses timing and device motion tomation from abusing their services. Current solutions information during password typing. require (i) either additional user actions (e.g., sporadi- The work that is most closely related to ours is the cally solve mathematical quizzes or pattern recognition) Invisible CAPPCHA [19]. Similar to zkSENSE, authors like CAPTCHAs or (ii) user behavioral data to be sent leveraged the different device acceleration appearing on to the server, thus raising significant privacy concerns. a finger touch and a software touch to make a decision In this paper, we propose zkSENSE: a novel con- about whether a user is a bot. However, Invisible CAP- tinuous human attestation scheme, which is both (i) PCHA is not fully implemented as it requires a secure privacy-preserving and (ii) friction-less for the user. execution environment and its accuracy is low when de- By leveraging the motion sensor outputs during touch vice is stable on a table. In addition, it only considers screen events, zkSENSE performs human attestation at simple tap and vibration events; its accuracy on more the edge, on the user’s very own device, thus avoiding to complicated touch events (e.g., drag, long press, and transmit any sensitive information to a remote server. double tap) is unclear In comparison, zkSENSE consid- By enclosing the classification result in zero-knowledge ers any types of touch events and works regardless of the proofs, zkSENSE guarantees the integrity of the attes- device movement. To improve the accuracy, zkSENSE tation procedure. We tested our system under different uses more data sources in addition to accelerometer and attack scenarios: (i) when device is resting (on a table), introduces context into the detection. (ii) when there is artificial movement from device’s vi- Privacy-Preserving and Provable ML: A potential bration, (iii) when the device is docked on a artificial approach to offer privacy-preserving machine learning is movement generating swinging cradle, and it was able to evaluate the model locally, without sending data to a to detect if an action was triggered by a human with 92% server. However, if, unlike , such approach is zkSENSE accuracy. Performance evaluation of our Android proto- taken without proving correct evaluation of the model, type demonstrates that each attestation takes around 3 then verification may be lost [93–95]. In cases such as seconds, with minimal impact on both CPU and battery bot detection the user’s interest might be of faking the consumption. evaluation model, and this may be vulnerable to user Proceedings on Privacy Enhancing Technologies 2021; ..:17–23

References [18] Ismail Akrout, Amal Feriani, and Mohamed Akrout. Hack- [1] Matthew Hughes. Bots drove nearly 40% of internet traffic ing Google reCAPTCHA v3 using Reinforcement Learning. last year - and the naughty ones are getting smarter. https: 2019. //thenextweb.com/security/2019/04/17/bots-drove-nearly- [19] Meriem Guerar, Alessio Merlo, Mauro Migliardi, and 40-of-internet-traffic-last-year-and-the-naughty-ones-are- Francesco Palmieri. Invisible CAPPCHA: A usable mech- getting-smarter/, 2019. anism to distinguish between malware and humans on the [2] Shailin Dhar Mikko Kotila, Ruben Cuevas Rumin. Com- mobile IoT. Computers and Security, 78:255–266, 2018. pendium of ad fraud knowledge for media investors. [20] Muhammad Asim Jamshed, Wonho Kim, and KyoungSoo https://www.wfanet.org/app/uploads/2017/04/ Park. Suppressing bot traffic with accurate human attesta- WFA_Compendium_Of_Ad_Fraud_Knowledge.pdf, 2017. tion. In Proceedings of the first ACM asia-pacific workshop [3] ThreatMetrix. H2 2018 cybercrime report. https: on Workshop on systems, pages 43–48, 2010. //www.threatmetrix.com/info/h2-2018-cybercrime-report/, [21] Tim Allen. Having a captcha is killing your conversion rate. 2018. https://moz.com/blog/having-a-captcha-is-killing-your- [4] Drew Phillips. What is securimage? https:// conversion-rate, 2013. www.phpcaptcha.org/, 2015. [22] Google. Safetynet recaptcha api. https:// [5] Intuition Machines, Inc. hcaptcha: Earn money with a developer.android.com/training/safetynet/recaptcha, 2021. captcha. https://www.hcaptcha.com, 2019. [23] Elie Bursztein, Steven Bethard, Celine Fabry, John C [6] Roberto Iriondo. Breaking captcha using machine learning Mitchell, and Dan Jurafsky. How good are humans at in 0.05 seconds. https://medium.com/towards-artificial- solving captchas? a large scale evaluation. In 2010 IEEE intelligence/breaking-captcha-using-machine-learning-in-0- symposium on security and privacy, 2010. 05-seconds-9feefb997694, 2018. [24] Ivan Enríquez. Why is captcha killing your conversion [7] Suphannee Sivakorn, Iasonas Polakis, and Angelos D rate? https://blog.arengu.com/why-captcha-is-killing-your- Keromytis. I am robot:(deep) learning to break semantic conversion-rate/, 2019. image captchas. In 2016 IEEE European Symposium on Se- [25] Richard Kahn. How the use of captcha can hurt user experi- curity and Privacy (EuroS&P), pages 388–403. IEEE, 2016. ence. https://www.anura.io/blog/how-the-use-of-captcha- [8] Jeff Yan and Ahmad Salah El Ahmad. A Low-cost Attack can-hurt-user-experience, 2020. on a Microsoft CAPTCHA. In Proceedings of the 15th [26] Interaction Design Foundation. Killing the captcha for better ACM conference on Computer and communications security, ux. https://www.interaction-design.org/literature/article/ pages 543–554. ACM, 2008. killing-the-captcha-for-better-ux, 2016. [9] Jarrod Overson. Bypassing captchas with headless chrome. [27] Meriem Guerar, Mauro Migliardi, Alessio Merlo, Mohamed https://medium.com/@jsoverson/bypassing-captchas-with- Benmohammed, and Belhadri Messabih. A completely au- headless-chrome-93f294518337, 2018. tomatic public physical test to tell computers and humans [10] Kevin Bock, Daven Patel, George Hughey, and Dave Levin. apart: A way to enhance authentication schemes in mobile uncaptcha: a low-resource defeat of recaptcha’s audio chal- devices. In 2015 International Conference on High Perfor- lenge. In 11th USENIX Workshop on Offensive Technologies mance Computing & Simulation (HPCS), 2015. (WOOT 17), 2017. [28] Thomas Hupperich, Katharina Krombholz, and Thorsten [11] Ruti Gafni and Idan Nagar. Captcha–security affecting user Holz. Sensor Captchas: On the Usability of Instrumenting experience. Issues in Informing Science and Information Hardware Sensors to Prove Liveliness. In International Con- Technology, 13:063–077, 2016. ference on Trust and Trustworthy Computing, 2016. [12] Josh Dzieza. Why captchas have gotten so difficult. https: [29] Babins Shrestha, Nitesh Saxena, and Justin Harrison. Wave- //www.theverge.com/2019/2/1/18205610/google-captcha- to-Access: Protecting Sensitive Mobile Device Services via ai-robot-human-difficult-artificial-intelligence, 2019. a Hand Waving Gesture. In Michel Abdalla, Cristina Nita- [13] Scott Hollier, Janina Sajka, Jason White, and Michael Rotaru, and Ricardo Dahab, editors, Cryptology and Net- Cooper. Inaccessibility of captcha: Alternatives to visual tur- work Security, 2013. ing tests on the web. https://www.w3.org/TR/turingtest/, [30] Anupam Das, Nikita Borisov, and Matthew Caesar. Track- 2019. ing mobile web users through motion sensors: Attacks and [14] Wei Liu. Introducing recaptcha v3: the new way to stop defenses. In NDSS, 2016. bots. https://webmasters.googleblog.com/2018/10/ [31] Jorge-L. Reyes-Ortiz, Luca Oneto, Albert Samà, Xavier introducing-recaptcha-v3-new-way-to.html, 2018. Parra, and Davide Anguita. Transition-Aware Human Ac- [15] FreePrivacyPolicy. Privacy policy for recaptcha. https:// tivity Recognition Using Smartphones. Neurocomputing, www.freeprivacypolicy.com/blog/recaptcha-privacy-policy/, 2016. 2019. [32] Rubén San-Segundo, Henrik Blunck, José Moreno-Pimentel, [16] Thomas Claburn. Google’s recaptcha favors – you guessed Allan Stisen, and Manuel Gil-Martín. Robust Human Ac- it – google: Duh, only a bot would refuse to sign into the tivity Recognition using smartwatches and smartphones. chocolate factory. https://www.theregister.co.uk/2019/06/ Engineering Applications of Artificial Intelligence, 2018. 28/google_recaptcha_favoring_google/, 2019. [33] Mohammad Malekzadeh, Richard G Clegg, Andrea Caval- [17] Katharine Schwab. Google’s new recaptcha has a dark side. laro, and Hamed Haddadi. Protecting Sensory Data Against https://www.fastcompany.com/90369697/googles-new- Sensitive Inferences. In Proceedings of the 1st Workshop on recaptcha-has-a-dark-side, 2019. Privacy by Design in Distributed Systems, W-P2DS’18. Proceedings on Privacy Enhancing Technologies 2021; ..:18–23

[34] Erhan Davarci, Betul Soysal, Imran Erguler, Sabri Orhun Benjamin Livshits. Vpn0: A privacy-preserving decentralized Aydin, Onur Dincer, and Emin Anarim. Age group detection virtual private network. In Decentralising the Internet with using smartphone motion sensors. In 2017 25th European IPFS and Filecoin, DI2F’21, 2021. Signal Processing Conference (EUSIPCO), 2017. [52] Eli Ben-sasson, Alessandro Chiesa, Christina Garman, [35] Jiexin Zhang, Alastair R Beresford, and Ian Sheret. Sen- Matthew Green, Ian Miers, Eran Tromer, and Madars Virza. sorID: Sensor Calibration Fingerprinting for Smartphones. In Zerocash: Decentralized anonymous payments from bitcoin. Proceedings of the 40th IEEE Symposium on Security and 2014. Privacy (SP). IEEE, 5 2019. [53] Nick Grosz. How icash protects delegate votes and iden- [36] Elias P. Papadopoulos, Michalis Diamantaris, Panagiotis tities in its proof of trust protocol. https://medium.com/ Papadopoulos, Thanasis Petsas, Sotiris Ioannidis, and Evan- @nickgrosz/how-icash-protects-votes-and-voter-identity-in- gelos P. Markatos. The long-standing privacy debate: Mobile its-proof-of-trust-protocol-7a06c38e4296, 2018. websites vs mobile apps. In Proceedings of the 26th Interna- [54] Gonçalo Pestana, Iñigo Querejeta-Azurmendi, Panagiotis tional Conference on World Wide Web, WWW ’17, 2017. Papadopoulos, and Benjamin Livshits. Themis: Decentral- [37] World Wide Web Consortium (W3C). Captcha alterna- ized and trustless ad platform with reporting integrity. arXiv tives and thoughts. https://www.w3.org/WAI/GL/wiki/ preprint arXiv:2007.05556, 2020. Captcha_Alternatives_and_thoughts, 2019. [55] Jan Camenisch and Markus Stadler. Efficient Group Sig- [38] Web Accessibility In Mind (WebAIM). Screen reader nature Schemes for Large Groups (Extended Abstract). In user survey #7 results. https://webaim.org/projects/ CRYPTO, 1997. screenreadersurvey7/, 2017. [56] Jens Groth. On the size of pairing-based non-interactive [39] Armin Sebastian. Buster: Captcha solver for humans. https: arguments. In Proceedings, Part II, of the 35th Annual //github.com/dessant/buster, 2019. International Conference on Advances in Cryptology, EURO- [40] Jennifer Tam, Jiri Simsa, Sean Hyde, and Luis V Ahn. CRYPT’16, 2016. Breaking audio captchas. In Advances in Neural Informa- [57] Mary Maller, Sean Bowe, Markulf Kohlweiss, and Sarah tion Processing Systems, pages 1625–1632, 2009. Meiklejohn. Sonic: Zero-knowledge snarks from linear- [41] Yuanxi Ou. What is shuabang, and should i be using it to size universal and updateable structured reference strings. promote my game? https://www.mobvista.com/en/blog/ Cryptology ePrint Archive, Report 2019/099, 2019. https: shuabang-using-promote-game/, 2018. //eprint.iacr.org/2019/099. [42] Cristina Stefanova. Black hat aso for mobile apps & games: [58] Ariel Gabizon, Zachary J. Williamson, and Oana Ciobotaru. What is it and how it works (and why you shouldn’t do it). Plonk: Permutations over lagrange-bases for oecumenical https://thetool.io/2018/black-hat-aso, 2018. noninteractive arguments of knowledge. Cryptology ePrint [43] Gabriel Machuret. Blackhat aso news 2016: Shuabang Archive, Report 2019/953, 2019. https://eprint.iacr.org/ – a notorious blackhat app store optimization provider in 2019/953. china. https://asoprofessional.com/blackhat-aso-news-2016- [59] B. Bünz, J. Bootle, D. Boneh, A. Poelstra, P. Wuille, and shuabang-a-notorious-blackhat-app-store-optimization- G. Maxwell. Bulletproofs: Short proofs for confidential trans- provider-in-china/, 2020. actions and more. In 2018 IEEE Symposium on Security and [44] Brave Software, Inc. Get rewarded for browsing and support Privacy (SP), 2018. your favorite content creators. https://brave.com/brave- [60] Jonathan Bootle, Andrea Cerulli, Pyrros Chaidos, Jens rewards/, 2019. Groth, and Christophe Petit. Efficient zero-knowledge [45] Anton Kivva. The banker that can steal anything. https:// arguments for arithmetic circuits in the discrete log set- securelist.com/the-banker-that-can-steal-anything/76101/, ting. Cryptology ePrint Archive, Report 2016/263, 2016. 2016. https://eprint.iacr.org/2016/263. [46] Mike Murray. Pegasus for android: the other side of the [61] Torben P. Pedersen. Non-interactive and information- story emerges. https://blog.lookout.com/pegasus-android, theoretic secure verifiable secret sharing. In Proceedings 2017. of the 11th Annual International Cryptology Conference on [47] Henry de Valence, Jack Grigg, George Tankersley, Filippo Advances in Cryptology, CRYPTO ’91, 1991. Valsorda, and Isis Lovecruft. The ristretto255 Group. [62] Jens Groth. Linear algebra with sub-linear zero-knowledge Internet-Draft draft-hdevalence-cfrg-ristretto-01, Internet arguments. In Shai Halevi, editor, Advances in Cryptology - Engineering Task Force. CRYPTO 2009, 2009. [48] Isis Agora Lovecruft and Henry de Valence. curve25519- [63] Jan Camenisch and Markus Michels. Proving in zero- dalek. https://crates.io/crates/curve25519-dalek, 2020. knowledge that a number is the product of two safe primes. [49] S Goldwasser, S Micali, and C Rackoff. The knowledge In Jacques Stern, editor, Advances in Cryptology — EURO- complexity of interactive proof-systems. In Proceedings CRYPT ’99, 1999. of the Seventeenth Annual ACM Symposium on Theory of [64] Amos Fiat and Adi Shamir. How to prove yourself: Prac- Computing, STOC ’85, 1985. tical solutions to identification and signature problems. [50] Manuel Blum, Paul Feldman, and Silvio Micali. Non- In Andrew M. Odlyzko, editor, Advances in Cryptology — interactive zero-knowledge and its applications. In Proceed- CRYPTO’ 86, 1987. ings of the Twentieth Annual ACM Symposium on Theory [65] Nathan Dowlin, Ran Gilad-Bachrach, Kim Laine, Kristin of Computing, STOC ’88, 1988. Lauter, Michael Naehrig, and John Wernsing. CryptoNets: [51] Matteo Varvello, Iñigo Querejeta Azurmendi, Antonio Applying Neural Networks to Encrypted Data with High Nappa, Panagiotis Papadopoulos, Goncalo Pestana, and Throughput and Accuracy. In Proceedings of the 33rd Inter- Proceedings on Privacy Enhancing Technologies 2021; ..:19–23

national Conference on International Conference on Machine [86] Gerardo Reynaga and Sonia Chiasson. The usability of Learning, ICML’16, 2016. CAPTCHAs on smartphones. In 2013 International Confer- [66] Thore Graepel, Kristin Lauter, and Michael Naehrig. ML ence on Security and Cryptography (SECRYPT), 2013. Confidential: Machine Learning on Encrypted Data. In Lec- [87] Google. Choosing the type of reCAPTCHA. https: ture notes in computer science, volume 7839, 2012. //developers.google.com/recaptcha/docs/versions, 2019. [67] Joppe Bos, Kristin Lauter, and Michael Naehrig. Private [88] Google Developers. reCAPTCHA v3. https:// Predictive Analysis on Encrypted Medical Data. Technical developers.google.com/recaptcha/docs/v3, 2018. Report MSR-TR-2013-81, 9 2013. [89] Lara O’Reilly. Google’s new CAPTCHA security lo- [68] Android Developers. Android debug bridge (adb). https: gin raises ’legitimate privacy concerns’. https:// //developer.android.com/studio/command-line/adb, 2020. www.businessinsider.com/google-no-captcha-adtruth- [69] Henry de Valence, Cathie Yun, and Oleg Andreev. Bullet- privacy-research-2015-2?r=US&IR=T, 2015. proofs. https://crates.io/crates/bulletproofs, 2020. [90] Alexander De Luca, Alina Hang, Frederik Brudy, Christian [70] Mihir Bellare and Phillip Rogaway. Random oracles are Lindner, and Heinrich Hussmann. Touch Me Once and I practical: A paradigm for designing efficient protocols. CCS Know It’s You!: Implicit Authentication Based on Touch ’93, 1993. Screen Patterns. In Proceedings of the SIGCHI Conference [71] ZoKrates community. Zokrates: A toolbox for zksnarks on on Human Factors in Computing Systems, CHI ’12, 2012. ethereum. https://github.com/Zokrates/ZoKrates, 2019. [91] Meriem Guerar, Mauro Migliardi, Alessio Merlo, Mohamed [72] str4d. Bellman: Zero-knowledge cryptography in rust. https: Benmohammed, Francesco Palmieri, and Aniello Castiglione. //github.com/zkcrypto/bellman, 2016. Using screen brightness to improve security in mobile social [73] Matteo Varvello, Kleomenis Katevas, Mihai Plesa, Hamed network access. IEEE Transactions on Dependable and Haddadi, and Benjamin Livshits. BatteryLab: a distributed Secure Computing, 15(4):621–632, 2016. power monitoring platform for mobile devices. In HotNets [92] Attaullah Buriro, Sandeep Gupta, and Bruno Crispo. Eval- ’19, 2019. uation of motion-based touch-typing biometrics for online [74] BatteryLab. A Distributed Platform for Battery Measure- banking. In 2017 International Conference of the Biometrics ments. https://batterylab.dev, 2019. Special Interest Group (BIOSIG), pages 1–5. IEEE, 2017. [75] Monsoon Solutions Inc. High voltage power monitor. [93] Theja Tulabandhula, Shailesh Vaya, and Aritra Dhar. https://www.msoon.com/online-store/High-Voltage-Power- Privacy-preserving Targeted Advertising. CoRR, abs/1710.0, Monitor-Part-Number-AAA10F-p90002590, 2019. 2017. [76] Jory Mackay. Screen time stats 2019: Here’s how [94] Mikhail Bilenko and Matthew Richardson. Predictive Client- much you use your phone during the workday. https: side Profiles for Personalized Advertising. In Proceedings of //blog.rescuetime.com/screen-time-stats-2018/, 2019. the 17th ACM SIGKDD International Conference on Knowl- [77] Alex Davidson. The privacy pass protocol. https: edge Discovery and Data Mining, KDD ’11, pages 413–421, //tools.ietf.org/html/draft-privacy-pass-00, 2019. New York, NY, USA, 2011. ACM. [78] Solly Ross Joel Martin, Samuel Mannehed and Pierre Os- [95] Saikat Guha, Bin Cheng, and Paul Francis. Privad: Practical sman. Novnc: Html vnc client library and application. Privacy in Online Advertising. In Proceedings of the 8th https://github.com/Genymobile/scrcpy, 2021. USENIX Conference on Networked Systems Design and [79] Novnc - the open source vnc client. https://github.com/ Implementation, NSDI’11, 2011. novnc/noVNC, 2021. [96] Drew Davidson, Matt Fredrikson, and Benjamin Livshits. [80] A A Chandavale, A M Sapkal, and R M Jalnekar. Algorithm Morepriv: Mobile os support for application personalization to Break Visual CAPTCHA. In 2009 Second International and privacy. In Proceedings of the 30th Annual Computer Conference on Emerging Trends in Engineering Technology, Security Applications Conference, ACSAC ’14, 2014. pages 258–262, 12 2009. [97] George Danezis, Markulf Kohlweiss, Benjamin Livshits, and [81] Ian J Goodfellow, Yaroslav Bulatov, Julian Ibarz, Sacha Alfredo Rial. Private Client-side Profiling with Random Arnoud, and Vinay Shet. Multi-digit number recognition Forests and Hidden Markov Models. In Proceedings of the from street view imagery using deep convolutional neural 12th International Conference on Privacy Enhancing Tech- networks. arXiv preprint arXiv:1312.6082, 2013. nologies, PETS’12, 2012. [82] Aimilia Tasidou, Pavlos S Efraimidis, Yannis Soupionis, Lilian Mitrou, and Vasilios Katos. User-centric, Privacy- Preserving Adaptation for VoIP CAPTCHA Challenges. 2012. Appendix [83] Google. Are you a robot? Introducing “No CAPTCHA re- CAPTCHA”. https://security.googleblog.com/2014/12/are- A Sub-linear Inner Product Zero- you-robot-introducing-no-captcha.html, 2014. Knowledge Proof [84] Yuan Zhou, Zesun Yang, Chenxu Wang, and Matthew Boutell. Breaking Google reCaptcha V2. J. Comput. Sci. In this section, we prove that the zero-knowledge proof Coll., 34(1):126–136, 10 2018. presented in section 3.3 provides perfect completeness, [85] Chamila Walgampaya, Mehmed Kantardzic, and Roman special honest verifier zero-knowledge and knowledge Yampolskiy. Real time click fraud prevention using multi- soundness. Firstly, we formally define these properties, level data fusion. In Proceedings of the World Congress on which we reproduce as defined in [60]. Engineering and Computer Science, 2010. Proceedings on Privacy Enhancing Technologies 2021; ..:20–23

A zero-knowledge proof system is defined by three returns a transcript of the prover’s communication. The probabilistic polynomial time algorithms, (K, P, V), the extractor E can ask the oracle to rewind the proof to generator, prover and verifier. The generator takes as any point and execute the proof again from this point input a security parameter written in unary form, 1λ, on with fresh challenges from the verifier. We define and builds the common input of a proof, pp ← K(1λ). b ∈ 0, 1 to be the verifier’s output in the first oracle exe- In our paper, we use only common inputs that do not cution, i.e., whether it accepts or not, and we think of s need to be honestly generated, meaning that the out- as the state of the prover. If the definition holds also for put of K can be publicly verified. The P and V al- unbounded P∗ and A we say the proof has statistical gorithms take as input (pp, u, w) and (pp, u) respec- knowledge soundness. The definition can then be para- tively. We denote the interaction between prover and phrased as saying that if the prover in state s makes a verifier, and the latter’s output (0 if valid or 1 other- convincing proof, then we can extract a witness. wise) by hP(pp, u, w)||V(pp, u)i. We consider relations R that consist of a three element tuple (pp, u, w), which Definition 4 (Special Honest-Verifier Zero-Knowledge). we refer as the common input, instance and witness A proof system is computationally special honest-verifier respectively. We define the set of all valid instances zero-knowledge (SHVZK) if there exists a PPT sim- as LR = {(pp, u)|∃w :(pp, u, w) ∈ R}. The protocol ulator S such that for all state-full interactive PPT (K, P, V) is called a zero-knowledge proof system if it adversaries A that output (u, w) such that (pp, u, w) ∈ R has perfect completeness, knowledge soundness and spe- and randomness φ for the verifier cial honest-verifier zero-knowledge as described below.  pp ← K(1λ); (u, w, φ) ← A(pp); First, we introduce the notion of negligible function.   Pr  viewV ← hP(pp, u, w)||V(pp, u, φ)i : − Definition 1 (Negligible function). A non-negative A(viewV ) = 1 function f : N → R is called negligible if for ev-  pp ← K(1λ); (u, w, φ) ← A(pp); ery γ ∈ N there exists a k0 ∈ N such that for all Pr  viewV ← S(pp, u, φ): k ≥ k0, f(k) ≤ 1/kγ.   A(viewV ) = 1 We now proceed with a formal definition of the prop- is negligible with respect to λ. We say the proof is statis- erties a proof system needs to have to be considered a tically SHVZK if the definition holds against unbounded zero-knowledge proof. adversaries, and perfect SHVZK if the probabilities are equal. Definition 2 (Perfect Completeness). A proof system is perfectly complete if for all PPT adversaries A This definition can be paraphrased as saying that for " # pp ← K(); (u, w) ← A(pp): every valid protocol run, a simulator can generate sim- Pr = 1 ulated random view which is indistinguishable from the (pp, u, w) ∈/ R ∨ hP(pp, u, w)||V(pp, u)i = 1 original run. Having formalised the properties, we pro- Paraphrasing, this means that whenever prover and ver- ceed with the proof of Theorem 1: ifier proceed with the protocol, the verifier will always Proof. We follow the lines of the proof presented in the validate the proof. work of Bünz et al. [59] to complete our proof. Com- pleteness is trivial. To prove special honest verifier zero- Definition 3 (Knowledge soundness). A proof system knowledge we construct a simulator that generates valid has (strong black-box) computational knowledge sound- statements which are indistinguishable from random. To ness if for all DPT P∗ there exists a PPT extractor E this end, the simulator acts as follows: such that for all PPT adversaries A ∗ C ∈R Zp  pp ← K(1λ); (u, s) ← A(pp); n ∗ l, r ∈R Zp  hP (s)||V(pp,u)i  Pr  w ← E (pp, u): t,ˆ τC, µ ∈R Zp b = 1 ∧ (pp, u, w) ∈/ R T2 ∈R G is negligible with respect to λ.  2 1/C tˆ τC −1 −C T1 = g h · V · T2 Here the oracle hP∗(s)||V(pp, u)i runs a full protocol ex- µ l r −1
1/C ecution from the state s, and if the proof is successful it S = h · g · h · A Proceedings on Privacy Enhancing Technologies 2021; ..:21–23

We can see that the simulated transcript, which we can combine to get the following representa- tion of T2: (S, T1,T2; C; l, r, t,ˆ τC, µ), C0+C000  1  C0+C00 L R
C0−C000 T2 = g h all elements except for S and T1 are random, and the latter two are computationally indistinguishable from ˆ0 ˆ00 ˆ0 ˆ000 τ 0 −τ 00 τ 0 −τ 000 with L = t −t − t −t and R = C C − C C . random due to the DDH assumption. Also, note that C0−C00 C0−C000 C0−C00 C0−C000 Extractions of T1 and V follow. S and T1 are generated following the verification equa- If for any transcript, we have that tions, hence this simulated conversation is valid. i i i2 Next, we construct an extractor to prove knowledge tˆ 6= c + t1C + t2C soundness. The extractor runs the prover with three dif- with i ∈ {0,00 ,000 }, then the extractor has again found ferent values of the challenge C, ending with the follow- a non trivial discrete logarithm relation. Let P (X) = ing valid proof transcript: hl(X), r(X)i. Due to the validity of the transcripts, we 2 0 0 0 ˆ0 0 0 have that P (X) equals t(X) = c+t1X +t2X at least in (S, T1,T2; C ; l , r , t , τC, µ ) 00 00 00 00 00 00 the different challenges. In other words, the polynomial (S, T1,T2; C ; l , r , tˆ , τ , µ ) C P (X) − t(X) has at least three roots, and is of degree 000 000 000 ˆ000 000 000 (S, T1,T2; C ; l , r , t , τC , µ ). 2, hence it must be the zero polynomial. Therefore, we Additionally, the extractor invokes the extractor of have that t(X) = P (X). This implies that the zero co- the inner product argument. This extractor is proved efficient of t(X), namely c, equals ha, bi and we have a to exist in the original paper of Bunz et al. [59]. For valid witness for the statement.  each proof transcript, the extractor first runs the in- ner product extractor, to get a witness l, r to the inner B Formal Analysis of zkSVM µ l r product argument such that P = h g h ∧ hl, ri = tˆ. In this section we formally define the properties we ex- With this witness, and using two valid transcripts, pect out of zkSVM, privacy and verifiability. We use one can compute linear combinations of (16), we can game based proofs to show that zkSVM indeed pro- α a b compute α, ρ, a, b, sL, sR such that A = h g h and vides these properties. We formally define them here, ρ s s S = h g L h R . Such an extraction of these values pro- and include the proofs subsequently. To model the ex- ceeds as follows: periments of privacy and verifiability, we define the fol- 0 0 0 0 00 00 00 00 lowing five functions: A · SC = hµ gl hr A · SC = hµ gl hr .

Combining both relations we have – Setup(λ): Which is defined exactly as in Proce-

1 dure 1. We omit the notation of the cryptographic  0 00 0 00 0 00  0 00 S = hµ −µ gl −l hr −r C −C . material, and consider it implicit. We represent the set of parameters of the SVM model (normalisation The extraction of A follows. mean, normalisation scale, SVM weight and SVM Using these representations of A and S, as well as intercept) by W . i i 0 00 000 l and r with i ∈ { , , }, we have that – GenProof(v): Generates the zkSVM proof by run- ning Procedure 2, 3, 4, and 5. Mainly, it runs all li = a + s Ci L steps of the proof except for the submission step. i i r = b + sRC . We simplify the representation of the resulting tu- If these do not hold for all challenges, then the prover ple by [SH , Θ, Score, rR], where Θ consists of all in- has found two distinct representations of the same group termediate proofs and commitments. element, yielding a non-trivial discrete logarithm rela- – SubmitReq([SH , Θ, Score, rR]): Submits the output tion. of GenProof(v) by sending it to the verifier (Proce- dure 6). Next, we extract the values Ti with i ∈ {1, 2} from equation (14) as follows: – VerifReq([SH , Θ, Score, rR]): Runs all procedures defined in the Verification phase of zkSVM, mainly ˆ0 0 0 02 t τC C C g h = V · T1 · T2 Procedures 7, 8, 9 and 10. ˆ00 00 00 002 t τC C C – EvalSVM(v): Generates the result, Score, corre- g h = V · T1 · T2 ˆ000 000 000 0002 sponding to v, as defined in the zkSVM proof, but t τC C C g h = V · T1 · T2 excluding the cryptographic mechanisms. Proceedings on Privacy Enhancing Technologies 2021; ..:22–23

priv,b playing the ballot privacy experiment with b = 1. We ar- ExpA,C (λ): W← Setup(λ) gue that each of these steps are indistinguishable, and b0 ← AOSubmit(W ) therefore the results follows. The proof proceeds along Output b0 the following sequence of games:

OSubmit(v1, v2): 1 1 1 1 priv,0 Let [SH , Θ , Score , rR] ← GenProof(v1,W ) Game G0 Let game G0 be the ExpA,C (λ) game (see 2 2 2 2 Let [SH , Θ , Score , rR] ← GenProof(v2,W ) Figure 8). Let Θ2 ← SimResult(S2 , Score1, r1 ) H R Game G1 Game G1 is as in G0, but now OSubmit return SubmitReq([Sb , Θb, Score1, r1 ]) H R always computes a simulation of Θ regardless of the

priv,b bit. Mainly, OSubmit proceeds as follows: Fig. 8. In the privacy experiment ExpA,C , the adversary A has access to the oracle OSubmit and controls zkSVM verifier. OSubmit(v1, v2): 1 1 1 1 Let [SH , Θ , Score , rR] ← GenProof(v1,W ) 2 2 2 2 B1 Privacy Let [SH , Θ , Score , rR] ← GenProof(v2,W ) Let Θ1 ← SimResult(S1 , Score1, r1 ) The goal of zkSVM is that no information is leaked H R Let Θ2 ← SimResult(S2 , Score1, r1 ) from the input vector other than the result of the SVM H R return SubmitReq([S1 , Θ1, Score1, r1 ]) model. To model this, in Figure 8 we define an exper- H R priv,b iment, ExpA,C , between an adversary, A, and a chal- The function SimResult proceeds by simulating all lenger, C. The latter chooses a bit b ∈ {0, 1}, uniformly zero knowledge proofs contained in Θ. Because all at random, which is given as input to the experiment. these proofs are Zero Knowledge Proofs, and hence The adversary controls the zkSVM verifier, and its goal have the Special Honest-Verifier Zero-Knowledge is to distinguish the submissions of two different in- property, there exists a simulation algorithm such put vectors. The adversary is given access to an ora- that A cannot distinguish between a real and a sim- cle, OSubmit(), which takes as input two vectors of size ulated proof. Note that at this point of the experi- n, runs GenProof() over them, and submits a result. ment, the commitment SH and all commitments in Note that it is the adversary who chooses the vectors Θ correspond to those of v1—only the zero knowl- over which the zkSVM is executed and may modify edge proofs in Θ are simulated. the weights of the SVM model outputted by Setup(). Game G2 Game G2 is as in G1, but now, instead of Therefore, this experiment models the malicious choice returning of the SVM parameters, as well as any possible choice 1 1 1 1 of input vector. Depending on the bit, b, the oracle sub- SubmitReq([SH , Θ , Score , rR]), mits the result of one vector or the other, by running the oracle OSubmit returns SubmitReq(). However, to avoid a trivial win by the ad- 2 2 1 1 versary, the submitted SVM score is always computed SubmitReq([SH , Θ , Score , rR]). over the first vector. Hence, the experiment simulates In G2 the view of the adversary is identical to the the proof of the second vector by running SimResult(). priv,1 one of Exp (λ). Only thing that remains is to The adversary may call this oracle as many times as it A,C prove that G1 and G2 are indistinguishable wishes. By the end of the experiment, the adversary out- 0 0 puts a bit, b ∈ {0, 1}. The adversary wins if b = b with Given the Special Honest-Verifier Zero-Knowledge prop- non-negligible probability with respect to the security erty of the proofs, we know that the simulated view is parameter, λ. random. Given that both simulations are equally dis- tributed, it is infeasible to distinguish between Θ1 and Theorem 2. There exists a algorithm, such SimResult Θ2. Similarly, given the perfectly hiding property of that no PPT adversary can win the privacy zkSVM Pedersen commitments, no adversary can distinguish experiment with colluding verifier with probability non- between S1 and S2 . Clearly the resulting view is in- negligibly better than 1/2 with respect to λ. H H dependent of b. And privacy follows.  Proof. To prove that zkSVM provides privacy we pro- ceed by a series of games. We start with the adversary B2 Verifiability playing the privacy experiment with b = 0, and after a The other goal of zkSVM is that an adversary cannot sequence of game step transitions, the adversary finishes convince a verifier that the result is not linked to the committed vector as defined by the protocol. To model Proceedings on Privacy Enhancing Technologies 2021; ..:23–23

that Diff commits to the difference of S and Expverif(λ): H A,C iter W← Setup(λ) SH , in all entries but the last, which contains a [v,SH , Θ, Score, rR] ← A(W ) zero, by verifying Π0. If EvalSVM(v) = Score return 0 – Procedure 8 checks that Avg commits to the sum of If VerifReq([SH , Θ, Score, rR]) = ⊥ µ the elements committed in SH by verifying ΠIP . It then return 0, else return 1. does the same for the difference vector. – Procedure 9 first checks that Gµ˜ and Hµ˜ commit verif Fig. 9. In the verifiability experiment Expb,A , the adversary A G to the sum committed in Avg by verifying ΠEq and needs to submit a result not corresponding to the committed H vector. ΠEq respectively. Then, it checks that HS commits S to the same values as SH by verifying ΠEq. With these verified commitments, the verifier can check verif 2 this, in Figure 9 we define an experiment, ExpA,C , be- that V ar commits to N hv − µ, v − µi, in other tween an adversary, A, and a challenger, C. Informally, words, that it commits to a factor of the variance. verifiability ensures that a result that is not the out- It does so by running the algebraic operations of come of the model evaluation over the committed vec- σ2 Equation (23), and verifying ΠIP . Finally, to check tor cannot have a valid proof. During the experiment the that Std commits to the standard deviation of the adversary has access to the weights of the model, and vector committed in SH , it simply needs to check generates an input vector, a result, and its correspond- that it contains the square root of V ar. It does so ing proof material. The adversary wins the experiment σ by verifying Πsqrt. The same is performed for the if the result does not correspond the the SVM execution difference vector. of the committed vector, and the verifier validates. We formally describe the experiment in Figure 9. Given that all these proofs are Zero Knowledge Proofs, which provide the knowledge soundness property, the Theorem 3. No PPT adversary can win the zkSVM commitments of the SVM features, Avg, Avg 0, Std, Std0, verifiability experiment with non-negligible probability do not contain the expected features of the vector com- with respect to λ. mitted in SH with negligible probability. It only remains to prove that the result indeed corresponds to the SVM Proof. At the end of the proof generation procedure, function executed with these features as inputs. the prover (in our case the adversary) outputs a com- mitment, SH , a tuple of intermediate cryptographic ma- – Procedure 10 first leverages the homomorphic prop- terial, Θ, and a score, Score, together with the random- erties of the commitment scheme. In this way, the ness associated to the commitment of the score, rR. The verifier obtains a commitment of the linear combina- result follows from the soundness property of ZKPs and tion of the values committed in Avg, Avg 0, Std, Std0. the binding property of Pedersen commitments. Let us This results in the commitment of the result, as extend the cryptographic material associated with the described in Equation (22). This operation is con- 0 0 vector. We have that Θ = [∆, M, M , Λ, Λ ], with ducted solely by the verifier, avoiding any possible attacks by the adversary. Finally, the verifier checks ∆ = Diff, Siter, Π , Π  , H Eq 0 that the submitted score, and the corresponding  µ  M = Avg, ΠIP , opening key, indeed correspond to the locally com- 0  0 µ 0 M = Avg , ΠIP , puted commitment. Only if this is true, the verifier  µ˜ µ˜ G H S validates. Λ = G ,H ,HS, ΠEq, ΠEq, ΠEq, σ2 σ i V ar, ΠIP , Std, Πsqrt , Given that the commitment scheme used in zkSVM 0  µ˜ 0 µ˜ 0 0 G 0 H 0 is computationally binding, a PPT adversary has no Λ = G ,H ,HS , ΠEq , ΠEq , more than negligible probability of submitting an open- S 0 0 σ2 0 0 σ 0i ΠEq , V ar , ΠIP , Std , Πsqrt . ing that does not correspond to the committed value. Given that EvalSVM and the proof of zkSVM com- The proof verifies all proofs. In particular pute the exact same operations over the input vector, the result that the adversary has no more than negli- – Procedure 7 first generates the iterated generators, iter gible probability of wining the verifiability experiment and checks that indeed SH and SH commit to the follows.  same opening, by verifying ΠEq. Then, it checks