Budapest University of Technology and Economics Faculty of Electrical Engineering and Informatics Department of Networked Systems and Services
Gergő Ládi TRANSPARENT ENCRYPTION FOR CLOUD-BASED SERVICES
ADVISOR Dr. Levente Buttyán
BUDAPEST, 2017 Table of Contents
Table of Contents ...... 2 Összefoglaló ...... 7 Abstract ...... 8 1 Introduction ...... 9 1.1 Definitions ...... 9 1.2 Problem Statement ...... 9 1.3 Challenges ...... 11 1.4 Outline...... 12 2 Goals, Tasks, Objectives, and Strategies ...... 13 2.1 Goals ...... 13 2.2 Tasks ...... 13 2.3 Objectives and Strategies ...... 15 3 Initial Research ...... 16 3.1 Related Work ...... 16 3.1.1 Publications ...... 16 3.1.2 Similar Software ...... 17 3.1.2.1 Boxcryptor ...... 17 3.1.2.2 Cipherdocs ...... 17 3.1.2.3 CloudFogger ...... 18 3.1.2.4 SeaFile ...... 18 3.1.2.5 Cryptomator ...... 18 3.1.2.6 Tresorit ...... 18 3.1.3 Summary of Related Work ...... 19 3.2 Enumerating Potential Services for Encryption ...... 19 4 Picking and Analysing Services ...... 21 4.1 Detailed Analysis of Evernote's Communication Protocol ...... 21 4.1.1 Protocol Analysis ...... 21 4.1.2 Message Analysis ...... 24 4.1.2.1 Initial Page Load ...... 24 4.1.2.2 Reading Note Contents ...... 25 4.1.2.3 Creating Notes ...... 26 4.1.2.4 Editing Notes ...... 26
4.1.2.5 Editing Reminders ...... 27 4.1.2.6 Deleting Notes ...... 29 4.1.3 Analysis Summary ...... 29 4.2 Detailed Analysis of Google Calendar’s Communication Protocol ...... 30 4.2.1 Protocol Analysis ...... 30 4.2.2 Message Analysis ...... 32 4.2.2.1 Initial Page Load ...... 32 4.2.2.2 Dynamic Loading ...... 34 4.2.2.3 Creating Events ...... 35 4.2.2.4 Editing Events ...... 37 4.2.2.5 Deleting Events ...... 37 4.2.3 Analysis Summary ...... 38 4.3 Quick Analyses ...... 39 4.3.1 Dropbox ...... 39 4.3.2 Dynalist ...... 40 4.3.3 OneNote (Online) ...... 40 4.3.4 SimpleNote ...... 40 4.4 Analysis Summary ...... 40 5 Designing a Transparent Encryption Layer ...... 42 5.1 Intercepting Traffic ...... 42 5.1.1 Hijacking DNS Queries ...... 42 5.1.2 Proxying Connections ...... 43 5.1.3 Handling Certificates ...... 44 5.1.3.1 The “Problem” with Certificates ...... 44 5.1.3.2 Becoming a Trusted Root Certificate Authority ...... 44 5.1.3.3 Validating the Provider’s Certificate...... 45 5.2 Inspecting and Altering Traffic ...... 45 5.3 Encrypting/Decrypting Messages ...... 46 5.3.1 Key Management ...... 46 5.3.2 Using Format Preserving Encryption ...... 46 5.3.2.1 Format-Preserving Encryption for Text ...... 48 5.3.2.2 Format-Preserving Encryption for Date and Time ...... 48 5.4 Design Summary ...... 48 6 Implementing a Prototype ...... 49
6.1 Intercepting traffic ...... 49 6.1.1 DNS Hijacking...... 49 6.1.2 Creating Certificates ...... 50 6.1.3 Implementing the Proxy ...... 51 6.2 Inspecting and Altering Traffic ...... 52 6.3 Encrypting/Decrypting Messages ...... 54 6.3.1 Key Management ...... 54 6.3.2 Initialization Vectors ...... 54 6.3.3 Format preserving encryption ...... 54 6.3.3.1 Format-Preserving Encryption for Text ...... 55 6.3.3.2 Format-Preserving Encryption for Date and Time ...... 56 7 Testing the Prototype ...... 57 7.1 Smoke Testing ...... 57 7.1.1 Smoke Testing the DNS Hijacking Component ...... 57 7.1.2 Smoke Testing the TLS Proxy...... 58 7.1.3 Smoke Testing the FPE Module ...... 59 7.2 Unit Testing ...... 60 7.2.1 Unit Testing the Filters...... 60 7.2.2 Unit Testing the FPE Module ...... 61 7.3 Integration Testing ...... 61 8 Conclusion ...... 63 9 Further Considerations ...... 64 9.1 Possible Threats ...... 64 9.1.1 Ever-Changing APIs ...... 64 9.1.2 New Security Measures ...... 64 9.2 Plans for Improvement ...... 64 9.2.1 Supporting Multiple Users ...... 64 9.2.2 Usage in Enterprise Environments ...... 65 9.2.3 More Services ...... 65 9.2.4 Linux Compatibility ...... 65 9.2.5 User Experience ...... 65 References ...... 66 Appendix ...... 69 A. Table of Abbreviations ...... 69
B. Table of Figures ...... 71 C. Table of Exhibits ...... 71 D. Exhibits ...... 72
HALLGATÓI NYILATKOZAT
Alulírott Ládi Gergő, szigorló hallgató kijelentem, hogy ezt a diplomatervet meg nem engedett segítség nélkül, saját magam készítettem, csak a megadott forrásokat (szakirodalom, eszközök stb.) használtam fel. Minden olyan részt, melyet szó szerint, vagy azonos értelemben, de átfogalmazva más forrásból átvettem, egyértelműen, a forrás megadásával megjelöltem.
Hozzájárulok, hogy a jelen munkám alapadatait (szerző(k), cím, angol és magyar nyelvű tartalmi kivonat, készítés éve, konzulens(ek) neve) a BME VIK nyilvánosan hozzáférhető elektronikus formában, a munka teljes szövegét pedig az egyetem belső hálózatán keresztül (vagy hitelesített felhasználók számára) közzétegye. Kijelentem, hogy a benyújtott munka és annak elektronikus verziója megegyezik. Dékáni engedéllyel titkosított diplomatervek esetén a dolgozat szövege csak 3 év eltelte után válik hozzáférhetővé.
Kelt: Budapest, 2017. 12. 17.
...……………………………………………. Ládi Gergő
Összefoglaló
Az idei elemzések szerint a felhő alapú szolgáltatások egyre nagyobb népszerűségre tesznek szert, mind vállalati közegben, mind az otthoni felhasználók körében. Legyen szó fájlok tárolásáról, levelezésről, naptár- és időkezelésről, jegyzetelésről, vagy akár jelszókezelésről, előszeretettel veszünk igénybe online szolgáltatásokat. Ez biztonsági szempontból számos veszéllyel jár – adataink elveszhetnek, sérülhetnek, vagy még rosszabb: illetéktelenek kezébe kerülhetnek.
Az elmúlt néhány évben többtíz biztonsági incidensről olvashattunk a hírekben, melyek áldozatai között voltak kicsi és nagy, híres és kevésbé ismert cégek is. Az incidensek során sokszor többszázezer felhasználó adatai szivárogtak ki és kerültek nyilvánosságra az interneten. Ez nem csak azért veszélyes, mert esetlegesen bizalmas információk (például magánjellegű üzenetek, üzleti titkok) kerülhetnek a versenytársak kezébe, de azért is, mert az adatok között szerepelhetnek jelszavak vagy jelszóval egyenértékű adatok. Ezen adatok birtokában egy támadó képessé válhat arra, hogy az áldozatok más szolgáltatásokhoz tartozó fiókjaihoz hozzáférjen, ezzel pedig még több esetlegesen érzékeny adatra tehessen szert.
A fenti probléma egyik lehetséges megoldása az úgynevezett transzparens titkosítás. Ennek működési elve az, hogy az adatokat még helyben, a szolgáltatóhoz való továbbítás előtt titkosítjuk (így oda már csak titkosított formában jutnak el), majd később, még mielőtt a kliens feldolgozná azokat, kititkosításra kerülnek, szintén helyben. Ezáltal, még ha betörés áldozata is lesz egy szolgáltató, vagy akár csak egy máshonnan megszerzett felhasználónév-jelszó párossal lép be egy támadó, csak a titkosított, számára értéktelen adathalmazt látja.
Diplomatervem keretében választok egy felhő alapú szolgáltatást, elemzem az általa használt kommunikációs protokollt, majd megtervezek és elkészítek egy szoftvert, amely képes biztonságot nyújtani a szolgáltató felé irányuló forgalom releváns üzeneteinek transzparens titkosításával.
Abstract
Recent surveys have shown that cloud services are becoming more and more popular, both in the enterprise sector and among individuals. Be it online file storage, e- mail, calendar & time management, note taking, or even password management, we rely heavily on online services. From a security standpoint, this poses several risks – data might be lost or corrupted, or even worse: accessed by unauthorized individuals.
In the past few years, tens of security incidents, hitting big and small, little-known and famous companies alike, were covered in the news. Many of these breaches resulted in several hundreds of thousands of user records being leaked and made available on the internet – some exclusively on the black market, others to the general public. This is not only dangerous because potentially confidential information (such as private messages or trade secrets) might get into the hands of competitors, but also because user records may contain passwords or password equivalents. Using said information, it might be possible for an adversary to get into the accounts of these victims for other services, gaining access to even more potentially sensitive information.
One of the possible solutions to this issue is employing transparent encryption, the principle of which is to encrypt information locally, before it is being sent to the cloud service provider (where it is stored in an encrypted form), and then, upon reception, decrypt it before it is processed by the local client. This way, even if the cloud service provider itself is compromised, or is accessed using stolen credentials, the attacker can obtain nothing but encrypted pieces of information.
Within this thesis, I am to choose a cloud service, analyse the communication protocol used, then design and implement a piece of software that can perform transparent encryption by identifying and modifying the relevant messages in transport.
8 1 Introduction
1.1 Definitions
For the purposes of this document, unless otherwise noted, the terms cloud service, cloud service provider, service provider, and provider refer to a service (and the company that provides said service) that is available through the internet and that lets its users upload and store user data.
User data shall refer to documents, images, and other files, as well as personal information including but not limited to names, addresses, telephone numbers, and dates of birth that need not be known by the provider in order to fulfil its purpose. For example, in the case of a web shop, a shopper’s address and telephone number are not considered user data as these are needed to ensure delivery, while in the case of a contact manager application, they are.
1.2 Problem Statement
Over the course of the past couple of years, we could see public cloud-based services gain ground over traditional self-hosted or serverless solutions. This shift towards public cloud services could be observed not only in the enterprise sector, but also among home users. Gartner's forecast, titled Public Cloud Services, Worldwide, 2014- 2020 corroborates these observations, and further adds that this process is not expected to stop in the following years, although the speed of change may begin decreasing, starting from 2020 [1]. A different publication by RightScale points out that the typical user, as of 2017, leverages 1.8 public cloud services on average as part of his daily routine and is experimenting with a further 1.8 services [2].
This growing interest resulted in several existing companies adapting their software and services to the cloud, as well as new companies entering the market, promising easy-to-use applications that provide access to your information, regardless of which one of your devices you're using. Services appeared providing online file storage and synchronization, calendar management, image sharing, note taking, or even password management. Either to speed up the process of development in order to be first, or simply to cut costs, security analyses were often skipped. As it was later revealed, the lack of security measures and improper security design were the two main reasons for most of
9 recent years' data breaches [3]. These breaches affected big and small companies alike, causing loss of fame, revenue, and their users' trust. To make matters worse, some of these incidents are not discovered until after several months, or even years have passed.
The risks of using public cloud services is fivefold:
1) Data loss: If a provider ceases operations without notifying clients, clients lose their files unless they have other copies of these. An example of this would be the sudden closure of MegaUpload, a file storage and sharing service, in 2011.
2) Direct data theft: If a provider is breached where potentially sensitive information is stored (e.g. files, images, or notes), these could be accessed by unauthorized individuals.
3) Indirect data theft: If any site or service is breached where the user hasn't stored anything valuable, the attacker may still gain access to the users' email addresses and passwords or equivalent authentication information (such as hashes of passwords). In this case, if the users had accounts with different providers where they used the same usernames and passwords, the attackers will be able to log in and access the users' sensitive information, even if the service itself was reasonably secure.
4) Insider access: The provider itself may access the users' sensitive information without their knowledge or permission. This could be a malicious employee or a system designed to perform data mining on user data to extract features deemed interesting in order to create user profiles to be sold or otherwise used for profit.
5) Nation state attackers: The provider may be forced to, or may decide to hand over user data to nation states or law enforcement agencies, which could put the users' lives at risk. A good example for this could be Arabic countries, where the internet is heavily regulated, and people having opposing views to the current political party are often chased down.
10 The first risk may be eliminated by having a proper backup plan (such as the 3-2- 1 strategy1) in place. The rest of these risks could also be eliminated, or at least greatly reduced by employing transparent encryption. The principle of transparent encryption is that, before being sent over the network by client applications unencrypted, data is intercepted on the client computer (or on a trusted device on the home network), and is encrypted by a separate piece of software. This encrypted data is then received and stored by the provider. When the client application needs this data later, it requests the data from the server, which then sends it back – still encrypted – to the client. Before it could be processed by the client application, is it intercepted again by the previously mentioned software, and is decrypted on-the-fly. Finally, the client application receives the data unencrypted, just as it was expecting it, then processes the data as needed. In case an attacker manages to get his hands on a file (in any of the manners detailed above) that was encrypted this way, he will have gained nothing but a blob of garbage (from his point of view). This approach can be extended to cover not only files, but also other kinds of potentially sensitive information, such as text fields, dates or credit card numbers.
Transparent encryption has the advantage that neither the client application nor the server-side code has to be modified in any way, therefore it can be used even if the service provider does not support such extra security measures, and the provider itself does not have to spend resources implementing said measures. In addition, if the encryption layer is implemented by independent developers, the provider does not have to be trusted (which may, otherwise, implement intentionally weak or flawed encryption, or leave backdoors). Furthermore, since transparent encryption is implemented as a separate piece of software, it can be licensed under a different licensing model than the original software – this is advantageous because the encryption software can be made open source, even if the underlying client application is closed source.
1.3 Challenges
When designing and implementing a transparent encryption layer, one may face several challenges that need to be overcome in order to succeed. The most crucial part is analysing and learning how the target service works and how the client application
1 The 3-2-1 strategy: have 3 copies of your critical data. These copies should reside on at least 2 different kinds of media (e.g. two on hard disks, one on an optical disk), and 1 copy should be kept off-site.
11 communicates with the web service, followed by the design of encryption methods that generate output that pass possible validation checks made by the providers.
Firstly, if the communication channel itself is encrypted, the encryption method has to be understood, circumvented, then reimplemented in the transparent encryption layer. In the event that the service uses a non-standard or proprietary encryption mechanism, these steps may end up rather time consuming. Secondly, in addition to being encrypted, the channel may also be authenticated or otherwise tamper-proofed, which, again, needs to be circumvented, and may even make it impossible to create a truly transparent proxy. Thirdly, the protocol spoken by the parties also has to be – at least partially – understood. This may turn out to be a custom, undocumented proprietary binary protocol, which could take several days to reverse. Fourthly, some legal agreements or country laws may not permit the disassembly and/or analysis of the client application. Finally, in certain cases, the cloud service expects the data to conform to a certain format and/or be in a specific range of values, and if encryption is used, the ciphertext will most likely not meet these requirements.
1.4 Outline
Within the next sections, I will choose and introduce a cloud-based service, find a way of intercepting its messages, analyse these, then design, implement, and test the prototype of a software that is capable of performing transparent encryption on the relevant messages.
In section 2, I will explain in detail the problem to be solved, and the goals to be reached. In section 3, I will research and provide an overview of similar works and results, as well as identify services that could be possible candidates for this project. In section 4, I will analyse at least one cloud-based software, its channels of communication, message types, then identify which (parts of) these need to be encrypted to ensure the security of user data. Then, in section 5, I will design a system that makes it possible to intercept and encrypt/decrypt messages identified in the previous section. Afterwards, in section 6, I will implement a prototype using the previous design. Then, in section 7, I will perform tests on the prototype. In section 8, I will summarize my work, then, finally, in section 9, I will explain how this prototype could be further improved, as well as list possible issues that might make it difficult to use or maintain such a solution.
12 2 Goals, Tasks, Objectives, and Strategies
2.1 Goals
Goals describe what must be achieved for the project to be considered successful. As defined in the thesis assignment description, I will have six top-level goals:
1) Choosing a cloud-based service that can be used to demonstrate transparent encryption
2) Analysing the communication protocol that is used by the service
3) Identifying the relevant protocol elements that should be encrypted in order to provide confidentiality
4) Identifying a transparent encryption scheme that could be used to provide confidentiality, and perhaps, adapting it to better suit the current task
5) Designing and implementing the encryption layer, and integrating it with the client-side application
6) Testing the implementation and summarizing the results
In addition to the above, I’ve chosen to add an extra goal as a sort of a zeroth step: researching similar or related existing solutions.
2.2 Tasks
Tasks are technically breakdowns of goals.
Goal 1) can be broken down to two main tasks:
a) Making a list of potential services, possibly with the help of search engines and peers
b) Choosing a service that is expected to use a limited set of relatively simple messages. It would also be ideal if this service was not completely unknown to me.
Goal 2) will consist of four distinct tasks:
13 a) Identifying the transport layer (OSI layer 4) protocol2 that is used by the application
b) Identifying whether the application uses a well-known higher-layer (e.g. OSI layer 7) protocol or not
c) Identifying whether the protocol messages are encrypted and/or covered by integrity protection
d) Identifying the structure of the protocol messages (if any)
Goal 3) is comprised of the following three tasks:
a) Identifying the data types that will need to be protected (e.g. files, text, images, phone numbers, etc.)
b) Identifying the messages in which these are transmitted
c) Identifying the location of these data elements within these messages
Goal 4) includes another three tasks:
a) Enumerating the algorithms that could be used
b) Choosing the most suitable group of algorithms
c) Identifying how the keys and other necessary parameters will be managed
Goal 5) can be broken down to another set of three:
a) Planning the architecture of the prototype, defining what components there are and how they interact with each other
b) Choosing a paradigm and a language the prototype will be implemented in
c) Implementing each component in the chosen language and paradigm
Finally, goal 6) will include the following four tasks:
a) Determining the type and number of tests needed
2 Open Systems Interconnection model: an abstract model that can be used to describe the means and methods of network communication between hosts. It consists of 7 independent layers, with each layer being responsible for a different role, such as physical or logical addressing, or retransmission of lost data.
14 b) Planning the tests, writing test cases
c) Performing the tests
d) Evaluating the results, drawing conclusions
In addition, the previously mentioned zeroth goal will be made up of two tasks:
a) Looking for related publications in conference archives and publication indexing services
b) Using search engines to find related software
2.3 Objectives and Strategies
Objectives define the deadline, while strategies define the order of completion.
In my case, the tasks are mostly linear, with each task depending on the previous ones.
• The literature research phase should be done first so as to avoid possible duplicate work. It is expected to take one or two weeks, depending on the findings.
• Goal 1) must be completed next as all others depend upon it. This should take at most one week.
• Goal 2) must follow afterwards, as none of the others are available at this point. This is task expected to take one week.
• Goals 3) and 4) may be done in parallel, although it would be preferable to finish goal 3) before starting goal 4). These are expected to take two and three weeks, respectively.
• Goal 5) depends on all previous goals and is the only one that can be completed next. It is expected to take four weeks.
• Finally, goal 6) can be completed. This should take two weeks.
15 3 Initial Research
3.1 Related Work
3.1.1 Publications
Based on searches I conducted using the IEEE Xplore Digital Library and Google Scholar, two of the biggest research databases, over 2 000 papers have been published to date that are related to cloud and encryption. Approximately 200 papers are connected to transparent encryption, while only a fifth of those are also related to cloud computing. There exist some solutions for specific use cases, such as transparently encrypting data stored in a local filesystem [4][5], in MongoDB databases [6], HDFS3 [7][8], or transmitted between virtual machines and their hosts [9][10].
Application Layer Encryption for Cloud by Saxena et al. [11] describes a similar, but not necessarily transparent method, user-layer encryption (as they name it). In 2012, Diallo et al. published CloudProtect [12], a middleware written in the Java programming language that can transparently encrypt certain data fields in Google Docs documents, as well as Google Calendar items, while also making it possible to share encrypted items with others. However, the solution presented in this paper only seems to consider unencrypted HTTP4 sessions, which are less and less common these days. Just recently, in 2017, Newport et al. described a system in their paper, A Secure Cloud Storage System for Small and Medium Enterprises [13], that is capable of encrypting files on the fly, then storing them in Dropbox, a cloud-based file storage service. Although the exact details of the method are not specified, I would infer from the examples that this solution works by creating a local folder, looking for changes in the files and folders within, then encrypting the changed files, putting them in an actual Dropbox folder, to be uploaded to the cloud.
3 Hadoop Distributed File System: a distributed file system that stores data on commodity machines and is often used in clustered environments.
4 Hypertext Transfer Protocol: a text-based application layer (OSI L7) protocol that is most commonly used by browsers to access web-based content.
16 3.1.2 Similar Software
The next phase of my research consisted of discovering software that was similar in functionality to what I aim to achieve within this thesis.
3.1.2.1 Boxcryptor The first solution I found was Boxcryptor. It is a closed-source software that creates a virtual drive that can be used for secure storage. When a file is written on this drive, it is encrypted instead, then stored in one of the supported cloud storage services (this approach is often called the overlay method as it overlays an existing solution). The free version only supports Dropbox as its back-end, while the paid version includes support for Google Drive, OneDrive, Box, Cubby, and several others [14]. It uses AES5 with a key length of 256 bits in CBC6 mode to encrypt files, with each file being encrypted with a different key. The AES keys are encrypted with the user’s 4096-bit RSA7 key, and are appended to the files [15]. Boxcryptor also supports master keys and sharing files with other users or groups. It works on Windows and Linux, as well as several mobile platforms.
3.1.2.2 Cipherdocs The next solution I came across was cipherdocs. It is an open-source project, available on GitHub8. According to the project documentation [16], it works with most cloud storage solutions, including Dropbox, Google Drive and OneDrive. Instead of the overlay method, it changes the file extensions of the encrypted files to .gpg, which, when opened, are decrypted into a temporary (local) folder, then re-encrypted and moved back
5 Advanced Encryption Standard: a commonly used symmetric encryption algorithm for which no known feasible attacks exist as of today
6 Cipher Block Chaining: a mode of operation of a block cipher, in which the input of the nth encryption function is not just the nth block of plaintext, but the nth plaintext block XORed (bitwise eXclusive OR) with the ciphertext of the (n-1)th block. For the first block, the first plaintext block is XORed with an initialization vector.
7 Rivest-Shamir-Adleman: a commonly used asymmetric encryption algorithm, the strength of which is based on the factoring problem (factorizing the product of two large primes)
8 A development platform that makes it easy to share code, enable other programmers to propose changes to the code, track issues, publish releases, automate testing, and manage projects.
17 if changed. It uses an OpenPGP9 implementation to encrypt files. Designed for single- user mode, sharing encrypted files is not supported. It works on Windows only.
3.1.2.3 CloudFogger A third provider I intended to check out was CloudFogger, but as of 4 Oct, 2017, only a notice appears on their website10, stating that the service is no longer available, recommending users to try Boxcryptor instead.
3.1.2.4 SeaFile The next solution to be investigated was SeaFile, an open-source, Git11-based storage system. It is self-hosted, meaning that the server has to be installed by, and the storage space has to be provided by the user, requiring more knowledge of computer systems than the previously introduced solutions. It supports sharing files among users and groups [17]. Neither the website nor their GitHub page mention any specifics of the encryption process. Supports Windows and Linux.
3.1.2.5 Cryptomator Another application I tried was Cryptomator. It is free and open-source, and acts as an overlay above Dropbox, Google Drive, and other services, just like Boxcryptor. It uses AES with 256-bit long keys, and encrypts file names as well as the structure of the folders. Written in Java, it works on Windows and Linux as well.
3.1.2.6 Tresorit Last, but not least, I checked out Tresorit. It turned out to be different: it is a stand-alone service that has its own client, and does not overlay an existing service. It supports 2- factor authentication, has a version history, and the more expensive plans include sharing files. It uses AES-256 for encryption. It is closed-source and has no free plan.
9 An e-mail encryption framework based on Phil Zimmermann’s software titled Pretty Good Privacy (PGP).
10 https://www.cloudfogger.com
11 A distributed version control system for files, often used by programmers.
18 3.1.3 Summary of Related Work
Based on the above, it can be concluded that the need for encryption for cloud- based services has already been recognized, resulting in several publications and implementations.
As for the papers, while many are related to my project, some of them employ non-transparent methods, some of them focus on enterprise-level solutions more than home users, and some of them just describe what should be done, but now how. It is apparent, however, that by combining existing proposals and extending the result with some of my ideas, it is possible to build a system that solves the problems detailed in the problem statement.
As for the implementations, it can be said that none of them are truly transparent in that they all require the end-user to change how he uses the underlying cloud service (for example, by having to store and manage files on a new virtual drive instead of Dropbox). Furthermore, it seems that all the solutions focus on securing files, and there are no well-known implementations to secure note-taking applications or calendars.
3.2 Enumerating Potential Services for Encryption
I used three different approaches to find cloud-based services that could possibly be made more secure using transparent encryption. First, I listed the services that I currently use, used before, or at least have heard about. Second, I asked some of my colleagues and friends about which services they used. Third, I used three different search engines, Google, Yahoo and Bing, querying for typical search terms such as online file storage, online calendar, note management, and
After compiling the lists from all the sources, I ended up with Table 3.1. The table shows the services found in alphabetical order, the type of the service, and the source where the entry came from.
Service Type Source Apple Cloud file storage colleagues Box.com file storage heard about Dropbox file storage used before Dynalist.io note/task management Yahoo Evernote note management used before Google Calendar calendar management used before 19 Google Drive file storage used before Google Keep calendar management heard about Note Taking Express note management Bing OneDrive file storage using it OneNote (online) note management colleagues Outlook (online) calendar management colleagues SimpleNote note management Google SpiderOak file storage colleagues Sync.com file storage Google Zoho Notebook note management Google Table 3.1 – A list of services that could potentially be encrypted transparently
20 4 Picking and Analysing Services
Having found that there exist several solutions that focus on securing online file storage services, but none that aim to secure calendar applications and note-taking services, I have chosen to focus on these latter two categories. Even though the assignment description only requires me to analyse one service, if time permits, I would like to examine more of them in order to gain insight on current trends. I expect this to help me design the transparent encryption layer later more efficiently.
The first service to be analysed was chosen to be Evernote. Evernote, as its name suggests, is a note-taking application that also supports reminders, task lists, and interactive content. Even though Evernote isn't the most popular service on the list, I have previously worked with its API12 as part of a related project. The API isn't exactly what I'd call simple, but being at least somewhat familiar with it makes Evernote an ideal candidate for starters.
4.1 Detailed Analysis of Evernote's Communication Protocol
4.1.1 Protocol Analysis
Evernote has a web-based, a desktop, and a mobile client. This would make it logical to assume that all of these clients use a common API, which is likely to be a web- based API. To confirm this hypothesis, I opened Firefox, a web browser, loaded the landing page (https://www.evernote.com), then pressed F12 to show the browser’s Developer Tools window. Although it might be called differently, all modern browsers have this feature today. This feature makes it possible to inspect and manipulate the DOM13 of websites, run/inject arbitrary JavaScript code locally, perform benchmarks, test the layout with various screen sizes and aspect ratios, and, of course, take a peek at what is being sent on the network (see Figure 4.1).
12 Application Programming Interface: a set of data structures and function declarations that describe how a service that implements this API may be consumed (interacted with).
13 Document Object Model: a model that treats an (X)HTML document as a tree, where each node represents an element in the document.
21
Figure 4.1 – Evernote's messages in Firefox's Developer Tools
It can be seen that the web client of Evernote uses HTTP over SSL14/TLS15 over TCP16 to send and receive messages.
To further confirm the hypothesis, I downloaded and installed the Evernote application from Play Store, the official application store of Google. After installation, I made sure that my Android-based phone was connecting to the same WiFi network my laptop was on, then opened Wireshark, a network packet capture/analysis application. After logging in to Evernote on the mobile, I could see in Wireshark that it was connecting to the same IP address as the browser, and that it was also using TLS. I could not see what was being transmitted, but this is expected since the traffic itself is encrypted.
Having confirmed the hypothesis, the analysis followed. Within its scope, I logged in, created two notes, set their title and contents, added a reminder to one of them, changed
14 Secure Sockets Layer: a set of cryptographic protocols that aim to secure communication channels. Typical services include encryption, integrity protection and authentication.
15 Transport Layer Security: a (more secure) successor of SSL.
16 Transmission Control Protocol: a connection-oriented transport layer (OSI L4) protocol that offers reliability via the retransmission of lost segments. It also reorders out-of-order segments and supports flow control.
22 the contents, changed the date of the reminder, created a third note, deleted it, and then examined the results:
• For all of the actions that create or change a note, AJAX17 calls are made to the Evernote API. The HTTP method (verb) used for these calls is always POST, no matter whether the action is a read, update, or delete operation. From this, I can conclude that the Evernote API is not a RESTful18 API.
• The endpoint for the relevant messages is always either https://www.evernote.com/shard/s###/enweb/notestore or https://www.evernote.com/shard/s###/enweb/notestore/ext, where ### is a three-digit number, possibly referring to the specific server on which my session exists.
• The response always has the MIME19 type text/json, although the response body is never a valid JSON20 object. It always begins with two forward slashes and the letters O, K (i.e. //OK), which, in JavaScript, denotes a comment, however, this is not valid in JSON. This is most likely used as a security measure against object hijacking attacks via script inclusion, which the two forward slashes break by making the entire response behave like a comment if included maliciously. The actual client implementations can just ignore the first four characters of each response, and parse the rest as a regular (and valid) JSON object.
17 Asynchronous JavaScript and XML: a means of performing asynchronous calls in web applications that is typically used to dynamically load page contents without having to reload the entire page.
18 Representational State Transfer: a REST (or RESTful) API is a stateless API that identifies the resource to be queried or manipulated in the URL, and uses HTTP verbs to specify the action to be carried out on the target resource.
19 Multipurpose Internet Mail Extensions: originally designed to describe the types and contents of files that are sent as e-mail attachments, MIME has been adopted to be used in other protocols as well.
20 JavaScript Object Notation: a JavaScript-like data format that is commonly used in modern web APIs, especially object-oriented ones.
23 • The request body always consists of several tens of data fields separated by pipe symbols ( | ). Some of the fields contain values such as java.lang.String/2004016611, hinting that these fields may be Java objects serialized into strings. It took me a while to figure out what framework or API serializes data like this, but I succeeded. The framework was identified as GWT21, while the protocol being spoken as GWT-RPC22 wire protocol. While there is no official documentation for this, it was reverse- engineered in 2012 [18]. Even though GWT-RPC parsers are only available in Java, this does not limit the languages that can be used to implement a transparent proxy for Evernote since we don’t need to parse and interpret each field in the message, only specific ones that are always in the same position.
• The messages have no replay or integrity protection. I was able to replay an API call using Firefox's Edit and Resend function. I changed several letters in the content of a note that was to be edited, then replayed the message. The server responded with a 200 OK message. After a reload, the note had its content successfully modified.
4.1.2 Message Analysis
4.1.2.1 Initial Page Load After logging in, when the site is first loaded, a call to the findNotesMetadata function is issued. As shown later, this function returns a list of notes that are owned by the user that is logged in.
7|0|10|https://www.evernote.com/focusclient/|78E137D3512D195F071EB90374365 42A|com.evernote.web.shared.GWTNoteStoreInterface|findNotesMetadata|com.ev ernote.edam.notestore.NoteFilter/3387378272|I|com.evernote.edam.notestore. NotesMetadataResultSpec/2285571585|[Z/1413617015|Etc/GMT-1||1|2|3|4|4|5|6|
21 Google Web Toolkit: an open-source framework that makes it possible to create JavaScript- based web applications in Java.
22 Remote Procedure Call: a means of implementing inter-process communication, in which the caller can invoke methods on remote servers, then process the results as if they were the results of local function calls.
24 6|7|5|8|4|1|1|1|0|0|0|0|0|0|0|2|0|9|10|0|50|7|8|11|1|0|1|1|0|0|1|0|1|1|0|1 |0|1|0|1|0|1|0|1|0|1|
This request does not contain anything sensitive, but we can see that the version of the GWT-RPC protocol being used is 7.
The response looks as follows:
//OK[150,2,0,0,0,'VjcO6_o',0,11,0,8,0,0,10,'A','VjcO30g',0,'A',0,0,0,0,'A' ,'A','A','A',0,0,0.0,0.0,0,0,0,0,0,0,6,0,0.0,0,0,0,0,0,0,0,0,0,0,0,0,12,2, 5,0,0,0,1,1,0,6,2,4,'VjcPCUY',0,9,0,8,0,0,7,'A','VjcO$Kw',0,'A',0,0,0,0,'A ','A','A','A',0,0,0.0,0.0,0,0,0,0,0,0,6,0,0.0,0,0,0,0,0,0,0,0,0,0,0,0,12,2 ,5,0,0,0,1,1,0,6,2,4,2,3,1,1,1,3,2,1,["com.evernote.edam.notestore.NotesMe tadataList/3192047339","[Z/1413617015","java.util.ArrayList/4159755760","c om.evernote.edam.notestore.NoteMetadata/2574771094","com.evernote.edam.typ e.NoteAttributes/74627218","me4626","b1dc0239-b0b1-47ee-a17e-b69b66ba70e4" ,"My second note","9a8dddfd-0d87-4e7c-b81d-086a5aa3f9ee","My first note"],0,7]
Here, I can see that I have two notes, surrounded with what appear to be GUIDs23. From that, I can infer that notes are identified by the GUIDs in the system. One has the GUID of b1dc0239-b0b1-47ee-a17e-b69b66ba70e4 and the title My second note, the other the GUID of 9a8dddfd-0d87-4e7c-b81d-086a5aa3f9ee and the title My first note. The note titles here are of interest since these may contain sensitive information.
4.1.2.2 Reading Note Contents The content of notes is retrieved using getHtmlNoteContent:
7|0|11|https://www.evernote.com/focusclient/|304280BD765DCDBF4A79609E10928 B10|com.evernote.web.shared.GWTNoteStoreExtensions|getHtmlNoteContent|java .lang.String/2004016611|java.util.List|S|Z|b1dc0239-b0b1-47ee-a17e-b69b66b a70e4|java.util.ArrayList/4159755760|/shard/s652/res/|1|2|3|4|5|5|6|5|7|8| 9|10|0|11|-1|0|
The request contains the GUID of the note whose contents should be retrieved, while the response carries the contents of the note:
//OK[1,["\x3Cbody class\x3D\"ennote\"\x3E\x3Cdiv\x3ESecond note content. \x3Cbr clear\x3D\"none\"/\x3E\x3C/div\x3E\x3C/body\x3E"],0,7]
23 Globally Unique Identifier: a 128-bit hexadecimal number that is supposed to uniquely identify an object. GUIDs usually are represented in the form 8-4-4-4-12, e.g. 12345678-90ab-cdef-1234- 567890abcdef
25 It may be observed that note bodies are considered and handled as HTML24, with the special characters (such as < and >) escaped. In a more readable format, the above note body would be:
Obviously, note contents are prime targets for encryption.
4.1.2.3 Creating Notes Notes are created by calling the createNote function.
7|0|14|https://www.evernote.com/focusclient/|304280BD765DCDBF4A79609E10928 B10|com.evernote.web.shared.GWTNoteStoreExtensions|createNote|com.evernote .edam.type.Note/4071998839|java.lang.String/2004016611|java.util.List|[Z/1 413617015|com.evernote.edam.type.NoteAttributes/74627218|me4626|Untitled|| java.util.ArrayList/4159755760|1|2|3|4|3|5|6|7|5|8|6|0|1|1|0|1|0|1|9|8|12| 0|0|0|0|0|0|0|0|0|0|0|1|0|0|10|0|0|0|0|0|0|0|0|0|0|A|A|A|A|0|0|0|0|A|0|0|0 |VjcXc9v|A|0|0|11|0|0|0|0|0|12|0|VjcXnr6|13|14|0|
The function creates a new instance of the note object, sets its title to Untitled, and its content to an empty string. There's nothing sensitive here.
The response:
//OK['VjcXnoo',155,8,0,0,0,0,0,7,0,6,'A','VjcXc5I',154,72,77,-51,-102,- 43,36,23,-32,115,44,30,-68,-25,63,34,73,16,5,0,'A',0,0,0,0,'A','A','A', 'A',0,0,0.0,0.0,0,0,0,0,0,0,4,0,0.0,1,0,0,0,0,0,0,0,0,0,0,0,12,2,3,1,1,1,0 ,1,1,1,6,2,1,["com.evernote.edam.type.Note/4071998839","[Z/1413617015","co m.evernote.edam.type.NoteAttributes/74627218","me4626","[B/3308590456","b3 ba7ec9-9b0a-4ec3-8bba-fea2d9d07537","Untitled"],0,7]
Here, it is shown that the new, third note was created and that it was assigned a GUID of b3ba7ec9-9b0a-4ec3-8bba-fea2d9d07537. There is nothing sensitive here, either.
4.1.2.4 Editing Notes Changes to note titles and contents are sent using the updateNoteIfUsnMatches function.
7|0|16|https://www.evernote.com/focusclient/|304280BD765DCDBF4A79609E10928 B10|com.evernote.web.shared.GWTNoteStoreExtensions|updateNoteIfUsnMatches| com.evernote.edam.type.Note/4071998839|java.lang.String/2004016611|java.ut il.List|[Z/1413617015|com.evernote.edam.type.NoteAttributes/74627218|me462 6|[B/3308590456|b1dc0239-b0b1-47ee-a17e-b69b66ba70e4|My second note|
24 Hypertext Markup Language: the markup language in which the layout of websites and web applications are written.
26 Second note content with updates.
The request contains the GUID of the note to be edited, the new title, and the new contents. The title and the contents to be encrypted here.
The response:
//OK[1,'VjcQk84',151,9,0,0,0,0,0,8,0,7,'A','VjcO$Kw',186,56,-126,-45,- 106,15,-115,-53,-4,75,84,33,48,-128,-12,-95,125,16,6,0,'A',0,0,0,0,'A', 'A','A','A',0,0,0.0,0.0,0,0,0,0,0,0,5,0,0.0,0,0,0,0,0,0,0,0,0,0,0,0,12,2,4 ,1,1,1,0,1,1,1,6,2,3,1,1,2,1,["com.evernote.edam.notestore.UpdateNoteIfUsn MatchesResult/226232967","[Z/1413617015","com.evernote.edam.type.Note/4071 998839","com.evernote.edam.type.NoteAttributes/74627218","me4626","[B/3308 590456","b1dc0239-b0b1-47ee-a17e-b69b66ba70e4","My second note"],0,7]
The GUID and the note title are echoed back.
The function name has the conditional clause if USN matches in its name, with USN most likely meaning Update Sequence Number. USNs are used in multi-client systems to ensure that an update cannot accidentally overwrite changes that were made by another update in the meantime.
4.1.2.5 Editing Reminders Reminders are set using the updateNote function.
7|0|12|https://www.evernote.com/focusclient/|78E137D3512D195F071EB90374365 42A|com.evernote.web.shared.GWTNoteStoreInterface|updateNote|com.evernote. edam.type.Note/4071998839|[Z/1413617015|com.evernote.edam.type.NoteAttribu tes/74627218|me4626|[B/3308590456|b1dc0239-b0b1-47ee-a17e-b69b66ba70e4|cd3 306ff-56c9-4ae8-9def-75ee96af1ee8|My second note|1|2|3|4|1|5|5|6|6|1|1|1|0| 1|1|1|7|6|12|0|0|0|0|0|1|0|1|0|0|0|0|0|0|8|0|0|0|0|0|0|0|0|0|0|A|VjcRAIZ| Vky2QMA|A|0|0|0|0|A|0|9|16|125|-95|12|-128|48|33|84|75|-4|-53|-115|15|-106 -45|-126|56|186|VjcO$Kw|A|10|0|11|0|0|0|0|0|12|153|VjcQk84|
The GUID of the note (and for some reason, its title) are sent to the server. The request also contains several alphanumeric fields of length 7 (highlighted in blue or red). I knew these fields contained dates and times somehow, but I wasn’t sure how. I figured out that the one highlighted in red stores the date of the reminder by changing the reminder date multiple times. Smaller changes to the date resulted in smaller changes in the value, same dates resulted in the same value, and a later date resulted in a value that succeeded the previous values in alphabetical order (in other words, I got a string that was “greater” if the date was also “greater” than the previous one). I had a feeling these were
27 base64-encoded25 integers, but decoding them in the usual manner or decoding them in reverse byte order both produced results that made no sense. The previously mentioned reverse-engineered documentation had no mention of this format, so I started digging around the source code of the GWT server implementation. In the class file com.google.gwt.user.server.Base64Utils.java [19], I found a function named longFromBase64 that converts these to long integers26. It turned out I was right about it being (a kind of) base64, and the reason I couldn’t decode these values was that they are compressed to save a few bytes. Based on the implementation, I wrote my own function to decode such strings:
static long longFromBase64(string base64EncodedValue) { long result = 0;
foreach (char character in base64EncodedValue) { result <<= 6; result |= base64ValueOf(character); }
return result; }
The base64ValueOf function (not detailed here) returns the value of a given character that is encoded in base64. For the first 62 characters, the standard mapping applies (A-Za-z0- 9), but for the last two characters, $ and _ are used (instead of the usual + and /) respectively. In addition, there is no padding.
Using the above function, it can be verified that the value Vky2QMA decodes to 1482616800000, which, when interpreted as a Unix timestamp27 in millisecond precision, yields 24 December 2016 22:00:00, which is almost the date I set the reminder for: it is off by exactly one hour. This could be attributed to a difference in the time zones. The other dates highlighted in the request are close to each other, and are likely to be the
25 An encoding scheme that is used to represent binary data in the form of text.
26 An integer that is wider (in bits) than the usual integer. Long integers are usually 64-bit long, while regular integers are 32 bits long, although this may also depend on the platform and the compiler.
27 An integer that stores the number of seconds that have elapsed since 1 January 1970, 00:00:00 to a certain date.
28 creation and last modification dates for the note. The reminder date is a candidate for encryption.
The response to the above request can be seen below:
//OK['VjcQk84',154,8,0,0,0,0,0,7,0,6,'A','VjcO$Kw',186,56,-126,-45,106,15, -115,-53,-4,75,84,33,48,-128,-12,-95,125,16,5,0,'A',0,0,0,0,'A','Vky2QMA', 'VjcRAIZ','A',0,0,0.0,0.0,0,0,0,0,0,0,4,0,0.0,0,0,0,0,1,0,1,0,0,0,0,0,12,2 ,3,1,1,1,0,1,1,1,6,2,1,["com.evernote.edam.type.Note/4071998839","[Z/14136 17015","com.evernote.edam.type.NoteAttributes/74627218","me4626","[B/33085 90456","b1dc0239-b0b1-47ee-a17e-b69b66ba70e4","My second note"] ,0,7]
The note GUID and title, as well as the reminder date are echoed back.
4.1.2.6 Deleting Notes Notes are also deleted with the updateNote function:
7|0|12|https://www.evernote.com/focusclient/|C16F92F30BE4F3D436A66C9BE4BCA B46|com.evernote.web.shared.GWTNoteStoreInterface|updateNote|com.evernote. edam.type.Note/4071998839|[Z/1413617015|com.evernote.edam.type.NoteAttribu tes/74627218|me4626|[B/3308590456|b0e3bbcc-6d9f-4227-b93a-9fb7313e18b7| cd3306ff-56c9-4ae8-9def-75ee96af1ee8|Delete Me!|1|2|3|4|1|5|5|6|6|1|1|1|1|1 |1|0|7|6|12|0|0|0|0|0|0|0|0|0|0|0|0|0|0|8|0|0|0|0|0|0|0|0|0|0|A|A|A|A|0|0| 0|0|A|0|9|16|-2|43|107|105|22|-95|66|90|-126|-70|121|20|38|-40|-83|54|165| WBdnUxw|WBdnZDH|10|0|11|0|0|0|0|0|12|217|WBdnXdo|
The GUID of the note, along with the title are sent to the server. The response is as follows:
//OK['WBdnXdo',218,8,0,0,0,0,0,7,0,6,'WBdnZDH','WBdnUxw',165,54,-83,-40, 38,20,121,-70,-126,90,66,-95,22,105,107,43,-2,16,5,0,'A',0,0,0,0,'A','A', 'A','A',0,0,0.0,0.0,0,0,0,0,0,0,4,0,0.0,0,0,0,0,0,0,0,0,0,0,0,0,12,2,3,0,1 ,1,1,1,1,1,6,2,1,["com.evernote.edam.type.Note/4071998839","[Z/1413617015" ,"com.evernote.edam.type.NoteAttributes/74627218","me4626","[B/3308590456" ,"b0e3bbcc-6d9f-4227-b93a-9fb7313e18b7","cd3306ff-56c9-4ae8-9def- 75ee96af1ee8","Delete Me!"],0,7]
Both are echoed back to the client.
Using the updateNote function for deletion hints that notes are not actually deleted in this step, but are simply marked as deleted instead.
4.1.3 Analysis Summary
Based on the above analysis, it can be stated that:
• Even though the request messages use a not-so-common format, since the relevant fields are always in the same position (considering pipe symbols as field separators), these can be found and replaced relatively easily.
29 • The relevant fields (and their types) to be encrypted are: note titles (text), note contents (text/html), and reminder dates (Unix timestamps, long integers).
• The message formats seen here do not match the message formats of either of the public APIs [20]. A third, undocumented API is used.
4.2 Detailed Analysis of Google Calendar’s Communication Protocol
Having analysed a note management application, I wanted to choose a calendar manager as the next one for diversity. While I don’t personally use Google Calendar, it is one of the bigger and better-known apps. Understanding how it works could prove to be useful in the future, so I’ve chosen Google Calendar’s as the second protocol to analyze.
4.2.1 Protocol Analysis
Like Evernote, Google Calendar also has a web-based, a desktop, and a mobile client, which, again, makes it logical to assume that all of these clients use a common, web-based API. This time I used Chrome’s developer tools to analyse traffic from the browser. Google Calendar loads lots of resources and is very noisy in terms of network traffic, but it can be seen that a web API is used (HTTP over TLS over TCP).
In addition, I downloaded and installed the Google Calendar application on my phone (I don’t use a stock ROM28, so I didn’t have it by default). In Wireshark, I could see the app connect to a different IP29 address (216.58.209.164 instead of 216.58.209.174), but this did not necessarily mean that a different service was used. Since the traffic is encrypted with TLS, there was no easy way to verify whether the messages were the same, but luckily, modern TLS clients support and use the SNI30 extension,
28 Read-Only Memory. In this context, ROM refers to the operating system image that runs on a mobile phone, and has little to do with the original meaning.
29 Internet Protocol: the most commonly used OSI layer 3 protocol in everyday networking.
30 Server Name Indication: a TLS extension that lets clients indicate which server they are trying to communicate with, all before any certificates are exchanged. This makes it possible to serve multiple
30 which contains the name of the server in plaintext that the client is attempting to connect to. As it can be seen on Figure 4.2, the SNI extension contains the server name calendar.google.com, which is the same as the one that was used by the browser version. Thus, it is very likely that the same API is used.
Figure 4.2 – SNI extension used by the Google Calendar mobile app
I continued my work with the analysis of the messages. I logged in, added two events, modified one of them, deleted the other one, then examined the results:
• For all of the actions that create, change, or delete an event, AJAX calls are made to the Google Calendar API. The HTTP method used for these calls is always POST, regardless of the type of operation. Based on this, it can be concluded that the Google Calendar API is not RESTful.
websites with TLS on the same IP address and port because the server can choose the appropriate certificate to use based on the value found in the SNI extension.
31 • The endpoint being used for the relevant messages is always https://calendar.google.com/calendar/endpoint, where endpoint specifies the action to be made.
• The response always has the MIME type text/javascript, and always begins with the four characters )]}’. Just like in the case of Evernote with the two forward slashes, these are most likely used as a security measure against object hijacking attacks via script inclusion. Here, including the endpoints would cause the browser to attempt to evaluate and run the response as JavaScript, which would fail with a syntax error on one of the parenthesis. An actual client implementation can just ignore the first line of each response, then parse the rest as a regular JavaScript object or array.
• The request bodies are passed as key-value pairs as form data, with shortened (but usually recognizable) key names.
• The messages seem to have no replay or integrity protection. Using the Tamper Chrome add-on, I edited and replayed a message, modifying the date of an event. The server accepted the message with a 200 OK response. After a reload, I could see that the event had its date successfully modified.
4.2.2 Message Analysis
4.2.2.1 Initial Page Load After logging in, the user is redirected to a URL31 that has the format https://calendar.google.com/calendar/r/view/year/month/day, where view is the time frame that is shown to the user (e.g. day, month, week, year). Year is given as a 4-digit number, and determines the year of the time frame. Month determines the month that is shown, and day determines the day that is shown. For example, if the URL is /calendar/r/ week/2017/12/14, then the week on which 14 December is (in 2017) is displayed.
The server’s response for the initial page load contains all the information about all the events in the selected time frame (plus a margin of approximately two weeks). The
31 Uniform Resource Locator: an address string that specifies from where and how a resource may be retrieved. For example, http://example.com/something is a URL.
32 window object is assigned a key named INITIAL_DATA, which holds an array of arrays and other values. The array with the index of 2 is another array that seems to contain more arrays that store user settings and the events for this time frame. Within, the array with the index of 2 holds a final array, the 0th index of which is a string that can be parsed as JSON. Parsing this JSON results in yet another array that has as many elements as the number of events. Each event is another array by itself, with the 0th element always being a single letter ‘a’, and the 1st element being another JSON string. At first, I had the suspicion that the letter ‘a’ would be the type of the event, but I could not get it to change, even though I tried several various combinations of settings. Parsing this JSON string, one gets a final array, with the 1st element containing the title of the event (Lunch with Jane Doe in this case), the 2nd and 3rd elements containing the start and end dates (14 December, 2017, 12:00-13:00 here), and finally, the 16th index containing the description (Description for this event in this case) (see Figure 4.3). Fields with the indices 0, 4, 5, 6, and 9 are base64-encoded, the first and last of them containing some sort of identifiers that include a part of the user’s login name (these were redacted for security reasons), and the middle three containing the user’s e-mail address (these were also redacted). I’m not exactly sure why this has to be stored three times, but I would guess that these are used for different purposes, for example one could mean who created the event, another could mean who changed it last, etc.
33
Figure 4.3 – Data structure of a Google Calendar event
The title, start and end dates, as well as the description are targets for encryption since these may contain sensitive information.
4.2.2.2 Dynamic Loading Events that were not sent by the server during the initial page load are loaded dynamically when the user navigates to a calendar page that has not been loaded yet. To retrieve such events, a POST request is sent to https://calendar.google.com/calendar/load as per Figure 4.4.
34
Figure 4.4 – Google Calendar dynamically loading events (request)
The request contains the date range for which the events should be retrieved (the range is sent in the emf parameter, and in this case, the range is 11 December, 2017 to 18 December, 2017). The lef parameter contained my account name in base64, therefore it was redacted. There is also a parameter named secid, hinting it is some kind of a security identifier, so its value was also redacted.
Responses to these requests contain a JavaScript array for each event in the given date range. Each array has two elements, the 0th one being the letter ‘a’, and the 1st one being a JSON string that parses to the same data structure that was presented earlier on Figure 4.3 in 4.2.2.1. From this point on, such arrays will be referred to as event arrays.
4.2.2.3 Creating Events Events are created by issuing a POST request (see Figure 4.5) to the https://calendar.google.com/calendar/event endpoint.
35
Figure 4.5 – Google Calendar creating an event (request)
The request contains CREATE in the action field, the current Unix timestamp in the cts field, the name of the newly created event (My New Event in this case) in the text field, my account name in the src field (plaintext, redacted), and finally, the start and end dates in the dates field, separated by a forward slash. The event title should be considered for encryption.
The response contains a single event array, containing the newly created event.
36 4.2.2.4 Editing Events Events are edited using the same endpoint that was used to create events, but with a different set of parameters (Figure 4.6).
Figure 4.6 – Google Calendar editing an event (request)
In this case, the action parameter has the value EDIT, and the eid field contains the event identifier that was assigned by the server upon creation. A new field, erev is included, containing the timestamp of the previous edit (supposedly acting as a USN). The cts field has the same function as before. Only fields that have been changed are sent to the server. In this case, the event’s title was changed from My New Event to My Renamed Event, and a description was added. Some sensitive information was redacted.
The response contains a single event array, containing the event that was changed.
4.2.2.5 Deleting Events Events are deleted using the https://calendar.google.com/calendar/deleteevent endpoint as shown on Figure 4.7.
37
Figure 4.7 – Google Calendar deleting an event (request)
The event to be deleted is specified using the eid parameter. Some sensitive information was redacted.
The response contains an array of arrays that have nothing useful in them. An example can be seen on Figure 4.8.
Figure 4.8 – Google Calendar deleting an event (response)
4.2.3 Analysis Summary
Based on the above analysis, I can conclude that:
38 • The API relies on relatively simple structures to represent events (even though it uses nested arrays and nested JSON extensively), meaning that virtually any JSON/Javascript parser library can be used for the encryption process later.
• The relevant fields (and their types) to be encrypted are: event titles (text), event descriptions (text), and start and end dates (date-time).
• Google Calendar only sends down events that are within (or very close to) to currently selected time frame. Therefore, when encrypting dates and times for events, care should be taken to encrypt them in a way such that the ciphertexts stay on the same week or month (depending on the use case).
• There is an anomaly in how the account name in the src parameters of the create and delete request are treated. For the event creation request, it is sent in plaintext, whereas for the deletion request, it is sent base64- encoded.
4.3 Quick Analyses
Having analysed two of the bigger applications, I still wanted to understand more about other services to get a broader picture, so I took a glance at some of the other software, without intending to deeply analyze messages.
4.3.1 Dropbox
Dropbox is one of the oldest cloud-based file storage providers. I’ve only been able to determine that it uses TLS, because it seemed to be protected against man-in-the- middle attacks, most likely via certificate pinning32. Only the desktop client was examined.
32 A security check that, in addition to requiring a valid certificate, also requires that the certificate be issued by one of the whitelisted certificate authorities or that it have a certain thumbprint or serial number.
39 4.3.2 Dynalist
Dynalist is primarily a to-do management application. While it has mobile apps, only the web-based version was evaluated. It leverages HTTP over TLS for transport, with a binary format for requests (to-do list updates) and JSON for responses.
4.3.3 OneNote (Online)
OneNote is a rich note-taking application that has an offline desktop version, an offline mobile version, and an online one as well. The online app uses HTTP over TLS for transport, with JSON for both requests and responses.
4.3.4 SimpleNote
I found SimpleNote to be quite similar to Evernote, with a less complicated UI33 and maybe slightly fewer features. It uses WebSockets34 over TLS, with JSON for requests as well as responses.
4.4 Analysis Summary
Based on the above analyses, it is now possible to construct a generalized model for the message flows of cloud-based services:
• When the client application is started (or the user enters the service’s website’s URL in the browser), a query is made to a DNS35 server to resolve the domain name to an IP address
• The name server resolves the hostname and returns the response
• The application (or the browser) initiates a TCP connection to this IP address.
33 User Interface: the part of an application with which users directly interact
34 A relatively new protocol that offers bi-directional in communication real time and has less protocol overhead compared to HTTP. Almost always used in tandem with web services.
35 Domain Name System: a hierarchical system that, among others, can be used to map domain names to IP addresses and vice versa
40 • Once a connection is established, the client (or the browser) attempts to secure the channel by negotiating TLS.
• When the channel is secured, the user is authenticated. If authentication succeeds, protocol messages may now be sent and received.
This process is depicted on Figure 4.9.
1: Where is cloudprovider.example.com?
2: It's at 10.1.2.3.
3: Create TCP session, DNS Server negotiate TLS
4: Data transfer
Client Cloud-based Service application at 10.1.2.3 Figure 4.9 – A generalized model for the message flows of cloud services
It can be further stated that JSON is the most prevalent solution of passing around data structures. Therefore, to achieve a widely usable transparent encryption layer, it has to be designed in a way that JSON, HTTP, and TLS are supported at the very least.
41 5 Designing a Transparent Encryption Layer
Having researched and modelled several applications, having identified message fields with sensitive information, it is time to design a system that is capable of intercepting traffic between the client and the server, finding the relevant messages in the data stream, then encrypting/decrypting the selected fields.
5.1 Intercepting Traffic
In order to encrypt and decrypt traffic transparently, one needs to be able to impersonate the service provider to the client, and the client to the service provider. This is most easily done with a MitM36 attack. Considering that all cloud services rely on DNS to locate the provider, the easiest way of achieving this is DNS hijacking.
5.1.1 Hijacking DNS Queries
A DNS server needs to be set up so as to hijack DNS queries. This server should work in the following way:
• It should have a list of domain names, for which, traffic needs to be hijacked.
• Requests that target these domain names should be resolved to a local IP address (where the transparent encryption service is running) instead of the actual address.
• However, when requested by the transparent encryption service, the server should return the actual addresses, so that the proxy can connect to the provider.
• Name resolution for domains that are not on the list should work as usual, regardless of who the requestor is.
36 Man-in-the-middle: an attack where the attacker positions himself between two communicating parties without their knowledge, gaining the ability to read, insert, alter, or erase messages.
42 The list should contain all the domain names that clients may use to connect to the provider. Some applications may use more than one domain, or more than one subdomain, therefore, care should be taken to enumerate these as precisely as possible.
The local DNS server should be configured to resolve addresses according to these requirements, and the local computer should be set to use the local DNS server instead of the one assigned originally.
5.1.2 Proxying Connections
With the above DNS server set up, requests to the cloud service will be arriving at the proxy instead. To handle these, a service needs to listen on TCP port 443 (the port of HTTPS37). Upon receiving an incoming connection, this service will then have to negotiate a secure (TLS) connection with the client, as well as establish a connection to the actual service provider, then secure that, too. At this point, there should be two duplex connections established, both secured: one between the client and the proxy, and one between the proxy and the actual provider (see Figure 5.1) (loosely based on a model by S. J. Murdoch and R. Anderson [21]).
Our Proxy TLS TLS
Cloud-based Client service application Figure 5.1 – Connections to and from the proxy
The proxy can now access messages coming from the client unencrypted, as well as traffic from the provider, also unencrypted. It is now possible to read or alter messages before forwarding them to the other party, as well as inject new messages or delete certain ones. In other words, a man-in-the-middle situation is established.
37 Secure Hypertext Transfer Protocol
43 5.1.3 Handling Certificates
5.1.3.1 The “Problem” with Certificates While the above may seem easy at first, terminating the TLS connection from the client needs further effort. To negotiate TLS for this connection, a security certificate38 is needed. One of the reasons of using such certificates is that they protect against the exact same kinds of man-in-the-middle attacks that I am trying to achieve here. Obviously, I will not be able to acquire certificates for domains that I do not actually own, so alternate approaches are needed. First, I could rely on the fact that some applications do not actually check the validity of certificates (or don’t validate it in every aspect), but simply require their existence, accepting virtually any certificate. This is, however, bad security design and should not be relied upon, as it may get fixed over time. Second, I could abuse the fact that most applications delegate the job of certificate validation to the operating system (or the browser, in the case of browser applications).
5.1.3.2 Becoming a Trusted Root Certificate Authority One of key the philosophies of PKI39 is that there are some entities that you trust (these are the so-called trusted root CAs40), and if they trust others, you also trust them. Trust appears in the form of digital signatures, which can be verified using asymmetric cryptographic algorithms. It if was possible to become such a trusted root CA somehow, it would also be possible to issue certificates for any domain, which would then verify as valid. In this case, any application asking the operating system (or browser) to validate the certificate would get the response that it is valid, and everything would proceed as usual.
Becoming a CA is relatively easy. First, I need to generate and sign a certificate for myself that says I that am allowed to issue (sign) other certificates with it. This will be a so-called self-signed certificate, which can be generated using the openssl library (available on both Linux and Windows) or the MakeCert (Windows only) utility. Once
38 Cryptographically verifiable evidence which proves that a peer is who he is claiming to be
39 Public Key Infrastructure: a system where the identity of entities (such as people or companies) is proven by digital certificates.
40 Certificate Authority: a central entity of PKI that can issue certificates to other entities
44 done, this certificate has to be imported into the operating system’s store of trusted root CAs. Some browsers, most notably Firefox and its forks, have their own lists of trusted CAs that are managed separately. This means that in case of cloud services that are also accessed from a browser, the root CA's certificate also has to be added to the list of trusted root CAs in these specific browsers.
Once this is complete, I can issue certificates for any domain (including that of the cloud provider), that will be accepted as valid.
5.1.3.3 Validating the Provider’s Certificate As discussed earlier, the proxy has to establish and secure a connection to the service provider as well. For this, the proxy does not need to have a certificate, however, it has to validate the provider’s certificate. Extensive care should be taken to perform this validation correctly, otherwise the proxy itself will become vulnerable to man-in-the- middle attacks by other parties.
5.2 Inspecting and Altering Traffic
With the proxy in place, it is now possible to inspect, and if necessary, alter messages that are sent between the parties. Since the protocol above TLS is HTTP, using HTTP libraries would be desirable to avoid having to deal with compression, encoding, as well as interpreting the headers, and separating them from the payload. As seen before, payloads are typically JSON/Javascript, sometimes XML41/SOAP42, for which usually many serialization/deserialization libraries exist for most programming languages. To avoid mistakes, these should be preferred over parsing messages by hand.
Having access to parsed messages should make it easy to identify the ones that need to be altered (these are going to be the ones that were identified in the analysis phase). Since some of the messages may need to be altered in multiple ways, I came up with the idea of building the internals of the proxy using the pipes and filters architecture. In this architecture, messages are emitted from the source into a pipe. This pipe is
41 Extensible Markup Language: a markup language with well-defined rules for encoding documents and messages (data structures).
42 Simple Object Access Protocol: a method of exchanging data structures and invoking remote procedures. Uses XML for messaging.
45 connected to a chain of filters (that are also connected by pipes), while the last pipe is connected to the sink (destination). Each filter may decide to alter the message it receives on its input before emitting the message on its output.
In my example, there need to be two chains of filters: one on the channel from the proxy to the service, and one on the channel from the service to the proxy. The job of each filter will be to recognize a specific protocol message, then alter the contents of the message as needed (i.e. encrypting data in the channel to the provider, then decrypting data in the channel to the client). The source will be a JSON (or XML) unserializer that outputs structured, processable data to the first filter, and the sink will be a JSON (or XML) serializer that converts the modified data structure back to a plaintext that can be sent over HTTP.
Using this model has the added advantage that if the structure of one of the protocol messages changes, only one specific filter has to be rewritten or changed, rather than several components in the system.
5.3 Encrypting/Decrypting Messages
5.3.1 Key Management
A general expectation of such an application would be that the users should be able to access their data no matter which computer of theirs they're using (provided they have the transparent encryption layer installed). Generating a key before the first use would work, but then users would have to make sure they carry the key with them so that they can copy it to each of their computers. To solve this issue, in addition to being able to generate random keys, users should be able to choose to have a key generated for them from a password, for example, using the PBKDF243 algorithm.
5.3.2 Using Format Preserving Encryption
Services typically perform format and range validation on anything that is submitted to the service. For this reason, the naïve idea of encrypting fields with a usual stream cipher, then sending resulting ciphertext to the service will not work, since the raw
43 Password-Based Key Derivation Function 2: a standard function that can be used to generate cryptographic keys from passwords deterministically, with a tuneable computational cost
46 binary data will not pass validation checks. While this could sometimes be worked around by applying base64 encoding to the binary data, this unnecessarily increases the length of the output, making it not preserve length, APIs often impose maximum length restrictions for fields, so this would not always work. This is where format preserving encryption algorithms (will be referred to as FPE from now on) are useful.
An encryption algorithm 퓕 is said to be format preserving if the domain and the range (the 퓜 message space) are the same (with the exception that the algorithm also takes a key parameter 퓚) [21].
퓕: 퓚 ⨯ 퓜→ 퓜 (1) Using such algorithms, it is possible to encrypt data in a way that the ciphertext passes format validations from simple length checks to more complex range or integrity checks.
The first format preserving algorithms with provable security were described by Black and Rogaway [23], who proposed three possible approaches:
• a prefix cipher-based construction that is only effective for small domains as we have to store a lookup table of a size that of the domain,
• a construction based on cycle walking, which does not use a lookup table, but is a recursive function that may take several cycles to complete (as such, its runtime is unpredictable), and
• a construction based on Feistel networks that does not need a lookup table, but may need cycle walking (however, the number of rounds can be limited by tweaking the parameters).
Based on the above, it can be seen that the Feistel network-based algorithms are the best for general use. Multiple implementations exist, typically differing in the number of rounds, the maximum supported range of values, and whether the algorithm takes a tweak44 or not. These algorithms, however, are only capable of transforming integers of a given range to other integers in the same range.
44 Similar to IVs: An extra parameter used during encryption that helps ensure that even if the key and the plaintext are the same, the ciphertext will be different.
47 5.3.2.1 Format-Preserving Encryption for Text To overcome the limitation of having only integers to work with, ranking functions can be used to map the elements of the message space to integers. For simple text fields, one can use a function that maps each letter to its (1-based) position in the alphabet, then encrypt this rank value using the cardinality of the alphabet as the maximum value parameter to the FPE algorithm.
5.3.2.2 Format-Preserving Encryption for Date and Time Similarly, date-and-time type fields can be expressed as the number of seconds that passed since a given reference date. This number, together with a desired maximum offset value can be used as parameters to an FPE algorithm [24].
5.4 Design Summary
In this section, I’ve designed several components of a system, that, when assembled, can be used to perform transparent encryption on a target service. I’ve introduced the issues that arise when designing such a system, then provided solutions to them.
The architectural diagram of the complete system can be seen in Exhibit 1 in the Appendix.
48 6 Implementing a Prototype
With the design phase finished, I moved on to implementing a prototype. As for the programming language, I’ve chosen C# for several reasons. Firstly, it has a great networking support with TLS included. Secondly, well-written JSON libraries are available. In addition, I have approximately ten years of experience with it.
As for the application to be encrypted, I’ve chosen Evernote. Based on the analysis, it has less complicated message formats, which would fit better for a prototype.
6.1 Intercepting traffic
6.1.1 DNS Hijacking
First, I needed to get the browser to connect to my service (to be written later) instead of the real Evernote servers, and I wanted to make sure that all other traffic is not affected. This can be achieved by setting up a proxy DNS server that resolves www.evernote.com (the only domain that is being used by the service) to the local host, 127.0.0.1, while proxying all other DNS queries to the original DNS servers (that return – hopefully – legitimate answers). Instead of using dedicated DNS server software, I opted to write my own proxy service, making use of the open-source Kapetan DNS library [25], which had all the features I needed. Unfortunately, the pre-built version would keep crashing upon startup, so I downloaded and compiled the project from source.
I still needed to have a way of differentiating between requests coming from the proxy service and the external applications. I came up with the idea of using the reserved .local top-level domain [26] for internal requests in a way that if the original hostname was a.example.com, I query for a.example.com._nospoof.local internally.
A part of my implementation can be seen on Figure 6.1.: the list of domain names to be spoofed is read from a file (DomainsToHijack.cfg), then for each domain-IP pair, an A record45 is added to the master zone file. In addition, an entry under the
45 A record: a DNS record that maps a domain name to an IP address
49 _nospoof.local domain is added, to make it possible to resolve these domains internally to the correct IP addresses.
Figure 6.1 – Selective DNS hijacking (C# code extract)
6.1.2 Creating Certificates
I generated a self-signed certificate to be used as the CA certificate, using MakeCert.exe, a certificate generation tool that is part of the WDK46. Afterwards, I proceeded to import the root certificate into Windows’ store of trusted root certificate authorities (Figure 6.2) to prevent security alerts from popping up. The certificate appeared in the list.
Figure 6.2 – The CA certificate in Windows’ Certificate Manager
46 Windows Development Kit: a set of tools and resources that are often needed by developers for Windows-related development
50 Then, I used MakeCert again, this time to generate a certificate for www.evernote.com, using the previous certificate to sign it. Checking it out, Windows said it was valid (Figure 6.3)
Figure 6.3 – The forged certificate is shown as valid
6.1.3 Implementing the Proxy
Once the certificates were ready, I wrote a small application that listens only on 127.0.0.1, on TCP port 443. Whenever a connection is opened, it connects to the actual Evernote website, essentially copying traffic between the two pipes. The application uses the AuthenticateAsClient and AuthenticateAsServer methods as provided by the SslStream class (Figure 6.4). The SslStreamProxy custom class is responsible for copying data between the pipes and invoking the filters. The code is written in a way such that multiple parallel connections from the client can be handled.
51
Figure 6.4 – Connection setup in the proxy (C# code extract)
6.2 Inspecting and Altering Traffic
Identifying interesting traffic turned out to be easy enough. For client-to-server connections, all that has to be done is checking for the presence of the interesting API function names (from section 4.1.2) and their surroundings. In case of a match, the sensitive contents of the message are extracted and sent to the crypto module for encryption. Even though there is no parser for the GWT-RPC protocol used here, the messages can be processed by splitting by the delimiting ‘|’ character. Any pipe characters that would appear in the message are escaped to \! by the framework, so messages are guaranteed not to have extra pipe characters. An example can be seen on Figure 6.5.
52
Figure 6.5 – A filter, proxy-to-server side (C# code extract)
Server-to-client messages are more difficult to recognize, but since each message has something unique about it, element count and known static values at fixed indices can be used to distinguish among them. An example can be seen on Figure 6.6.
Figure 6.6 – A filter, server-to-proxy side (C# code extract)
For each protocol message that was identified earlier, a filter like the one above was created.
53
6.3 Encrypting/Decrypting Messages
6.3.1 Key Management
I wrote a simple application that can use the built-in cryptographically secure random number generator class RNGCryptoServiceProvider to generate a random key. It can also generate a key using PBKDF2 from a password entered by the user (implemented in the also built-in Rfc2898DeriveBytes class) (Figure 6.7).
Figure 6.7 – Key generation using PBKDF2 (C# code extract)
6.3.2 Initialization Vectors
While somebody having two notes of the exact same title or contents is rare, it is a possibility. Without IVs, two messages having the same title or contents would result in the same ciphertext (under the same key), which would leak information to the attackers. My idea here was to use the notes' GUIDs as the IVs since these appear in each message that has to be encrypted or decrypted.
6.3.3 Format preserving encryption
In the case of Evernote, there are two types of data to encrypt: text (titles and note bodies) and dates (reminders). These types are enforced by Evernote, therefore it is not possible to pass binary values (such as the direct outputs of ciphers) to the API. Format preserving algorithms are invaluable here as they can transform a plaintext of a format
54 and range to a ciphertext of the same format and range, thereby making sure that the ciphertext passes the validation filters.
For the general part of the solution, I used an implementation of Bellare & Rogaway's Feistel network based encryption block (FE1, from the DotFPE library [27]). In their paper [22], they prove that the algorithm is just as strong as the underlying key generation algorithm that is used to feed it.
As for generating the key stream, I use an AES cipher in CTR mode with the 256- bit key from section 6.3.1 as the key and the IV from section 6.3.2 as the IV.
6.3.3.1 Format-Preserving Encryption for Text The FE1 algorithm cannot be used directly on strings, but it can be used as input to a rank-then-encipher approach-based algorithm. I implemented such an algorithm for strings consisting of printable ASCII characters only (Figure 6.8, only the encryption is shown).
Figure 6.8 – FPE implementation for ASCII text (C# code extract)
55 The algorithm is rather slow and inefficient as it encrypts/decrypts characters one by one (this could be optimized), but for the purposes of this prototype, it is good enough.
6.3.3.2 Format-Preserving Encryption for Date and Time Evernote allows dates between 1000-01-01 and 9999-12-31, and dates are stored in Unix timestamps with millisecond precision [28]. For this purpose, the FE1 encryptor can be used without modifications if the modulus is set to the difference of 9999-12-31 and 1000-01-01, and if 1000-01-01 is subtracted from every date before encryption and added again after decryption. However, I chose to implement the algorithm described in section 5.3.2.2, and used that instead. The implementation is shown on Figure 6.9.
Figure 6.9 – FPE implementation for DateTime (C# code extract)
56 7 Testing the Prototype
Once the prototype was mostly finished, the testing phase followed. For testing, I used three different approaches: smoke testing, unit testing, and integration testing. Being a prototype, I did not expect it to be free of bugs, however, I wanted the basic features to work in order to demonstrate the concept of transparent encryption.
7.1 Smoke Testing
Smoke testing is the act of manually testing a piece of software to see if it works at all. Smoke tests are generally not very thorough, but are quick to perform, and usually tell you quick if there are bigger issues. Every single component was smoke tested, several times during development, and as least once afterwards.
7.1.1 Smoke Testing the DNS Hijacking Component
The DNS hijacker was the first component to be completed, and was the easiest to test. I set the virtual machine I was testing in to use 127.0.0.1 as the DNS server, then used the command line tool nslookup to see if it was working. As it can be see on Figure 7.1, the test was successful. Google.com was resolved to its actual IP address, but evernote.com was hijacked to point to 127.0.0.1. Success.
Figure 7.1 – Testing the DNS hijacker
57 7.1.2 Smoke Testing the TLS Proxy
Having a working DNS hijacker, it was now possible to test if the TLS proxy was working, simply by connecting to evernote.com from a browser.
First, I tried using Firefox. The site didn't load. Firefox displayed a message that the site was attempting to use an invalid security certificate (Figure 7.2). I should have expected that, I even mentioned that Firefox uses its own list of trusted CAs, so even though I added the CA to the list of trusted ones in Windows, it didn’t help.
Figure 7.2 – Firefox displaying a warning for the MitM certificate
Next, I tried Chrome. The site loads, with no warnings appearing (Figure 7.3).
Figure 7.3 – Chrome loads Evernote and does not show warnings
58 Clicking the green padlock, it is possible to check which certificate is being seen by Chrome. The one I’d generated earlier appeared (Figure 7.4). Success.
Figure 7.4 – Verifying that Chrome sees my certificate
In addition, I loaded Google and two other TLS-protected websites, just to verify that nothing else is being hijacked. They were not hijacked and all appeared to be using their own certificates.
7.1.3 Smoke Testing the FPE Module
I wrote two simple test cases in a console window.
• The first one displays a string, then encrypts it, displays it in its encrypted form, then decrypts it, and finally, it shows the result.
• The second one does the same, except with a variable of type DateTime instead of a string.
59
Figure 7.5 – Smoke testing the FPE encryption algorithms
Both tests showed that the input was encrypted, and the formats did not change, as expected. The ciphertext could also be decrypted, and it was decrypted to the original plaintext. Success.
7.2 Unit Testing
Unit tests are tests that focus on a small section of a module, usually a single function. These are more detailed than smoke tests, and can be run automatically before a release. If a modification to the code accidentally breaks a previously working functionality, it can be detected this way. Unit tests have three phases:
1) Arrange: the environment is prepared for the test. Instances of objects that will be tested are instantiated, variables are set, etc.
2) Act: the function or the sequence of functions to be tested is called.
3) Assert: checking whether the outcome meets the expectations and that there were no side effects.
7.2.1 Unit Testing the Filters
I wrote four unit tests for each filter: one checks that a matching outbound message is processed correctly, one checks that the matching inbound message is processed correctly, and the other two tests whether non-matching messages in either direction are processed (they should not be). One of the four tests for the GetHtmlNoteContentFilter class can be seen on Figure 7.6.
60
Figure 7.6 – Unit testing a filter (C# code extract)
Each filter was tested individually. Some issues were found and corrected.
7.2.2 Unit Testing the FPE Module
I wrote two unit tests for the text encryption functions of the FPE module, and another two for the date encryption functions. The first tests whether an encryption- decryption sequence results in the same plaintext, while the second tests test whether the format of the input is preserved. The tests succeeded, as shown on Figure 7.7.
Figure 7.7 – Passing unit tests for the FPE module
7.3 Integration Testing
Integration testing aims to test the product as a whole.
As a final test, I signed in to Evernote with my test account and without the DNS hijacker active. I deleted all my notes, then signed out. Afterwards, I switched DNS servers, activated the hijacker and the TLS proxy, logged in again, and added a note with the title An example note and the contents Something goes here.. The note appeared correctly. I then logged in with my Chrome as well, where I could see the same thing.
61
Figure 7.8 – The encrypted note appears unencrypted through the proxy
Finally, I logged in from the host machine that did not have its DNS server changed. There, I could still see my note, except all the text appeared garbled.
Figure 7.9 – The encrypted note appears garbled to attackers
I also tried adding a reminder for the note later, but it seems that the UI can't actually handle the date range that was shown in the API documentation. I entered the current date, which got encrypted to the year of 1502 (correctly), but the UI displayed it as 2002. It seems like only the last two digits are considered. Nevertheless, the tests succeeded.
62 8 Conclusion
Cloud-based services are popular and will stay popular in the near future. They bring with themselves several risks from a security standpoint that are often underestimated. The aim of this work was to find and elaborate a method to increase the security of cloud-based applications, even in cases where the cloud service provider cannot be trusted at all.
Within this project, I researched related publications and similar software, then enumerated services where transparent encryption could increase the level of security. Then, I selected six services for analysis, two of which I analysed in great detail: I identified the relevant outgoing and incoming messages and their structure. Afterwards, I designed a system that enables transparent encryption using DNS hijacking, performing man-in-the-middle attacks against TLS sessions, then altering the previously identified messages as needed. Format-preserving encryption algorithms are used to make sure that the encrypted contents pass server-side validation. Then, I implemented a system based on the previous design that protects user content on the note-taking site Evernote. Finally, I designed and ran smoke, unit, and integration tests, with success.
63 9 Further Considerations
9.1 Possible Threats
This subsection lists reasons why it may be undesirable or inconvenient to employ the approach presented in this thesis.
9.1.1 Ever-Changing APIs
For the system to work properly, the filters have to be able to recognize the message types and data structures. If the cloud service provider keeps making frequent changes to the API, the maintainers of the encryption layer will also have to make frequent changes to the code of the filters. In the meantime, their data might be inaccessible to the users, and unencrypted information might leak to the provider. The latter issue may be worked around by letting through only known message types.
9.1.2 New Security Measures
New security measures such as certificate pinning or HSTS47 might make it impossible to perform the MitM attack. Disabling these might require the application to be patched, at which point the method is no longer transparent as the client had to be altered.
9.2 Plans for Improvement
This subsection lists ideas how this project could be continued, how the software could be improved.
9.2.1 Supporting Multiple Users
Currently, the prototype is… well, a prototype. It has its shortcomings, for example, it supports only one user at a time per computer. If two or more users use the service simultaneously, their notes will be encrypted with the same key. Multi-user support might be interesting in small office – home office environments.
47 Hypertext Strict Transport Security: a security mechanism that can be used to enforce HTTPS and make certificate checks stricter.
64 9.2.2 Usage in Enterprise Environments
As it is now, the software is designed to run on the users’ computers. It could be adapted to enterprise scenarios in which it could be used as a single central proxy, protecting everyone’s data simultaneously. It could be improved to detect individual user sessions and protect them by different keys.
9.2.3 More Services
In its current form, the prototype only supports one service, Evernote. It could be adapted to support more services, possibly in a more modular setup.
9.2.4 Linux Compatibility
While in theory, .NET can be run on Linux, the project was not built with Linux compatibility in mind. Certain or all features may not work as expected on Linux.
9.2.5 User Experience
Currently, the prototype consists of separate modules that need to be configured, started, and managed separately. A central UI could be written to make it easier to use for inexperienced users. In addition, a user’s manual could be written.
65 References
[1] Sid Nag, Lai-ling Lam et al: Forecast: Public Cloud Services, Worldwide, 2014- 2020, 2Q16 Update – Gartner, G00310051, 29 June 2016
[2] Kim Weins, RightScale, Inc.: Cloud Computing Trends: 2017 State of the Cloud Survey, https://www.rightscale.com/blog/cloud-industry-insights/cloud- computing-trends-2017-state-cloud-survey (published: 15 February, 2017)
[3] Nate Lord, Digital Guardian: The History of Data Breaches, https://digitalguardian.com/blog/history-data-breaches (published: 17 July 2017)
[4] Yang Li, Li Yeli, Zheng Liangbin: Transparent encryption based on network file system filtering driver, 2011 International Conference on Electric Information and Control Engineering (ICEICE), 2011
[5] Xiang He, Yihong Long, Liheng Zheng: A Transparent File Encryption Scheme Based on FUSE, 12th International Conference on Computational Intelligence and Security (CIS), 2016, pp. 642-645
[6] Xingbang Tian, Baohua Huang, Min Wu: A transparent middleware for encrypting data in MongoDB, 2014 IEEE Workshop on Electronics, Computer and Applications, 2014, pp. 906-909
[7] Thanh Cuong Nguyen, Wenfeng Shen, Jiwei Jiang, Weimin Xu: A Novel Data Encryption in HDFS, 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, 2013, pp. 2183-2187
[8] Madhvaraj M. Shetty, Manjaiah D.H.: Data security in Hadoop distributed file system, International Conference on Emerging Technological Trends (ICETT), 2016
[9] Xiaowei Li, Hongxiang Sun, Qiaoyan Wen: An approach for secure- communication between XEN virtual machines, 2012 IEEE 2nd International Conference on Cloud Computing and Intelligent Systems (CCIS), 2012, pp. 283- 286
[10] Yubin Xia, Yutao Liu, Haibo Chen: Architecture support for guest-transparent VM protection from untrusted hypervisor and physical attacks, 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA2013), 2013
[11] Amitabh Saxena, Vikrant Kaulgud, Vibhu Sharma: Application Layer Encryption for Cloud, 2015 Asia-Pacific Software Engineering Conference (APSEC), 2015, pp. 377-384
[12] Mamadou H. Diallo, Bijit Hore, Ee-Chien Chang, Sharad Mehrotra, Nalini Venkatasubramanian: CloudProtect: Managing Data Privacy in Cloud
66 Applications, 2012 IEEE 5th International Conference on Cloud Computing (CLOUD), 2012, pp. 303-310
[13] Jason Newport, Basie von Solms: A secure cloud storage system for small and medium enterprises, 2017 IST-Africa Week Conference (IST-Africa), 2017, pp. 1-6
[14] Secomba GmbH: Boxcryptor - Supported Providers, https://www.boxcryptor.com/en/providers/ (retrieved: 29 September, 2017)
[15] Secomba GmbH: Boxcryptor – Technical Overview, https://www.boxcryptor.com/en/technical-overview/ (retrieved: 29 September, 2017)
[16] Sid Shetye: cipherdocs, https://github.com/sidshetye/cipherdocs (published: 14 September, 2017)
[17] Jiaqiang Xu and other contributors: SeaFile, https://github.com/haiwen/seafile (retrieved 30 September, 2017)
[18] Brian Slesinsky: The GWT-RPC wire protocol, https://docs.google.com/document/d/1eG0YocsYYbNAtivkLtcaiEE5IOF5u4LUol 8-LL0TIKU/ (published: 10 December, 2012)
[19] Google Inc: com.google.gwt.user.server.Base64Utils.java, http://grepcode.com/file/repo1.maven.org/maven2/com.google.web.bindery/reque stfactory-server/2.4.0/com/google/gwt/user/server/Base64Utils.java (retrieved: 8 October, 2017)
[20] Evernote Corporation: Evernote Developer Documentation, https://dev.evernote.com/doc/ (retrieved: 2 December, 2017)
[21] Steven J Murdoch, Ross Anderson: Access denied: The practice and policy of global internet filtering, The MIT Press, ISBN 9780262251440, 2008, pp. 58-63
[22] John Black, Phillip Rogaway: Ciphers with arbitrary finite domains, RSA Conference, Cryptographers’ Track, 2002, pp. 114-130
[23] Mihir Bellare, Thomas Ristenpart, Phillip Rogaway, Till Stegers: Format- Preserving Encryption, International Workshop on Selected Areas in Cryptography, 2009, pp. 295-312
[24] Zheli Liu, Chunfu Jia, Jingwei Li, Xiaochun Cheng: Format-preserving encryption for DateTime, 2010 IEEE International Conference on Intelligent Computing and Intelligent Systems (ICIS), 2010, pp. 201-205
[25] Mirza Kapetanovic and other contributors : Kapetan – a DNS library written in C#, https://github.com/kapetan/dns (retrieved: 7 December, 2016 )
[26] S. Chesire, M. Krochmal: Multicast DNS, Internet Engineering Task Force (IETF) RFC 6762, 2013, ISSN: 2070-1721
67 [27] Unknown author(s): DotFPE - A free format-preserving encryption implementation for .net, https://dotfpe.codeplex.com/ (published: 7 October 2012)
[28] Evernote Corporation: Thrift module: Types, Typedef: Timestamp, https://dev.evernote.com/doc/reference/Types.html#Typedef_Timestamp (retrieved: 7 December, 2016)
68 Appendix
A. Table of Abbreviations
Abbrev. Definition Page # AES Advanced Encryption Standard 15 AJAX Asynchronous JavaScript and XML 21 API Application Programming Interface 19 CBC Cipher Block Chaining 15 DNS Domain Name System 38 DOM Document Object Model 19 FPE Format-Preserving Encryption 45 GUID Globally Unique Identifier 23 GWT Google Web Toolkit 22 HDFS Hadoop Distributed File System 14 HSTS Hypertext Strict Transport Security 63 HTML Hypertext Markup Language 24 HTTP Hypertext Transfer Protocol 14 HTTPS Secure Hypertext Transfer Protocol 42 IP Internet Protocol 29 JSON JavaScript Object Notation 21 MIME Multipurpose Internet Mail Extensions 21 MitM Man-in-the-middle 40 OSI Open Systems Interconnection 12 PBKDF2 Password-Based Key Derivation Function 2 45 PGP Pretty Good Privacy 16 PKI Public Key Infrastructure 43 REST Representational State Transfer 21 ROM Read-Only Memory 29 RPC Remote Procedure Call 22 RSA Rivest-Shamir-Adleman 15 SNI Server Name Indication 29 SOAP Simple Object Access Protocol 44
69 SSL Secure Sockets Layer 20 TCP Transmission Control Protocol 20 TLS Transport Layer Security 20 UI User Interface 38 URL Uniform Resource Locator 31 USN Update Sequence Number 25 WDK Windows Development Kit 49 XML Extensible Markup Language 44
70 B. Table of Figures
Figure 4.1 – Evernote's messages in Firefox's Developer Tools ...... 22 Figure 4.2 – SNI extension used by the Google Calendar mobile app ...... 31 Figure 4.3 – Data structure of a Google Calendar event ...... 34 Figure 4.4 – Google Calendar dynamically loading events (request) ...... 35 Figure 4.5 – Google Calendar creating an event (request) ...... 36 Figure 4.6 – Google Calendar editing an event (request) ...... 37 Figure 4.7 – Google Calendar deleting an event (request) ...... 38 Figure 4.8 – Google Calendar deleting an event (response) ...... 38 Figure 4.9 – A generalized model for the message flows of cloud services ...... 41 Figure 5.1 – Connections to and from the proxy ...... 43 Figure 6.1 – Selective DNS hijacking (C# code extract) ...... 50 Figure 6.2 – The CA certificate in Windows’ Certificate Manager ...... 50 Figure 6.3 – The forged certificate is shown as valid ...... 51 Figure 6.4 – Connection setup in the proxy (C# code extract)...... 52 Figure 6.5 – A filter, proxy-to-server side (C# code extract) ...... 53 Figure 6.6 – A filter, server-to-proxy side (C# code extract) ...... 53 Figure 6.7 – Key generation using PBKDF2 (C# code extract) ...... 54 Figure 6.8 – FPE implementation for ASCII text (C# code extract) ...... 55 Figure 6.9 – FPE implementation for DateTime (C# code extract) ...... 56 Figure 7.1 – Testing the DNS hijacker ...... 57 Figure 7.2 – Firefox displaying a warning for the MitM certificate ...... 58 Figure 7.3 – Chrome loads Evernote and does not show warnings ...... 58 Figure 7.4 – Verifying that Chrome sees my certificate ...... 59 Figure 7.5 – Smoke testing the FPE encryption algorithms...... 60 Figure 7.6 – Unit testing a filter (C# code extract) ...... 61 Figure 7.7 – Passing unit tests for the FPE module ...... 61 Figure 7.8 – The encrypted note appears unencrypted through the proxy ...... 62 Figure 7.9 – The encrypted note appears garbled to attackers ...... 62
C. Table of Exhibits
Exhibit 1 – Architectural diagram of the designed system ...... 72
71 D. Exhibits
Exhibit 1 – Architectural diagram of the designed system