Exploiting Facebook, Flickr, and Picasa : Utilizing Photo Sharing

University of Konstanz Department of Computer and Information Science Distributed Systems Group

Master Thesis

Exploiting Facebook, Flickr, and Picasa: Utilizing Photo Sharing Websites as Cloud Storage Backends

Scientiﬁc Thesis in Fulﬁllment of the Requirements for the Degree of Master of Science (M.Sc.)

Author: Wolfgang Miller (01/612437)

First Assessor: Professor Dr. Marcel Waldvogel Second Assessor: Professor Dr. Dietmar Saupe Advisor: Sebastian Graf, M.Sc.

July 2013 Wolfgang Miller

Exploiting Facebook, Flickr, and Picasa: Utilizing Photo Sharing Websites as Cloud Storage Backends, c July 2013

Abstract

A major application of cloud computing is to supply storage that is often advertised by providers as being reliable, available, functional, and cost efﬁcient because it relies on a massive infrastructure available in the cloud. Photo sharing websites rely on similar infrastructures to handle the amount of uploaded image data; however, in contrast to conventional cloud storage providers, they offer free storage capacities for images. Due to the fact that images can be used as data containers to store arbitrary information, it is possible to exploit these services as free cloud storage. In this thesis we describe our approach for the development of a fully functional photosharing website driven cloud storage system based on twelve encoding approaches. These encoding approaches allow for adjusting our system to the requirements of a speciﬁc photo sharing website and to gain the best trade-off between the actual data ratio of an image and its robustness against image compression. The system’s performance has been tested with the three major photo sharing websites Facebook, Flickr, and Picasa Web Albums, and has been compared with Amazon’s professional cloud storage service AWS S3. The results show that the system can be an alternative in scenarios in which the overall performance is not that important, such as backup systems. Our developed photosharing website driven cloud storage system is accessible as an open source and implemented as part of the widely used jClouds framework.

III IV

IV Acknowledgements

My special thanks goes to my family, my parents Willi and Anneliese Miller, my brother Sebastian Miller, and my grandmother Adelheid Rothmund, with whom I lived during the ﬁrst years of studying in Konstanz. They made it possible for me to study by giving me always great support.

I would like to thank Professor Dr. Marcel Waldvogel and the members of the DiSy working group for the excellent support. I would also like to thank Professor Dr. Dietmar Saupe for taking the time to assess this thesis.

Special thanks goes to Sebastian Graf. Throughout my studying I became to know and appreciate Sebastian in several roles: As a fellow student, a lecturer, a colleague, the su- pervisor of this thesis, and most importantly as a valuable friend. Thanks Sebastian for everything and good luck for your little family!

During my time in Konstanz, I met many nice people whom I studied with, and with whom I spend very good times. A special thanks to all my friends, who made me feel at home in Konstanz. Particularly I want to mention my master clique: Anja Fauth, Laura Lorenz, Volker Rehberg, Markus Hankh, Alexander Walter, Benno Geißelmann, Jochen Budzinski, Lin Shao, and Roland Jungnickel.

I also want to give special thanks to my very good friends Johanna Harde and Andreas Kraft, who gave me a great support during the writing of this thesis.

Last but not least, I would like to especially thank Anna Dowden-Williams for her more than valuable input in writing this thesis in English. Thank you Anna for all the time you invested in helping me and for your friendship.

V VI

VI Contents

1. Introduction1 1.1. Motivation ...... 1 1.2. Contribution of the Thesis ...... 2 1.3. Key Points of the Thesis ...... 3 1.4. Related Work ...... 4 1.4.1. Image Steganography ...... 5 1.4.2. 2D-Codes ...... 6 1.4.3. Projects with similar Orientation ...... 8 1.4.4. Conclusion and Comparison ...... 9

2. Encoding Approaches 11 2.1. Intention ...... 12 2.2. First Encoding ...... 13 2.2.1. Encoding Workﬂow ...... 14 2.2.2. Decoding Workﬂow ...... 15 2.2.3. Results ...... 16 2.3. Advanced Encoding ...... 17 2.3.1. Basics on Numeral Systems ...... 18 2.3.2. Relationship between Radix γ and Data Ratio ...... 18 2.3.3. Results ...... 20 2.4. Single-Layer-Approaches ...... 21 2.4.1. Encoding ...... 22 2.4.2. Decoding ...... 26 2.4.3. Results ...... 31 2.5. Multi-Layer-Approaches ...... 32 2.5.1. Encoding ...... 34 2.5.2. Decoding ...... 39

VII VIII Contents

2.5.3. Results ...... 39 2.6. Images and Colors ...... 40 2.6.1. Types of Images ...... 40 2.6.2. Predeﬁned Colors Single-Layer-Approaches ...... 42 2.6.3. Generated Colors Multi-Layer-Approaches ...... 46 2.7. Summary ...... 47 2.7.1. Benchmarks ...... 48 2.7.2. Results ...... 51

3. Implementation of our Photosharing Website driven Cloud Storage System 53 3.1. Photosharing Website driven Cloud Storage System ...... 53 3.1.1. Error Correcting Code (Reed Solomon) ...... 54 3.1.2. Areas and Sectors of generated Images ...... 56 3.1.3. Integration in jClouds and System Workﬂows ...... 59 3.2. Facebook ...... 62 3.2.1. Features ...... 63 3.2.2. Implementation ...... 63 3.2.3. Benchmarks ...... 64 3.2.4. Results ...... 67 3.3. Flickr ...... 67 3.3.1. Features ...... 68 3.3.2. Implementation ...... 69 3.3.3. Benchmarks ...... 69 3.3.4. Results ...... 74 3.4. Picasa Web Albums ...... 74 3.4.1. Features ...... 74 3.4.2. Implementation ...... 75 3.4.3. Benchmarks ...... 75 3.4.4. Results ...... 78 3.5. Comparison Facebook, Flickr, and Picasa Web Albums ...... 79 3.5.1. Feature Comparison ...... 79 3.5.2. Overhead Comparison ...... 81 3.5.3. Performance Comparison ...... 82 3.5.4. Results ...... 84

4. Conclusion 87 4.1. Results ...... 87

VIII Contents IX

4.2. Future Work ...... 88 4.2.1. Benchmarks with larger Data and Flickr’s new Service ...... 88 4.2.2. Enhance Images with Visual Information ...... 89 4.2.3. Automatic Determination of Best Encoding Parameters ...... 90 4.2.4. Step from Images to Videos ...... 90

Appendices 91

A. Random Input Bytes 93

B. Encoding Approaches Naming Convention 95

Bibliography 97

IX X Contents

X 1 Introduction

1.1. Motivation

In the last years, cloud computing has become one of the most influencing and widely discussed trends in the IT sector. Big players like Apple, Amazon, Google, and Microsoft developed new cloud services and pushed the idea to outsource to third party providers launching marketing campaigns to gain the attention of the end consumer market. Con- sequently, today the term “cloud computing“ is well-known not only by computer enthusiasts. The pull-effect of the cloud trend also brought up new business-concepts and start-up companies with fresh ideas. The credo seems to be “everything is better with the cloud“. Through this background a huge variation of services around cloud-computing have mushroomed. From services for companies to exclude a lot of their internal IT- infrastructure to trustful third parties, e.g. the local provider Adlon Datenverarbeitung Sys- tems GmbH [Adl13], to collaborative file storage for end-users, e.g. like Dropbox [Dro13]. Even gaming, whereas the complex graphics of video games are rendered in the cloud and users stream the precalculated scenes via client. A fitting example would therefore be Sony’s cloud-based gaming service Gaikai [com13].

Besides additional computing power, the major application of cloud services is to supply storage. This storage is often advertised as being reliable, available, functional, and cost efﬁcient. Provider offers for end-users extend from services like the mentioned Dropbox or Apple’s iCloud [App13] to external storage, data backup, synchronizing and sharing general data, to professional big data NoSQL storage like Amazon’s AWS S3 [Ama13] or Microsoft’s Azure [Mic13]. With the cloud storage another type of service exploited the availability of large storage areas, namely: photo sharing websites.

Photo sharing websites are storage providers specialized to store and share photos. How-

1 2 1.2. Contribution of the Thesis ever, these sites rely on massive server infrastructures that utilize the same reliable storage techniques like other cloud storage services, they do not require fees. Examples for such services are Facebook [Fac13a], Yahoo’s Flickr [Fli13a], and Picasa Web Albums [Goo13a] from Google.

Taking these facts about photo sharing websites into account, we questioned whether it is possible to exploit these services and use them as free cloud storage for any kind of data. Whereby the major issue was ﬁnding a way of how to negotiate the constraint of only being able to upload photos. In this thesis the development of a fully functional photosharing website driven cloud storage system will be shown. The last part of the thesis will provide a closer look at the available providers implemented in our system, namely Facebook, Flickr, and Picasa Web Albums, and we will analyze how well they perform in our cloud storage scenario.

Flickr has shown that it is worth thinking about utilizing photo sharing websites as cloud storage. On May 20th, 2013 Flickr completely renewed its service offer [Fli13c]. Since that day Flickr allows to upload and store image data up to a size of 1TB for free instead of their previous monthly limit of 300MB. Another important change is that they now allow to access the original uploaded images and no longer apply JPEG-compression, which, as we will see, makes it a lot easier to store data. Since Flickr’s restart took place during the time this thesis was written, we only refer to features before the restart. Nevertheless our system supports all tools to also use the new Flickr service as well.

1.2. Contribution of the Thesis

In a nutshell, this thesis introduces a way of utilizing photo sharing websites as cloud storage backends for no-sql stores only. It answers the question of how offers of certain image providers can be exploited to store any kind of data without charge. The contribution covers the following points:

Twelve different encoding approaches, allowing to easily adapt our system to arbi- • trary photo sharing website providers and their handling of uploaded images (e.g. JPEG-compression).

Details about the implementation of our fully functional photosharing website driven • cloud storage system, e.g. with system workﬂows.

2 1.3. Key Points of the Thesis 3

Analysis of the in this system available photo sharing website providers Facebook, • Flickr, and Picasa Web Albums and their performance as cloud storage provider.

1.3. Key Points of the Thesis

The goal of this thesis is to show a way of how to make photo sharing websites usable as cloud storage backends imitating a blobstore. This necessitates finding a solution to the question of how to store arbitrary buckets of data on specific provider’s storage and also how to work the other way around and to regain access to the data again. Since we are dealing with photo sharing websites this data has to be somehow exchanged through images. The following figure summarizes how data should flow in such a system:

Image upload encode

00100101 110110111 Data Cycle Photo Storing Website 10101010 1000101

decode download Image

Figure 1.1.: Data Cycle of a Photo Sharing Website driven Cloud Storage System

On the left hand side we have buckets ﬁlled with data on a local machine symbolized by binary strings, whereas on the right hand side we see the cloud storage of a photo sharing website represented by a cylinder. Free data interchange between these two major points is crucial to support a fully functional storage. Let us take a closer look on the cycle and how the tasks of storing and restoring data work:

Store Data: We start on the left hand side with data that should be stored. This data ﬁrst has to be encoded into an image. After that step the image with the embedded

3 4 1.4. Related Work

data is uploaded and hosted on the photo sharing website.

Restore Data: We start on the right hand side. First the image must be downloaded from the photo sharing website. Next the data that is embedded in the image has to be decoded. At the end the data previously stored is regained.

In the presented cycle we see four traversal steps that are crucial for the data ﬂow between local host and photo sharing website: encoding & decoding as well as uploading & downloading. The following two chapters of this thesis can be roughly divided by these steps:

Encoding & Decoding: In Chapter2 we will introduce twelve different encoding ⇒ approaches and give a detailed look on how the encoding of bytes into images works and vice versa. Each of these approaches has its own set of features like data capacity and utilized image colors. This amount of approaches is needed, because as we will see, every photo sharing website treats uploaded images differently. Some of the vendors apply image compression and force the use of encodings with lower data ratio to preserve the data of the images. As a result not every approach is applicable for every photo sharing website.

Uploading & Downloading: Chapter3 focuses on the application of the before dis- ⇒ cussed encoding approaches. It will show details about the implementation of our photosharing website driven cloud storage system, e.g. how the general upload and download workﬂows look like. Afterwards we will introduce the three in our system available photo sharing websites Facebook, Flickr, and Picasa Web Albums. We will furthermore discuss every vendor in detail and draw a comparison between the three main market players of photo sharing websites.

1.4. Related Work

There are three main ﬁelds the related work can be assigned to: image-steganography, 2D-Codes, and projects with similar orientation.

4 1.4. Related Work 5

1.4.1. Image Steganography

A topic related to the idea of storing data in images is image-steganography - the art of hiding additional data in an image without being visually noticed.

JPEG Compression Immune Steganography Using Wavelet Transform

A work of the image-steganography-ﬁeld that especially attracted our attention is [XSSL04] and its JPEG-compression immune approach to store data in images. Due to some photo sharing websites applying JPEG-compression on uploaded images, compression is a serious threat to the stored data. In [XSSL04] a way of storing bits in an image in such a manner that the hidden data in the image cannot be destroyed by JPEG-compression, not even with the highest compression settings, is introduced. The trade-off creating such a compression immune image is a very poor data ratio. An image container using this approach can only store poor 0.00025 Bpp (bytes per pixel). That is the reason why this approach is not applicable in our photo sharing website driven cloud storage scenario where we want to deal with data of several megabytes. Also we do not need to hide our data behind an image. As we will see later, our encoding approaches have a data ratio between at least 0.125 Bpp up to 3 Bpp, which, when compared, is an increase of the data ratio between a factor of 500 up to a factor of 12, 000.

A Web based Covert File System

An idea for a covert file system called “CovertFS” is introduced in [BKI07]. This file system should be built on public available media hostings and sharing services and allows its users to store their files hidden inside of media without being noticed by the providers. As an example for such a media service they refer to the photo sharing website Flickr. Their goal is to utilize image-steganography to add hidden file system information in images and upload these images to Flickr. Thereby the data ratio of the utilized image- steganography-technique is only specified vaguely, e.g. about 10% of an image’s actual size should be usable as storage, for example a 40KB image can contain 4KB actual data.

Even though [BKI07] wants to exploit photo sharing websites as storage, their purpose is completely different. They want to hide information and secure it as much as possible, even if this comes with a great lack of performance, e.g. they want to map every 4KB

5 6 1.4. Related Work disk block of their ﬁle system to a separate image. This is very different from our cloud storage scenario, which aims at providing the best possible data ratio and performance.

It has to be kept in mind that this work is based on theoretical thoughts. A practical application of the approach would possibly enhance some of the theories.

1.4.2. 2D-Codes

A set of techniques that share the similar use-case of expressing data with blocks of pixels, respectively images, are 2D-Codes. These codes are designed to embed data in images in such a way that it can be recognized and processed by appropriate reading equipment, e.g. mobile phones with cameras.

QR-Codes and DataMatrix-Codes

Two popular 2D-Code-techniques are the nowadays in context of mobile phone widely used QR-Codes [QRc13b] (“Qick Response” Codes) and the DataMatrix-Codes [fS06]. Fig- ure 1.2 shows an example output for both codes, storing the same array of 18 random bytes (this array is referenced in AppendixA and will be used throughout the thesis to make the output images of different approaches visually comparable):

(a) QR-Code (b) DataMatrix-Code

Figure 1.2.: Example Output 2D-Codes: QR-Code and DataMatrix-Code

QR-Codes exist in several predeﬁned versions with respect to their resolution / used mod- ules and capacity. At a maximum it can have a resolution of 177 177 pixels (QR-Code version 40). Combined with the minimal available error-correction× code of level “L” a QR- Code can store up to 23,648 bits at maximum [QRc13a]. This corresponds to a data ratio

6 1.4. Related Work 7 at best of about 0.09 Bpp, iff the QR-Code is used to its capacity. That is an important point since, if the∼ input data reaches a deﬁned threshold, the next greater version of QR- Codes has to be utilized to store it. This can result in an extremely unfavorable data ratio, for example the in Figure 1.2a shown QR-Code. 17 bytes would have ﬁt into a version 1 QR-Code with a resolution of 21 21 pixels, but the threshold is reached with 18 bytes and a version 2 QR-Code has to be× used that has a resolution of 25 25 pixels. This code would be capable of storing 32 bytes, but is used to only store 18× bytes, meaning that the data ratio in this case is poor with about 0.03 Bpp - presenting only one third of the best ratio. ∼

For DataMatrix-Codes it is similar, there are also predeﬁned DataMatrix-Code versions with respect to capacity and resolution. At best they can have a maximum size of 144 144 pixels and store in team play with ECC 200 up to 12,448 bits. That equals 1,556× bytes and a data ratio of about 0.075 Bpp. ∼ Summing up, the 2D-Codes have been a good inspiration for our own encoding with respect to utilizing pixels to embed data in images, but the inconstant data ratio and the limitation for predeﬁned image container sizes prompted us to search for a better own solution. Moreover, in our photo sharing website scenario we do not need to take a detour over reading equipment like cameras, and can therefore spare a lot of error-correction code.

Quick Layered Response (QLR) Codes

An enhanced 3D-Code version of QR-Codes is introduced in [DD12]. Their so called “QLR- Codes” improve the capacity of QR-Codes by a factor of 3 through utilizing RGB images and their layers as additional dimension. To achieve this capacity win, the input data is split into three different equal sized QR-Codes. Every one of these three QR-Codes represents an RGB layer. Whereby the ﬁrst QR-Code encodes its data in red pixels, the second in green pixels and the third in blue pixels. Finally, the three layers are merged together into an RGB image that represents the resulting QLR-Code and contains all three sub- QR-Codes. This method, gaining additional data capacity, is also applied by some of our encoding approaches and will be discussed in detail in Chapter 2.5.

7 8 1.4. Related Work

Unsynchronized 4D Barcodes (Coding and Decoding Time-Multiplexed 2D Colorcodes)

Another evolution of 2D-Codes is presented in [LB07]. The main idea behind this approach is to improve the capacity of a regular cellphone-readable 2D-Codes by adding the dimensions, color, and time to the regular used dimensions width and height. The DataMatrix-Code is here used as originator. The authors argue that consumer-cameras have a clear limitation, which means that increasing the resolution of the barcodes to gain more data volume, is not an option. To achieve this goal they propose animated GIF-images [W3C13a] that hold an array of 3D DataMatrix-Codes. The GIF-images display the containing barcodes in a row in an endless loop. To access the information stored in those GIF-images, they have to be captured via a mobile camera (e.g. of a cellphone) and afterwards decoded by the device. Every single barcode consist of three layers of DataMatrix-Codes, each layer is represented by one of the three secondary colors of the RGB color space (red, green, and blue). In contrast to the previously introduced QLR- Codes, these layers are not used to gain additional storage, instead they are utilized to improve the robustness of the codes: Every DataMatrix-Code-frame has two redundan- cies.

1.4.3. Projects with similar Orientation

PNG-Store

[Sei13] introduces the idea to store JavaScript and CSS ﬁles in PNG-images [W3C13b] to utilize its lossless compression as alternative to the usually used GZip compression. To achieve this goal, 8 bit PNG-images are used as data containers, whereby the encoder has a data ratio of 1 Bpp. A continuation of this idea is the PNG Store [Hen13]. In this work also 24 bit PNG-images are used with a data ratio of 3 Bpp.

Both encoding methods are similar to two of our encoders introduced later in this thesis; nevertheless the goals of our projects differ vastly. Their aim is to gain compression through storing data in images, whereas we want to store data on photo sharing websites even at a cost of overhead. Depending on the used provider, we maybe have to deal with lossy JPEG-compression and therefore also need encoders in our portfolio that have a lower bytes per pixel ratio but are more robust against compression.

8 1.4. Related Work 9

Filr

Simultaneously to the previously mentioned restart of Flickr and its new service offering 1TB photo storage for free, a new project appeared called “Filr” [Tom13]. It shares the similar goal as our project to exploit Flickr as free storage. Filr is a command line tool that allows to encode and upload ﬁles to Flickr. Its encoder accords to the one used in the PNG Store project with a data ratio of 3 Bpp.

In contrast to our photosharing website driven cloud storage system presented in this thesis, this is a very limited tool. It only allows the encoding and uploading of single ﬁles with a limitation of 15MB. The ﬁles then have to be downloaded manually and encoded again. In contrast, our system works as blob store and handles all that automatically. Moreover, our system is adaptable for any photo sharing website and can deal with JPEG- compression if necessary. Filr only works with Flickr and its service allowing access to the original images.

1.4.4. Conclusion and Comparison

A comparison between the eight discussed approaches is shown in Table 1.1:

Approach: Use Case: JPGc Bpp: JPEG Comp. Immune Steg. Using Hide data in images compression im- 2.5 10 4 Wavelet Transform [XSSL04] mune − × A Web based Covert File System Hidden ﬁle system based on public 1 10 1 [BKI07] media hosting (e.g. Flickr) − ∼ × Embed data in images in such way 2 QR-Codes [QRc13b] that it can be processed by appropri- 9 10− ate reading equipment × 2 DataMatrix-Codes [fS06] “ 7.5 10− Quick Layered Response (QLR) Codes Enhance QR-Codes capacity with × 2.7 10 1 [DD12] RGB colors − × Enhance DataMatrix-Codes capacity 2 Unsynchronized 4D Barcodes [LB07] by utilizing additional dimensions 7.5 10− #f r. color and time × × PNG-Store [Hen13] Use PNG-images as GZip alternative 3 × Filr [Tom13] Store single ﬁles on Flickr 3 × Table 1.1.: Overview Approaches Related Work

The first column shows the name of the approach and its reference, the second column describes briefly the use case of the specific approach, the third column shows whether

9 10 1.4. Related Work the approach is capable of dealing with JPEG-compression, and the fourth column shows the best available data rate in bytes per pixel.

We see that the approaches that have the best data ratios are [Hen13] and [Tom13]. The disadvantage of both is that they are not capable of working with JPEG-compressed images.

On the other hand [XSSL04] as the JPEG-compression immune approach in this portfolio of approaches, is worlds apart from being usable with respect to its data ratio.

The approaches [QRc13b], [fS06], [DD12], and [LB07] utilize 2D-Code techniques and have an acceptable data ratio. At the same time, they are usable in a scenario where certain kind of JPEG-compression to images is applied.

[BKI07]’s encoding speciﬁcation are very vaguely described and therefore difﬁcult to anticipate.

10 2 Encoding Approaches

This chapter describes how to encode bytes into the pixels of an image, as well as how to regain such encoded bytes again. Twelve different encoding approaches are presented that serve this purpose. These encoding approaches can be divided in two main cate- gories, the approaches using a single layer of colors (SL-approaches) and the approaches using multiple layers of colors (ML-approaches). Figure 2.1 gives an overview over all encoding approaches’ output images generated with the same input data (the bytes used to generate this output can be found in AppendixA):

SL2 SL3 SL4 SL7 SL16 SL256

ML2 ML3 ML4 ML7 ML16 ML256

Figure 2.1.: Sample Output of all Encoding Approaches

As can be seen, the resulting images differ in their size and the number of colors utilized. Every encoding approach has its unique features and is therefore useful with respect to

11 12 2.1. Intention the later discussed application and the treating of uploaded images by speciﬁc photo sharing websites (e.g. JPEG-compression).

On the following pages, we are presenting all important steps taken to achieve the resulting portfolio of encoding approaches. Following will be a summary of benchmark comparisons between the different approaches.

2.1. Intention

The first test subject to exploit was Yahoo’s photo sharing website Flickr [Fli13a]. As tool for the implementation, we used the programming language JAVA in team play with the flickrj-android API [Yu13] as middleware to get access to Flickr’s web API. First we looked at how Flickr treats images and uploaded some test images stored in lossless image file format PNG [W3C13b]; then downloaded the hosted version again. It showed that Flickr applies JPEG-compression [UNI13] on uploaded images (changed since Flickr restarted its service, see Chapter 3.3). The downloaded images differed from the originals with respect to color fidelity of its single pixels as well as their overall disk space usage.

Therefore we did some research in the ﬁeld of making images more robust against JPEG- compression and found papers related to image-steganography. Particularly [KTC09]’s JPEG-compression immune approach seemed to be interesting for our purpose of handling Flickr’s post-processing. With their method it is possible to regain information embedded in images without bit errors, even if the highest level of JPEG-compression has been applied to these images. The problem of this approach with respect to our goal of utilizing Flickr’s storage is, like previously mentioned, its poor data ratio.

Further investigations led us to 2D-Codes. Common available 2D-Code approaches solve the task of embedding data in images by utilizing blocks of pixels. We focused on the nowadays widely used QR-Codes [QRc13b]. Tests with test cases applied to Flickr and uploaded QR-Code images with random input bytes showed that QR-Codes can be interpreted by its decoders even after Flickr’s post-processing. This means that the data stored in these images survives the applied JPEG-compression. As a result QR-Codes could be used to host data on Flickr. Nevertheless, a big trade-off with QR-Codes and 2D-Codes in general, is that they are optimized to be camera-readable, meaning that they take a lot of effort with respect to readability and error correction. In our photo sharing website

12 2.2. First Encoding 13 scenario images will be directly interpreted by computers and do not have to take a detour over cameras. Therefore, 2D-Codes waste a lot of image information for in our case unnecessary features that renders them impractical with respect to efﬁciency, although, they are more efﬁcient compared to [KTC09]’s approach.

At this point, we realized that the available approaches do not really satisfy the requirements of our scenario and decided to develop our own encoding approaches optimized for our purposes. The work with 2D-Codes provided some good inspiration of how data can be embedded in pixels and consequently we focused on pixels as atomic storing units for our own approaches.

2.2. First Encoding

The main idea for the ﬁrst encoding was that, if we have an image that consists of the colors black and white, every pixel in such an image has two possible states: the state black and the state white. A common bit has also two states: the state 0 and the state 1. Therefore both can be used as substitute for each other.

We deﬁne:

1. There are two color classes: the class black and the class white. Black stands for the binary number 0, whereas white is represented by the binary number 1. 2. Every 8 pixel in a row represent one coherent data unit (byte) in the image. 3. Every pixel that has a color value that is smaller or equal than white (white repre- 2 sents the highest color value available in a speciﬁc image) is interpreted as pixel of color class black and every color value that is greater is interpreted as pixel of color class white:

white cb ( ) < cw ≤ 2 Whereas cb equals color class black and cw class white.

This encoding will hereafter be denoted as SL2 (details about naming conventions are discussed later in this chapter).

13 14 2.2. First Encoding

2.2.1. Encoding Workow

Figure 2.2 shows the corresponding encoding workﬂow to this ﬁrst encoding approach

(as input the random bytes listed in AppendixA were used, cf. resulting image with SL2 Figure 2.1):

0 1 0 0 0 1 0 1 6910 010001012

+ 0 1 0 0 0 0 0 1 6510 010000012

+ 1 1 0 1 1 0 1 0

-3810 11011010 2

+ 1 0 1 1 1 0 1 0 11011010 -7010 2

+ 1 0 1 1 1 0 0 0 -7210 101110002

Input Step 1 Step 2 Step 3 Bytes convert to binary map Bits to pixels add parts together

Figure 2.2.: Encoding Workﬂow - From an Array of Bytes to the ﬁnal Image

To generate an image from a given array of input bytes, we go sequentially through it and then encode one byte after another into a pixel row. The resulting rows are then fused together to the ﬁnal image. In detailed steps:

(Step 1) At first the given input bytes are converted from the decimal system to their corresponding binary representation (negative two’s complement values are interpreted unsigned). As an example we take the first input byte value 6910. This value is converted to the binary system, resulting in the 7 bit long string 10001012. We specified that every encoded byte row consists of 8 digits, respectively pixels, thus omissible leading bits with value 0 are appended, so that the resulting bit string fits the length of 8 digits. In this example we get 010001012 as a result.

(Step 2) The received binary representation of the byte then can be mapped to pix-

14 2.2. First Encoding 15 els. Like we deﬁned: 0 will be encoded as black pixel (b) and 1 as white pixel (w). As a result we, derive from the binary number 01001012 the pixel row bwbbbwbw.

(Step 3) Finally we add all pixel rows together and get an image that contains all input bytes encoded in its pixels.

2.2.2. Decoding Workow

How to regain bytes stored in an image shows the general decoding workﬂow in Fig- ure 2.3:

010001012 6910 0 1

010000012 6510 0 1

11011010 2 -3810 1 1 0 1 0

101110102 -7010 0

Input Step 1 Step 2 Step 3 Image Inspect Color and regain Bit convert to Decimal MERGE bytes

Figure 2.3.: Decoding Steps - From Image to the regained Array of Bytes

The starting point in the decoding process is an image that contains embedded bytes. In the example image these bytes equal the bytes that were used in the encoding workﬂow, but the actual image is different. It is a JPEG-compressed version of the original output image (e.g. downloaded from a photo sharing website like Flickr) and therefore contains deviating pixel colors. To decode the embedded data from this image, the following steps are required:

(Step 1) It is deﬁned that every data unit is stored in a row of 8 pixels, thus the image is traversed sequentially from top to bottom and each such row is processed. Due

15 16 2.2. First Encoding to the color values of pixels can deviate from given color classes (in this case black and white), every pixel color in such a row then is inspected and via threshold assigned to its nearest color class. In the workflow this inspection is symbolized by a magnifier. For example the first eight pixels of the input image: the magnified pixel value has a grey color value of 210. Since this value is greater than white , respectively greater a value of 2 127, this pixel is interpreted as pixel of the color class white. Next the determined color class is substituted by its corresponding bit. Received bits are then appended to the in previous iterations extracted bits. As a result, we gain the binary representation of the byte stored in a specific row, for example from image’s first row we gain 010001012.

(Step 2) The received binary string that represents the embedded byte is then converted back to the decimal system. In this case we receive the number 6010 from the binary string 010001012.

(Step 3) All extracted bytes are merged together. As a result, we receive the original array of input bytes that has been stored before.

2.2.3. Results

Analyses with several test cases and random test data showed that the here introduced encoding (SL2) makes it possible to store arbitrary bytes on the photo sharing website Flickr without being harmed by its post-processing, respectively its JPEG-compression. The data ratio of such a generated image is 1 byte to 8 pixels. In comparison with common available approaches, e.g. like 2D-Codes, this is a great capacity win with respect to the generated image dimensions. For example the in Figure 1.2 shown QR-Code and

DataMatrix-Code hold the same array of bytes as we used in the SL2 workﬂow examples, but both need many more pixels to store it. Figure 2.4 illustrates a visual comparison between the three outputs (the array of input bytes can be found under AppendixA):

16 2.3. Advanced Encoding 17

25 px

20 px

18 px

25 px 20 px 8 px

(a) QR-Code (b) DataMatrix (c) SL2

Figure 2.4.: Output Comparison QR-Code vs. DataMatrix-Code vs. Own Encoding (SL2)

The comparison in detail:

an array of 18 input bytes needs an image with the pixel dimensions of 25 25 • to be represented in a QR-Code by using the lowest error-correction code available,× which is a consumption of 625 pixels or a pixel ratio of 1 byte to 35 pixels. ∼ the same data can be stored in DataMatrix-Code with 20 20 pixels. Those are in • sum 400 pixels and the ratio is 1 byte to about 22 pixels.× ∼

the SL2 in contrast just needs a sum of 144 pixels and has a data ratio of 1 byte to 8 • pixels. These are 481 pixels less or only 23% of the pixels that are needed for the encoding with QR-Code and 256 pixels less∼ than the DataMatrix-Code, respectively only 36% of the DataMatrix-Code’s pixel usage. ∼

These results make the SL2 a good solution of how to embed data in images and host it on photo sharing websites. Therefore it is used as a basis for the other encodings later introduced in this thesis.

2.3. Advanced Encoding

The next logical step to enhance the data capacity of the previously described SL2, is by utilizing images with more colors instead of only black and white. For example a greyscale

17 18 2.3. Advanced Encoding image as data container and an additional grey color could be utilized for the encoding. As a result, there are three different possible states available: The state black, the state grey, and the state white. This allows the switch from the binary numeral system, which expresses itself in two states to the less familiar ternary system consisting of three states: the state 0, the state 1, and the state 2.

The increased amount of utilized colors, respectively digits and the data ratio are depending on each other. To be able to determine this dependence and how the choice of the utilized numeral system inﬂuences the data ratio, some basics on numeral system follow.

2.3.1. Basics on Numeral Systems

A natural number z N0 consisting of n place-values in a numeral system of base γ can ∈ be expressed by the following polynomial (information derived from [KW05], p.33ff):

n 1 X− i z = ziγ (2.1) i=0

Whereas 0 zi < γ. The unique base γ of a numeral system is the so called radix. It determines≤ the amount of digits available in a speciﬁc system that reach from 0 to γ 1. Due in our context the digits of a number represented in a speciﬁc numeral system− are mapped to the colors of an image, γ also determines the amount of colors that are utilized to encode a number of such a numeral system.

2.3.2. Relationship between Radix γ and Data Ratio

The choice of radix γ inﬂuences the data ratio of an according encoding through deﬁning the length of one pixel chunk that is able to represent a byte. In the following, we want to determine how many place-values, respectively pixels, are needed to do this representation.

To determine the maximum number zmax to be possibly displayed with a speciﬁc radix γ and n available place-values, we set in Equation 2.1 zi = γ 1 for each i 0, . . . , n 1 , − ∈ { − }

18 2.3. Advanced Encoding 19 because (γ 1) equals the highest digit available in a speciﬁc numeral system: − n 1 n 2 zmax = ((γ 1) γ − ) + ((γ 1) γ − ) + ... + (γ 1) (2.2) n − n×1 n 1 −n 2 × − = (γ γ − ) + (γ − γ − ) + ... + (γ 1) (2.3) n − − − = γ 1 (2.4) − As a result, we ﬁnd that with n place-values it is possible to display numbers between 0 to γn 1. − In the next step, we want to determine the lowest amount of place-values n needed to represent the values of one byte. Since 255 is the greatest possible value of a byte, it n must apply that zmax = γ 1 255. We derive: − ≥ γn 1 255 (2.5) − ≥ γn 256 (2.6) ≥ n logγ(256) (2.7) ≥

The smallest possible natural number n that satisﬁes the Inequation 2.7 is n = logγ(256) . d e From now on we denote by

pSL = logγ(256) (2.8) d e the lowest amount of pixel needed to represent one byte in a speciﬁc numeral system with radix γ.

We are now considering the ternary system (radix γ = 3) as an example. In this case, pSL = log3(256) = 6, meaning that 6 pixels are needed to embed a byte in an image utilizingd the ternarye system. This example also shows that not always the full possible number range of a numeral system with the minimal needed place-values to represent one byte is exploited. With the ternary system and 6 place-values it would be possible to 6 represent a range between 0 and zmax = 3 1 = 728, but a byte only uses the range from 0 to 255. Such unexploited ranges are− true for choices of radix γ of which 256 is not dividable without a remainder.

19 20 2.3. Advanced Encoding

2.3.3. Results

Referring back to the idea to improve the introduced encoding technique’s data ratio by utilizing greyscale images with the three colors black, grey, and white as data containers instead of the formerly used black & white images. This enables the switch from the binary system (γ = 2) to the ternary system (γ = 3).

For this new encoding we deﬁne that:

1. There are three color classes: color class black stands for the digit 0, the color class grey for the digit 1, and the color class white for the digit 2 2. Every group of 6 pixels in a row represent one coherent data unit (byte) in the image. 3. The pixel color values are interpreted as follows:

white white cb ( ) < cg (2 ) < cw ≤ 3 ≤ × 3 Whereas cb equals color class black, cg color class grey, and cw color class white.

This encoding approach is from now on denoted as SL3. Figure 2.5 shows a comparison between the SL2 and the SL3:

11111000 0, 1 0, 1, 2 2 8 pixels

-810 Binary Ternary 100012 System System 3 6 pixels

(a) Pixel-mapping (b) Example Encoding

Figure 2.5.: Comp. Encoding Approaches: Binary System (SL2) and Ternary System (SL3)

In Figure 2.5a it is displayed how the mapping using three colors and the ternary system works in comparison to the previously discussed encoding using two colors and the binary system. Figure 2.5b shows the gain of this switch. Instead of 8 pixels that are needed to represent one byte, it is now possible to encode a byte with 6 pixels. This is 1/4th less pixel usage for storing one bytes with the SL3 than it was needed with the SL2 (cf. also

Figure 2.1 SL2 vs. SL3).

20 2.4. Single-Layer-Approaches 21

2.4. Single-Layer-Approaches

We have previously discussed the basic encoding approach SL2 and how to advance the data ratio of this technique by utilizing more colors, e.g. through a third color with the help of the less familiar ternary system leading to the SL3. Both of these encoding approaches are part of our set of six single-layer-approaches (SL-approaches). Four other encoding approaches with different γ complement this portfolio.

Since γ is unique to every approach and represents its utilized colors and digits simultaneously (e.g. like we see in Figure 2.5a), it can be used to distinguish between particular encoding approaches. Approaches itself are named by their features, depending on whether it is a single-layer (SL) or a multi-layer (ML) approach (multi-layer-approaches are discussed Chapter 2.5) and its radix γ. Therefore the black and white approach using the binary system is denoted as SL2, whereas the approach using the colors black, white, and grey, and the ternary system is denoted as SL3 approach. Details to the naming conventions can be found in AppendixB.

The data ratio of a speciﬁc SL-approach can be determined with with Equation 2.8, were pSL equals the pixels that are needed to store one byte with a speciﬁc SL-approach. The SL-approaches itself cover only those encodings properly that gain an improvement through their bytes per pixel ratio with respect to encodings that use a smaller radix γ (equivalent to less digits, respectively less colors) to embed bytes. Encoding approaches that utilize more colors to encode, but need same amount of pixels to store a byte with respect to encoding approaches with less colors, are uninteresting in our context, since more colors mean less robustness against JPEG-compression and that the output images require a greater disk space. For example the SL4 approach that uses the quaternary system is able to represent a byte within 4 pixels and has a better data ratio to that of the

SL3 that needs 6 pixels for a byte. The SL4 therefore is part of the SL-approaches. On the other hand, an encoding approach utilizing γ = 5 would also need 4 pixels like the SL4 to embed a byte in an image, but with the disadvantage that more colors have to be utilized than it is the case with the SL4. This encoding is therefore not covered in our set of SL-approaches.

Taking these requirements into account, the resulting portfolio of SL-approaches contains the following six encoding approaches: SL2, SL3, SL4, SL7, SL16, and SL256. A summary of these encoding approaches and their features is shown in Figure 2.6:

21 22 2.4. Single-Layer-Approaches

p Byte / Pixel

1 2 8 1 3 6 1 4 4 1 7 3 1 16 2

256 1

Figure 2.6.: All SL-Approaches and their Features

The column γ stands for the radix utilized by the SL-approach, whereas column p represents the actual pixels needed to store one byte. The pixels displayed correspond to the output of the speciﬁc approaches encoding a byte with the value 69 (For details about the colors utilized and how they are deﬁned, please refer to Chapter 2.6.2 about colors). The last column shows the actual data ratio of the SL-approaches. We see that with the portfolio of these six SL-approaches, bytes per pixel ratios can be achieved that reach from 1/8 Bpp up to 1 Bpp.

In the following, we will take a detailed look on how the encoding and decoding works with respect to its general algorithms.

2.4.1. Encoding

To encode bytes using an encoding approach of the introduced SL-approaches, it is important that certain preconditions are fulﬁlled:

the image container to embed the input bytes has to have a sufﬁcient amount of • pixels available. The pixel usage can be calculated by determining the amount of

pixels pSL that are used to store one byte with the chosen radix γ (see Equation 2.8) and multiplying it with the length of the input byte array. The result then is taken into consideration at the point the dimensions of the image container are speciﬁed.

the image type of the image container must also be able to serve γ-different pixel •

22 2.4. Single-Layer-Approaches 23

states / colors (details follow in Chapter 2.6.1).

a deﬁned set of γ-different colors must be available to map the digits of a number • to colors (details follow in Chapter 2.6.2).

If these preconditions are met, the following algorithm can be used to encode bytes with the presented SL-approaches:

General Encoding Algorithm Single-Layer-Approaches

Algorithm1 shows the general SL-approach algorithm to encode bytes into the pixels of an image.

Algorithm 1 storeBytesInImage(img: Image, w: Int, h: Int, γ: Int, bs: Byte []): Image Require: img is capable to store all given bytes 1: ppb := getPixelsPerByte(γ) 2: col := null 3: for y := 0 to h 1 do 4: rix := w y− 5: for x := 0×to w 1 do 6: pix := rix + −x 7: pos := pix mod ppb 8: if pos == 0 then 9: if not bs.moreBytes? then 10: return img 11: end if 12: col := getColorsFromByte(bs.nextByte(), γ, ppb) 13: end if 14: img.drawPixel(x, y, col[pos]) 15: end for 16: end for 17: return img

The function storeBytesInImage() takes ﬁve invocation parameters: The image img to store the given bytes in, its width w, its height h, the radix to be used for the encoding γ, and an array ﬁlled with bytes to be stored bs. For the algorithm to be successful it is required that img has enough pixel space with respect to the bytes to be encoded. The function returns the given image with the given array of bytes embedded.

23 24 2.4. Single-Layer-Approaches

Assign Initial Variables , Al gorithm 1 →

Ln1: At ﬁrst the function getPixelsPerByte() is invoked to determine how many pixels with the given radix γ are needed to store one byte in the image (see Equation 2.8). The returned value is assigned to variable ppb. Ln2: Next, the variable col is assigned. It will later take an array ﬁlled with color objects that represents the current byte and is initialized empty with null.

Loop over the Image’s Pixels , Al gorithm 1 →

Ln3: After the initialization step, a nested loop over the image’s ( img) pixels starts. The pixels will be traversed from left to right, from the upper-leftmost pixel to the lower-rightmost pixel. Variable y stands for the current y-coordinate position in the image. The outer loop assigns the variable y to 0 and will be entered as long as y is smaller than the image height h. Ln4: Every time the outer loop is entered, the variable rix is assigned to the product of the image width w and the current y-coordinate position y. Therefore rix represents the sum of all full lines that have before been traversed in iterations of the loop. Ln5: Next follows the inner loop. Variable x stands for the current x-coordinate position in the image. The inner loop assigns x to 0 and runs as long as x is smaller than the width w of the image. Ln6: When the inner loop is entered, the variable pix is assigned to the addition of rix and x. Thus, pix represents the position of the current pixel in the image-traversal process. Ln7: In the next step the variable pos is assigned to the value of the current pixel position pix modulo the amount of pixels needed for one byte ppb. Thus pos indicates the position inside a coherent pixel chunk of the encoding. Ln8: pos’s value is then utilized to determine whether the current pixel position is the start of a new byte in the image’s pixels. If pos equals 0, the start of a new byte is indicated and the body of the if statement is entered. Ln9: After the if statement is entered, it is checked whether the array bs contains further bytes. Ln 10: If it does not, all bytes have been encoded in the image. The algorithm ends and the image img is returned to the caller.

24 2.4. Single-Layer-Approaches 25

Ln 12: Else, if there are still bytes left that have to be embedded in the image, the subroutine getColorsFromByte() is invoked. This function translates a given byte into an array of colors. It takes the next byte to be embedded from the array bs, the approach’s radix γ, and ppb. Since this subroutine is crucial for the algorithm, we will have a closer look in the following paragraph:

Translate Byte in Colors , Al gorithm 2 →

Algorithm 2 getColorsFromByte(b: Byte, γ: Integer, ppb: Integer): Color[] 1: col := array(ppb) 2: nsb := convertToNumeralSystem(b, γ) 3: while nsb.length() < ppb do 4: nsB := string.concat(nsb, 0) 5: end while 6: for i := 0 to ppb 1 do 7: id x := convertFromNumeralSystem(− nsb[i], γ) 8: col[i] := getColorFromIndexPosition(id x) 9: end for 10: return col

getColorsFromByte() takes three arguments: The byte to be encoded b, the radix to be used for the encoding γ, and the amount of pixels needed to store one byte ppb. After processing, the routine returns an array ﬁlled with colors that represent the given byte b.

Ln1: First, a new array of color objects with the length of ppb is created and allocated to the variable col. Ln2: Next, the given byte is transferred into its representation in the given numeral system via invoking the function convertToNumeralSystem(). It takes two arguments: The byte that has to be transferred b and the radix of the numeral system γ. The function returns a string representation of the byte b in the chosen numeral system. This string is assigned to variable nsb. Ln3-5: Then follows a while loop. It guarantees that the amount of characters in string nsb is equal to length of pixels that are reserved for one byte in the image. It is entered as long as the length of the string nsb is smaller than ppb, and every time it is entered, a character with the value “0” is appended to the existing string nsb. Ln6: In the next step the actual translation is done for which a for loop is entered. It initializes the variable i with 0 and runs over all characters of the string nsb.

25 26 2.4. Single-Layer-Approaches

Ln7: In the body of the loop the function convertFromNumeralSystem() is invoked to convert the character at nsb’s index position i to its decimal representation. It takes the current character of nsb and the radix γ as parameters. Its returning value is assigned to variable id x. This value lies between 0 and γ 1 and can be used to map a digit value to a value of γ colors. − Ln8: Then the function getColorFromIndexPosition() is invoked. This function takes the index position id x as argument and returns the matching color. (The pool of colors for the approaches is predeﬁned see also: Chapter 2.6.2). The returned color is assigned to the color array col at index position i. Ln6: If i is still less or equal ppb another iteration starts. Ln 10: Else getColorsFromByte() returns the created color array col that represent the value of b to the caller.

Draw colors to image , Al gorithm 1 →

Ln 12: Now we are back in Algorithm1 Line 12. The returned color array from function getColorFromByte() is allocated to the variable col. Ln 13: In this step, the actual drawing of the byte’s color representation to the image’s pixels takes place. It is implemented by invoking the drawPixel() function of the image img. As arguments this function takes the coordinates of the current pixel x and y and the pixel color, which corresponds to col at index position pos. After drawPixel() is processed, the current pixel is set to its determined color. Ln5: If x is after the increment still less the image width w, another iteration of the inner loop starts, else the inner loop ends. Ln3: If y afterwards is less the image height h, another iteration of the outer loop begins, else the nested loop stops and all bytes of the input array bs have been successfully encoded in the image’s pixels. Ln 17: The created image img holds all bytes of bs and is returned to the caller. The general SL-approach encoding algorithm ends at this point.

2.4.2. Decoding

Once bytes have been encoded in an image, another important task is to perform the operation the other way around and to extract bytes embedded in an image. The scenario that we are facing is that an image is hosted on a certain provider and regained

26 2.4. Single-Layer-Approaches 27

by downloading it. The downloaded image contains via a speciﬁc SL-approach encoded bytes that can be extracted by the following algorithm:

General Decoding Algorithm Single-Layer-Approaches

Algorithm3 shows how to decode bytes that have been encoded by a SL-approach. To regain the in the image stored bytes, we invoke the function extractBytesFromImage().

Algorithm 3 extractBytesFromImage(img: Image, w: Int, h: Int, γ: Int) : Byte [] 1: ppb := getPixelsPerByte(γ) 2: bs := array( ) 3: chs := createString()∞ 4: for y := 0 to h 1 do 5: rix := w y− 6: for x := 0×to w 1 do 7: pix := rix + −x 8: pc := img.getColor(x, y) 9: ch := getCharacterFromPixelColor(pc, γ) 10: chs := chs.append(ch) 11: if pix mod ppb == ppb 1 then 12: bs.append(convertFromNumeralSystem(− chs, γ)) 13: chs := createString() 14: end if 15: end for 16: end for 17: return bs

extractBytesFromImage() takes four invocation-parameters: The image from which the previously embedded bytes have to be restored img, the width of the image w, the height of the image h, and the radix of the numeral system that was used to embed the bytes into the image γ.

Assign Initial Variables , Al gorithm 3 →

Ln1: At ﬁrst the function getPixelsPerByte() is called to determine how many pixels in a row represent one byte in the image. It takes the radix γ as parameter and calculates the bytes per pixel ratio. Its return value is assigned to variable ppb.

27 28 2.4. Single-Layer-Approaches

Ln2: Next, an array with an adequate size is initialized that is able to hold all extracted bytes that are stored in the image, it will later contain the extracted bytes. (A solution to precisely determine the amount of bytes that an image contains, could be a header section that stores meta data about the encoded data of an image. More details about such a header section that is included in our implementation of our photosharing website driven cloud storage system, is outlined in Chapter 3.1.2). Ln3: Next follows the assignment of an empty string to variable chs, which is established by calling the function createString(). chs, will be used to cache the extracted characters from coherent pixel chunks.

Loop over the Image’s Pixels , Al gorithm 3 →

Ln4: After the initial variables are assigned to its values, a nested loop starts over the image’s (img) pixels. Like in the encoding, the pixels are traversed from left to right, from the upper-leftmost pixel to the lower-rightmost pixel. Variable y stands for the current y-coordinate position in the image. The outer loop assigns the variable y to 0 and will be entered as long as y is smaller than the image height h. Ln5: Every time the outer loop is entered, the variable rix is assigned to the product of the image width w and the current y-coordinate position y. It represents all full pixel lines that have been traversed in previous iterations of the loop. Ln6: Next follows the inner loop. Variable x stands for the current x-coordinate position in the image. The inner loop assigns x to 0 and runs as long as x is smaller than the width w of the image. Ln7: When the inner loop is entered, the variable pix is assigned to the addition of rix and x. Thus, pix represents the position of the current pixel in the image-traversal process. Ln8: Then the image’s function getColor() is called to read the current pixel color. This function takes the current x- and y-coordinate and returns the color value. Its returned color is assigned to the variable pc. Ln9: In the next line the function getCharacterFromPixelColor() is invoked. It determines the corresponding character to the current pixel color and takes pc and γ as parameter. Since this is crucial to our decoding, we will have a closer look as to how this function works in the following paragraph:

28 2.4. Single-Layer-Approaches 29

Retrieve Character from Color , Al gorithm 4 →

Algorithm 4 getCharacterFromColor(pc: Color, γ: Integer): String 1: pr := pc.getRed(), pg := pc.getGreen(), pb := pc.getBlue() 2: id x := 1 3: sdi := −1 4: for i :=−0 to γ 1 do 5: cr := COL[i−].getRed(), cg := COL[i].getGreen(), cb := COL[i].getBlue() 6: cdi := Math.abs(cr pr) + Math.abs(cg pg) + Math.abs(cb pb) 7: if sdi == 1 or sdi >− cdi then − − 8: sdi := cdi− 9: id x := i 10: end if 11: end for 12: return convertToNumeralSystem(id x, γ)

The function getChracterFromColor() takes two parameters, the current pixel color pc and the radix of the used numeral system γ. It utilizes the approaches’ speciﬁc array of predeﬁned colors COL, representing the colors that are used as substitute for the embedded number’s digits (details about colors are discussed in Chapter 2.6). It returns the character the current pixel color is representing.

Ln1: At first the color pc is split into its RGB components and assigned to the variables pr, pg, pb by invoking the color’s functions getRed(), getGreen() and getBlue(). Ln2: Next the variable id x is set to 1. Since we cannot trust the color value, for example, the image might have gone− through JPEG-compression, id x later represents the index of the color that is determined to be the best fitting color of the predefined color array COL. This index then stands for the most likely digit in the utilized numeral system that the color is substitute for. Ln3: Then the variable sdi is set to 1. It will represent the color of COL with the nearest distance since the current iteration− of the following loop. Ln4: The for loop initializes the variable i with 0 and is entered until all colors of COL have been traversed. Ln5: In the body of the loop, first the red, green, and blue components of the predefined color at the index position i of COL are set to the variables cr, cg and cb. Ln6: Then follows the determination of the deviation of the current color of COL from the pixel color pc. This is done by subtracting the RGB components of both colors from each other and then adding the absolute values of this calculation together.

29 30 2.4. Single-Layer-Approaches

The resulting value is assigned to the variable cdi. The closer cdi is to 0, the nearer are both colors. Ln7: An if statement follows that checks whether the current color’s distance value cdi is closer to the given color than the colors that have been selected in the iterations of the loop before, whereby it is checked if sdi equals 1, which means it is the ﬁrst iteration of the loop, or sdi is greater than cdi. − Ln8-9: If that is the case, cdi is assigned to sdi to be the closed value till the current iteration of the loop, and id x stores the corresponding index position of the iteration i. Ln4: If i is still smaller than γ, another iteration of the loop starts. Ln 12: Else the loop ends and the nearest color is determined. It takes the index position id x and the radix γ as parameter and translates the index value to the character that represents this value in the given numeral system. This character then is returned to the caller.

Regain Byte from extracted Characters , Al gorithm 3 →

Ln9: Back in Line9 of Algorithm3, the returned value of function getCharacterFromPixelColor() is assigned to variable ch. Ln 10: The character ch is then appended to the before extracted characters and assigned to chs. Ln 11: An if statement follows that checks whether enough characters were extracted and stored in chs to regain a byte. This is accomplished by taking the value of pix modulo ppb. If it is equal to the bbp 1, a full pixel chunk has been extracted. Ln 12: If the body of the if statement is entered− the function convertFromNumeralSystem() is invoked to translate the characters of chs to the decimal system. It takes the string with the extracted characters chs and the numeral system’s radix γ and returns the extracted byte’s value. This byte then is appended to the byte array bs. Ln 13: Afterwards the string chs is reset to an empty string. Ln6: If x is after the increment still less the image width w, another iteration of the inner loop starts, else the inner loop ends. Ln4: If y afterwards is less than the image height h, another iteration of the outer loop begins, else the nested loop stops and all bytes of the input image img have been successfully decoded.

30 2.4. Single-Layer-Approaches 31

Ln 17: As a result all extracted bytes are stored in the array bs. It then is returned to the caller. The general SL-approach decoding algorithm ends at this point.

2.4.3. Results

After having introduced the SL-approaches in this subchapter, Table 2.1 shows an overview of the available data ratios that can be gained from these approaches:

Encoding Approach:

SL2 SL3 SL4 SL7 SL16 SL256 Data Ratio: 1/8 1/6 1/4 1/3 1/2 1 Table 2.1.: Summary Data Ratio SL-Approaches (in Bytes per Pixel)

Looking back at available techniques like 2D-Codes and their data ratios, this is a great improvement. This portfolio also makes it possible to adjust the best compromise between data ratio and robustness against compression, depending on the image host.

A practical application of these encodings is shown in Figure 2.7. It displays the output images of all SL-approaches for the 18 example input bytes of AppendixA:

SL =2 SL =3 SL =4 SL =7 SL = 16 SL = 256

Figure 2.7.: Example Output SL-Approaches

We see how the data ratio of the speciﬁc approaches impacts the dimensions of the out-

put images. Whereas the SL2 needs 144 pixels to store the 18 input bytes, the SL256 only needs 18 pixels for the same data (details about the image types and colors utilized by the different encoding approaches are discussed in Chapter 2.6).

31 32 2.5. Multi-Layer-Approaches

2.5. Multi-Layer-Approaches

In this subchapter six multi-layer-approaches (ML-approaches) are introduced. These encoding approaches enhance the achievable bytes per pixel ratio, with respect to their corresponding SL-approach, further.

The discussed SL-approaches use different amounts of colors to encode bytes into an image. The more colors an approach can utilize, respectively the greater γ is, the better the bytes per pixel ratio of generated output images. To improve the data ratio further, the three color components of RGB images can be utilized. Each pixel in such an image has its red, its green, and its blue component. The displayed color of such an image’s pixels is then the result of the mixture of these color components. The idea behind ML- approaches is to exploit these components as layers with additional encoding space. This technique combined with the SL-approaches, enhances the bytes per pixel ratio for each approach by a factor of 3.

Each of the RGB layers is treated as a separate image. The given bytes are split over the three layers and encoded into it. In contrast to the SL-approaches and working with γ-different RGB colors, the ML-approaches work with γ-different colors for each RGB layer. Lastly, the resulting red, green, and blue images are combined together to the ﬁnal ML-image. Figure 2.8 shows an example how the encoding for ML-approaches works by comparing the outputs of the SL2-approach and the ML2-approach (like in the examples before, the random bytes listed in AppendixA were used as input):

SL2

Layer 1 (Red) Layer 2 (Green) Layer 3 (Blue) ML2

+ + =

Figure 2.8.: Comparison between the SL2-Approach and the ML2-Approach

32 2.5. Multi-Layer-Approaches 33

In the upper left corner, we show the ﬁrst three pixel rows of the output image generated by the SL2-approach. Below we see the three layers of the RGB image, which in addition result in the ﬁnal image produced by the ML2-approach. Since the SL2-approach and the

ML2-approach both utilize the same radix γ to encode bytes, the resulting pixel patterns in the output images can be visually compared. It is demonstrated how the encoded bytes of the black & white image generated by the SL2-approach are distributed over the RGB image of the ML2-approach. The ﬁrst row of the black & white image is stored in the blue layer, the second in the green layer, and the third in the red layer, and so on. In contrast to the black & white image, which differs between black and white, the colors used to encode the pixels in the RGB image differ in each layer between black and the layer color. After the input bytes are encoded in the three layers, these layers are combined to the resulting image of the ML2-approach. Because the resulting image consists of three layers, meaning there is three times more space available to encode bytes into, the ML2- approach needs three times less pixels than the output image of the SL2-approach.

We can adjust the formula to calculate the bytes per pixel ratio used for SL-approaches (see Equation 2.8) to ﬁt for our ML-approaches by taking the three times more available encoding space into account. This results in the following equation:

logγ(256) pML d e (2.9) = 3

The pixels pML that are needed to represent one byte in a speciﬁc ML-approach equal the ceiling of the logarithm of 256 to base γ divided by 3, because of the three utilized color layers.

A summary of all ML-approaches and their speciﬁc features is shown in Figure 2.9:

33 34 2.5. Multi-Layer-Approaches

Layers p Byte / Pixel

3 2 8 1 3 2 3 4 4

7 1

3 16 2

256 3

Figure 2.9.: All ML-Approaches and their Features

The ﬁrst column stands for the radix γ utilized by the speciﬁc ML-approach. The second and the third column are dependent on each other. Both show the pixel output of the six ML-approaches for the three input byte values 69, 65, and -38. The second column shows the three different RGB layers a pixel consists of, whereas the third column contains the actual pixel output by combining these layers to the actual color.

As is shown in the fourth column, with the ML-approaches it is possible to gain data ratios between 3/8 Bpp and 3 Bpp. In the following, we will take a look on how the encoding and decoding works for ML-approaches.

2.5.1. Encoding

To encode bytes with ML-approaches important preconditions have to be fulﬁlled:

the image container that is used to embed given bytes into must at least have • enough pixels available to store all input bytes with the selected ML-approach. The needed amount of pixels can be determined by calculating the ML-approach’s pixel need for one byte by using the Equation 2.9 and multiplying the result with the length of the input array.

the image container must be an RGB image (for details see Chapter 2.6.1). Without • it is obviously not possible to exploit the RGB component as additional data layers.

34 2.5. Multi-Layer-Approaches 35

a set of γ-different deﬁned colors must be available to encode the bytes with the • speciﬁc encoding approach (details follow in Chapter 2.6.3).

If these preconditions are met, the following algorithm can be used to encode bytes utilizing the presented ML-approaches:

35 36 2.5. Multi-Layer-Approaches

General Encoding Algorithm Multi-Layer-Approaches

Algorithm5 shows the general multi-layer-algorithm to encode bytes into the pixels of an image:

Algorithm 5 storeBytesInImageML(img: Image, w: Int, h: Int, γ: Int, bs: Byte []): Image 1: ppl := getPixelsPerByte(γ) 2: col := null 3: for y := 0 to h 1 do 4: rix := w y− 5: for x := 0×to w 1 do 6: pix := rix + −x 7: pos := pix mod ppl 8: if pos == 0 then 9: for la y := 0 to LAY 1 do 10: if not bs.moreBytes()− then 11: if la y == 0 then 12: return img 13: else 14: break 15: end if 16: end if 17: lco := getColorsFromByteLayered(bs.nextByte(), γ, ppl, la y) 18: if la y == 0 then 19: col := lco 20: else 21: for i := 0 to col.length() 1 do 22: col[i] := col[i] + lco[i−] 23: end for 24: end if 25: end for 26: end if 27: img.drawPixel(x, y, col[pos]) 28: end for 29: end for 30: return img

The function storeBytesInImageML() takes four invocation parameters: The image img in which to store the given bytes, its width w, its height h, the radix that is used for the encoding γ, and the array ﬁlled with bytes to be stored bs. It returns an image containing the input bytes encoded with the speciﬁc ML-approach.

36 2.5. Multi-Layer-Approaches 37

Assign Initial Variables , Al gorithm 5 →

Ln1: At ﬁrst the function getPixelsPerByte() is invoked. It takes the utilized radix γ as parameter to calculate how many pixels are needed in each layer to store one byte. The returned value is assigned to variable ppl. Ln2: Next, the variable col is assigned. Later it will take an array with colors that represents the current three bytes in the RGB layers and is initialized with null.

Loop over the Image’s Pixels , Al gorithm 5 →

Ln3: After the initialization step of the variables, a nested loop over the image’s ( img) pixels starts. The pixels will be traversed from left to right, from the upper-leftmost pixel to the lower-rightmost pixel. Variable y stands for the current y-coordinate position in the image. The outer loop assigns the variable y to 0 and will be entered as long as y is smaller than the image height h. Ln4: Every time the outer loop is entered, the variable rix is assigned to the product of the image width w and the current y-coordinate position y. It represents the sum of all full pixel lines of the image that have been traversed in previous iterations. Ln5: Next follows the inner loop. Variable x stands for the current x-coordinate position in the image. The inner loop assigns x to 0 and runs as long as x is smaller than the width w of the image. Ln6: When the inner loop is entered, the variable pix is assigned to the addition of rix and x. Thus pix represents the position of the current pixel in the image-traversal process. Ln7: In the next step, the variable pos is assigned to the value of the current pixel position pix modulo the amount of pixels needed for one byte in each layer ppl. Ln8: pos’s value is then utilized to determine whether the current pixel position is the start of a new pixel chunk or not. If pos equals 0 the start of a new byte is indicated and the body of the if statement is entered.

Encode Bytes into the Color Layers of an RGB Image , Al gorithm 5 →

Ln9: Afterwards a for loop follows that runs over all RGB layers. This loop initializes the variable la y with 0 and runs as long as la y is smaller than LAY (LAY is a constant

37 38 2.5. Multi-Layer-Approaches

with value 3, standing for #RGB layers). Ln 10: After the loop is entered, a check follows whether the input byte array bs contains more bytes to encode or not. Ln 11: If that is the case, it is tested whether the current iteration of the loop equals the first RGB layer by checking if la yer equals 0. Ln 12: If that is true, all bytes have been embedded in the image. img is returned to the caller and the algorithm ends at this point. Ln 13: Else if not, the loop over the RGB layers is stopped. Meaning there are less than #LAY bytes to embed in this iteration. The algorithm jumps to Line 27. Ln 17: If there are still bytes left that have to be embedded in the image, the subroutine getColorsFromByteLayered() is invoked to determine the color value for the current layer. As parameters it takes the next byte to be embedded from the input array bs, the radix γ, the amount of pixels need for one byte in one layer ppl, and the current layer la y. It returns an array containing the colors for the current layer. For example, if we are in the red layer of an RGB image, it works with #γ different red colors to represent the byte. The same applies to both of the other RGB layers. (For a detailed look of a similar function for the SL-approaches, see Algorithm2). The returning value is assigned to variable lco. Ln 18: Then an if statement follows that tests whether the current iteration contains colors of the first layer or not. This is done by checking if la y equals 0. Ln 19: If that is the case the color array col is set to the first layer’s color array lco. Ln 20-22: Else the determined colors of the layers are merged together to the final color. This is done by adding the current layer’s colors to the existing layer colors col. At the end of the layer loop over the three RGB layers (Line9), col represents the pixel colors for all three RGB layers. Ln9: If not all layers have been passed, another iteration of the layer loop is done, else the loop ends. Ln 27: Now follows the step in which the determined colors are actually drawn to their pixels. This is done by invoking the image’s function drawPixel(). It takes x, y, and the current color consisting of three independent encoded RGB layers constituting the parameter. Ln5: If x is after the increment still less than the image width w, another iteration of the inner loop starts, else the inner loop ends. Ln3: If y afterwards is less than the image height h, another iteration of the outer loop begins, else the nested loop stops and all bytes of the input image img have been successfully decoded.

38 2.5. Multi-Layer-Approaches 39

Ln 30: In the last step img is returned to the caller. The algorithm to encode bytes via a speciﬁc ML-approach ends at this point.

2.5.2. Decoding

The decoding of images encoded by ML-approaches works pretty similar to the decoding of SL-approaches that was discussed in detail in Chapter 2.4.2. Therefore a decoding algorithm for the ML-approaches is omitted.

The main steps of the decoding can be summarized as follows:

loop over all pixels from left to right, from top to bottom. • every traversed pixel color is read. If enough colors are collected to represent a full • pixel chunk, an encoding step follows.

the decoding of ML-approaches differs in one major point to the decoding of SL- • approaches: One color consists of three independent layers to be decoded. There- fore, a loop over the layers is done and every layer is treated analogous to a color introduced in the procedure for SL-approaches.

the decoded bytes are appended to an array.At the end of the procedure all encoded • bytes are stored in this array.

2.5.3. Results

In this subchapter we have introduced the ML-approaches. These approaches improve the data ratio of the corresponding SL-approaches by a factor of 3. Table 2.2 shows an overview of the available data ratios that are gained from these approaches:

Encoding Approach:

ML2 ML3 ML4 ML7 ML16 ML256 Data Ratio: 3/8 1/2 3/4 1 3/2 3 Table 2.2.: Summary Data Ratio ML-Approaches (in Bytes per Pixel)

With the ML-approaches, it is possible to produce data ratios between 3/8 Bpp up to

39 40 2.6. Images and Colors

3 Bpp.

Figure 2.10 shows the application of the ML-approaches. It shows the output images of these approaches for the 18 example input bytes of AppendixA:

ML =2 ML =3 ML =4 ML =7 ML = 16 ML = 256

Figure 2.10.: Example Output ML-Approaches

We see that the amount of pixels needed to store one byte, are dramatically reduced. For example, the ML256 only needs 6 pixels, whereas the SL-approach SL2 needs 144 pixels to store the same 18 bytes. This results in an improvement of 96% less pixel usage.

In the next subchapter we will take a closer look at the images that are used as containers to encode bytes and their respective colors.

2.6. Images and Colors

The introduced encoding approaches use images as data containers and utilize deﬁned pixel colors to store bytes. We will discuss the available image types and the colors utilized to encode bytes through speciﬁc approaches.

2.6.1. Types of Images

We distinguish between three different primary image types: black & white images, greyscale images, and RGB images.

These images types differ in the amount of available colors, which also inﬂuences the actual disk space an image needs to be stored. Like the name suggests, a black & white image can only display the two colors black and white, whereas a greyscale image consists of 256 different grey colors, and an RGB image is able to display 16, 777, 216 different colorful colors. More technically:

40 2.6. Images and Colors 41

a black & white image’s pixel is represented by 1 bit • a greyscale image’s pixel is represented by 1 byte • a RGB image’s pixel is represented by 3 bytes • Tests with photo sharing websites and their applied JPEG-compression showed that there is a correlation between the amount available colors of an image and the disk size of such an image. Moreover, there is also correlation of how well the color information in such an image survives the compression. For example, greyscale images with the same colors were less robust than RGB images. Thus, it is worth thinking about which image type to use in combination with which type of encoding approach in order to achieve the best ratio between actual disk space usage and robustness.

Since the ML-approaches are depending on RGB layers, they are only working with RGB images. This question is even more exciting for SL-approaches. All of these approaches allow for encoding in greyscale images. However, the SL2 is the only approach that is able to utilize black & white images, since it is limited to two colors. Figure 2.11 shows to which image type we assigned speciﬁc approaches in our photosharing website driven cloud storage system:

G B

RGB Three Bytes per pixel Greyscale One Byte per pixel SL7 ML2 Black&White One bit per pixel SL16 ML3 SL3 SL ML4 SL2 4 ML SL 7 256 ML16 ML256

Figure 2.11.: Types of Images with Approaches working

As can be seen, the SL2 works with black & white images, whereas the SL3, the SL4, and

41 42 2.6. Images and Colors

the SL256 utilize greyscale images. The SL7, the SL16 and all ML-approaches team play with RGB images.

The SL7 and the SL16 are assigned to RGB image containers since tests showed that the distance between the grey colors are too small to survive JPEG-compression applied by photo sharing website providers. Instead of 7, respectively 16 grey-scale colors, we used the more distinct colors of an RGB image to gain better robustness against compression.

The SL256 on the other hand is assigned to the greyscale images even if it utilizes more colors than the SL7 and the SL16. This is due to the fact that the SL256 and also the ML256 are playing a special role in our set of encoding approaches. Both approaches are not depending on specially deﬁned colors, but instead use the full range of colors available for a pixel with their assigned type of image container. As mentioned before, the color of a pixel of a greyscale image is described by 1 byte. The SL256 uses this full byte to encode bytes from an input byte array. The same applies to the ML256, which uses the full 3 bytes of an RGB image to embed input bytes.

2.6.2. Predened Colors Single-Layer-Approaches

The SL-approaches SL2, SL3, SL4, SL7, and SL16 all have in common that they are utilizing a manageable amount of colors (between two and sixteen) to encode bytes. Therefore it is possible to have a closer focus on their portfolio of certain colors and predeﬁne them. This is an important feature of these approaches if we have to deal with JPEG- compression, e.g. applied by certain photo sharing website to uploaded images.

During the JPEG-compression process the original colors of an image are converted from RGB color space to the YCbCr color space and vice versa. These color space conversions are jointly responsible for loss of color information, since colors that have a maximum distance in the RGB color space are not guaranteed to have the same distance in the YCbCr color space. It can be determined that the less colors are used and the greater the distance between these colors in both color spaces, the more robust is an approach to a certain kind of compression.

42 2.6. Images and Colors 43

Color Selection Scheme

Good information about which colors to use are found in Kato’s “Novel Colour Selec- tion Scheme for 2D Barcode” [KTC09]. It introduces a color scheme consisting of 10 recommended RGB colors that are designed to gain the best available robustness against deviation of colors through JPEG-compression. The idea behind Katos’s color scheme is based on choosing colors that preserve their distance between their neighboring colors across the RGB color space and the YCbCr color space.

Kato experimented with the eight color vertices of the RGB color cube red (R), green (G), blue (B), cyan (C), magenta (M), yellow (Y), black (K), and white (W) and their suscep- tibility to compression. He found that colors from a plane with the corners KRWC provide the best robustness, which allowed him to generate his color scheme. Figure 2.12 shows this plane and the 10 recommended colors represented once in the RGB color space and once in the YCbCr color space. In order to gain enough colors that allow also encoding with the SL16, additional 6 colors on the plane have been selected for our purpose and are also displayed in this ﬁgure (both graphics are based on graphics from [KTC09]):

43 44 2.6. Images and Colors

1 B

0,8 W 0,6 Blue 0,4 M G 0,2

0 0 K 0,2 0,4 Y Red 0,6 1 0,8 0,8 0,6 0,4 1 0,2 10 by Kato deﬁned colors R 0 Green 6 additional deﬁned colors

(a) RGB color space

R 1

0,8 M

RGB color space

0,6 K Y

Cr W 0,4 B

0,2 G

0 0 0 0,2 C 0,2 0,4 0,4 0,6 0,6 0,8 0,8 1 1 10 by Kato deﬁned colors Y 6 additional deﬁned colors Cb

(b) YCbCr color space

Figure 2.12.: 10 Colors derived from [KTC09] and 6 additional deﬁned Colors

From Kato’s color plane we can derive the following 9 colors in normalized range [0,1]: (0,0,0), (0.5,0,0), (1,0,0), (0,0.5,0.5), (0.5,0.5,0.5), (1,0.5,0.5), (0,1,1), (0.5,1,1), (1,1,1)

44 2.6. Images and Colors 45

Kato recommends the color yellow (1,1,0) as the 10th since it has proven to be especially insusceptible during testing.

We derived 6 additional colors from the color plane, in addition to the colors recommended by Kato in order to also use the SL16 with custom colors. These colors are: (0,0.25,0.25), (0.5,0.25,0.25), (1,0.25,0.25), (0,0.75,0.75), (0.5,0.75,0.75), (1,0.75,0.75)

Color Assignment

Combined with the in the previous subchapter introduced mapping of our approaches to the three available image types, this results in the following color assignments:

SL16

(0.5,0,0) (0.5,1,1) SL7 (1,1,0) (0,0.25,0.25) (0,1,1) SL3 SL SL2 4 (0.5,0.25,0.25) (1,0,0) (0.5,0.5,0.5) (0,0,0) (0.25,0.25,0.25) (1,0.25,0.25) (0,0.5,0.5) (1,1,1) (0.75,0.75,0.75) (0,0.75,0.75) (1,0.5,0.5) (0.5,0.75,0.75) (1,0.75,0.75) Black & White Image

Greyscale Image

RGB Image

Figure 2.13.: Predeﬁned colors for SL-Approaches SL2, SL3, SL4, SL7, and SL16

The ﬁgure’s color values are given in the normalized range [0,1]. Underlined colors are derived from [KTC09] as to be the ten best colors in case of robustness against JPEG- compression.

As shown, the sets of predeﬁned colors for the different approaches are interdependent.

In fact with exception of the SL4-approach, the colors of the approaches with a smaller radix γ are a proper subset of the approaches with a greater γ. In other words the colors of SL2 are a proper subset of the SL3, whereas the colors of the SL3 are a proper subset of the SL7, and the colors of the SL7 in turn are a proper subset of the SL16.

45 46 2.6. Images and Colors

2.6.3. Generated Colors Multi-Layer-Approaches

One difference between the SL- and the ML-approaches is that the ML-approaches do not work with predeﬁned colors. Instead the colors are generated by an algorithm that splits the color range of an RGB layer into γ-different colors with equal distance. This algorithm is described in the following:

Algorithm to determine Colors for ML-Approaches

Algorithm 6 determineLayeredColors(γ: Integer): Color[][] 1: col := Color[LAY ][γ] //LAY equals #RGB layers 2: dis := 256 γ 3: for i := 0 to÷γ 1 do 4: val := i dis− 5: for la y :×= 0 to LAY do 6: if la y == 0 then 7: col[la y][i] := Color(val,0,0) 8: else if la y == 1 then 9: col[la y][i] := Color(0,val,0) 10: else 11: col[la y][i] := Color(0,0,val) 12: end if 13: end for 14: end for 15: return col

The function determineLayeredColors() takes the number of Colors per layer, respectively the radix γ utilized by the encoding approach, as invocation parameter. It returns a two-dimensional array whereas the ﬁrst dimension stands for the three layers of an RGB image and the second dimension holds the corresponding colors of the layer.

The main steps can be summarized as follows:

initialize an array with appropriate size to store all determined colors (col) • calculate maximal distance between each color in one RGB layer by dividing 256 • through γ (dis).

start a nested loop, whereas the outer loop iterates over the amount of colors and •

46 2.7. Summary 47

when it is entered the current color value val is determined. The inner loop then iterates over the single layers.

every time the inner loop is entered, new colors for the current layer are created • by invoking the function Color() with the current color value val. Its parameter stands for the three RGB components and can have values between 0 and 255. A red layer color for example would then be Color(25,0,0).

at the end all determined colors are stored in col. • Result

An Example for the output of this algorithm is shown in Figure 2.14:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 R

(0,0,0) (17,0,0) (34,0,0) (51,0,0) (68,0,0) (85,0,0) (102,0,0) (119,0,0) (136,0,0) (153,0,0) (170,0,0) (187,0,0) (204,0,0) (221,0,0) (238,0,0) (255,0,0) G

(0,0,0) (0,17,0) (0,34,0) (0,51,0) (0,68,0) (0,85,0) (0,102,0) (0,119,0) (0,136,0) (0,153,0) (0,170,0) (0,187,0) (0,204,0) (0,221,0) (0,238,0) (0,255,0) B

(0,0,0) (0,0,17) (0,0,34) (0,0,51) (0,0,68) (0,0,85) (0,0,102) (0,0,119) (0,0,136) (0,0,153) (0,0,170) (0,0,187) (0,0,204) (0,0,221) (0,0,238) (0,0,255)

Figure 2.14.: Sample Colors generated by Algorithm6 for the ML16 Approach

We see that the distance between colors is distributed equally for each layer. The main disadvantage of this technique is that these colors are not selected in a custom way like SL-approaches and only take the RGB color space into account.

2.7. Summary

In this chapter we have introduced twelve different encoding approaches. All of these approaches have unique features that enable us to later face the issue of speciﬁc photo sharing websites’ treating of uploaded images and gain the best data ratio without data loss. Table 2.15 summarizes all 12 encoding approaches and their features:

47 48 2.7. Summary

Encoder γ #L Bpp ppB Image Type

SL2 2 1 1/8 8px Black&White SL3 3 1 1/6 6px Greyscale SL4 4 1 1/4 4px Greyscale SL7 7 1 1/3 3px RGB SL16 16 1 1/2 2px RGB SL256 256 1 1 1px Greyscale ML2 2 3 3/8 8/3px RGB ML3 3 3 1/2 6/3px RGB ML4 4 3 3/4 4/3px RGB ML7 7 3 1 3/3px RGB ML16 16 3 3/2 2/3px RGB ML256 256 3 3 1/3px RGB

Figure 2.15.: Summary of the Features of the different Encoding Approaches.

The first column shows the specific encoding approach, the second column its utilized radix γ, the third column shows the amount of layers, the forth column denotes the bytes per pixel ratio, the fifth column contains the amount of pixels needed to store a byte, and the sixth column shows the image type assigned for this approach.

To compare the introduced encoding approaches additional benchmarks have been generated that more closely illuminate their application:

2.7.1. Benchmarks

The following benchmarks are based on random input bytes. The input increases expo- x nential with the size of 2 | x N, 10 x 20 and therefore ranges between 1KiB and 1MiB. The results itself base∈ on averaged≤ values≤ of 50 runs for each x. This will was applied for all following benchmarks in this paper.

First, we will compare the encoding and decoding times of the approaches. Afterwards we will take a look on the overhead produced when comparing the encapsulated data with the size of its image container.

48 2.7. Summary 49

Encoding Times

Figure 2.16 shows the time needed to encode bytes with a speciﬁc encoding approach (logarithmic scale):

Enc. Time: SL2 Enc. Time: SL7 Enc. Time: ML2 Enc. Time: ML7 50000 50000 Enc. Time: SL3 Enc. Time: SL16 Enc. Time: ML3 Enc. Time: ML16 Enc. Time: SL4 Enc. Time: SL256 Enc. Time: ML4 Enc. Time: ML256 5000 5000 500 500 Time [ms] Time [ms] 50 50 5 5

210 211 212 213 214 215 216 217 218 219 220 210 211 212 213 214 215 216 217 218 219 220

Data Input [byte] Data Input [byte]

(a) Encoding Times SL-Approaches (b) Encoding Times ML-Approaches

Figure 2.16.: Encoding Times all Encoding Approaches

We see that the encoding times and the utilized radix γ correlate. This is due to the fact that the higher γ, the higher is the data ratio and therefore the less steps are needed to embed certain bytes. This means that the resulting image containers have smaller dimensions and less pixels have to be traversed during the encoding steps, which results in an overall faster encoding.

Looking at the SL-approaches in Figure 2.16a, we see that the SL256 performs especially well compared to the encoding times of the other SL-approaches. The reason is, as previously mentioned, that the SL256 and the ML256 use the full broadband of their image container’s pixel range. Thus, they do not need to deal with special deﬁned colors and conversions between numeral systems, bytes can be stored directly as pixel value, which makes it possible to implement these approaches with faster algorithms than the ones previously introduced. Storing the pixel values directly, has been done in both imple- mentations of the encoding approaches in our photosharing website driven cloud storage system. Especially the SL256 beneﬁts from this custom algorithm.

Decoding Times

Next we look at the decoding times. Figure 2.17 shows the decoding times for all approaches (logarithmic scale):

49 50 2.7. Summary

Dec. Time: SL2 Dec. Time: SL7 Dec. Time: ML2 Dec. Time: ML7

Dec. Time: SL Dec. Time: SL 2000 Dec. Time: ML3 Dec. Time: ML16 2000 3 16 Dec. Time: SL4 Dec. Time: SL256 Dec. Time: ML4 Dec. Time: ML256 500 500 100 100 Time [ms] Time [ms] 20 20 5 5 2 2

210 211 212 213 214 215 216 217 218 219 220 210 211 212 213 214 215 216 217 218 219 220

Data Input [byte] Data Input [byte]

(a) Decoding Times SL-Approaches (b) Decoding Times ML-Approaches

Figure 2.17.: Decoding Times all Encoding Approaches

We see that the decoding correlates to the encoding times. The higher γ, the smaller the dimensions of the image, the less pixels have to be traversed, and as a result, the faster the decoding.

The decoding times again show the performance superiority of the SL256 and ML256 and their custom implementation. Both approaches do not need to convert a pixels’ color values to regain the original stored byte, instead the color value corresponds to the actual byte’s value. This enables us to omit certain steps in the decoding process, which compared to other approaches is less time costly.

Overhead

Another important measure to classify the approaches, is the overhead size of generated output images. We proceed by determining the ratio between image container disk size and its containing data, therefore, we calculate the division between disk space usage of the speciﬁc image containers through the size of the data that is stored inside.

Figure 2.18 shows the overhead comparison between the certain approaches (logarithmic scale):

50 2.7. Summary 51 2.0 2.0 SL2 B&W SL4 GS SL16 RGB ML2 RGB ML4 RGB ML16 RGB SL3 GS SL7 RGB SL256 GS ML3 RGB ML7 RGB ML256 RGB 1.8 1.8 1.6 1.6 1.4 1.4 1.2 1.2 Overhead of Uploaded Image to Input Size Overhead of Uploaded Image to Input Size Overhead 1.0 1.0 210 211 212 213 214 215 216 217 218 219 220 210 211 212 213 214 215 216 217 218 219 220

Data Input [byte] Data Input [byte]

(a) Overhead SL-Approaches (b) Overhead ML-Approaches

Figure 2.18.: Overhead all Encoding Approaches

The graphs show how the selection of image types and their team play with a certain approach inﬂuence the overhead of output images.

As shown, the SL2 has the smallest overhead than all approaches, even better than the

SL256 and ML256. That is due to the fact that the two colors of such image can be better compressed than images with more colors. It can be said that the less colors are used in an image, the better smaller the overhead ratio.

This also applies to the SL-approaches SL3, SL4, and SL256. These approaches are used in team play with greyscale images as data containers and are superior to their corresponding ML-approaches with respect to their overhead, although, their actual data ratio is only a third compared to the ML-approaches.

2.7.2. Results

The data ratio of an approach is correlating with the processing time of encoding and decoding. The higher the radix γ, the higher the data ratio, and the faster the processing.

A likewise beneﬁcial result is that the overhead of output images is strongly depending on the container type of the image. It showed that the SL2 is able to produce the smallest overhead, even though it has the worst data ratio of all in our ﬁeld of approaches.

Important is that these measures are based on lossless compressed PNG-images. As we

51 52 2.7. Summary will see in Chapter3 about the application of our encoding approaches, this overhead advantage can be reversed through JPEG-compression.

Figure 2.19 summarizes the applying of the twelve introduced encoding approaches by showing the output for the input bytes of AppendixA and its type of image container:

SL =2 SL =3 SL =4 SL =7 SL = 16 SL = 256

Black & White Image Greyscale Image RGB Image

ML =2 ML =3 ML =4 ML =7 ML = 16 ML = 256

Figure 2.19.: Example Output for all Approaches

The output enables to visually compare the data ratio of the encoding approaches used to encode the array of 18 sample bytes.

In the next chapter, we will discuss the application of the introduced encoding approaches.

52 3 Implementation of our Photosharing Website driven Cloud Storage System

This chapter is about our developed cloud storage system. It addresses the previously introduced encoding approaches as key part of the system.

(a) Facebook (b) Flickr (c) Picasa

Figure 3.1.: Logos of the three major Photo Sharing Websites

The applicability of this system is shown by exploiting the photo sharing websites Face- book, Flickr, and Picasa Web Albums as cloud storages.

It is important to know that Flickr completely renewed its service during the writing of this thesis on May the 20th 2013 [Fli13c]. All feature information and measures relating to Flickr are based on Flickr’s service before this date.

3.1. Photosharing Website driven Cloud Storage System

The important steps to render the introduced encoding approaches applicable to our concrete scenario and to produce a fully functional photosharing website driven cloud storage system are discussed in the following.

The subsections in detail:

53 54 3.1. Photosharing Website driven Cloud Storage System

1. In the ﬁrst subsection we will brieﬂy address the optional applied reed solomon [RS60] error-correction code (ECC) that we included in our system. The reed solomon implementation we adapted for this purpose is derived from Google’s open source barcode library ZXing [ZXi13].

2. The second subsection demonstrates the areas and sectors an image contains when generated with our system. It will show why more than the actual data is stored an output image and why these sectors are crucial for the working of our system.

3. In the last subsection, we will outline the concrete implementation of our cloud storage system as part of the widely used cloud API jClouds. That requires that we ﬁrst have a look at the API itself and afterwards look at the concrete upload- and download workﬂows our system is relying on.

3.1.1. Error Correcting Code (Reed Solomon)

As mentioned before, some photo sharing websites post-process uploaded images and apply JPEG-compression, which harms the embedded data. In our group of providers this is done by Facebook and [formerly] Flickr. To solve this problem, we have developed twelve different encoding approaches, which allow us to ﬁnd the right balance between a preferably high bytes per pixel ratio and robustness against compression. It is valid to say that the less colors are used for the encoding, the greater is the distance between the single colors, and the more robust is the output image against compression. However, there are circumstances in which the failure rate of a speciﬁc approach/provider combination is proportionately small. Such a case is Flickr with its [former] service and the two ML-approaches ML7 and ML16.

The displayed plot in Figure 3.2 shows the relative byte failure rate of images hosted on Flickr that were generated by the approaches SL256, ML7, ML16, and ML256:

54 3.1. Photosharing Website driven Cloud Storage System 55 1.0 0.8 0.6 SL256 ML16 ML7 ML256 0.4 Failure Rate [%] Failure 0.2 0.0

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.2.: Flickr: Failure Rates of SL256, ML7, ML16, and ML256

The x-axis shows the amount of random input bytes, and the y-axis shows the percent- age of extracted failure bytes. In contrast to the SL256 and ML256, which demonstrate a failure rate between 70% and 80% (lower failure rates may adulterated by the random input bytes and the small input), the approaches ML7 and ML16 have a constantly low rate of failures of about 3%.

Nevertheless, ML7 and ML16 produce failures and are therefore not effective in team play with [former] Flickr. A switch to the next lower encoding without decoding failures, the ML , is costly with respect to the data ratio. From a bytes per pixel ratio of 3 for the 4 2 ML , we would fall back to a ratio of 3 for the ML , which is a loss of half of the data 16 4 4 rate. Consequently, in cases of such controllable failure rates it is better to invest some overhead bytes to be reserved for error correction. We chose to add 10% overhead bytes for the reed solomon ECC. This enables us to correct failure rates up to 5%. Utilizing this technique, it is possible to also use the ML7 and ML16 with Flickr, although, they produce some byte failures. Encoding approaches with ECC applied to their input bytes are from now on denoted with a lowered “E” after their γ-value. In this case ML7 and ML16 with

ECC enabled are denoted as ML7E and ML16E (see also AppendixB for naming conventions in this thesis).

To calculate our gain from the ECC in the Flickr scenario, we assume that we want to

55 56 3.1. Photosharing Website driven Cloud Storage System

store 1024 bytes on Flickr. If we take the ML4, which is the approach with the highest bytes per pixel ratio without decoding failures and its bytes per pixel ratio of 3 , it costs us 4 1366 pixels to embed these bytes. The alternative would be the ML16E. Because of the

10% ECC overhead, we have to embed 1103 bytes. The ML16E itself has a bytes per pixel ratio of 3 , thus we need 736 pixels to store the bytes. As a result the ML saves us 630 2 16E pixels with respect to the ML4.

3.1.2. Areas and Sectors of generated Images

We are now turning to the areas and sectors contained in the images generated by our cloud storage system.

Until now, we focused on the pure output of speciﬁc encoding approaches for the given input bytes. This part of the images is from now on denoted as bytes sector. As we will see, the bytes sector is just one of the seven sectors in our images, even if the most important one. Additional sectors are appended by our cloud storage system to store crucial information about the image and its encoding. These sectors increase the applicability of images as data containers in our setting. The created images can be divided into two areas containing depending sectors. Figure 3.3 illustrates how a created image can be split into its areas and sectors:

56 3.1. Photosharing Website driven Cloud Storage System 57

Size Sector (32px) Encoding Area Sector (42px) (8px) ML? Sector (1px) ECC? Sector (1px)

Bytes Sector Data Area

ECC Sector

Unused Sector

(a) Output Image ML2E (b) Output Image’s Main Sectors and Sections

Figure 3.3.: Sectors of generated Images

On the left hand side in Figure 3.3a, we see the output of the ML2E for the random input bytes of AppendixA. The right hand side of Figure 3.3 displays this image divided into its two main areas with the depending sectors, the encoding area and the data area. In the following we will take a detailed look at these areas.

Encoding Area

The ﬁrst 42 pixels are reserved for the encoding area. In this area information about the encoding of the embedded data is stored. This information makes it possible to later extract the data embedded, even without knowing further details about the encoding used to create the image. Since the encoding area is crucial for decoding, it has to be stored as robust as possible, meaning that this area is always encoded with the SL2, which is the most robust encoder available.

The encoding area can be further divided into four sectors:

1. The ﬁrst sector is called size sector and has a length of 32 pixels. It stores the amount of bytes that are embedded in the image. The size value is important to determine

57 58 3.1. Photosharing Website driven Cloud Storage System

up to which pixel the image data is stored. Hence the SL2 is used to encode the encoding area, supporting a maximum size value of 232 is supported. In this example the size of the embedded data is 22 bytes (10110). 2. The second sector is the γ sector. It stores the γ-value used by the encoder of the data area. Since a byte can have values between 0 and 255, it is γ = value(γ sector)+1. Therefore in this example γ = 2. 3. The third sector is named ML? sector. It holds a Boolean value whether the encoder was a ML-approach or not. In this example this value is set true. 4. The fourth and last sector of the encoding area is the ECC? sector. It holds a Boolean value whether the encoded bytes also contain error-correction code bytes or not. This is set true in this example.

If we summarize the information stored in the example’s encoding area, we ﬁnd that the encoder is an ML-approach with γ = 2 and ECC enabled. In short: The encoder is the ML2E. We also know that a sum of 22 bytes are embedded in the data area.

Data Area

As the name suggests, this area contains the actual data embedded in the image. In contrast to the encoding area, the data area fully depends on the encoding of parameters and has a variable size.

The data area can be further divided into three sectors:

1. The ﬁrst sector is the bytes sector. It contains the actual encoded bytes. 2. The second sector is the ECC sector, which contains the bytes of the error-correction code. This sector only exists if ECC is enabled 3. The third and last sector of the data area is the unused pixels sector. It consists of overhead pixels that are not used. The existence and the size of this sector depends on how well the data ﬁts into the image with chosen encoder. Additionally, some providers restrict the image to minimum dimensions, e.g., images to be hosted on Facebook have to have a height of at least 5 pixels.

By utilizing the information gathered from the encoding area, it is possible to decode the data stored in the data area.

58 3.1. Photosharing Website driven Cloud Storage System 59

3.1.3. Integration in jClouds and System Workows

One of the major decisions for the implementation was to include our project into the jClouds API [jCl13]. jClouds is an open source library for the programming language JAVA with the purpose of providing a portable abstraction that makes it possible for developers to address different cloud providers without being forced to apply their concrete API’s. Instead, once they included jClouds into their project there are only some parameters needed to change to switch from one cloud provider to another. Until now jClouds supports 30 different cloud providers, amongst others AWS S3 [Ama13] from Amazon or Azure [Mic13] from Microsoft.

For our project we forked jClouds and extended it with our approach of a photosharing website driven cloud storage system. At this time, the storage providers supported by our version of jClouds are the photo sharing websites Facebook, Flickr, and Picasa Web Albums. The integration provided us with the opportunity to equally compare the performance of the subjected photo sharing websites with each other and also with professional cloud storage providers. Moreover, our technique can be utilized without barriers. Due to the fully documented interfaces, it is also possible to add more photo sharing website providers at a small expense. All performance measures about our approaches and their application are derived from this implementation. Our fully functional and documented jClouds fork is hosted on the social coding platform GitHub and available under: [MG13].

The key points of how our implementation deals with data to be uploaded and hosted on a photo sharing website and vice versa, how data is regained from a concrete photo sharing website, will be outlined in the following.

Upload Workow

To utilize photo sharing websites as cloud storage backends, the key questions are how to store data objects in images and how to host these images on a speciﬁc photo sharing provider afterwards. Figure 3.4 shows a step-by-step workﬂow of how these tasks are solved in the project’s implementation:

59 60 3.1. Photosharing Website driven Cloud Storage System

Object byte[] Split byte[][] Next Fragment byte[] → byte[]

Reed-Solomon ECC Hosted on Yes Yes Frags Encoding enabled Photo Sharing left Website Store Encoding No No Stop properties all other HTTP 200 Upload PNG- Picture return Image Encoding

Figure 3.4.: Upload Workﬂow

The workflow starts at the blue area in the upper left corner. First the bytes of the data object to be hosted are extracted and stored into a byte-array. If the size of the byte-array is greater than the amount of bytes one image of a specific photo sharing website/encoding combination can hold, the byte-array is split into equal-sized fragments that fit best into the images to be generated. The split bytes of the original array are stored into a two- dimensional byte-array, whereas the first dimension stands for the number of fragments, and the second holds the bytes of each fragment.

Next follows the encoding of the fragments shown in the green area of the figure. For each fragment in the two-dimensional byte-array a loop is done to embed the fragment’s data into an image and to afterwards host this image on the predefined photo sharing website. In the first step of the loop there is a check whether ECC shall be used or not. If ECC is enabled, the byte-array of the current fragment runs through a reed solomon encoding which appends several additional ECC-bytes to the given byte-array. Next, the properties of the encoding that will be used to embed the fragment’s bytes are stored into the first 42 pixels of the output-image, the so called encoding area of the image (for further details see also: Chapter 3.1.2). These properties contain all important information to later decode the data embedded in the output-image. Since this area is crucial, it is always encoded with the most robust encoder available, the SL2, independent from the chosen encoder of the input bytes. In the next step, the bytes of the current fragment are encoded into pixels with the given encoding properties and stored right after the first 42 pixels of the encoding area, into the so called data area of the image.

60 3.1. Photosharing Website driven Cloud Storage System 61

Now the image contains all data of the current fragment and is ready to be uploaded to the given photo sharing website. The upload step is shown in the red area of the ﬁg- ure. In this step, a HTTP-POST request with the image-data is send to the provider. If the HTTP-return is other than status code 200, which means a successful transmission, this step is looped until the host acknowledges that the image was sent properly.

The upload-step is followed by the hosting-step and shown in the yellow area of the ﬁgure. Once an image was uploaded to the provider and internally processed, a check follows whether there are fragments left that are not uploaded yet. If fragments are left, the loop starts again with the next fragment; if not, the complete data object is successfully uploaded and represented by the images hosted at the photo sharing website provider. The program stops at this point.

Download Workow

Once data objects are hosted on photo sharing websites, another important question is how to regain access to these objects. Figure 3.5 illustrates the workﬂow of this task and shows how to regain original data objects from hosted images:

Hosted on Download HTTP JPEG-/PNG- Determine Photo Sharing Fragment return Image Decoding Website

all other Object Image → byte[] Decoding Cache Fragment Reed-Solomon Yes byte[] byte[] Yes Decoding Frags ECC Combine No Fragments left byte[] No enabled

Figure 3.5.: Download Workﬂow

The workﬂow starts at the yellow area in the upper left corner, which symbolizes images containing data objects hosted on a photo sharing website. To separate images belonging together, virtual photo albums and naming conventions can be used, e.g., if a data object due to its size is fragmented over several images. If a speciﬁc data object is requested, a loop over all images containing fragments of this data object has to be done. Every

61 62 3.2. Facebook fragment needs to be downloaded and decoded, in order to at least combine all data fragments of the original data object.

The download step is shown in the red area of the ﬁgure. For every fragment, respectively concrete image, a HTTP-GET request is send to the image host. If the host answers with status code other than 200, this step is looped until the image is fetched.

The download step is followed by the decoding step shown in the green area of the workﬂow. First the encoding properties of the image have to be determined to guarantee a successful decoding of the in the image embedded data. Consequently, the encoding area of the image is evaluated. It contains information about the length of the stored bytes, the encoding’s utilized radix γ, whether it is a ML-approach or not, and whether ECC is used or not. Next these encoding properties are used to select the right decoding and extract the bytes embedded in the data area of the image. If ECC was used, a reed solomon decoding on the extracted bytes is performed as well. A check follows, whether fragments of the requested data-object are still left. In that case the bytes extracted from the current fragment are cached in a two-dimensional byte-array. The ﬁrst dimension stands for the amount of fragments of the requested data-object and the other dimension stands for the bytes extracted from a particular fragment. After the bytes of the current fragment are cached another iteration starts with the next fragment from download-step to decoding-step.

If, however, all fragments are downloaded, decoded, and no fragment of the requested data-object is left, a combine-step as shown in the blue area of Figure 3.5 follows. In this step all the cached fragments contained in the two-dimensional byte-array are combined to a normal byte-array representing the bytes of the data-object. Afterwards this array is converted into the requested data-object. The program stops at this point.

3.2. Facebook

With 1.11 billion members, Facebook [Fac13a] is currently the largest social network in the world [Fac13b]. It was launched in February 2004. Due to the fact that uploading photos and sharing them with other people is one of the most popular features, it is also the world’s largest photo sharing website. Every week users upload about 1 billion images, respectively 60 terabyte of images to Facebook [BKLaPV10]. In this section, we

62 3.2. Facebook 63 will discuss how well Facebook’s massive infrastructure can be used as free additional storage by utilizing it with our photosharing website driven cloud storage system.

3.2.1. Features

Provider: Facebook Costs: free Upload Limit: unlimited JPEG-compression: (quality factor: 0.84) ∼ Max. Resolution: 2048 2048 min. image× width: 2048px, Other Restrictions: min. image height: 5px

Usable Encoders: SL2, SL3, SL4

Table 3.1.: Features of Facebook

Table 3.1 sums up the in our context interesting features of Facebook as storage provider:

Positive: Facebook is absolutely free and does not limit the amount of images that • a registered user can upload to his/her account. Additionally, the maximum image resolution of 2048 2048 allows us to use image data containers of suitable size. × Negative: Facebook applies image compression on uploaded images. With a JPEG • quality factor of around 0.84 (determined with [Has13]), it has the strongest compression in our ﬁeld of providers. As we will see, this results in only three effectively usable SL-approaches.

Other Restrictions: Facebook forces the uploading of images with a minimal reso- • lution of 5 2048, else the image ratio is changed and ends up in unexploitable image containers.×

3.2.2. Implementation

For the inclusion of Facebook as one of the selectable providers for our system, we utilized the RestFB API [All13]. It is a JAVA library that supports the access to Facebook’s Graph API [Fac13c].

63 64 3.2. Facebook

3.2.3. Benchmarks

Failure Rates

Facebook applies JPEG-compression to uploaded images. As a direct consequence, there are encoders that are not effectively usable with Facebook and produce decoding failures. To measure these failures, an array of random bytes is encoded with a speciﬁc encoding approach and uploaded to Facebook. After the image has been post-processed and is hosted on Facebook, the image is downloaded and the inside stored bytes are decoded. The decoded array is then compared with the original array and all altered bytes are counted as a failure byte. Figure 3.6 shows all encoding approaches that produce byte failures and their speciﬁc failure rate: 1.0 0.8 0.6 0.4 Failure Rate [%] Failure SL ML ML

0.2 7 2 7 SL16 ML3 ML16 SL256 ML4 ML256 0.0

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.6.: Failure Rates Facebook

We see that the failure rates of the failure producing approaches all lay above 60%. This high ratio of failures makes it impossible to work with our ECC to improve the set of working approaches. The ECC could just compensate a 5% failure rate (see also Chap- ter 3.1.1). Consequently, only the SL-approaches SL2, SL3, and SL4 work with Facebook.

64 3.2. Facebook 65

Overhead

Another consequence of the usage of JPEG-compression is that the image size of the original uploaded PNG-images differ from the hosted images. In normal cases the images become less in size, however, we will see that in our case the compression does not compress at all and instead increases the size of uploaded images. Figure 3.7 shows a comparison between the overhead of uploaded ﬁles in contrast to their corresponding downloaded ones (logarithmic scale): 10 Upload: SL2 Download: SL2 Upload: SL3 Download: SL3 Upload: SL4 Download: SL4 5 2 Overhead of Image to Input Size [%] of Image to Input Size Overhead 1

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.7.: Overhead of uploaded Images on Facebook

The result is that images post-processed by Facebook’s JPEG-compression have a dramatically increased overhead. The SL2 performs the worst with a rise of around 5.27 in the middle. This is remarkable because the original images generated by the SL2 have had the smallest overhead and the from Flickr downloaded ones have the largest overhead. This correlates with a switch of the color palette of these images. The uploaded data containers are black & white images in the PNG ﬁle format using 1bit as their color palette, whereas the downloaded ones are RGB images in compressed JPEG ﬁle format with a 24bit color palette. Nevertheless also the best ratio with a rise of around 3.15 in the middle for the SL4 is disappointing. Like JPEG-compression in general, the by Facebook applied compression does not do well dealing with images that have sharp edges and pixel transitions like the output images of our encoding approaches. JPEG-compression

65 66 3.2. Facebook is laid out for photos with areas of very similar colors and performs badly if there are none of those areas. This additional constant overhead is a disadvantage to the trafﬁc between Facebook and the potential user of our system who wants to regain previously stored data.

Performance

We will look at the performance of Facebook as storage provider of our photosharing website driven cloud storage system. Figure 3.8 shows the upload and download times of the single encoding approaches in team play with Facebook compared to Amazon’s professional cloud service AWS S3 (logarithmic scale):

Upload: AWS Download AWS Upload: SL2 Download: SL2 Upload: SL3 Download: SL3

20000 Upload: SL4 Download: SL4 5000 Time [ms] 1000 200

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.8.: Facebook vs. AWS S3: Upload and Download Time

It can be determined that overall the SL4 performs best in team play with Facebook. This correlates with the results of the overhead measures. Taking both into account, the SL4 is the most efﬁcient approach and recommended to be used with Facebook.

A comparison of the time it takes to upload the same amount of data between Facebook utilizing the SL4 and Amazon’s AWS S3, yields that Facebook needs about an average of 36 times longer than AWS S3. The standard deviation is about 20 for the measured ∼values. It can be said that the greater the data uploaded, the greater∼ is the performance distance between Facebook with SL4 and AWS S3. In the worst case, Facebook needs even

66 3.3. Flickr 67 about 67 times longer than AWS S3. This is true for the inputs of 219 bytes and 220 bytes. For∼ the download it is an average factor of about 10 times that Facebook is slower than AWS S3. The here standard deviation yields just about∼ 3. ∼ 3.2.4. Results

Although, Facebook has the advantage of being absolutely free without limits for the amount of uploaded data, its disadvantage is the lack of performance. The time to upload data to Facebook is at average about 36 times, at worst even about 67 times, higher than the professional AWS S3. For∼ the download it is about an average∼ of 10 times. Due to its JPEG-compression, Facebook also has to struggle with a massive overhead∼ for uploaded images. At best, it bloats image containers over 3.5 times the size the actual data inside. Moreover, the JPEG-compression minimizes the options of encoders that can be used. With a rather poor data ratio of 1 Bpp the SL is the best approach available. 4 4

As a result, Facebook is not usable as high performance storage. Nevertheless, it is a free service and this may be the strongest argument. In cases of non-time-critical scenarios the disadvantages can be overlooked. A point not mentioned is that since Facebook is a social network, the sharing of the uploaded data with friends be managed easily (e.g. deﬁning constraints with whom to share).

3.3. Flickr

Flickr [Fli13a] is a popular photo sharing website owned by Yahoo. It started in February 2004 and was acquired by Yahoo in March 2005. In contrast to Facebook and its focus on social features, Flickr’s primary aim is to offer photo enthusiasts a place to share their high quality photos with like-minded persons. The other orientation is also reflected in a much smaller user base. The latest figures released from Yahoo are from August 2011 [Yah13]. At this time Flickr had 51 million users, and although, this figure has increased since then, it is far behind Facebook’s user base of over 1 billion registered people. Also the amount of uploaded photos is worlds apart from Facebook’s figures. On August the 4th 2011 Flickr [Fli13b] celebrated the 6 billionth uploaded photo, although, Flickr users uploaded in five years only the amount of image data, Facebook users generally upload in one week. Nevertheless Flickr is smaller than Facebook, it is still a giant within its territory of pure photo sharing websites and also has a strong infrastructure. In the following we

67 68 3.3. Flickr will see how well it works in the context of our system.

3.3.1. Features

Old Services (before May 20th 2013) New Services Provider: Flickr ? Flickr Pro Flickr Flickr Pro 3 month: $6.95 Costs: free 1 year: $24.95 free 1 year: $499 2 years: $44.95 300MB per month unlimited Upload Limit: / / 1TB 2TB 30MB per photo 50MB per photo JPEG-compression: (quality factor: 0.96) - - - Max. Resolution: 2048 2048 ∼ 4096 4096 - - Other Restrictions: image× min. width: 2048px - × - - SL , SL , SL , SL , SL Usable Encoders: 2 3 4 7 16 All All All ML2, ML3, ML4, ML7E , ML16E

Table 3.2.: Features of Flickr and Flickr Pro

Like mentioned before, Flickr changed its services on May the 20th. Table 3.2 summarizes the features of Flickr’s services before and after this date. All information bases on Flickr’s services before this date:

In contrast to the other available photo sharing websites in our set of supported providers, Flickr distinguishes between a free and a paid service. The basic service is at no cost and the advanced service Flickr Pro is fee bound. Since the advantage of our system is that the storage is for free, we focused our tests on the basic service of Flickr, in the table marked with an asterisk (?).

Positive: The basic Flickr service is free of charge and provides a resolution of at • max 2048 2048, which allows for utilizing image containers with a proper size. Also Flickr×applies JPEG-compression to uploaded images, with a quality factor of about 0.96 (determined with [Has13]) it uses a quite less compression than for example Facebook, which allows us to use approaches up until the ML16E.

Negative: Only Flickr Pro users are able to access the original ﬁles of the images • uploaded. People that use Flickr’s basic service, can instead only use the compressed versions of these. Additionally, Flickr users with a basic account are limited to a monthly upload of 300MB (the upload for paying members is unlimited).

68 3.3. Flickr 69

Other Restrictions: Images uploaded to Flickr have to have a width of 2048px to • avoid that the aspect ratio of uploaded images is changed.

3.3.2. Implementation

Flickr has been included into our photosharing website driven cloud storage system with the help of the ﬂickrj-android API [Yu13]. It is a JAVA library that supports the access to Flickr’s REST-based API [Fli13d].

3.3.3. Benchmarks

Failure Rates

The failure rates of Flickr have been discussed in Chapter 3.1.1, which refers to the optional applied ECC in our system. Results are that due to the low failure rates of the ML7 and the ML16, it is possible to use ECC to make these approaches also usable with Flickr.

Overhead

Flickr in its basic version applies JPEG-compression to uploaded images. Figure 3.9 (logarithmic scale) and Table 3.3 allow a closer look at this overhead:

69 70 3.3. Flickr 3.0 SL2 SL4 SL16 ML3 ML7E SL3 SL7 ML2 ML4 ML16E 2.5 2.0 1.5 1.0 Overhead of Hosted Image to Input Size [%] of Hosted Image to Input Size Overhead 210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.9.: Flickr: Overhead of Uploaded Images

Figure 3.9 shows the overhead of hosted images with respect to their encapsulated data. Table 3.3 additionally shows a comparison of the data containers’ average overhead before and after the upload to Flickr. The change of the overhead and the arithmetic mean between upload and download overhead are also recorded.

Encoder Up.Overh. Dl.Overh. Change A. Mean

SL2 1 1.56 +0.56 1.28 SL3 1.36 1.78 +0.42 1.57 SL4 1.17 1.5 +0.33 1.34 SL7 1.49 1.54 +0.05 1.52 SL16 1.39 1.49 +0.1 1.44 ML2 1.21 1.27 +0.06 1.24 ML3 1.34 1.43 +0.09 1.39 ML4 1.17 1.23 +0.06 1.2 ML7E 1.44 1.45 +0.01 1.45 ML16E 1.34 1.35 +0.01 1.35

Table 3.3.: Flickr: Upload and Download Overhead Comparison

It can be observed that the SL3 with an average overhead of about 1.78 times its actual embedded data performs the worse. Another fact is that the SL2 has the by far strongest increase of overhead after being uploaded and hosted on Flickr. The original PNG-image has about zero overhead, whereas the overhead of the hosted JPEG-image is over one third of its actual data. This results from the JPEG-compression and its changes in colors,

70 3.3. Flickr 71 respectively the addition of from black and white deviating colors. The original uploaded images were black & white images with a color depth of 1bit, the downloaded ones have the full 24bit color palette of an RGB image. The same is true for all original images with less than the complete palette.

Overall the ML2 and the ML4 come off best with an average overhead of 1.24 respectively 1.2. It can also be determined that the ML7E and the ML16E have the smallest image growth with an average change of about +0.01 .

Performance

We are now looking at the performance of Flickr with respect to the time it takes to host images and the time to download hosted images. Figure 3.10 contains two graphs comparing the performance of Flickr with the different working encoding approaches (logarithmic scale):

Upload: SL2 Upload: ML2 10000 Download: SL2 Download: ML2 Upload: SL3 Upload: ML3 Download: SL3 Download: ML3 Upload: SL4 Upload: ML4 Download: SL4 Download: ML4

50000 Upload: SL7 Upload: ML7E Download: SL7 Download: ML7E

Upload: SL16 Upload: ML16E 6000 Download: SL16 Download: ML16E 20000 4000 Time [ms] Time [ms] 10000 2000 5000

210 211 212 213 214 215 216 217 218 219 220 210 211 212 213 214 215 216 217 218 219 220

Data Input [byte] Data Input [byte]

(a) Flickr Upload (b) Flickr Download

Figure 3.10.: Flickr: Upload and Download Performance

Figure 3.10a displays the time it takes to upload an image generated by a speciﬁc encoding approach to Flickr. The other direction is illustrated in Figure 3.10b. It shows how long it takes for a download of an on Flickr hosted image. If the two graphs are compared, it can be determined that the upload takes between 2 to at max 10 times longer than the download. Therefore, the upload is the bottleneck in the data exchange between our system and Flickr.

This fact is reﬂected in Figure 3.17a in which the arithmetic mean of upload and download time is calculated and used as input (logarithmic scale):

71 72 3.3. Flickr

UL/DL−Mean: SL UL/DL−Mean: ML

50000 2 2 UL/DL−Mean: SL3 UL/DL−Mean: ML3 UL/DL−Mean: SL4 UL/DL−Mean: ML4 UL/DL−Mean: SL7 UL/DL−Mean: ML7E UL/DL−Mean: SL16 UL/DL−Mean: ML16E 20000 Time [ms] 10000 5000

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.11.: Flickr: Upload/Download Arithmetic Mean

The graph looks very much the same than the graph of the upload time, because the downloads’ duration values have less inﬂuence. It can also be determined that the actual size of an image seems to be far less important on Flickr’s upload processing time than the amount of pixels an image contains. Table 3.17b shows a comparison between the size and the amount of pixels of output images generated by the with Flickr working approaches:

Encoder Color D. Size (Byte) #Pixels ? SL2 1Bit 1, 053, 496 8, 390, 656

SL3 8Bit 1, 391, 133 6, 293, 504

SL4 8Bit 1, 200, 682 4, 196, 352

SL7 24Bit 1, 524, 527 3, 147, 776

SL16 24Bit 1, 448, 143 2, 099, 200

ML2 24Bit 1, 252, 817 2, 797, 568

ML3 24Bit 1, 409, 883 2, 099, 200

ML4 24Bit 1, 200, 378 1, 398, 784

ML7E 24Bit 1, 410, 257 1, 187, 840 ? ML16E 24Bit 1, 354, 827 792, 576

Table 3.4.: Flickr: Encoder Comparison (Input 1MiB)

As input 210Bytes or 1MiB of random bytes were used. The with a star (?) marked values are best in their category. With almost no overhead, the SL2 is by far the best approach when it comes to the actual size of the generated images, but with over 8 million pixels

72 3.3. Flickr 73

it has the greatest pixel usage of all encoders. In this category the ML16E wins with about 0.8 million pixels usage. When we now look at the time that is needed to upload and host images generated by both approaches to Flickr, we see that the SL2 performs the worse, whereas the ML16E performs the best. Moreover, there is a correlation between upload time and pixel usage. For all image containers applies that the less pixels they contain that have to be processed by Flickr, the less time it takes to host an image. Con- cludingly we can say that by taking the results of the overhead measures into account, the SL4 without ECC and the ML16E with ECC can be determined to be the recommended encoding approaches to be used with Flickr.

In Figure 3.12 the time measure results for the recommended approaches SL4 and ML16E in team play with Flickr are compared to the results of Amazon’s AWS S3 (logarithmic scale):

Upload: AWS Download: AWS Upload: SL4 Download: SL4 Upload: SL16E Download: SL16E 10000 2000 Time [ms] 500 200

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.12.: Comparison Flickr vs. Amazon’s AWS S3

It can be determined that the with Flickr used approaches ML4 and ML16E take about 32 times, respectively 28 times longer than AWS S3 for the same amount of data. ∼For the upload this value∼ is about 10 times more for both. ∼

73 74 3.4. Picasa Web Albums

3.3.4. Results

Positive to notice about Flickr is that its basic version is a free service and that even if it applies JPEG-compression on uploaded images, it is using a high JPEG quality factor of about 0.96. This is also reﬂected in available encoders allowing to utilize up to the

SL4 and the ML16E. Flickr underperforms with respect to the time it takes to upload and host images. With at best an average of 28 times slower than the Amazon’s AWS S3, it is worlds apart from being high performance∼ storage. Another constraint is the limit of available hosting storage to 300MB a month, which means that uploading large ﬁles in one upload is not possible. Even though Flickr could be used for example as append only storage.

3.4. Picasa Web Albums

The third provider in our photosharing website driven cloud storage system’s portfolio of available storage hosts is Picasa Web Albums [Goo13a]. It was released in June 2006 [Len13] and can be seen as Google’s alternative to Yahoo’s Flickr.

3.4.1. Features

Provider: Picasa Web Albums Costs: free 5,120MB / Upload Limit: +25GB for 2.49$ per month / +100GB for 4.99$ per month JPEG-compression: - 800 800 Max. Resolution: / 2048× 2048 (if account is connected to Google+) Other Restrictions: - × Usable Encoders: All

Table 3.5.: Features of Picasa Web Albums

Table 3.5 shows an overview of the in our case important features of Picasa Web Albums:

Positive: Picasa Web Albums is the only photo sharing website in our portfolio that • allows accessing the original uploaded images. This implicates that all of our en-

74 3.4. Picasa Web Albums 75

coding approaches are usable with Picasa Web Albums. The amount of free web space is with 5,120MB also a good base for greater uploads. Another positive point is that the images’ maximum resolution with 2048 2048 offers good data con- × tainers (required that the speciﬁc account is linked to Google+).

Negative: Picasa Web Albums only allows to upload images with a size of 800 800 if • × the corresponding Google account is not linked to Google’s social network Google+.

Other Facts: Additional web space can be purchased. Extra 25GB cost 2.49$ per • month, extra 100GB 4.99$ per month.

All measures following are done with a Google account linked to Google+.

3.4.2. Implementation

In contrast to the other photo sharing websites in our portfolio, Google offers their own APIs for different programming languages, meaning that we did not use a third party API and instead integrated Picasa Web Albums with the help of Google’s Picasa Web Albums API for JAVA [Goo13b].

3.4.3. Benchmarks

Failure Rates and Overheads

Since Picasa Web Albums allows the access to original uploaded images, there are neither failures in the hosted images throughout all approaches, nor do the overheads of hosted images differ from the overheads originally produced by our approaches.

Performance

We will take a closer look at the performance of Picasa Web Albums in team play with the different encoding approaches. Figure 3.13 shows the according upload and download times (logarithmic scale):

75 76 3.4. Picasa Web Albums

Upload: SL2 Upload: ML2 Download: SL2 Download: ML2

Upload: SL3 Upload: ML3 12000 Download: SL3 Download: ML3 Upload: SL4 Upload: ML4 Download: SL4 Download: ML4 50000 Upload: SL7 Upload: ML7 Download: SL7 Download: ML7 Upload: SL16 Upload: ML16 Download: SL16 Download: ML16 Upload: SL Upload: ML Download: ML 256 256 8000 Download: SL256 256 6000 20000 Time [ms] Time [ms] 4000 10000 5000

210 211 212 213 214 215 216 217 218 219 220 210 211 212 213 214 215 216 217 218 219 220

Data Input [byte] Data Input [byte]

(a) Upload Time (b) Download Time

Figure 3.13.: Upload and Download Times Picasa Web Albums

Like for Facebook and Flickr also for Picasa Web Albums the upload times are significantly higher than the download times. In case of Picasa Web Albums it is an average factor of about 3.2 times. In Figure 3.13a we see the uploading time of images generated with the different∼ approaches. As the graph suggests, also for Picasa Web Albums it is valid to say that the smaller the dimensions of an uploaded image are, the faster it is processed. The actual file size has less influence. Best performs the ML256 with an average upload time of about 5, 622ms, whereas the worse is the SL2 with an average time of 16, 930ms. This may∼ be due to the fact that Picasa Web Albums offers the uploaded ∼files also in different formats and JPEG-compression, meaning that the uploaded files are processed even if the access to the original uploaded image is available. The actual download times of Picasa Web Albums are shown in Figure 3.13b.

Due to the fact that the download is much faster than the upload, it has less inﬂuence on the arithmetic mean between upload and download shown in Figure 3.14 (logarithmic scale):

76 3.4. Picasa Web Albums 77

UL/DL−Mean: SL2 UL/DL−Mean: ML2 UL/DL−Mean: SL3 UL/DL−Mean: ML3 UL/DL−Mean: SL4 UL/DL−Mean: ML4 UL/DL−Mean: SL7 UL/DL−Mean: ML7 UL/DL−Mean: SL16 UL/DL−Mean: ML16 20000 UL/DL−Mean: SL256 UL/DL−Mean: ML256 Time [ms] 10000 5000

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.14.: Comparison Picasa Web Albums vs. AWS S3

It can be determined that the encoding approaches gaining the best performance results in team play with Picasa Web Albums are the SL256 and the ML256 with an average upload- download time of about 4, 392ms, respectively 4, 189ms. In contrast, the SL2 does ∼ ∼ the worse with an average time of about 10, 772ms. Therefore, the SL256 and ML256 are the recommended encoding approaches∼ to be used with Picasa Web Albums.

In the following, the two for Picasa Web Albums recommended encoding approaches

SL256 and ML256 are compared with Amazon’s AWS S3. Figure 3.15 shows the resulting graph (logarithmic scale):

77 78 3.4. Picasa Web Albums 5000 2000 Upload AWS Download: AWS Upload: SL256 Download: SL256 Upload: ML256 Download: ML256 Time [ms] 500 200

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.15.: Download Time Picasa Web Albums

It can be determined that the upload of data to Picasa Web Albums takes about 25 ∼ times for the SL256, respectively 23 times longer for the ML256, than the upload of the same amount of data to AWS S3∼. The download performs better, while both encoding approaches need about 10 times longer than AWS S3. ∼ 3.4.4. Results

Because Picasa Web Albums does allow the access to the original images that have been uploaded, it is possible to use all available encoding approaches. Additionally, the given free upload space of about 5GB storage is a good reason to use Picasa Web Albums. By being about 23 times slower in team play with the ML256 encoding approach than Amazon’s AWS∼ S3, Picasa Web Albums may not be usable in high performance scenarios, but it shows nevertheless a good result compared to the measures of the other photo sharing websites Facebook and Flickr.

Less advantage may be the demand of Google to link the user’s Google account with Google+ in order to use images with a resolution of 2048 2048 instead of only 800 800. × ×

78 3.5. Comparison Facebook, Flickr, and Picasa Web Albums 79

3.5. Comparison Facebook, Flickr, and Picasa Web Albums

In the previous sections we have discussed the providers Facebook, Flickr, and Picasa Web Albums that are integrated in our photosharing website driven cloud storage system. The individual results are summed up and compared with each other:

3.5.1. Feature Comparison

Provider: Facebook Flickr Picasa W.A. Costs: free free free 300MB 30MB 5.120MB Upload Limit: unlimited month / photo JPEG-compression: (qf: 0.84) (qf: 0.96) - ∼ ∼ 800 800 Max. Resolution: 2048 2048 2048 2048 / 2048× 2048 (G. ) × × + min w.: 2048px × Other Restrictions: / min w.: 2048px - min h.: 5px SL 1 Bpp ML 3 Bpp 256 Rec. Encoder(s): SL 1 Bpp 4 4 ML ⇒ 3 Bpp 4 4 ML ⇒ 2 Bpp ECC 256 ⇒ 16E 3 ⇒ ⇒ −

Table 3.6.: Features Comparison between Facebook, Flickr, and Picasa Web Albums

Table 3.6 shows the features of the three photo sharing websites. The first and most important point is that all three providers are free services. This guarantees that independent of the choice of provider, storage is cost free. Furthermore, all three providers have in common that the available maximum resolution for images is 2048 2048, which allows image data containers of comfortable size. However Picasa Web Albums× has the disadvantage that the specific Google account has to be connected with Google+ to have access to this resolution, else it is limited to a resolution of only 800 800. The other restrictions with a given minimum image width or height can be neglected,× since they only have an effect if the input data is very small and the image produced gets greater than the pixels used to encode the specific data.

The main differences between the providers exists with respect to the upload limitations, the application of JPEG-compression and the resulting usable encoding approaches:

79 80 3.5. Comparison Facebook, Flickr, and Picasa Web Albums

Facebook obviously has the best offer when it comes to the available storage. On the • other hand Facebook processes uploaded images and compresses them with a JPEG quality factor of about 0.84. This is the strongest compression applied in our ﬁeld of providers and therefore∼ limits Facebook in the amount of usable encoders. The recommended encoder is the SL with a bytes per pixel ratio of only 1 Bpp. 4 4

Flickr’s basic service in contrast limits the upload to 300MB of image data per • month. It also applies JPEG-compression but with higher JPEG quality factor ( 0.96) than Facebook, which allows us to use encoding approaches that have a bet-∼ ter bytes per pixel ratio. Its recommended encoders are the ML with a ratio of 3 4 4 Bpp and the ML with a ratio of 3 Bpp (minus the 10% ECC added to the input 16E 2 byte array).

Picasa Web Albums as the third provider allows for the use of about 5GB free image • storage. It has the great advantage that it is the only provider in our set of providers who does allow to access the original uploaded images. Therefore image data containers uploaded to Picasa Web Albums do not face any kind of compression, which allows us to use all available encoding approaches. The recommended encoding

approaches are the SL256 (1 Bpp) and the ML256 (3 Bpp).

A more detailed summary of the available encoding approaches for the three photo sharing websites providers is shown in Table 3.7:

Encoder Facebook Flickr Picasa W.A.

SL SL2 SL3 ? SL4 SL7 - SL16 - ? SL256 - -

ML ML2 - ML3 - ? ML4 - ML7 - ( ) ? ML16 - ( ) ? ML256 - - Table 3.7.: Applicable Painters on Facebook, Flickr, and Picasa Web Albums

Cells with a check “ ” indicate that this encoding approach is available for the speciﬁc

80 3.5. Comparison Facebook, Flickr, and Picasa Web Albums 81 provider. Cells that have a check surrounded by braces “( )” inform that the speciﬁc approach/provider combination works, but only if ECC is applied. The cells marked with asterisks “?” represent the recommended encoding approaches for the speciﬁc provider.

3.5.2. Overhead Comparison

The three photo sharing websites differ vastly with respect to the growth of the overhead between the original uploaded images and the by the providers processed images. Figure 3.16 shows a comparison between Facebook, Flickr, and Picasa Web Albums (logarithmic scale): 10 Facebook UL: SL4 Flickr−ECC UL: ML16E Facebook DL: SL4 Flickr−ECC DL: ML16E Flickr UL: ML4 Picasa W.A.: SL256 Flickr DL: ML Picasa W.A.: ML

5 4 256 2 Overhead of Images to Input Size [%] of Images to Input Size Overhead 1

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.16.: Overhead Comparison of Hosted Images

It is easily seen that Picasa Web Albums performs best. This is very obvious since it allows to access the original uploaded images, which means that the overhead equals the overhead the images had before the upload. The recommended encoding approaches SL256 and ML256 produce an negligible average overhead of 1.03 respectively 1.04 in our test series. ∼ ∼

Flickr in team play with its recommended encoding approaches ML4 and ML16E performs second best. Due to its JPEG-compression, the overhead grows after the images have been uploaded, but the average growth is rather small with +0.06 for the ML4 and +0.01 for the ML16E. The ML4 has an average upload overhead of 1.17 and an average download ∼

81 82 3.5. Comparison Facebook, Flickr, and Picasa Web Albums

overhead of 1.23 and the ML16E has an average upload overhead of 1.34 and an download average∼ download overhead of 1.35 . ∼ ∼ Facebook has to deal with the worst overhead because it has the strongest JPEG-compression in our ﬁeld of providers. The average upload overhead for the SL4 is 1.22. Facebook growth this average overhead to an average value of 4.37 for the hosted∼ images. This ∼ corresponds to an average growth of +3.15 additional overhead.

3.5.3. Performance Comparison

Next we address the performance comparison between the three photo sharing websites with respect to the time needed to upload and download images. Figure 3.17 contains two graphs that show the upload and download measures for the Facebook, Flickr, and Picasa Web Albums in team play with their recommended encoding approaches (logarithmic scale):

UL Facebook: SL4 DL Facebook: DL4 UL Flickr: ML4 DL Flickr: ML4 UL Flickr: ML16E DL Flickr: ML16E 20000 UL Picasa W.A.: SL256 10000 DL Picasa W.A.: SL256 UL Picasa W.A.: ML256 DL Picasa W.A.: ML256 5000 Time [ms] Time [ms] 5000 2000 1000 1000 210 211 212 213 214 215 216 217 218 219 220 210 211 212 213 214 215 216 217 218 219 220

Upload Input [byte] Download Input [byte]

(a) Comparison Upload Time (b) Comparison Download Time

Figure 3.17.: Upload and Download Time for Facebook, Flickr, and Picasa Web Albums

It is obvious that Facebook overall performs the worse when it comes to the upload and download times. Only for small inputs it does have an advantage over the two other photo sharing websites. Its average upload time is about 9, 721ms and about 3, 192ms for the download. ∼ ∼

Flickr does better. Its average upload times in team play with the ML4 and the ML16E are about 9, 377ms, respectively 7, 990ms. The download utilizing both encoding approaches∼ are about 2, 715ms, respectively∼ 2, 804ms. ∼ ∼

82 3.5. Comparison Facebook, Flickr, and Picasa Web Albums 83

Picasa Web Albums performs best. In contrast to both other providers it takes about the same time to upload/download data with all tested sizes between 1KiB and 1MiB. The average upload times in team play with the SL256 and the ML256 are 6, 249ms, respectively 5, 622ms. The average download times for both encoding approaches∼ are about 2, 534∼ ms and 2, 755ms. ∼ ∼ In the next step we take these measures and compare them with measures we have done for the same amount of data in team play with Amazon’s AWS S3. Figure 3.18 shows the comparison of the arithmetic mean of the upload and download times (logarithmic scale):

UL/DL−Mean AWS UL/DL−Mean Flickr: ML16E UL/DL−Mean Facebook: SL4 UL/DL−Mean Pic. W.A.: SL256

20000 UP/DL−Mean Flickr: ML4 UL/DL−Mean Pic. W.A.: ML256 5000 2000 Time [ms] 500 200 100

210 211 212 213 214 215 216 217 218 219 220

Data Input [byte]

Figure 3.18.: Comparison with AWS S3: Arithmetic Mean of Upload and Download Times

It can be determined that Facebook performs the worse with an average of about 36 times the time it takes to upload the same amount of data to AWS S3. ∼

Flickr does a better job needing an average of 32 times longer with the ML4 and an ∼ average of 28 times longer with the ML16E. ∼ Winner in the performance race is Picasa Web Albums. Its average time to upload the test data is about 25 times higher in team play with the SL256 and about 23 times ∼ ∼ with the ML256.

83 84 3.5. Comparison Facebook, Flickr, and Picasa Web Albums

The download times are for all photo sharing website providers about 10 times the time it takes do download the same data from AWS S3. ∼

The following Table 3.8 summarizes the results of the benchmark comparison between Facebook, Flickr, and Picasa Web Albums:

Provider: Facebook Flickr Picasa Web Albums

Encoder: SL4 ML4 ML16E SL256 ML256 Avg. Overh. Upload: 1.22 1.17 1.34 1.03? 1.04 Avg. Overh. Download: ∼ 4.37 ∼ 1.23 ∼ 1.45 ∼ 1.03? ∼ 1.04 ∼ ∼ ∼ ∼ ? ∼ ? Overh. Growth (UL vs. DL): +3.15 +0.06 +0.01 0 0 Avg. Upload Time: 9, 721ms 9, 377ms 7, 990ms 6, 249ms 5, 622ms? AWS vs. Provider Upload: ∼ 36 ∼ 32 ∼ 28 ∼ 25 ∼ 23? AWS S3 Download Time: 3,× 192ms 2,× 715ms 2,× 804ms 2,× 534ms? 2,× 755ms AWS S3 vs. Provider Download: ∼ 10? ∼ 10? ∼ 10? ∼ 10? ∼ 10? × × × × × Table 3.8.: Summary Comparison between Facebook, Flickr, and Picasa Web Albums

Cells that are marked with a asterisk “?” represent the best result in the speciﬁc category.

3.5.4. Results

All three providers in our photosharing website driven cloud storage system offer a free service that can be exploited as cheap storage. If we sum up all facts and measures, Pi- casa Web Albums seems the photo sharing website of choice if the speciﬁc Google account is linked to Google+ so that the maximum resolution of 2048 2048 can be utilized. Moreover, it has no compression and allows uploading up to 5,120MB× of image data, which for example in combination with the SL256 and its overhead ratio of about 1.03 are about 4,970MB of free cloud storage. Especially when it comes to the performance∼ of upload and download times, Picasa Web Albums is the winner of the comparison.

Flickr with its basic service does second best. It has the strictest upload limitation with 300MB a month and although, it applies JPEG-compression it retains a fair JPEG quality factor. The limit makes it impossible to upload a great number of data at once, but it could be used as append only storage, where the data grows over time (n.b. taking Flickr’s new introduced service offers of 1TB for free and access to original uploaded images into account, it can be assumed that currently Flickr performs even better with respect to available storage, applicable encoding approaches, and overall performance).

84 3.5. Comparison Facebook, Flickr, and Picasa Web Albums 85

Facebook has only one advantage that is, however, in comparison a strong advantage, allowing to upload images without storage space limit. This comes with a rather high JPEG-compression and a lack of performance.

Compared with the professional AWS S3, our photosharing website driven cloud storage system cannot compete when it comes to upload and download performance. This is the price that is paid for the free space. For this reason our system ﬁts better in scenarios where performance is not that important, for example, if used as backup system.

85 86 3.5. Comparison Facebook, Flickr, and Picasa Web Albums

86 4 Conclusion

4.1. Results

In this thesis we have introduced a way of how to exploit photo sharing websites as cloud storage backends. We have analyzed twelve encoding approaches that enable to encode a given array of input bytes into the pixels of an image. Based on these approaches, a photosharing website driven cloud storage system was build that is applicable to be cus- tomized to any available photo sharing website. This system then has been tested and benchmarked with the photo sharing website providers Facebook, Flickr, and Picasa Web Albums.

The results show that with our system it is possible to exploit photo sharing websites as additional cloud storage, however, only with a trade-off relative to performance. Nev- ertheless, in scenarios like backup systems, our approach can be used as free storage.

Progression in the ﬁeld of photo sharing websites, like Flickr’s change in service, under- lines that the idea to exploit photo sharing websites as storage is worth to think about. 1TB of free space and access to the original uploaded images are incredible offers. This also motivated other people for the mentioned project Filr, which has in comparison to our project a very similar purpose.

A comparison between the in the related work discussed approaches (see Chapter 1.4) and our introduces photosharing website driven cloud storage system is presented in Ta- ble 4.1:

87 88 4.2. Future Work

Our approach is listed in the last row. We have proven that our system is capable to provide the best bytes per pixel ratio, and in contrast to other approaches, also allows to work with providers that apply JPEG-compression. Another advantage is that our system is fully integrated in the jClouds project and therefore imitates a blob store and is not limited to single ﬁles e.g. like Filr. Overall our system is the best choice in the context of exploiting photo sharing websites.

Within the framework of this work we have also generated a technical report which can be accessed at: [GMW13]

4.2. Future Work

4.2.1. Benchmarks with larger Data and Flickr's new Service

The benchmark graphs introduced in this thesis base on random input data of sizes between 1KiB up to 1MiB. It could be interesting to widen this data range and measure how the encoding approaches scale with larger data.

Another interesting question is how Flickr’s new service offers inﬂuence the performance

88 4.2. Future Work 89 of the introduced system in team play with Flickr.

4.2.2. Enhance Images with Visual Information

For the human eye the images generated by our introduced encoding approaches only consist of “randomly“ colored pixels added together. It does not contain any visual information, any patterns, or symbols. A photo sharing website that wants to prevent abuse of their storage could use algorithms that are specially designed to detect this kind of computer generated images with no further optical information in it. This could be a serious threat to our hosted images. A way to protect our hosted images against being easily detected, is to combine them with simple pixel symbols, e.g. a heart, a stick figure, a star, a smiley, etc. The system could have a predefined pool of such symbols and their pixels. To keep the waste of pixels as small as possible, these pixel symbols should be a good compromise between optical perceptibility and pixel usage. During the encoding process the encoder could choose randomly from this pool, such that every image is unique with respect to its symbols and their adjustment. To produce additional variety, the colors from which to draw the different symbols, could alternate between the amount of colors available for a specific encoding approach. Figure 4.1 shows how such an image could look:

Figure 4.1.: Sample ML2 Image enriched with Symbols

Due to solve the problem of decoding such enriched images, an additional sector in the encoding area of the images could be added (for details see: Chapter 3.1.2). In this sector the utilized symbols, their order and colors could be stored.

89 90 4.2. Future Work

4.2.3. Automatic Determination of Best Encoding Parameters

In this thesis we introduced the best working encoding parameters for the specific photo sharing websites. These encoding parameters were determined manually, which means testing all approaches, watching for failures in the from the photo host returned images, and comparing the results. An improvement would be to fully automatize this process, so that if parameters of a specific photo sharing website change, this changes could be easily adapted. More importantly, this way our approaches could easily be applied to other photo sharing websites and their specifications.

4.2.4. Step from Images to Videos

This thesis is entirely about images and how to exploit photo sharing websites as free storage providers. The next major step would be to apply the here introduced approach on videos and exploit video sharing websites, e.g. like Google’s Youtube.

90 Appendices

A Random Input Bytes

The following array of 18 random input bytes were used to create output images in various examples of this thesis:

[69, 65, -38, -70, -72, -36, 80, 15, -7, -18, 106, 70, -86, -38, 47, 56, 65, -18] Figure A.1.: Random sample Bytes (Two’s Complement)

93 94

94 B Encoding Approaches Naming Convention

The encoding approaches in this thesis are named by their unique features. The ﬁrst part of their denotation describes whether it is single-layer-approach (SL) or multi-layer- approach (ML). The second part is a lowered ﬁgure that represents the γ-value of the approach. This leads to the following naming convention:

{SL (single-layer) | ML (multi-layer)}γ value − Figure B.1.: Approaches Naming Convention

For example the multi-layer-approach using a γ-value of γ =2 is denoted as ML2-approach.

Reed Solomon Error Correcting Code

To show whether ECC is applied to the input bytes, the γ-value of the approach is followed by a lowered “E”. For example the ML16 with ECC is denoted as ML16E-approach.

95 96

96 Bibliography

[Adl13] Adlon - empowering business with it. http://www.adlon.de/, April 2013.

[All13] Mark Allen. RestFB API. http://restfb.com/, April 2013.

[Ama13] Amazon. Amazon web services, cloud computing: Compute, storage, database. http://aws.amazon.com/, April 2013.

[App13] Apple. icloud. http://www.icloud.com/, April 2013.

[BKI07] Arati Baliga, Joe Kilian, and Liviu Iftode. A web based covert ﬁle system. In Proceedings of the 11th USENIX workshop on Hot topics in operating systems, HOTOS’07, pages 12:1–12:6, Berkeley, CA, USA, 2007. USENIX Associa- tion.

[BKLaPV10] Doug Beaver, Sanjeev Kumar, Harry C. Li, and Jason Sobel an Peter Vajgel. Finding a needle in Haystack: Facebook’s photo storage. In "USENIX OSDI", 2010.

[com13] Sony Computer Entertainment company. Gaikai. http://www.gaikai. com/, April 2013.

[DD12] Thomas Dean and Charles Dunn. Quick layered response (qlr) codes. Technical report, Department of Electrical Engineering, Stanford Univer- sity, 2012.

[Dro13] Dropbox. http://www.dropbox.com/, April 2013.

97 98 Bibliography

[Fac13a] Facebook. http://www.facebook.com/, April 2013.

[Fac13b] Facebook. Facebook reports ﬁrst quarter 2013 results. http://investor. fb.com/releasedetail.cfm?ReleaseID=761090, April 2013.

[Fac13c] Facebook. Graph api. http://developers.facebook.com/docs/ reference/api/, April 2013.

[Fli13a] Flickr from yahoo! http://www.flickr.com/, April 2013.

[Fli13b] Flickr. 6,000,000,000. http://blog.flickr.net/en/2011/08/04/ 6000000000/, April 2013.

[Fli13c] Flickr. A better, brighter ﬂickr. http://blog.flickr.net/en/2013/05/ 20/a-better-brighter-flickr/, May 2013.

[Fli13d] Flickr. Flickr Services. http://www.flickr.com/services/api/, April 2013.

[fS06] ISO/IEC: International Organization for Standardization. Information technology automatic identiﬁcation and data capture techniques data matrix bar code symbology speciﬁcation, 2006.

[GMW13] Sebastian Graf, Wolfgang Miller, and Marcel Waldvogel. Utilizing photo sharing websites for cloud storage backends. Technical report, Distributed Systems Group, Department of Computer and Information Science, Univer- sity of Konstanz, 2013. http://kops.ub.uni-konstanz.de/handle/ urn:nbn:de:bsz:352-234273.

[Goo13a] Google. Picasa web albums. http://picasaweb.google.com/home/, April 2013.

[Goo13b] Google. Picasa web albums data api. http://developers.google.com/ picasa-web/, April 2013.

[Has13] Calvin Hass. Jpegsnoop - jpeg ﬁle decoding utility. http://www. impulseadventure.com/photo/jpeg-snoop.html, May 2013.

98 Bibliography 99

[Hen13] Cal Henderson. PNGStore - Embedding compressed CSS & JavaScript in PNGs. http://www.iamcal.com/png-store/, 2013.

[jCl13] jClouds. Java API for accessing cloud services. http://jclouds.org, February 2013.

[KTC09] Hiroko Kato, Keng T Tan, and Douglas Chai. Novel colour selection scheme for 2d barcode. In International Symposium on Intelligent Signal Processing and Communication Systems, 2009.

[KW05] W. Küchlin and Andreas Weber. Einführung in die Informatik - objektorien- tiert mit Java. Springer-Verlag, 2005. Dritte überarbeitete Auﬂage.

[LB07] Tobias Langlotz and Oliver Bimber. Unsynchronized 4d barcodes: coding and decoding time-multiplexed 2d colorcodes. In Proceedings of the 3rd international conference on Advances in visual computing - Volume Part I, 2007.

[Len13] Philipp Lenssen. Google picasa web albums live. http://blogoscoped. com/archive/2006-06-14-n55.html, April 2013.

[MG13] Wolfgang Miller and Sebastian Graf. jClouds Fork: Photo Sharing Web- sites as additional Cloud Storage Providers. https://github.com/disy/ jclouds, April 2013.

[Mic13] Microsoft. Windows azure: Microsoft’s cloud platform. http://www. windowsazure.com/, April 2013.

[QRc13a] Information capacity and versions of the qr code. http://www.qrcode. com/en/about/version.html, April 2013.

[QRc13b] Qrcode.com. http://www.qrcode.com/en/, April 2013.

[RS60] Irving S. Reed and Gustave Solomon. Polynomial codes over certain ﬁnite ﬁelds. In Journal of the Society for Industrial and Applied Mathematics, 1960.

[Sei13] Jacob Seidelin. Compression using Canvas and PNG-

99 100 Bibliography

embedded data. http://blog.nihilogic.dk/2008/05/ compression-using-canvas-and-png.html, 2013.

[Tom13] Ricardo Tomasi. Filr. http://github.com/ricardobeat/filr, May 2013.

[UNI13] INTERNATIONAL TELECOMMUNICATION UNION. Recommendation t.81. http://www.w3.org/Graphics/JPEG/itu-t81.pdf, April 2013.

[W3C13a] W3C. Graphics interchange format(sm) - version 89a. http://www.w3. org/Graphics/GIF/spec-gif89a.txt, April 2013.

[W3C13b] W3C. Portable network graphics (png) speciﬁcation (second edition). http://www.w3.org/TR/2003/REC-PNG-20031110/, April 2013.

[XSSL04] Jianyun Xu, Andrew H. Sung, Peipei Shi, and Qingzhong Liu. Jpeg compression immune steganography using wavelet transform. In Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC’04) Volume 2 - Volume 2, 2004.

[Yah13] Yahoo. Flickr advertising solution. http://advertising.yahoo.com/ article/flickr.html/, April 2013.

[Yu13] Toby Yu. Flickrj-Android. https://code.google.com/p/ flickrj-android/, April 2013.

[ZXi13] ZXing. ZXing Multi-format 1D/2D barcode image processing library. http: //code.google.com/p/zxing/, February 2013.

100 List of Algorithms

1. storeBytesInImage(img: Image, w: Int, h: Int, γ: Int, bs: Byte []): Image . 23 2. getColorsFromByte(b: Byte, γ: Integer, ppb: Integer): Color[] ...... 25 3. extractBytesFromImage(img: Image, w: Int, h: Int, γ: Int) : Byte [] .... 27 4. getCharacterFromColor(pc: Color, γ: Integer): String ...... 29 5. storeBytesInImageML(img: Image, w: Int, h: Int, γ: Int, bs: Byte []): Image 36 6. determineLayeredColors(γ: Integer): Color[][] ...... 46

101 102 List of Algorithms

102 List of Figures

1.1. Data Cycle of a Photo Sharing Website driven Cloud Storage System . . .3 1.2. Example Output 2D-Codes: QR-Code and DataMatrix-Code ...... 6

2.1. Sample Output of all Encoding Approaches ...... 11 2.2. Encoding Workﬂow - From an Array of Bytes to the ﬁnal Image ...... 14 2.3. Decoding Steps - From Image to the regained Array of Bytes ...... 15

2.4. Output Comparison QR-Code vs. DataMatrix-Code vs. Own Encoding (SL2) 17

2.5. Comp. Encoding Approaches: Binary System (SL2) and Ternary System (SL3) 20 2.6. All SL-Approaches and their Features ...... 22 2.7. Example Output SL-Approaches ...... 31

2.8. Comparison between the SL2-Approach and the ML2-Approach ...... 32 2.9. All ML-Approaches and their Features ...... 34 2.10.Example Output ML-Approaches ...... 40 2.11.Types of Images with Approaches working ...... 41 2.12.10 Colors derived from [KTC09] and 6 additional deﬁned Colors ...... 44 2.13.Predeﬁned colors for SL-Approaches SL2, SL3, SL4, SL7, and SL16 ..... 45

2.14.Sample Colors generated by Algorithm6 for the ML16 Approach ...... 47 2.15.Summary of the Features of the different Encoding Approaches...... 48 2.16.Encoding Times all Encoding Approaches ...... 49 2.17.Decoding Times all Encoding Approaches ...... 50 2.18.Overhead all Encoding Approaches ...... 51 2.19.Example Output for all Approaches ...... 52

3.1. Logos of the three major Photo Sharing Websites ...... 53

3.2. Flickr: Failure Rates of SL256, ML7, ML16, and ML256 ...... 55 3.3. Sectors of generated Images ...... 57 3.4. Upload Workﬂow ...... 60

103 104 List of Figures

3.5. Download Workﬂow ...... 61 3.6. Failure Rates Facebook ...... 64 3.7. Overhead of uploaded Images on Facebook ...... 65 3.8. Facebook vs. AWS S3: Upload and Download Time ...... 66 3.9. Flickr: Overhead of Uploaded Images ...... 70 3.10.Flickr: Upload and Download Performance ...... 71 3.11.Flickr: Upload/Download Arithmetic Mean ...... 72 3.12.Comparison Flickr vs. Amazon’s AWS S3 ...... 73 3.13.Upload and Download Times Picasa Web Albums ...... 76 3.14.Comparison Picasa Web Albums vs. AWS S3 ...... 77 3.15.Download Time Picasa Web Albums ...... 78 3.16.Overhead Comparison of Hosted Images ...... 81 3.17.Upload and Download Time for Facebook, Flickr, and Picasa Web Albums . 82 3.18.Comparison with AWS S3: Arithmetic Mean of Upload and Download Times 83

4.1. Sample ML2 Image enriched with Symbols ...... 89

A.1. Random sample Bytes (Two’s Complement) ...... 93

B.1. Approaches Naming Convention ...... 95

104 List of Tables

1.1. Overview Approaches Related Work ...... 9

2.1. Summary Data Ratio SL-Approaches (in Bytes per Pixel) ...... 31 2.2. Summary Data Ratio ML-Approaches (in Bytes per Pixel) ...... 39

3.1. Features of Facebook ...... 63 3.2. Features of Flickr and Flickr Pro ...... 68 3.3. Flickr: Upload and Download Overhead Comparison ...... 70 3.4. Flickr: Encoder Comparison (Input 1MiB) ...... 72 3.5. Features of Picasa Web Albums ...... 74 3.6. Features Comparison between Facebook, Flickr, and Picasa Web Albums .. 79 3.7. Applicable Painters on Facebook, Flickr, and Picasa Web Albums ...... 80 3.8. Summary Comparison between Facebook, Flickr, and Picasa Web Albums . 84

4.1. Comparison: Approaches Related Work vs. our System ...... 88

105