Enhancement of Open Source Monitoring Tool for Small Footprint Databases

Dissertation

Submitted in partial fulfillment of the requirements for the degree of

Master of Technology in Computer Science and Engineering

Submitted by

Sukh Deo Roll No. 123050061

Under the Guidance of

Prof. Deepak B. Phatak

Department of Computer Science and Engineering Indian Institute of Technology Bombay Powai, Mumbai, India 400076 June, 2014 Dissertation Approval Certificate

Department of Computer Science and Engineering

Indian Institute of Technology, Bombay

The dissertation entitled “Enhancement of Open Source Monitoring Tool for Small Footprint Databases”, submitted by Sukh Deo (Roll No: 123050061) is approved for the degree of Master of Technology in Computer Science and Engineering from Indian Institute of Technology, Bombay.

Prof. Deepak B. Phatak Dept. of CSE, IIT Bombay Supervisor

Internal Examiner External Examiner

Place: IIT Bombay, Mumbai Date: June, 2014 Declaration

I declare that this written submission represents my ideas in my own words and where other’s ideas or words have been included, I have adequately cited and referenced the original sources. I also declare that I have adhered to all principles of academic honesty and integrity and have not misrepresented or fabricated or falsified any idea/data/fact/source in my submission. I un- derstand that any violation of the above will be cause for disciplinary action by the Institute and can also evoke penal action from the sources which have thus not been properly cited or from whom proper permission has not been taken when needed.

Signature

Name of Student

Roll number

Date

3 Acknowledgement

I would like to thank my guide, Prof. Deepak B. Phatak, for the consistent encouragement and guidance throughout the project. Due to his consistent encouragement and right directions, we are able to do this project work. I would like to thank Mr. Nagesh Karmali for his continuous inputs in my work. I would also like to thank, Firuza Aibara, Prashant Kumar Singh, Arun Kumar, and B. Raju for proof reading this report and their valuable feedback. Abstract

Nowadays, mobile devices are not only used for the basic purpose like calls and messages, but also for many others purposes and applications. We can say that mobile devices are siblings of personal computers. Their usage patterns are very different as compared to conventional PCs, since the mobile devices are targeted for general users. They do have some great features, e.g., lightweight and efficiency, but has some drawbacks like limited memory and battery issues. People now wants to use more and more applications to make life easier, but mobile device is able to manage the limited number of application. There is a need to measure the performance of mobile device.

Our work starts with simple questions like the: does storage affect the performance of mobile devices? What kinds of databases are required for mobile devices? What are the parameters to measure the performance of databases? How to measure the performance of mobile device and its possible tools. Finally, we will enhance the performance of monitoring tool for small foot print database. Contents

Acknowledgement ...... 4

1 Introduction 5 1.1 Motivation ...... 5

2 Literature Survey 7 2.1 Literature Survey on Small Footprint Databases ...... 7 2.1.1 Berkeley DB ...... 7 2.1.2 Perst DB ...... 9 2.1.3 SQLite DB ...... 9 2.1.4 Comparison among Berkeley DB, Perst DB, and SQLite DB ...... 10 2.2 Literature Survey on monitoring tool ...... 10 2.2.1 IOzone ...... 11 2.2.2 AndroBench ...... 12 2.2.3 MobiBench ...... 14 2.3 Conclusion ...... 14

3 Android Framework and SQLite Database 15 3.1 Android Framework ...... 15 3.1.1 Linux Kernel ...... 16 3.1.2 Libraries ...... 16 3.1.3 Android Runtime ...... 16 3.1.4 Application Framework ...... 16 3.2 Journal Mode of SQLite ...... 17 3.2.1 Rollback Journal ...... 17 3.2.2 Truncate Journaling Mode ...... 18 3.2.3 Persist Mode ...... 19 3.2.4 Write-Ahead Log(WAL) Mode ...... 19

4 MobiBench and Aakash Tablet Specifications 20 4.1 Overview of MobiBench ...... 20 4.1.1 Measure Tab ...... 20 4.1.2 History Tab ...... 21 4.1.3 Setting Tab ...... 21 4.2 Aakash Tablet Specification ...... 23

1 5 Test Result of MobiBench on Aakash Tablet 24 5.1 Testing Results on Aakash Tablet ...... 24 5.1.1 Test Cases for SQLite ...... 24 5.1.2 Test Result ...... 24 5.2 Battery consumption during testing ...... 26

6 Modification Done in MobiBench 27 6.1 Source code Analysis of MobiBench ...... 27 6.1.1 Possible Enhancement ...... 29 6.2 Modifications done to MobiBench Database ...... 30

7 Test Result after Modification of MobiBench on Aakash tablet 32 7.1 Testing Result after changes ...... 32 7.2 Result ...... 33 7.3 Battery consumption during testing ...... 33

8 Conclusion and Future Work 36 8.1 Conclusion ...... 36 8.2 Future Work ...... 37

2 List of Figures

1.1 Smart-phone’s Users[1] ...... 6

2.1 Architecture of Berkeley DB[2] ...... 7 2.2 Architecture of SQLite[3] ...... 8 2.3 Structure of AndroBench Measure tab[4] ...... 13 2.4 AndroBench Setting tab[5] ...... 14

3.1 Android Framework [6] ...... 15 3.2 Simple architecture of Android framework ...... 16 3.3 Component require for Facebook application ...... 17 3.4 Roolback Journal of SQLite[7] ...... 18

4.1 Over all view of MobiBench[5] ...... 20 4.2 Setting tab[5] ...... 21

5.1 Overall performance of Aakash2 ...... 25 5.2 Database performance of Aakash2 ...... 25 5.3 Database performance of Aakash2 ...... 26

6.1 Control flow of MobiBench ...... 28 6.2 Default table schema ...... 28 6.3 Email system table schema ...... 29 6.4 Bowser history table schema ...... 29 6.5 WhatsApp table schema ...... 29 6.6 Choose table ...... 30

7.1 Database performance of Browser table in Aakash2 ...... 34 7.2 Database performance of Email System in Aakash2 ...... 34 7.3 Database performance of WhatsApp in Aakash2 ...... 35

8.1 Database performance using WAL mode ...... 36

3 List of Tables

5.1 Default setting of Mobibench ...... 24 5.2 Test cases for SQLite ...... 25 5.3 Database Performance of Aakash2 ...... 26

7.1 Database Performance of Browser History table in Aakash2 ...... 32 7.2 Throughtput Result of Email system table in transaction/s ...... 33 7.3 Throughtput Result of WhatsApp table in transaction/s ...... 33

4 Chapter 1

Introduction

1.1 Motivation

Today’s mobile devices are multi-functional devices. It provides usage for many applications, both business and consumer purposes. Smart phones and tablets allow people to access Internet for email, messages, and web browsing. A business person can install many applications like streamline work-flow, save time, keep data accessible, automate daily tasks, and many others. Consumers can view new goods and their prices on Internet, and many more tasks can be done by mobile devices, i.e., smart phones and tablets. We can work away from the office using these devices and can synchronize with our personal computer and can get the updated information.

Due to the huge facility and light-weight property of mobile devices, there has been a huge market. People want to use many applications, but it has some limitations[8], like small memory and battery backup. It cannot manage large number of applications in these devices[9]. Today many research going on the application and service, device architecture, and operating system area[10] to make the optimal performance. Traditionally, storage was not viewed as a critical component of these devices, in term of expected performance, but now we got result that per- formance of mobile device depends on mobile database.

Performance of mobile devices is to be to continuously improve to fulfill the desire of users to install and execute more applications. As we can see, some applications depend on the storage performance, i.e., quality of audio/video playback while number of applications are downloaded in background, storage of web browser history, and browsing speed all of which depends on stor- age performance. It is also required to use good file systems and I/O schedulers for optimizing the storage performance and to use the high speed of NAND flash memory.

In Android mobile devices, SQLite is used both for system and application data, because SQLite is an embedded database. Example, if an Android application developer wants to make the Android application then he must use the database as SQLite for storing the related infor- mation like store the contacts, message, bookmarks of the web browser and many more. As we know each application uses separate database, speed of application depends on the speed of

5 Figure 1.1: Smart-phone’s Users[1] database and the performance of mobile device depends on application that is installed. We came to the following conclusions that the factor of performance of mobile device depends on:

• Mobile databases.

• The hardware like, SD card, internal memory and processor.

Due to demand of mobile devices in market, there is need to measure its performance, so that we can utilize maximum resources. Based on this motivation, we did the literature surveys on mobile databases and its monitoring tool, that we will discuss in the second chapter. We will also perform test on Aakash2 using MobiBench and we will also discuss possible way to enhance the performance of MobiBench and its modification.

6 Chapter 2

Literature Survey

2.1 Literature Survey on Small Footprint Databases

I did literature study in several small footprint databases. Each database has its own charac- teristics, based on which we can decide more preferable database. We will analyze the features of Berkeley DB, Perst DB, and SQLite DB.

Figure 2.1: Architecture of Berkeley DB[2]

2.1.1 Berkeley DB

Berkeley DB is an embedded database[11]. It is open source and written in C language with callable APIs for C#, C++, PHP, and so on[12]. Berkeley DB originated at The University of California Berkeley, but later developed at Oracle Corporation. It supports concurrent access by multiple users. It used to store the data, key/value technique. For example, we can use it to store the user name and password, i.e., user name as key and password as value. It is not a

7 relational database, but we can store multiple data using one key. Architecture of Berkeley DB is shown in figure 2.1. It has three layers. The first layer is the application program, which is directly connected to access methods and recovery.

Access Methods

Berkeley DB provides three types of access methods:

• B+tree: B+tree stores records in leaf page. Key in the tree is stored in sorted order. B+tree reduce tree size, and give better search performance.

• Hash : Berkeley DB uses extended linear hashing.

• Fixed or variable length records : in this method, each record has a record number, which serves as a key. For example, if we want to store a text file, then each line is treated as a record.

Figure 2.2: Architecture of SQLite[3]

Use of Berkeley DB

Berkeley DB provides concurrent access to databases[13]. It can be used as a standalone appli- cation, linked by other application. For example, in mobile applications, we can use Berkeley DB to store name and mobile number. This contact number can be seen whiling dialing the number, receiving call, or even while sending or receiving message to/from that number.

Limitations

• It does not support RDBMS. It has no schema.

8 • It stores only the record key/value technique, so that we cannot compare the data other than key value.

2.1.2 Perst DB

Perst Database is an open source embedded database, based on Java object oriented architecture.[14]. The supported features are as follows[15]:

• Classic B-Tree implementation

• R-tree indexes for spatially-oriented applications such as GIS and navigation

• Time Series class to efficiently deal with small fixed-size objects

• Specialized versions of collections for thick indices (indices with many duplicates), and bit indices (keys with a restricted number of possible values).

It is application specific database for storing real object information. Example, we can use Perst DB for Restaurant searching application, because we can access and store information using making object.

2.1.3 SQLite DB

SQLite is open source database and very light weight databases. If we enable all the features of SQLite in our Apps, size of SQLite Library will be less than 500KB. If we disable optional feature[16], i.e., WAL mode or Delete Mode, the size of SQLite library will be 300KB. It supports the relational database like SQL syntax, transaction, and SQL statement. That’s why it is available on most of the android device. This is an overview of SQLite.

• SQLite Data Types : SQLite supports only three data types[17]

– TEXT (like String) – INTEGER (like int) – REAL (like double)

• SQLiteOpenHelper Class : It is a helper class used to manage database creation and version management. We override the onCreate() and onUpgrade() methods in this class.

– onCreate() : this method is called, if the database does not exists. – onUpgrade () : this method is called, if the database version is increased.

• SQLiteDatabase Class : SQLite Database is the base class for working with a SQLite database. It provides various methods like: Open (), Insert (), Update (), and Close ().

• ContentValues : ContentValues are used to insert or update database entries. Records are stored as key-value pair in ContentValues. Where, key represents Column-Id and value represents data for that Column-Id.

9 Architecture of SQLite

The architecture of SQLite[3] explains the working of SQLite. This information is very useful, if we want to change inner code of SQLite. Block diagram is shown in figure 2.2.

• Interface : this module provides external interface to SQLite library. It is implemented in main.c.

• SQL command process : this module has three parts.

– Tokenize: This module breaks the SQL statement into tokens and calls the parser module. – Parser : this module, parses the meaning of tokens based on the context. – Code generator : after assembling the meaning of token, it generates code that will be used in virtual machine.

• Virtual Machine : It executes the generator code.

• B-tree : SQLite maintains the data using B-tree. A separate B-tree is made for each table. All B-trees are stored in same file.

• Pager : B-tree’s modules request data in the form of chunks. The default size is 1024KB. Page cache takes care of rollback journal and information.

• OS Interface : It provides facility for OS service, i.e., disk access, and critical section problems.

In NAND based flash memory, write operation is ten times slower than read operation, because of write before erase and garbage overheads[18]. In SQLite, rollback Journal file system is used[19]. It is good for small transaction and sequential writes. It is costly for random write transactions. For large transaction, we can use shadow paging technique, as it is good for random write.

2.1.4 Comparison among Berkeley DB, Perst DB, and SQLite DB

SQLite is a native database. It has a schema, so that we can design our own tables as we want. Berkeley DB is an embedded database[20]. It stores the data by key/value schema. Perst DB is an object oriented, embedded database. It is application specific database for storing real object information. We cannot really compare properly, because each has some special property. Since, SQLite has schema and it is embedded database of Android application, so there is huge usage of SQLite database.

2.2 Literature Survey on monitoring tool

IO characteristics of Android based mobile devices are different from desktop computers. We require monitoring tools to measure the storage performance of mobile devices. We have studied different monitoring tools that are used to monitor the embedded databases. Based on existing

10 monitoring tools, we will discuss what possible features can be enhanced. IO Characteristics: There are four types of IO characteristics:

• IO type (Read vs. Write).

• IO size (KB).

• Spatial locality (Sequential vs. Random)

• Temporal locality (Hot vs. Cold).

Synchronous IO : Works on polling basis[21]. While polling, if data is available, then we get the data. Otherwise application will block and retry for the data. Buffered IO : Default IO of programs is in buffer, i.e., they are used from a temporary storage to the requesting program. It will then increase the throughput of the application. We should analyze all the characteristics of an Android device, based on the above characteristics, we can measure correct workload of application process.For example, we need to measure the number of synchronous write operation that takes place in metadata as well as Journal file. Some existing workload does not identify the file types, thus, accuracy level is not good. We will discuss workload tool based on the above four characteristics.

2.2.1 IOzone

IOzone is benchmark tool for file system[22]. It is good for measuring the performance of a file system in computer platform. It is generally used by computer vendors. It measures the IO performance of accessing read, write, fread, and fwrite operations of a single file.

• Write : this test measures the performance of writing a new file. It contains overhead of creation of new file and metadata. Metadata contains the overhead address like directory information and space allocation.

• Re-write : this test measures the performance of writing a file that is already written. It takes low cost, as there is no overhead of metadata.

• Read : this test measures the performance of reading the file.

• Re-Read : this test measures the performance of reading the file that was recently read.

• Random Read and Random Write : this test measures performance of read/write, which requires random access file. It includes overhead of seek time and latency.

• IOzone can test file write and read operations, using library function fwrite and fread. It can also test recently fwrite and fread operations, using freread and frewrite.

• Mmap : this maps a file into a user address space. It provides easy access to the data for applications and does not require msync() to waiting for the data.

11 Limitations

It is file system benchmark tool[23].

• It does not include fsync(). But, SQLite forces to commit data via fsync() to commit on Journal modes. Thus, we cannot measure the correct performance of android device.

• In Android devices[24], there is no way of getting the super user permission in user level for remounting and un-mounting file, but IOzone needs super user permission. So, it cannot measure the performance of Android devices.

• IOzone must write sequential write operation, before running any other benchmarks, be- cause SQLite uses Journal mode.

Based on these limitations, we cannot measure the performance of SQLite in an Android device.

2.2.2 AndroBench

AndroBench is a storage benchmarking tool for Android mobile device[24]. It measures the sequential and random IO performance and throughput of various transaction of SQLite, i.e., in- sert, delete, and update. It has two methodologies, i.e., microbenchmarks and SQLite Benchmarks[24].

Microbenchmarks

AndroBench uses four micro-benchmarks for measuring the performance of sequential and ran- dom IO. For measuring the sequential read performance the step are given below:

• AndroBench creates a file (32MB by default) in target partition.

• The file is sequentially read with fixed buffer size (256kb by default).

• As write operation consumes lot of time; the file is reduced to 2MB for sequential write, which is similar to read operations.

• for random read/write operation : AndroBench measures the number of IO operations per second. The size of each read/write operation is 4KB default. 32MB of file is used for read and 2MB for writes.

For measuring the correct performance, it takes the average of each of the three transactions.

Measure

For measure the sequential read/write IO operation, below step are required:

• first, goes to measure tab.

• second, clicks the micro button.

It will measure the performance with default configurations. Java does not provide bypass buffer cache, so it is implemented by C.

12 Figure 2.3: Structure of AndroBench Measure tab[4]

SQLite Benchmarks

For testing the performance of SQLite transaction, AndroBench creates a table, which has 17 columns (twelve integer types and five text types) in target location. It performs three types of tests, i.e., insert, update, and delete. By default, the numbers of transactions are set to 300, but we can change from setting button. First, it release cache page by using SQLite- Database.releaseMemory (). Changing Parameters: Users can change, target partition, file size, and buffer size used in micro benchmarks, and SQLite benchmarks. After successfully completion of all benchmarks, it transmits the measurement result to the central server. Result is associated by model name and parameter. It measures the performance by transaction per second. Figure 2.3 shows the structure of measuring tab. We can measure the overall performance, SD card, and SQLite transaction by clicking specified button. The setting tab is shown in figure 2.4. This tab has provides features, we can change buffer size, file size, and partition location as desired.

History Tab

In this tab has two buttons:

• View history : Using this button, we can see history of measured performance.

• Clear history : Using this button, we can clear the history.

13 Figure 2.4: AndroBench Setting tab[5]

Limitation of AndroBench

Although, AndroBench is a powerful tool for measuring the performance of android device, it has some limitations[8], which are listed below: • It does not allow changing synchronization option such as O SYNC, O DIRECT, and Mmap.

• It has no method to measure the performance of fsync() call.

• Androbench does not support multi-threading benchmark environment. Based on the above mentioned limitations, AndroBench is not able to measure the correct performance.

2.2.3 MobiBench

MobiBench is open source mobile storage performance benchmark tool[23]. It has combined features of IOzone and AndroBench and it is specially designed for Android mobile device. We will discuss more in chapter 4.

2.3 Conclusion

Since, MobiBench is an open source monitoring tool and it is designed for Android device, so we can enhance the performance of MobiBench.

14 Chapter 3

Android Framework and SQLite Database

3.1 Android Framework

Android is an open source and Linux based operating system. It is specially designed for mobile devices such as tablet and smart phones. A developer only needs to develop the application for Android and it will run on all the devices which have Android operating system.

Figure 3.1: Android Framework [6]

Android is powerful operating system that provides many features such as beautiful GUI, storage as SQLite, connectivity, messaging, resizing, multi tasking, and many more.

As we are interested in the performance of the Android device, first, we have to understand the Android environment. This will help us to understand the performance bottleneck of An-

15 droid IO stack. Figure 3.1 shows the overall structure of Android framework. In first part, we have number of applications that user really uses. These applications are directly connected to application framework that will provide required system library of that application.

Let us discuss some basic component of Android framework.

3.1.1 Linux Kernel

Linux kernel is bottom up layer and it is directly connected to the hardware such as eMMC, SD card, and peripheral devices. It provides:

• Process management

• Memory management

• Manage network interface

• Device management such as camera, display, and keyboard etc.

3.1.2 Libraries

Android developers can use the functionalities provided by these libraries. Function is:

• Webkit provides web browser

• SQLite for storage and sharing data for applications

• SSL provides web security

3.1.3 Android Runtime

This is like Java Virtual Machine, designed for optimization of application operations. Application

3.1.4 Application Framework

It helps developer to form Java classes and SQLite use of their applications. This is top layer of Android architecture, where user can in- stall their own applications. As we are inter- ested to know the performance bottleneck of EXT4 mobile devices, figure 3.2 shows the way to write data from application to internal mem- ory(eMMC) or external memory(SD card). We can say that every Android application Device driver

eMMC/SD card 16 Figure 3.2: Simple architecture of Android framework must use SQLite as database. In mobile de- vice environment, we have two main appli- cations of mobile device that is highly popu- lar of smart phone, i.e., Facebook and What- sApp. Facebook uses more than 80% of insert operation[25].

• Facebook Apps : Earlier mobile Face- book users were able to update the sta- tus, only using text message. Later, when smart phone got smarter, they were able to open the Facebook mobile site, i.e., “m.facebook.com” in mobile browser. But, now Android version of phone has fully fea- tures of Facebook as desktop browser provides. This has happened, because of continuous increase in efficiency of smart phone and now facebook offer more Apps and add-on to Facebook users. Figure 3.3 demonstrate that how Facebook works. It does not write the data (i.e., chat stuff, images, and information update) in SD card, but it uses internal memory (eMMC).

• WhatsApp : This is similar to Face- book application, but it is specially de- signed for mobile devices.

3.2 Journal Mode of SQLite

In SQLite, there are generally four types of mode that provide different way to transaction Facebook GUI that are[19]:

• Rollback journal(Delete mode)

• Truncate Mode SQLite

• Persist Mode

• Write ahead logging(WAL) EXT4 file system Let us discuss how transaction mode work when an application want to insert the data to SD card or internal memory (eMMC). eMMC as internal memory Rollback journal is the default recovery mechanism[26]. Figure 3.3: Component require for Facebook ap- 3.2.1 Rollback Journal plication

In SQLite, when the process want to change the database file, it first creates a separate

17 rollback journal file and writes the original content of the database pages that are to be altered into the rollback journal file. It con- tains a header that records the original size of the database file. So if a change causes the database file to grow, we will still know the original size of the database. The page number is

Figure 3.4: Roolback Journal of SQLite[7] stored together with each database page that is written into the rollback journal. After the original page content has been saved in the rollback journal, the pages can be modified in user memory. Before making changes to the database file itself, we must obtain an exclusive lock on the database file. Once an exclusive lock is held, we know that no other processes are reading from the database file, and it is safe to write changes into the database file. Usually those changes affect operating systems disk cache, and not the mass storage. Another flush must occur to make sure that all the database changes are written into nonvolatile storage.

After the database changes are all safely completed on the mass storage device, the rollback journal file is deleted which is as shown in figure 3.4.

3.2.2 Truncate Journaling Mode

This is same as Rollback Journal mode, but instead of deleting the rollback journal file, it simply erases[27]. In this method, we can save the meta-data creation time.

18 3.2.3 Persist Mode

Persist mode does not delete the journal file after committing. Instead, the header of the journal is overwritten with zeros. This will prevent other database connections from rolling. This is useful when deleting or truncating is much expensive than overriding[27].

3.2.4 Write-Ahead Log(WAL) Mode

The traditional rollback journal works by writing a copy of the original unchanged database content into a separate rollback journal file and then writing changes directly into the database file[26]. In the event of a crash or rollback journal, the original content contained in the rollback journal is played back into the database file to revert the database file to its original state. The COMMIT occurs when the rollback journal is deleted. The WAL approach inverts this. The original content is preserved in the database file and the changes are appended into a separate WAL file. A COMMIT occurs when a special record indicating a commit is appended to the WAL. Thus a COMMIT can happen without ever writing to the original database, which allows readers to continue operating from the original unaltered database while changes are simultaneously being committed into the WAL. Multiple transactions can be appended to the end of a single WAL file.

Advantages:

• WAL is significantly faster in most scenarios

• WAL provides more concurrency as readers do not block writers and vice versa. Reading and writing can proceed concurrently

• Disk I/O operations tends to be more sequential using WAL

• WAL uses fewer fsync() operations and is thus less vulnerable to problems on systems where the fsync() system call is broken

In this chapter, we have discussed Android framework and how an Android apps use SQLite. We have also discussed the various transaction modes of SQLite and its performance. Next chapter, we will discuss Mobibench and testing on Aakash Tablet.

19 Chapter 4

MobiBench and Aakash Tablet Specifications

4.1 Overview of MobiBench

MobiBench is also called workload generator. It is implemented in two different versions. First one is shell application and second one is Android application. MobiBench provides similar environment to smart-phones, to create the workload and measure ability to execute the number of transactions per second[24]. There are three types of menu button available in MobiBench for users.

Figure 4.1: Over all view of MobiBench[5]

4.1.1 Measure Tab

In mobile device, there are two types of operations, i.e., database and file operation. MobiBench is able to measure both types of operations by generating the real kind of load. Figure 4.1 show the available option for user. We have four buttons that are:

20 • ALL : While clicking the ALL button, MobiBench generate the two types of operation, i.e., File IO and SQLite simultaneously. It will give the result as transactions per second.

• File IO : While clicking the File IO button, it generates the only file types of load such as read and write and give the result as KB/s or IOPS.

• SQLite : While clicking the SQLite button, it generates the query of insert, update, and delete operations and gives the result as transactions per second. Performance of SQLite also depends on PRAGRMA options, e.g., synchronous mode or journal mode. It generates three results, i.e., throughput, CPU utilization, and number of context switches. Units of throughput in SQLite operation is transactions per second.

• Custom : It runs only those set of tests that user choose from the Setting tab. Generally this button customizes the test.

4.1.2 History Tab

Figure 4.2: Setting tab[5]

History tab keeps the record of previous runs of benchmarks. Whenever we click the record, it will show the result from database.

4.1.3 Setting Tab

Figure 4.2 shows setting tab. In this tab, we can select various operations of file IO and SQLite transaction. In file IO, we have following options:

• Sequential writes

• Sequential read

• Random write

21 • Random read

In SQLite, we have following options:

• Insert

• Update

• Delete

It also provides us options to change partition, number of threads in a benchmark, IO size, File size, IO modes, number of transaction, and journal modes of database.

• Partition : Changes target partition of the device from “/data to following partitions.

– /data : /data partition of internal storage. – /sdcard: /sdcard partition of internal storage. – /extSdCard: /extSdCard partition of external storage.

• Number of Threads : Default mode runs in single thread and it allows to increase the number up to 100 threads

• Custom check box : Using check box, user can select specific tests in File IO and SQLite benchmarks.

• File size: Default file size for write and read is 1MB and 32MB, respectively, but we can change accordingly as we want.

• IO Size : Changes the size of IO issued in File IO benchmark. Default size is 4KB.

• Mode (File IO) : Changes synchronization mode for File IO benchmark. Default mode run in O DIRECT mode. As we can change modes available are as follows:

– Normal : Buffered IO – O SYNC : Synchronous IO – sync : SQLite use to flush the data via sync () system call. – O DIRECT : Direct IO – mmap : mmaps creates file to virtual address space. Then, the program copies the specified size record, e.g., 4KB, to the mmaped memory region. Upon completing copy of all memory copy, this mode flushes file through msync() to storage device.

• Transaction (SQLite) : We can set number of SQL transaction, where default number of transactions is 100. Maximum number of transaction is 1,00000.

• SQLite Journal : We can Change SQLite journal mode. Default mode of SQLite is Truncate mode. We have other mode also, i.e., WAL, Delete mode, Persist mode, Memory mode, and OFF mode. We can select one among them.

22 4.2 Aakash Tablet Specification

Aakash tablet is low cost tablet, it is specially designed to enhance the learning and teaching skills of both student and teacher. The Indian Government has a vision to provide such products to all 220 million students across the country over the next few years[28].

The first version of Aakash tablet was launched in 2011 and second version of this tablet was launch in April, 2012[29]. Now, third version of Aakash tablet is expected to be launched in the near future. With its super specifications and incredibly low price, it will once again set new standards for tablet computers all over the world[28]. The following table shows the specification of second[30] and third version of Aakash tablet.

Feature Aakash2 Aakash4 RAM 256 MB 512MB Processor 700 MHz Cortex A8 processor Cortex A7 Dual Core; 1.2 GHz Internal Storage 2GB 8GB Expandable Storage 8GB 32GB Camera Front facing Front VGA / Rear 2MP Battery 3200 mAh (4 hours) 4000 mAh Operating System Android 2.3.3 Android 4.2.1 Jelly Bean SIM calling No Yes

As we know, Aakash4 is not available in market; we will measure the performance of Aakash2 using MobiBench in the next chapter.

23 Chapter 5

Test Result of MobiBench on Aakash Tablet

5.1 Testing Results on Aakash Tablet

As we discussed the specification of Aakash2 and available features of Mobibench. In this chapter, we will measure the database performance of the Aakash tablet. In Mobibench, we

No. of File size(w) FileSize(r) IO Size No. of Jornalal Mode File Mode threads (MB) (MB) (kb) transaction 1 10 32 4 100 Truncate O Direct

Table 5.1: Default setting of Mobibench have an option to choose the desire test cases to measure the performance, but we also have a default setting which has been shown in the table 5.1. Initially, I tested the overall performance of Aakash2 using default settings, whose results have been shown in figure 5.1 and test result of database is shown in figure 5.2.

5.1.1 Test Cases for SQLite

As our aim is to know the database performance of Aakash2, we have to choose good test cases. Table 5.2 shows test cases for database testing. In this table, we choose three different modes of SQLite and also number of transactions, i.e., 100, 1000, and 5000. Mode of SQLite is already discussed in the Chapter 2 and based on test case which is on table 5.2, we will measure the performance of Aakash2 tablet.

5.1.2 Test Result

Results of database testing on Aakash2 are shown in table 5.2. If we take the average of the insert, delete, and update of each journal mode of testing result, we will get the result which is shown in figure 5.3. From the results we can say that WAL mode is best for overall performance for database operations on Aakash2 and its performance is nearly same as persist mode.

24 Figure 5.1: Overall performance of Aakash2

Figure 5.2: Database performance of Aakash2

Test Cases No. of Threads No. of Transactions Journal Mode 1 1 100 Delete 2 1 1000 Delete 3 1 5000 Delete 4 1 100 Persist 5 1 1000 Persist 6 1 5000 Persist 7 1 100 WAL 8 1 1000 WAL 9 1 5000 WAL

Table 5.2: Test cases for SQLite

25 Throughput of Throughput of Throughput of Test. Case Insert Update Delete 1 17 16 16 2 18 14 16 3 16 16 17 4 34 46 60 5 62 56 72 6 69 49 57 7 36 104 46 8 61 81 64 9 71 99 94

Table 5.3: Database Performance of Aakash2

Figure 5.3: Database performance of Aakash2

5.2 Battery consumption during testing

I installed the “batterychecker” which is an Android application[31], that tells the battery status of Android device. During the testing of Aakash2, battery status was 51% and at the end of the test, battery status became 40%.

As a result, we can say that battery consumption of database performance on Aakash2 using the given the test cases is approximate ly 11%.

26 Chapter 6

Modification Done in MobiBench

6.1 Source code Analysis of MobiBench

In previous chapter, we saw the available features of MobiBench and tested results on Aakash Tablet. Before talking about possible enhancement on MobiBench, we will discuss the developer view of MobiBench and its behavior. I went through the source code; in Android version of MobiBench has three packages that are: • Database : It has one Java class, i.e., NotesDbAdapter.java. That is for:

– Create the Database, i.e., MobiDB. – Store the result after testing.

• MobiBench : It has 10 Java classes and used for even handling of the overall behavior. In this package, some Java classes are used for set the flag variable and used to call the JNI file, i.e., mobibench.c.

• ResultListControl : It has also 10 Java classes and it is used for show the result after the getting the result. In spite of this package, we have one mobibench.c in JNI folder. All the important function like, thread main db() and thread main(), implemented in mobibench.c. Whenever we need these functions, we call from Mobibench package. We can see the simple control flow graph of MobiBench in figure 6.1, it shows the overall control flow of MobiBench.

As we are interested on database part, so we have to know the table schema used in Mo- biBench. It uses only one table for testing the database operation that is defined in mobibench.c and table schema show in figure 6.2. Whenever we want to measure the database performance, we have to call the thread main db() and the following activity happens inside the function. • Insert : There is one macro variable define for insert string, i.e., #define INSERT STR aaaaaaaaaabbbbbbbbbbccccccccccddddddddddeeeeeeeeeeffffffffffgggggggggghhhhhhhhhhiiiiiiiiiijjjjjjjjjj. Whenever we want to measure the performance of insert transaction, INSERT STR will be inserted into the table using id as a primary key.

27 Figure 6.1: Control flow of MobiBench

Figure 6.2: Default table schema

• Update : There is one macro variable define for update string, that is, #define UP- DATE STR ffffffffffgggggggggghhhhhhhhhhiiiiiiiiiijjjjjjjjjjaaaaaaaaaabbbbbbbbbbccccccccccddddddddddeeeeeeeeee. Whenever we want to measure the performance of update transaction, its update the old string to UPDATE STR using primary key.

• Delete : It simply deletes all the tuples available from table.

After this function, it calls the other function like CPU usage and gets the performance as a throughput.

28 Figure 6.3: Email system table schema

Figure 6.4: Bowser history table schema

Figure 6.5: WhatsApp table schema

6.1.1 Possible Enhancement

After analyzing the MobiBench, We got the following result:

29 • MobiBench uses only one table and it has only three attributes.

• It has macro variable for inserts and updates operations.

• It does not perform any random string generation for inserting the data.

Figure 6.6: Choose table

Based on this result, we can say that MobiBench is very simple and not realistic. To make it more realistic, we can do the following possible enhancements.

• When we talk about the real application, there might be many tables for single application and might be many attribute for single table. To make more realistic, we can provide many table to users based on real life application’s table.

Based on this solution, I have added some tables in MobiBench, so that user can choose any one table for testing.

6.2 Modifications done to MobiBench Database

In MobiBench Setting tab, I gave choose table option, that contains four table, that is shown in figure 6.6 and user can choose any one of them. The description of table is listed following:

• Default : Tt contains tblMyList table that is already existed in MobiBench and schema for this table shown in figure 6.2.

• Email : This is based on the real email system, that we use in daily life, schema for this table is shown in figure 6.3.

• Browser history : For holding the history of browser, it contains well define schema that is shown in figure 6.4.

• WhatsApp : Table schema of WhatsApp is complex that is shown in figure 6.5.

30 I have also added random string generation for insert and update operation, that will make the MobiBench more realistic. Whenever user will select a table, MoBiBench will set that table for testing.

31 Chapter 7

Test Result after Modification of MobiBench on Aakash tablet

7.1 Testing Result after changes

As we know, we added three more tables related to real life applications, we will discuss each of the performance in Aakash2 using different modes of SQLite.

Database Performance

As we know, I have given the table chooser option in setting tabs. For database testing, I choose each table and performed those test cases that are defined in table 5.2. These are the test result, which is performed on Aakash2.

• Browser History : Table 7.1 shows browser history table performance result.

Throughput of Throughput of Throughput of Test. Case Insert Update Delete 1 16 25 33 2 23 25 27 3 26 26 27 4 93 201 57 5 90 79 106 6 91 66 105 7 25 93 104 8 80 103 103 9 104 133 150

Table 7.1: Database Performance of Browser History table in Aakash2

• Email System : Table 7.2 shows the result of email system of database operation.

• WhatsApp : Table 7.3 shows the result of WhatsApp performance

32 Throughput of Throughput of Throughput of Test. Case Insert Update Delete 1 18 37 25 2 22 28 29 3 26 27 28 4 98 52 57 5 63 43 73 6 75 56 81 7 28 56 140 8 33 53 92 9 52 70 78

Table 7.2: Throughtput Result of Email system table in transaction/s

Throughtput of Throughtput of Throughtput of Test. Case Insert Update Delete 1 15 45 25 2 25 28 27 3 25 28 25 4 44 43 38 5 49 45 40 6 52 50 45 7 32 57 56 8 36 60 53 9 37 63 53

Table 7.3: Throughtput Result of WhatsApp table in transaction/s

7.2 Result

When we take the average of the insert, update, and delete throughput of the resulted table, then we got the result graph of Browser history, Email system, and WhatsApp, that is shown in figure 7.1, 7.2, and 7.3 respectively

7.3 Battery consumption during testing

During testing of the database performance of the given three table(i.e., Browser history, Email system, and WhatsApp), it consumed approximate 40% of battery of Aakash2.

33 Figure 7.1: Database performance of Browser table in Aakash2

Figure 7.2: Database performance of Email System in Aakash2

34 Figure 7.3: Database performance of WhatsApp in Aakash2

35 Chapter 8

Conclusion and Future Work

8.1 Conclusion

With the tremendous increase in usage of smart-phones and not only for basic purposes such as calls, messages, but also for heavy applications like multimedia, games, social networking apps, etc. Due to such a high demand, now a days new applications are released every day. We analyzed the way Android application works and we found out that the performance of Android device depends on the performance of database, i.e., SQLite. There is a need to measure the

Figure 8.1: Database performance using WAL mode performance of database for mobile devices.

Based on this motivation, we studied monitoring tools for SQLite Database, i.e., MobiBench and we performed tests on Aakash2 tablet. We got the result that WAL journal mode gives best throughput among all the available modes on SQLite.

When we studied the source code of MobiBench, we came to know that database model of

36 MobiBench is not realistic. For making it more realistic, we added three tables, i.e., Browser history, Email system, and WhatsApp and we gave an option to the user to choose a table, so that the user can choose one of the realistic base table to perform the testing. After modifying the MobiBench, we tested Aakash2 using each table, and we found out that WAL mode is the best among all journal modes. We can see the performance of different table in figure 8.1.

8.2 Future Work

MobiBench uses only one table at a time for database testing. We can extend the following feature of MobiBench.

• We can add more tables related to a single database, so that a single transaction is required to access more than one table.

• We can give a “file import” option to the user, so that they can import their own table schema to perform any test.

37 References

[1] “Smartphone user in india.,” 2013. URL:http://statspotting.com/here-is-why-you-should- build-smartphone-apps-for-the-indian-market/[last accessed: April, 24, 2014].

[2] “Architecture of berkeley db,” URL:http://www.aosabook.org/en/bdb.html[last accessed: Sept, 29, 2013].

[3] “Architecture of sqlite,” URL:http://www.sqlite.org/arch.html[last accessed: Sept, 29, 2013].

[4] “Structure of androbench measure tab,” URL:http://www.androbench.org/wiki/File:Androbench bench.png[last accessed: Oct, 2, 2013].

[5] “Structure of androbench setting tab,” URL:http://www.androbench.org/wiki/File:Androbench setting.png[last accessed: Oct, 2, 2013].

[6] Android framework. URL:http://www.linuxinsider.com/story/61961.html[last accessed: April, 24, 2014].

[7] “Roolback Journal.” http://www.sqlite.org/draft/atomiccommit.html. [Online; last accessed Oct, 12, 2013].

[8] “Limitation of mobile device,” URL:http://www.wireless-center.net/Mobile-and- Wireless/2505.html[last accessed: Oct, 2, 2013].

[9] H. Kim, N. Agrawal, and C. Ungureanu, “Revisiting storage for smartphones.,” TOS, vol. 8, no. 4, p. 14, 2012. URL:http://doi.acm.org/10.1145/2385603.2385607[last accessed: Aug, 30, 2013].

[10] H. Kim, N. Agrawal, and C. Ungureanu, “Revisiting storage for smartphones.,” TOS, vol. 8, no. 4, p. 14, 2012. URL:http://doi.acm.org/10.1145/2385603.2385607[last accessed: Aug, 30, 2013].

[11] “Berkeley db,” URL:http://www.oracle.com/technetwork/products/berkeleydb/ overview/index.html[last accessed: Oct, 2, 2013].

[12] F. Wikipedia, “Berkeley db,” http://en.wikipedia.org/wiki/Berkeley DB[last accessed: Sept, 29, 2013].

38 [13] K. B. Michael A. Olson and M. Seltzer, “Berkeley db,” Proceedings of the FREENIX Track:1999 USENIX Annual Technical Conference, 1999. URL:http://static.usenix.org/event/usenix99/full papers/ olson/olson.pdf[last accessed: Sept, 29, 2013].

[14] “Perst db,” URL:http://www.mcobject.com/[last accessed: Sept, 29, 2013].

[15] “Mcobject releases perst vs. sqlite benchmark on android,” URL:http://www.drdobbs.com/database/mcobject-releases- perst-vs-sqlite-benchm/205207102[last accessed: Sept, 29, 2013].

[16] “Sqlite features.,” 2013. URL:https://www.sqlite.org/about.html/[last accessed: April, 24, 2014].

[17] “Sqlite,” URL:https://www.sqlite.org/[last accessed: Sept, 29, 2013].

[18] C.-H. Wu, T.-W. Kuo, and L.-P. Chang, “An efficient b-tree layer implementation for flash-memory storage systems.,” ACM Trans. Embedded Comput. Syst., vol. 6, no. 3, 2007. URL:http://doi.acm.org/10.1145/1275986.1275991[last accessed: Aug, 30, 2013].

[19] “journal in sqlite,” URL:http://www.sqlite.org/lockingv3.html#rollback[last accessed: Sept, 29, 2013].

[20] “Berkeley db vs. sqlite db,” URL:http://db-engines.com/en/system/Berkeley+DB%3BSQLite[last accessed: Oct, 2, 2013].

[21] “Synchronous io vs. buffered io,” URL:http://stackoverflow.com/questions/ 1450551/buffered-i-o-vs-unbuffered-io[last accessed: Oct, 2, 2013].

[22] B. of open source in storage, “Iozone filesystem benchmark,” URL:http://www.iozone.org/docs/IOzone msword 98.pdf[last accessed: Sept, 29, 2013].

[23] J. H. S. L. Sooman Jeong, Kisung Lee and Y. Won, “Androstep: Android storage perfor- mance analysis too,” 2013. http://www.esos.hanyang.ac.kr/files/publication/conferences/ international/AndroStep.pdf[last accessed: Sept, 29, 2013].

[24] J.-M. Kim and J.-S. Kim, “Androbench: Benchmarking the storage perfor- mance of android-based mobile devices.,” in ICFCE (S. Sambath and E. Zhu, eds.), vol. 133 of Advances in Soft Computing, pp. 667–674, Springer, 2011. URL:http://csl.skku.edu/papers/fce12a.pdf:[last accessed: AUG, 30 2013].

[25] I/o stack optimization for smartphones. s. jeong, k. lee, s. lee, s. son, and y. won ed. URL:https://www.usenix.org/conference/atc13/technical- sessions/presentation/jeong.

[26] P. Patodi, Page Replacement Mechanism for Small Foot-Print Database in Android Devices. 2013. URL:http://www.it.iitb.ac.in/arndg/dokuwiki/images/5/5d/Stage2Report.pdf.

[27] “Truncate journaling mode,” URL:http://www.sqlite.org/pragma.html[last accessed: June, 10, 2014].

39 [28] Aakash tablet Specifications. Last accessed:June 10,2014. http://www.akashtablet.com/aakash4-tablet/.

[29] Aakash tablet Specifications. Last accessed:June 10,2014. http://aakash.org.in/158/aakash- 2-tech-specifications-of-the-cheapest-tablet-pc-in-the-world.

[30] Aakash 2 Specifications. Last accessed:June 10,2014. http://aakash.org.in/158/aakash-2- tech-specifications-of-the-cheapest-tablet-pc-in-the-world.

[31] “Battery checker apps,” URL:https://github.com/ninadchilap/batterycheck[last accessed: June, 10, 2014].

40