International Journal of Mathematics and Computer Applications Research (IJMCAR) ISSN 2249-6955 Vol. 3, Issue 2, Jun 2013, 175-182 ©TJPRC Pvt. Ltd.

IMPLEMENTATION OF ANDROID BASED MOBILE PHONE CRAWLERS

SRASHTI GUPTA1 & KUNAL GUPTA2 1Student, Amity University, Noida, Uttar Pradesh, India 2Assistant Professor, Amity University, Noida, Uttar Pradesh, India

ABSTRACT

As mobile applications become more difficult, precise development tools and frameworks as well as cost valuable testing techniques and tools will be necessary to assure the development of secure, high-quality mobile applications. This paper concentrates on the mobile crawling in Android platform, and represents a technique for rapid crawling of Android applications. The procedure is based on a crawler that automatically searches a model and obtains result that can be automatically executed. This technique is supported by a tool for both crawl the data and generating the result. In the paper we present an example of using the technique and the tool for crawl a file which has mp3, txt etc extension by using Android application that preliminary shows the effectiveness and usability of the proposed approach. By using that crawler system the data is secure and easy to handle in an organization.

KEYWORDS: Android Application, Crawling, Mobile Phone Crawlers, Security

INTRODUCTION

The mobile progress community is at a tipping point. Mobile users insist more choice, more prospects to customize their phones, and more functionality. Mobile operators want to provide value-added content to their subscribers in a convenient and profitable way. Mobile developers want the liberty to develop the powerful mobile applications users demand with minimal roadblocks to victory. Finally, handset manufacturers want a steady, protected, and reasonable platform to power their devices [3]. Up until now particular mobile platform has sufficiently addressed the needs of all the parties. Enter Android, which is a possible game-changer for the mobile development community. An innovative and open platform, Android is well positioned to address the growing needs of the mobile marketplace.

It proposes an alternative approach to Mobile crawling based on mobile phone crawlers. Crawler mobility permits for more difficult crawling algorithms and avoids some of the inefficiencies linked with the brute force strategies exercised by current crawlers. By looking at the common structure of traditional search engines, it recognized that their planning is strictly centralized while the data being accessed by search engine crawlers is extremely scattered. If here apply a stationary crawler, firstly it would download many pages from Mobile Phones, which are detached immediately because they do not meet the theme area of the search engine.

Perceptibly, this behavior is not very attractive because network bandwidth is exhausted by downloading unrelated information. Here it illustrate mobility in the context of Mobile crawling as the capability of a crawler to transfer itself to every Mobile Phone of notice before assembling pages on that Phone. After finishing the collection process on a particular Phone, the crawler together with the collected data travels to the next Phone or to its Network. Mobile crawlers are managed by a crawler manager (CM) which supplies each crawler with a record of target Mobile Phones and monitors the position of each crawler. In addition, the crawler manager provides the necessary functionality for pull out the collected data from the crawler for use by the indexer. 176 Srashti Gupta & Kunal Gupta

Figure 1: Architecture of Proposed Android Based Mobile Crawler Powerful Development Framework

Easily optimize a single binary for phones, tablets, and other devices. Android gives each and everything to user which they needed to build best in class familiarity. It gives a single application representation that organize apps broadly to hundreds of millions of users diagonally a wide range of devices—from phones to tablets and beyond. Android tools for creating apps that look great and take benefit of the hardware capabilities available on each device [13]. It repeatedly adapts your (UI) to look its best on each device, while giving you as much control as you want over your UI on dissimilar device types.

For Example, It can create single app binary that’s optimized for both phone and tablet form factors. It announce UI in lightweight sets of XML resources, one set for parts of the UI that are common to all form factors and other sets for optimizations precise to phones or tablets [14]. At run time, Android applies the correct resource sets based on its screen size, density, locale, and so on. To help you develop efficiently, Android Developer Tools offer a full Java IDE with superior features for developing, debugging, and packaging Android apps [2]. Using the IDE, you can expand on any available Android device or create virtual policies that emulate any hardware configuration.

QUALITIES FOR MOBILE PHONE CRAWLER

The main benefit of the mobile crawling approach is that it permits to allocate crawling functionality within a distributed system such as the Web. Particularly, the following four qualities plays major role in this system:

Localized Data Access

Due to the HTTP request/response pattern, downloading the contents from a Web server engross important overhead due to request messages which have to be sent for every mobile page separately. This approach still requires one HTTP request per document but there is no need to broadcast these requests over the system any longer. Using a mobile crawler we reduce the HTTP overhead by shifting the crawler to the supply of the data. The crawler can then issue all HTTP requests locally with respect to the HTTP server.

Remote Page Selections

Mobile crawlers utilize the query shipping approach of database systems because all the information desirable to identify the relevant data portion is transferred directly to the data source along with the mobile crawler. After the query has been performed remotely, only the query result is transferred over the network and can be used to establish the preferred index without requiring any extra analysis. Implementation of Android Based Mobile Phone Crawlers 177

Remote Page Filtering

Remote page filtering enlarges the concept of remote page selection to the contents of a Web page [4]. The thought behind remote page filtering is to allow the crawler to control the granularity of the data it retrieves. Depending on the ratio of relevant to irrelevant information, important portions of network bandwidth are wasted by transmitting ineffectual data [11].

A mobile crawler defeats this problem since it can filter out all irrelevant page portions keeping only information which is applicable with respect to the search engine the crawler is working for.

Remote Page Compressions

In order to decrease the quantity of data that has to be transmitted back to the crawler organizer, it introduces remote page compression as another basic feature of mobile crawlers [2].

In order to decrease the bandwidth required to transfer the crawler along with the data it holds back to the search engine, the mobile crawler applies compression procedures to reduce its size earlier to transmission.

SOFTWARE USED IN CRAWLING

In this section be going to to present the software which is used in different mobile phone systems. The basic fundamental technologies are HTML, XHTML, WAP, WML, CSS, ECMA Script.

Table 1: Browsers in Various Mobile Systems

Browser Origin Used in Android browser Google Nexus BlackBerry Browser Research in Motion - Palm installed on all newer Palm Treos and PDAs Mobile Microsoft Samsung newer versions for mobile Mozilla Nokia Maemo and for Android Acquired by Research in Motion - No longer supports Iris Browser Mobile Inc. or Linux Capable of reading HTML and reformat for small Mobile Opera Software screens, installed on many phones NetFront ACCESS Co.,ltd - Obigo Browser Obigo AB 100% owned by Teleca AB PlayStation Portable Sony Sony PSP Myriad Browser (Previously Myriad Group Acquired from Openwave in 2008 Openwave ) Nokia, Samsung, LG Electronics, KYOCERA and other Polaris Browser Infraware Inc. and cellular phone in USA, China, Korea, etc Renders Flash 10, Ajax and Silverlight content. supports Mobile Browser Skyfire Windows Mobile 5/6.x,Symbian S60 and Android. Apple Inc on iPhone, iPod Touch and iPad on Samsung, LG Electronics and other and uZard Web Logicplant Co., Ltd. cellular phones in Korea Kindle Basic Web Amazon.com Blackberry WebOS Browser Palm - xScope xScope Mobile on android systems

Table 1 record the compilation of various browsers in use by special Mobile phones systems from the mobile crawler using GPRS.

178 Srashti Gupta & Kunal Gupta

A TECHNIQUE FOR ANDROID MOBILE PHONE CRAWLERS

Figure 2: The Crawling Algorithm DEVELOPING APPLICATION FOR MOBILE PHONE CRAWLER Commonly Used Packages With Android, mobile developers no longer have to reinvent the controls. Instead, developers use recognizable class libraries exposed through Android’s Java packages to execute common tasks such as graphics, database access, network access, secure communications, and utilities (such as XML parsing) [13].

The Android packages include support for

 Common user interface widgets (Buttons, Spin Controls,Text Input)

 User interface layout

 Secure networking and Web browsing features (SSL,WebKit)

 Structured storage and relational databases (SQLite)

 Powerful 2D and 3D graphics (SGL and OpenGL ES 1.0)

 Audio and visual media formats (MPEG4, MP3, Still Images)

 Access to optional hardware such as Location-Based Services (LBS), Wi-Fi, and Bluetooth.

Developing Android Applications

The Android SDK provides a widespread set of application programming interfaces () that is both modern and robust. Android phones core system services are uncovered and available to all applications. When approved the appropriate permissions, Android applications can share data surrounded by one another and access shared possessions on the system securely. Implementation of Android Based Mobile Phone Crawlers 179

Android Programming Language Choices

Android applications are written in Java. For now, the Java language is the developer’s only choice on the Android platform. There has been some assumption that other programming languages, such as C++, might be added in future versions of Android. No dissimilarity Made Between Native and Third-Party Applications.

Unlike other mobile phone crawler development platforms, there is no difference between native applications and developer-created applications on the Android platform. Make available the application is granted the appropriate permissions, all applications have the similar access to core libraries and the underlying hardware interfaces. Android handsets transport with a set of native applications such as a Web browser and contact manager [12]. Third-party applications might put together with these core applications and even extend them to provide a rich user experience.

Figure 3: Java Android Code Which Indicates if this File Represents a Directory on the Underlying File System

Figure 4: Java Android Code for Crawl the Define Extention Files RELATED WORK Android Application Framework The Android application framework presents everything necessary to implement your average application. The Android application lifecycle involves the following key components:

 Activities are functions the application performs.

 Groups of views define the application’s layout.

 Intents inform the system about an application’s plans.

 Services allow for background processing without user interaction.

 Notifications alert the user when something interesting happens. 180 Srashti Gupta & Kunal Gupta

Android Applications can interact with the operating system and underlying hardware using a collection of managers. Each manager is responsible for maintenance the state of some underlying system service. For example, there is a Location Manager that make possible interaction with the location-based services available on the handset [6]. The View Manager and Window Manager manage user interface fundamentals. Applications can interact with one another by using or acting as a Content Provider. Built-in applications such as the Contact manager are content providers, allowing third- party applications to access contact data and use it in an infinite number of ways.

Figure 5: Mobile Phone Crawler Android Simulator SECURITY AND PERMISSIONS

The integrity of the Android platform is maintained through a variety of security measures.

Applications as Operating System Users

When an application is installed, the operating system creates a new user summary associated with the application. Each application runs as a different user, with its own private files on the file system, a user ID, and a secure operating environment [11]. The application executes in its own process with its own occurrence of the Dalvik VM and under its own user ID on the operating system.

Explicitly Defined Application Permissions

To access shared possessions on the system, Android applications register for the specific constitutional rights they require. Some of these privileges enable the application to use phone functionality to make calls, right to use the network, and control the camera and other hardware sensors. Applications also require permission to contact shared data containing private and personal information such as user preferences, user’s location, and contact information.

Applications might also impose their own permissions by declaring them for other applications to use. The application can announce any number of different permission types, such as read-only or read-write permissions, for finer control over the application.

Limited Ad-Hoc Permissions

Applications that act as content providers might want to provide some on-the-fly permissions to other applications for specific information they want to share openly [7]. This is done using ad-hoc granting and revoking of access to specific resources using Uniform Resource Identifiers (URIs). URIs is index specific data assets on the system, such as text. Here is an example of a URI that provides the phone numbers of all contacts: content: //contacts/phones Implementation of Android Based Mobile Phone Crawlers 181

To understand how this authorization process works, let’s look at an example. Let’s say we’ve got an application that keeps track of the user’s public and private birthday wish lists [3]. If this application wanted to share its data with other applications, it could grant URI permissions for the public wish list, allowing application authorization to access this list without explicitly having to ask for it.

Application Signing for Trust Relationships

All Android applications packages are signed with a certificate, so users know that the application is authentic. The private key for the documentation is held by the developer. This helps establish a trust relationship between the developer and the user [8]. It also allows the developer to control which applications can grant access to one another on the system. No certificate authority is necessary; self-signed certificates are acceptable.

CONCLUSIONS

Mobile software expansion has evolved over time. Android has emerged as a new mobile development platform, building on past successes and avoiding past failures of other platforms. Android was designed to sanction the developer to write innovative applications [10]. The platform is open source, with no up-front fees, and developers enjoy many benefits over other competing platforms [1]. Now it’s time to dive deeper and start writing Android code, so you can evaluate what Android can do for you. By using android platform user get information

REFERENCES

1. Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana," A GUI Crawling-based technique for Android Mobile Application Testing", 2011 Fourth International Conference on Software Testing, Verification and Validation Workshops.

2. Pavalam S. M., S. V. Kasmir Raja, Jawahar M., and Felix K. Akorli," Web Crawler in Mobile Systems",International Journal of Machine Learning and Computing, Vol. 2, No. 4, August 2012.

3. Android Developers. The Developer’s Guide. Available at: http://developer.android.com/. Last accessed Jan. 08, 2013

4. Android Emulator, available at: http://developer.android.com/guide/developing/tools/emulator.html Last accessed January 8, 2013

5. Joachim Hammer, Jan Fiedler,"Using Mobile Crawlers to Search the Web Efficiently", in International Journal of Computer and Information Science, 1:1, pages 36-58, 2000.

6. Jan Fiedler and Joachim Hammer,"USING THE WEB EFFICIENTLY: MOBILE CRAWLERS", In Seventeenth Annual International Conference of the Association of Management (AoM/IAoM) on Computer Science, Maximilian Press Publishers, San Diego, CA, pages 324-329, August 1999.

7. Jianxia chan,Wei wu, Chunzhi Wang," A Mobile Phone Information Search Engine Based on Heritrix and lucene", The 7th International Conference on Computer Science & Education (ICCSE 2012) July 14-17, 2012.Melbourne, Australia.

8. Anbukodi.S, Muthu Manickam.K,," Reducing Web Crawler Overhead using Mobile Crawler", PROCEEDINGS OF ICETECT 2011 978-1-4244-7926-9/11/$26.00 ©2011 IEEE. 182 Srashti Gupta & Kunal Gupta

9. Hiroshi Takeno, Makoto Muto, Noriyuki Fujimoto," Developing a Web Crawler for Massive Mobile Search Services", Proceedings of the 7th International Conference on Mobile Data Management (MDM'06) 0-7695-2526- 1/06 $20.00 © 2006 IEEE .

10. Md. Faizan Farooqui, Dr. Md. Rizwan Beg and Dr. Md. Qasim Rafiq," AN EXTENDED MODEL FOR EFFECTIVE MIGRATING PARALLEL WEB CRAWLING WITH DOMAIN SPECIFIC AND INCREMENTAL CRAWLING", International Journal on Web Service Computing (IJWSC), Vol.3, No.3,September 2012.

11. Ismet Aktas, Florian Schmidt, Muhammad Hamad Alizai, Tobias Drüner, Klaus Wehrle," CRAWLER: An Experimentation Platform for System Monitoring and Cross-Layer-Coordination", 978-1-4673-1239-4/12/$31.00 c 2012 IEEE.

12. Pavalam S. M., S. V. Kasmir Raja, Jawahar M., and Felix K. Akorli," Web Crawler in Mobile Systems", International Journal of Machine Learning and Computing, Vol. 2, No. 4, August 2012.

13. Android Developers. The Developer’s Guide. Available at: http://www.vogella.com//. Last accessed apr. 08, 2013

14. Android Emulator, available at: http://en.wikipedia.org/wiki/Android_ (operating_system) Last accessed march 25, 2013

AUTHOR’S DETAILS

Ms. Srashti Gupta is doing her M.Tech. from Computer Science and Engineering, Amity School of Engineering and Technology, Amity University, Uttar Pradesh. She would be pursuing PHD in Networking Field. Her research area includes Networking.

Ms. Kunal Gupta pursuing PHD from C B Raman University. Presently he is working as an Assistant Professor in CSE Department, Amity University, Noida, Uttar Pradesh, India. His Research area includes Networking.