Towards Automatic Consistency Checking between Web Application and its Mobile Application
1,2,3 Xiangping Chen Zhensheng Xu1,2 1 Institute of Advanced Technology, Sun Yat-sen 1 School of Information Science and Technology, Sun University, National Engineering Research Center of Yat-sen University, National Engineering Research Digital Life, Guangzhou, 510006, China Center of Digital Life, Guangzhou,510006, China 2 Shenzhen Engineering Laboratory for Mobile Internet 2 Research Institute of Sun Yat-sen University in Application Middleware Technology, Shenzhen, Shenzhen, Shenzhen, 518057, China 518060, China [email protected] 3 Research Institute of Sun Yat-sen University in Shenzhen, Shenzhen, 518057, China [email protected]
Abstract—With the increasing usage of mobile devices in daily news by clicking the element “News” in the homepage, it is life, a lot of websites delivered another mobile application improper to provide the list of international news through the version of application with the same services. If the mobile “Information” element in the interface of mobile application. application is inconsistent with the web application, it may In addition, the list of international news should be the same. imply possible error or misleading information in the mobile For a web application containing hundreds and thousands application. This paper proposes an approach for automatic pages, the implementation of its mobile application is consistency checking between mobile application and its web time-consuming with a lot of repetitive work. There may rise application. Our approach starts from the generation of page inconsistency problem that the mobile application include: (1) and link information by crawling content from web application error page that shows information which is not included in and simulating user actions in Android application. Page the web application; (2) error element that displays matching and consistency checking algorithms are proposed and used to find out inconsistency in page, element and link. misleading information; (3) error link that is linked to the The validation tool is used in 3 mobile application containing wrong page. Although, not all the inconsistencies exist more than 300 pages. The result shows that our approach can between a mobile application and its web application are find out inconsistency with acceptable precision and recall. faults. For example, a mobile application may provide a special sale for mobile users only. However, inconsistency Keywords-Consistency checking, Black-Box, Mobile information is still useful to make sure that extra Application functionalities are added. Current works[1-5] focus on consistency checking of I. INTRODUCTION different software models in an application. The problem in With the increasing usage of mobile devices in daily life, this paper is how to find out the consistencies between two the number of mobile applications increased very quickly. implementations with the same design model/requirements. By July 2013, the number of applications in Google play has Some works proposed testing methods and tools[7-10] for rose through 1 million1. Android applications. However, they do not notice the With this trend, a lot of websites delivered another consistency problem. mobile application version with the same services. Almost In this paper, we propose an approach for automatic all the top 500 global sites2provide its corresponding android consistency checking between mobile application and its application3. web application. We abstract both web application and Because the web application version and mobile mobile application as a set of pages and link relationships application version are provided for the same target users, between pages. Our approach starts from the generation of the functional requirements of the mobile application are page and link information by crawling content from web already fixed before implementation. The requirement is application and simulating user actions in Android usually the same as the web application, or subset of the application. We define the consistency relationship between requirements of the web application. In addition, these two applications and propose page matching and consistency functionalities are required to be provided in the same way. checking algorithm to find out inconsistencies in page, For example, if a user can access the list of international element and link. The tool is used in 3 mobile applications containing more than 300 pages. The result shows that our approach can find out inconsistencies with acceptable 1http://www.techweb.com.cn/world/2013-07-25/1312444.shtml precision and recall. Our approach can help software 2http://www.alexa.com/topsites 3 https://play.google.com/store
53 engineers and maintainers in finding possible errors during development and maintenance. The remainder of the paper is organized as follows: Section 2 gives an illustrative example to show inconsistency problem; Section 3 gives the definition of consistency and a general overview of our solution, the definition of consistency and a general overview of our solution, which is further detailed in the Section4. Section 5 presents the evaluation of our approach; Section 6 discusses the limitations of our work, Finally, Section 7 discusses some related works before Section 8 concludes our work.
II. ANILLUSTATIVE EXAMPLE For the OSChina website (http://www.oschina.net), its android application is delivered to provide the same functionality as its website. When we click the button with Figure 2. Corresponding page in the linked page text “推荐阅读” in the main interface of mobile application, III. APPROACH OVERVIEW we can see the interface shown in Fig 1(a). In the home page of the web application, we can find a A. The Consistency Requirementsbetween Web Application list of articles, as shown in Fig 1(b). These articles are listed and Mobile Application without detail information. In the mobile application, the A web application or mobile application can be author, the type of paper, publishing time and number of abstracted as AP = （P, L）where P is the set of web pages, L reviews of an article is listed in the interface. is the set of link between web pages. A page in P includes a list of elements. The elements are divided into two types: elements with link and element without link. P = (El, Ee).
link=
54 they are used to provide the same information or functionality. description(e) for the element e cannot be complete. For example, for an element which is a picture, it is hard to generate its description. The extracted description from the application may affect the accuracy for finding a possible consistent page. 2）（ , ∈∀∈∀ PpPp i mobile j web consistent ∈∧∈→ ....),( EpEpEpEppp ejeiljliji We assume that a page in a mobile application contains less information and functionality than its corresponding page in the web application version. A page pi in a mobile application is consistent with a page pj in its web application when all the elements included in pi are also included in pj.
3）（ i mobile , j ∈∀∈∀ LlLl web consistent ll ji ),( → consistent jisjsi ..).,.( elelplpl ∧=∧ consistent plpl tjti )).,.( A page in a web application may consistent with multiple pages in the mobile application version. We define that a link Figure 3. Approach Overview li in a mobile application is consistent with a link lj in a web application when (1) their source pages are consistent;(2)the IV. CONSISTENCY CHECKING target pages are consistent; (3) the related elements are equal. A. Generating Page and Link Information The consistency relationship between APweb and APmobile is defined as: In our approach, we view the application as a black-box and simulate user actions to generate the page and link )1( ∈∃∈∀ PpPp consistent pp ),(,, i mobile j web ji information. The generation starts from the homepage or )2( i mobile (, j ∈∃∈∀ LlLl web，consistent ll ji )),( ∨ main interface of the application. ∈∃ Pp consistent ppl ),.(,( ∧ consistent ppl )),.( For each page, we generate descriptions for its elements. j web jsj jtj Because the descriptions are used in the following For each page pi belongs to Pmobile, there exists a comparison, they should be described in the same way. Text corresponding consistent page pj in Pweb; For each li in Lmobile, and image are two of the most important information for user. if li ’s source and target pages are consist with the same page In real application, the pictures used in web applications and in APweb , there exists a corresponding consistent link lj in mobile applications are usually different because they are Lweb. used for different size of displayers. For the image It should be noted that inconsistency between web information, understanding and generating the description of application and mobile application do not imply an error in a picture is too hard. As a result, we only use text the implementation. A lot of mobile applications provide information as the description of page elements. For an specific functionalities for mobile user only. However, the element in page p, it is recorded using the page id, the text inconsistencies are worth of notice during development and information, and its related link. Text and link can be left evolutional process for software engineers. blank if the element does not contain text information or it is B. Approach Overview an element without a link. Element with no text and link information will be deleted because the comparison is Because the implementations of an application in meaningless. different platforms are different, the first step of our The required input for next step, searching possible approach is to generate the description of the mobile corresponding page, is a page and all its related pages. As a application and web application using the same model. The result, width-first search algorithm is better when exploring description includes page and link information. the pages. When exploring the pages, our approach records Considering consistency requirement in implementing a the related elements and the ids of source pages and target mobile application for an existing web application, pages for links. The search omitted a link when the link’s inconsistency may occur at the page level and the element target page is not included in the application or it is already level. Our approach first searches all possible pages in web explored before. The search stops when all the pages are applications to find out a possible consistent page for a page explored or the search depth reaches a pre-defined value. in mobile application. Because the link is related to an Because the programming language for web applications element in the page, the consistency in the element level and mobile applications are different, they have different considers both elements with and without a link. Our types of components in the pages. Our approach employs approach compares the elements and the links in sequence. existing parser/tool to extract information from the
application.
55 For the web application, our algorithm traverses pages corrspondi i = ppng matched i pp ),(|{)'( ≥ matched pp ji )',( from the home page and uses a Java HTML parser Jsoup[11] to extract information from each page. Jsoup provides APIs possible j ' possible ppPpPp j ∧≠∧∈∧∈ matched i pp ≠ }0),(' for fetching web page and extracting the text and link If the number of matched elements between pages pi’ and pj’ information from a page. Because a page in the web are all zero, there is no corresponding page for pi’. application is standalone, we can traverse the pages by using a width first search. During information extraction, we do C. Element and Link Consistency Checking not deal with dynamic content which is generated during We divided the inconsistencies into three types. For a execution of Javascript. These complex situations will be page p in mobile application, there is ： considered in our future work. (1) Page inconsistency: no corresponding page For an android application, our approach simulates user correspond ping )( = φ actions to traverse all possible pages. Because a page (2) Element inconsistency (interface) in the android application is not a standalone page, ∉∧∈∃ fetching a page require the simulation of user actions to .(, e eEpee correspond Eping e )).( access this page from main interface. If the traverse is carried (3) Link inconsistency out using a width-first search, all the user actions simulated .., s .., elppll ∉∧=∃ correspond ).( Eping l ∧ for a page will be repeated carried out when traversing its related pages. As a result, our algorithm traverses page in correspond pling t ).( ≠ correspond ping ))( Android application using the depth-first search. The inconsistent pages can be found in the page We employs an Android test automation framework consistency checking process. An inconsistent page includes Robotium [12] to carry out the simulated user actions. information and functionalities which are not included in the Robotium is an Android test automation framework that web application. Developers can use this result to see if support for native and hybrid applications. In real application, mobile app specific functionalities are implemented, and traversing pages in Android application is time-consuming. analyze the degree of coupling between these new For example, we extract 90 pages from an Android functionalities and existing functionalities. application in about 1 hour. The max depth for the depth first Element inconsistencies are recorded in the page search can be changed according to how complex the consistency checking stage when the algorithm tried to application is. compute the number of matched elements. An element inconsistency indicates possible differences in the way to B. Page Consistency Checking provide information in different applications. The example Based on the extracted page and link information, our introduced in section 2 is a classic example of element approach traverses all the pages in the mobile application inconsistency. In some cases, the developer may use image and finds out their corresponding pages. For a page pi∈ and text to show the same information in different Pmobile, its corresponding page is a page pj∈Pweb, when the applications. This may bring some undetectable consistency possibility for consistency(pi, pj) is higher than the possibility relationship and reduce the accuracy of our method. of consistency(pi, pj’), pj’∈Pweb and pj’ ≠ pj . Link inconsistency is validated based on the We suppose that the corresponding page of the main corresponding relationship between pages. Because there interface of a mobile application is the main page of the web may exist more than one page in the mobile application with application. Page matching starts from finding corresponding the same corresponding page in the web application, the pages linked from the main interface. Because we omitted links between these pages are not matched to a link whose target page is explored before. For a page p, there corresponding link in the web application. The checking is only one link whose target page is p. filters this kind of links. Other inconsistent links are caused For pages pi, pi’ ∈ Pmobile , pj ∈ Pweb, if by link inconsistent.
∈∃ =∧= pplpplLl '.., , corresponding(pi)=pj, mobile itis V. EXPERIMENTS corresponding(pi’)∈{pj } ∪ {p| p= e.link.ps ∧ e∈ .Ep lj We use our consistency tool to validate the following ∈∧ Pp }. android applications with their web applications. We choose web three application in our experiment. Because we view both The matching is carried out in the set of all possible web application and Android application as black-boxes, the matched pages Ppossbile={p| p= e.link.ps ∧ e ∈ .Ep lj first step of our experiment is to find out the website address and download the Android application package file (APK). ∈∧ Pp web }. For pj’∈Ppossbile , e∈pi TABLE I. EXPERIMENT PROJECTS ∈∃ j epe text = e'..,'',1 text (1) matched pe )',( = Name Website address Android Version j ∈∀ ≠ j epe text e'..,'',0 text app Size (M) (2) matched pp ji )','( = ∑matched j ∈ pepe i ',)',( oschina http://www.oschina.net 2.2 1.7.6.5 Based on the number of matched elements, we can find zol http://www.zol.com.cn 7.2 3.5.2 chinadaily http://www.chinadaily.com.cn 3.3 3.2.0 the corresponding page by comparing text information.
56 Because the generation of page and link information Project name Precision Recall Precision Precision Page page element link from the android application is time-consuming, we inconsistency inconsistency inconsistency inconsistency pre-defined the depth during explosion. Table II shows the oschina 1/4 1/2 187/243 40/49 page size and execution time in extracting page and link Zol 2/5 2/4 138/174 5/7 information. The pre-defined depth for explosion of chinadaily 13/17 13/19 14/20 7/11 Android application is 3. average 47.2% 56.1% 75.4% 72.2% The precision in checking inconsistencies between pages TABLE II. THE SIZE AND EXECUTION TIME OF GENERATION is relatively low. Because when our algorithm finds a page without corresponding page, it stops matching all its related Project name Number of Execution Number of pages Execution pages. A page inconsistency usually involves more than one pages Time (Android app） Time (web app） page. The recall of page inconsistency is caused by the oschina 4070 23min 145 50min matching of some meaningless elements in the page, for Zol 12430 107min 116 31min example, logo in the page. chinadaily 1727 23min 104 36min The extracted page and link information is saved in a tree VI. DISCUSSION AND LIMITATION structure. Fig 4(a) shows part of the generated information of During the experiment, we found some important the main interface of Android application, and Fig 4(b) problems that have not yet resolved in this paper. shows part of the generated information of the home page of First, for the application running in the internet, the web application in the project of oschina. content in both web application and mobile application is changing all the time. The changes of content during the process of information extraction bring inconsistency problem. Our approach will generate confusing result if the application is updating during consistency checking. Second, our approach employs a testing tool Robotium[12] to extract page and link information from Android application，as introduced in section 4. In the implementation, our tool supports extraction of information for pre-defined and simple user-defined view types. User-defined types are now widely used in the implementation of Android application, our approach cannot work well in complicated user-defined type.
VII. RELATED WORK Consistency checking is widely used in the field of model-driven software engineering and development. (a) Android app (b ) web app Inconsistencies between multiple views in the same phase or Figure 4. Example of generated information different phases of software lifecycle, may bring follow-on errors and unnecessary rework. Based on the page and link information generated, our Most works [1][2][3] focus on detecting consistency for tool compare the consistency relationship between Android UML models because UML is widely used and the applications with their web applications. Table III shows the consistency of models affects the quality of generated code. inconsistency problem we found for each projects. These works focus on both intra and inter level consistencies. Some works are proposed considering different aspects in TABLE III. NUMBER OF INCONSISTENT PAGES consistency validation. Blanc[4] proposed a meta-model Project name page Element Link inconsistency independent consistency validation method. In his approach, inconsistency inconsistency model is represented as sequences of elementary pages elements pages links oschina 4 139 1680 13 76 construction operations, structural and methodological Zol 33 58 613 15 25 consistency rules can then be expressed uniformly as logical chinadaily 17 58 85 16 36 constraints on such sequences. Egyed’s work [5] focus on an We choose 30 pages from each Android application automated approach for detecting and tracking randomly, and manual validate the consistency relationship inconsistencies in real time. The performance of his method with the web application. The result shows the precision and is not noticeably affected by the model size and common recall of our consistency validation results. The average consistency rules but only by the number of consistency precision of our result is 75.4% and 72.2% respectively for rules. Vierhauser et al.[6] proposed an approach for inconsistency in the element and link level, 47.2% for incremental consistency checking for product lines. It can be inconsistency in the page level. used to check the consistency of variability models and the consistency of the models with the underlying code base. TABLE IV. VALIDATION RESULT To the best of our knowledge, there is not an existing method for consistency checking between a mobile
57 application and its corresponding web application. Most Program for Basic Research(No.JC201105201042A), the works focus on the testing of mobile applications, which Special Foundation for Development of Internet Industry provide basis for our approach. of Shenzhen(No. 20120731105450), the Fundamental Hu et al.[7] present an approach for automating the Research Funds for the Central Universities, Science and testing process for Android applications, with a focus on Technology Planning Project of Guangdong GUI bugs. The approach detects GUI bugs by automatic Province (No.2012B010900089). generation of test cases, feeding the application random events, instrumenting the VM, producing log/trace files and REFERENCES analyzing them post-run. Azim et al. [8] propose two [1] Lucas F J, Molina F, Toval A. A systematic review of UML model strategies: targeted exploration and depth-first exploration, consistency management. Information and Software Technology, for the automatic exploration of the Android applications so 2009, 51(12): 1631-1645. as to improve the coverage at two granularity levels: activity [2] Usman M, Nadeem A, Kim T, et al. A survey of consistency checking techniques for uml models. Advanced Software Engineering and Its (high-level) and method (low-level). Amalfitano et al. [9] Applications, 2008. ASEA 2008.: 57-62. proposed an automated technique that tests Android apps via [3] Egyed A, Letier E, Finkelstein A. Generating and evaluating choices their Graphical User Interface (GUI). Its tool Android for fixing inconsistencies in UML design models. 23rd IEEE/ACM Ripper[10] is based on a user-interface driven ripper that International Conference onAutomated Software Engineering, 2008: automatically explores the application’s GUI with the aim of 99-108. exercising the application in a structured manner. In addition, [4] Blanc X, Mounier I, Mougenot A, et al. Detecting model there are useful testing tools for Android application. inconsistency through operation-based model construction. ACM/IEEE 30th International Conference onSoftware Troyd[11] allows developers to write Ruby scripts that can Engineering,ICSE08, 511-520, 2008. drive the Android application. Robotium[12] is an Android [5] Egyed A. Automatically detecting and tracking inconsistencies in test automation framework that support for native and hybrid software design models. IEEE Transactions on Software Engineering, applications. Our implementation of consistency validation 2011, 37(2): 188-204. tool is built based on Robotium. [6] Vierhauser M, Grünbacher P, Egyed A, et al. Flexible and scalable consistency checking on product line variability models. Proceedings VIII. CONCLUSION of the IEEE/ACM international conference on Automated software engineering. ACM, 2010: 63-72. In this paper, we propose our approach based on the [7] Azim, Tanzirul, and Iulian Neamtiu. Targeted and depth-first inconsistency problem in developing and using mobile exploration for systematic testing of android apps. Proceedings of the application in daily life. This paper proposes an approach for 2013 ACM SIGPLAN international conference on Object oriented automatic checking consistency between web application programming systems languages & applications. ACM, 2013. and its mobile application. Our approach starts from the [8] Amalfitano, Domenico, et al. Using GUI ripping for automated generation of page and link information by crawling content testing of Android applications. Proceedings of the 27th IEEE/ACM from web application and simulating user actions in Android International Conference on Automated Software Engineering. ACM, application. Page matching and consistency checking 2012. [9] Amalfitano, Domenico, et al. A toolset for GUI testing of Android algorithms are proposed and used to find out inconsistency in applications. Software Maintenance (ICSM), 2012 28th IEEE page, element and link. International Conference on. IEEE, 2012. Because the number of inconsistency found in our [10] Hu, Cuixiong, and Iulian Neamtiu. Automating GUI testing for experiment is too large for comprehension. In the future Android applications.Proceedings of the 6th International Workshop work, we will try to define inconsistency pattern based on on Automation of Software Test. ACM, 2011. mining the inconsistency data. Repetitive inconsistency [11] Hu C, Neamtiu I. Automating GUI testing for Android applications. simplified as a pattern will improve the usability of our Proceedings of the 6th International Workshop on Automation of , approach for real software development. Software Test 2011: 77-83. [12] Google Code. Robotium. January, 2014. ACKNOWLEDGMENT URL http://code.google.com/p/robotium/. [13] Jinseong Jeon and Jeffrey S. Foster. Troyd. January, This effort is supported by National Natural Science 2012.URL https://github.com/plum-umd/troyd. Foundation of China(No. 61100002), the NSFC Guangdong [14] Android GUI Ripper Joint Fund (No. U1201252), Shenzhen Technology R&D Wiki.URL http://wpage.unina.it/ptramont/GUIRipperWiki.htm
58