World Applied Sciences Journal 35 (7): 1083-1089, 2017 ISSN 1818-4952 © IDOSI Publications, 2017 DOI: 10.5829/idosi.wasj.2017.1083.1089

The Development of Visual Analyzer For Analysing Proxy Logs Files

Fauziah Ab.Wahab, Mokhairi Makhtar, Syadiah Nor Wan Shamsudin, Mohamad Afendee Mohamed, Nurul Fasihah Che Azmi

Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Malaysia

Abstract: Proxy Log Visual Analyzer (PLVA) is one of the important systems for network administrator which concern about the information from the proxy server log file. Proxy log files are generated from proxy servers to record operations that occur in the computer networks. Proxy log analyzer applications that have been proposed earlier exhibit limitation, which is no interactive analysis and just analyzing general IP without knowing the owner of the IP. Therefore this project is to implement interactive monitoring and analysis of log files from proxy server. In this paper, the design and implementation of the Proxy Log Visual Analyzer (PLVA) was tested with simulated data. The prototype was developed using PHP as a programming language. The prototype is able to analyze the log files from proxy server by categorizing the files accordingly. Thus, the system has potential to support a better understanding of events occurred at proxy server and the monitoring of internet activities in an organization will be more efficient.

Key words: Network monitoring Proxy server Log files Network management Packet analysis

INTRODUCTION be stored in the proxy server log file based on their own standard format. Normally, an admin has to open Network monitoring is the information collection the particular log file each time to check the information. function of network management [1, 2]. The purpose of The process may take times since the admin has to network monitoring is collecting of useful information understand what structure of the format to get the log from various parts of the network so that the network can file’s information. The objective of Proxy Log Visual be managed and controlled using collected information. Analyzer (PLVA) is to display log files from the proxy Internet traffic data can be collected from various sources server into a simple format for easier understanding by the such as routers, gateway or proxy server [3]. The proxy admin. The standard format of the log file is hard to server acts as an intermediate server that relays requests understand especially by new employee work as an admin between a client and a server [4]. It acts as a middleman responsible for analyzing data. between the two ends of the client/server network Log file analysis is involved heavily in both Software connection and also works with browser servers or other development and maintenance phases [7]. Logs is a application in HTTP protocol. An administrator can keep message or record that are stored in a file. Usually, this a record of user’s Internet activity and help in resource record represents a running application process. A log file provisioning and monitoring based on the log file. is a file that contains a list of events, which have been A log file can be located in three different places logged by a computer. Log files are often generated which are Web Servers, Web proxy Servers and Client during software installation and are created by Web browsers [5, 6]. In this paper, log files are retrieved from servers, but they can be used for many other purposes as Web proxy Server. Current logs at the proxy server are in well. Most log files are saved in a plain text format, which the text format. There is a problem to view log file minimizes their file size and allows them to be viewed in a basically and the user may not be able to understand the basic text editor. In Linux, there are three log files available structure of the log file and hence they may obtain wrong which are access.log, cache.log and store.log. Each log information from it. Normally, the information like IP has their own content that might help admin debugging address, date and time user accessed to the website will the potential problem. Web servers use log files to record

Corresponding Author: Fauziah Ab.Wahab, Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Besut Campus, 22200, Terengganu, Malaysia. 1083 World Appl. Sci. J., 35 (7): 1083-1089, 2017

Fig. 1: Example of log file

Fig. 2: Proxy Log Explorer analysis. data about website visitor. This information typically There are many existing systems that have similar includes the IP address of each visitor, the time of the function, to do log analysis. The application that has visit and the pages visited. Data in the log file can be almost similar properties and serves the same purpose used for analysis. For example in data forensic, the are reviewed. There are available for analyzing log files investigator can use this analysis to discover evidence. and generate a report in terms of users, IP numbers, site The log file may also keep track of what resources were visited and so on. Figure 1 shows an example of a log file. loaded during each visit, such as images, JavaScript or Literature reviews were done by reviewing existing CSS files [8]. software on the internet because of the lack of reading

1084 World Appl. Sci. J., 35 (7): 1083-1089, 2017

Fig. 3: Example output of SARG (Squid Analysis Report Generator) material in the journal. Below are some of the existing individual machines on the network and it can watch on software that has been reviewed. which websites the network’s users are accessing. SARG is efficient with the task such as creating reports based on Proxy Log Explorer: Proxy Log Explorer is an squid logs. Squid log files are used to generate a report in application used for monitoring the efficiency of HTML. Figure 3 shows example output of SARG. corporate internet usage of Proxy server and as a log Languages available are more than ten such as English, analyzer software that processes raw proxy log files. Catalan, Dutch, German, Hungarian, Indonesian, Italian, The program generates statistic from log files and Japanese, Polish, Turkish, Spanish, etc. Its programming supports more than five proxy log file format such as language is written in C. Squid log file format, Netscape Proxy log file format, Proxy+ log file format, SuperLumin Networks nemesis, Calamaris: Calamaris is a tool for analyzing squid’s Microsoft Internet Security and Acceleration Server log access.log and produces report of cache activity in ASCII file format, CCProxy v2010 log file format and other W3C or HTML format. Calamaris is a Perl script which Extended format. Figure 2 shows the Proxy Log Explorer generates nice statistics out of Squid or Oops log files. It analysis. is invoked daily before the proxy rotates its log files and mails the statistics or puts them on the web. Calamaris SARG (Squid Analysis Report Generator): SARG (Squid parse the log files of a wide variety of Web Proxy Server Analysis Report Generator) is an open source tool that such as Squid, NetCache, Inktomi Traffic Server, Proxy allows user to analyses the squid log files and generates server, Novell Internet Caching System, Compaq beautiful reports in HTML format with information about Tasksmart or Netscape/iPlanet Web Proxy Server [10]. users, IP addresses, top accessed sites, total bandwidth Calamaris generates reports about peak-usage, request- usage, elapsed time, downloads, access denied websites, methods, status-report of incoming and outgoing daily reports, weekly reports and monthly reports [9]. requests, second and top-level destinations, content- SARG is exactly the kind of tool that is very handy tool types and performance [11]. Figure 4 shows an example of for viewing how much internet bandwidth is utilized by Calamaris analyzer of TCP statistic.

1085 World Appl. Sci. J., 35 (7): 1083-1089, 2017

Fig. 4: Example output of Calamaris

MATERIALS AND METHODS Framework: A framework is a logical structure intended to provide a comprehensive representation This section explained the details of the methodology of an information technology enterprise that is used in this project. The methodology is defined as the independent of the tools that used in this project. analysis of the principles or procedure of inquiring in a It displays overall structure of data available in the particular field. Project methodology is being used to organization, regardless of the physical systematically solve all the problems arising in the system technology involved. Figure 6 shows a framework analysis to ensure this project complete and achieve its for PLVA. In summary, the log file generated by the objectives. It is important to decide which method is proxy server will be stored in the database. suitable for the project. There are a lot of software To analyze specific user, admin should register development methodologies exist which are waterfall user first and all information stored in the model, evolutionary, spiral and so on. However, a database. Through information in the database, prototyping-based methodology known as Rapid admin can view overall statistical category and Application Development (RAD) cycle has been chosen also choose to view Top 5 URL based on specific for this project due to the advantages obtained by other categories. approaches. The prototype is a concrete representation of part or all of an interactive system. It is a tangible artifact, The features of the prototype are: not an abstract description that requires interpretation This application is used for an organization that own [12]. Figure 5 shows a framework of a prototype model static IP that used for the development. There are four phases This project allows admin to know the specific user involved in the development of prototype system and their access. planning phase, analysis phases, design phases and This analyzer will generate an overall statistic of implementation phases. application, image, video and others.

1086 World Appl. Sci. J., 35 (7): 1083-1089, 2017

Fig. 5: RAD cycles - Prototype methodology

Fig. 6: Framework of PLVA

RESULTS AND DISCUSSIONS ways to handle this is by developing a user-friendly analyzer for easier access and reading log files and In this section, the functions and the designs of present it in graphical format. It becomes necessity for PLVA are discussed. Proxy Log Visual Analyzer (PLVA) administrator to have an effective Proxy Log Visual uses log files to record data about website visitors such Analyzer application to monitor network activity simply as timestamp, IP address, URL, usage statistic and or records any traffic logs with user-friendly efficiently. analysis and being able to print a report. Most of log files This analyzer allows admin to analyze what kind of are saved in a plain text format and allowed to be viewed information is being accessed based on log file from proxy in a basic text editor. The problems occur when the admin server. Figure 7 shows an interface of an overall statistics user have to view formal log file each time when admin of application, image, video and others. Figure 8 show needs to obtain particular information. One of the best results top five applications in specific category.

1087 World Appl. Sci. J., 35 (7): 1083-1089, 2017

Fig. 7: Statistic of Overall Log File

Fig. 8: Results Top five applications in Specific Category

CONCLUSION server and also review about the architecture of the existing system. As a conclusion, this prototype will help network administrators to monitor and analyze network traffic REFERENCES effectively. This system also helps network forensic investigators to identify internet abuse among workers 1. Khan, R., S.U. Khan, R. Zaheer and M.I. Babar, 2013. more easily as the system is capable of analyzing based An Efficient Network Monitoring and Management on the specific user. Throughout this paper, a lot of System. International Journal of Information and discussions have been made that describe log files, proxy Electronics Engineering, 3(1): 122-126.

1088 World Appl. Sci. J., 35 (7): 1083-1089, 2017

2. Verma, A., B. Kishan and R. Jain, 2012. Network 8. Wenwu, L., L. Guimei, L. Hongjun and Y. Qiang, 2002. Monitoring, Management and Enhancement Using Cut-and-Pick Transactions for Proxy Log Mining. VPN, 1(5): 391-394. Hong Kong University of Science and Technology: 3. Kamath, K.M., H.S. Bassali, R.B. Hosamani and Department of Computer Science. L. Gao, 2001. Policy-aware algorithms for proxy 9. Ravi, S., 2014. SARG-Squid Analysis Report placement in the Internet. Proc. SPIE 4526, Scalability Generator and Internet Bandwidth Monitoring Tools. and Traffic Control in IP Networks, 157(July 25, 2001). Available at https://www.tecmint.com/sarg-squid- doi:10.1117/12.434392 analysis-report-generator-and-internet-bandwidth- 4. Oracle Corporation. 2011. Firewall and Proxy Server monitoring-tool/ Support. In Siebel Security Guide (pp: 50). United 10. Beermann, C., 2013. Calamaris Home Page. Retrieved States: Oracle. from Cord Virtual Home: http://cord.de/calamaris- 5. Krishnamoorthi, K.S., 2009. Identifying User Behavior home-page by Analyzing Web Server Access Log Files. 11. Crovella, M., 2000. Performance Characteristics of the International Journal of Computer Science and World Wide Web, In: G. Haring, C. Lindemann, M. Network Security, 9(4): 327-332. Reiser (Eds.) Performance Evaluation: Origins and 6. Jeffery, C.L., S.R. Das and G.S. Bernal, 1996. Proxy- Directions, LNCS. 1769:219-232, Springer. Sharing Proxy Server. IEEE Proceeding of First 12. Michel, B.L., and W.E. Mackay, 2010. Prototyping Annual Conference on Emerging Technologies and Tools and Techniques. INRIA: Université Paris-Sud. Applications in Communications, pg 116-119. doi: Access at https:// www.researchgate.net/ 10.1109/ETACOM.1996.502490 file.PostFileLoader.html?id=587740bbb0366d811c07 7. Jayathilake, D., 2011. A Mind Map Based Framework 92b3&assetKey=AS%3A449625007104002%401484 for Automated Software Log File Analysis. 210363312 International Conference on Software and Computer Applications 9: 1-6.

1089