IRC Channel Data Analysis Using Apache Solr Nikhil Reddy Boreddy Purdue University
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
DETECTING BOTS in INTERNET CHAT by SRITI KUMAR Under The
DETECTING BOTS IN INTERNET CHAT by SRITI KUMAR (Under the Direction of Kang Li) ABSTRACT Internet chat is a real-time communication tool that allows on-line users to communicate via text in virtual spaces, called chat rooms or channels. The abuse of Internet chat by bots also known as chat bots/chatterbots poses a serious threat to the users and quality of service. Chat bots target popular chat networks to distribute spam and malware. We first collect data from a large commercial chat network and then conduct a series of analysis. While analyzing the data, different patterns were detected which represented different bot behaviors. Based on the analysis on the dataset, we proposed a classification system with three main components (1) content- based classifiers (2) machine learning classifier (3) communicator. All three components of the system complement each other in detecting bots. Evaluation of the system has shown some measured success in detecting bots in both log-based dataset and in live chat rooms. INDEX WORDS: Yahoo! Chat room, Chat Bots, ChatterBots, SPAM, YMSG DETECTING BOTS IN INTERNET CHAT by SRITI KUMAR B.E., Visveswariah Technological University, India, 2006 A Thesis Submitted to the Graduate Faculty of The University of Georgia in Partial Fulfillment of the Requirements for the Degree MASTER OF SCIENCE ATHENS, GEORGIA 2010 © 2010 Sriti Kumar All Rights Reserved DETECTING BOTS IN INTERNET CHAT by SRITI KUMAR Major Professor: Kang Li Committee: Lakshmish Ramaxwamy Prashant Doshi Electronic Version Approved: Maureen Grasso Dean of the Graduate School The University of Georgia December 2010 DEDICATION I would like to dedicate my work to my mother to be patient with me, my father for never questioning me, my brother for his constant guidance and above all for their unconditional love. -
Application Log Analysis
Masarykova univerzita Fakulta}w¡¢£¤¥¦§¨ informatiky !"#$%&'()+,-./012345<yA| Application Log Analysis Master’s thesis Júlia Murínová Brno, 2015 Declaration Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Júlia Murínová Advisor: doc. RNDr. Vlastislav Dohnal, Ph.D. iii Acknowledgement I would like to express my gratitude to doc. RNDr. Vlastislav Dohnal, Ph.D. for his guidance and help during work on this thesis. Furthermore I would like to thank my parents, friends and family for their continuous support. My thanks also belongs to my boyfriend for all his assistance and help. v Abstract The goal of this thesis is to introduce the log analysis area in general, compare available systems for web log analysis, choose an appropriate solution for sample data and implement the proposed solution. Thesis contains overview of monitoring and log analysis, specifics of application log analysis and log file formats definitions. Various available systems for log analysis both proprietary and open-source are compared and categorized with overview comparison tables of supported functionality. Based on the comparison and requirements analysis appropriate solution for sample data is chosen. The ELK stack (Elasticsearch, Logstash and Kibana) and ElastAlert framework are deployed and configured for analysis of sample application log data. Logstash configuration is adjusted for collecting, parsing and processing sample data input supporting reading from file as well as online socket logs collection. Additional information for anomaly detection is computed and added to log records in Logstash processing. -
Analysis of Web Logs and Web User in Web Mining
International Journal of Network Security & Its Applications (IJNSA), Vol.3, No.1, January 2011 ANALYSIS OF WEB LOGS AND WEB USER IN WEB MINING L.K. Joshila Grace 1, V.Maheswari 2, Dhinaharan Nagamalai 3, 1Research Scholar, Department of Computer Science and Engineering [email protected] 2 Professor and Head,Department of Computer Applications 1,2 Sathyabama University,Chennai,India 3Wireilla Net Solutions PTY Ltd, Australia ABSTRACT Log files contain information about User Name, IP Address, Time Stamp, Access Request, number of Bytes Transferred, Result Status, URL that Referred and User Agent. The log files are maintained by the web servers. By analysing these log files gives a neat idea about the user. This paper gives a detailed discussion about these log files, their formats, their creation, access procedures, their uses, various algorithms used and the additional parameters that can be used in the log files which in turn gives way to an effective mining. It also provides the idea of creating an extended log file and learning the user behaviour. KEYWORDS Web Log file, Web usage mining, Web servers, Log data, Log Level directive. 1. INTRODUCTION Log files are files that list the actions that have been occurred. These log files reside in the web server. Computers that deliver the web pages are called as web servers. The Web server stores all of the files necessary to display the Web pages on the users computer. All the individual web pages combines together to form the completeness of a Web site. Images/graphic files and any scripts that make dynamic elements of the site function. -
Universidad Pol Facultad D Trabajo
UNIVERSIDAD POLITÉCNICA DE MADRID FACULTAD DE INFORMÁTICA TRABAJO FINAL DE CARRERA ESTUDIO DEL PROTOCOLO XMPP DE MESAJERÍA ISTATÁEA, DE SUS ATECEDETES, Y DE SUS APLICACIOES CIVILES Y MILITARES Autor: José Carlos Díaz García Tutor: Rafael Martínez Olalla Madrid, Septiembre de 2008 2 A mis padres, Francisco y Pilar, que me empujaron siempre a terminar esta licenciatura y que tanto me han enseñado sobre la vida A mis abuelos (q.e.p.d.) A mi hijo icolás, que me ha dejado terminar este trabajo a pesar de robarle su tiempo de juego conmigo Y muy en especial, a Susana, mi fiel y leal compañera, y la luz que ilumina mi camino Agradecimientos En primer lugar, me gustaría agradecer a toda mi familia la comprensión y confianza que me han dado, una vez más, para poder concluir definitivamente esta etapa de mi vida. Sin su apoyo, no lo hubiera hecho. En segundo lugar, quiero agradecer a mis amigos Rafa y Carmen, su interés e insistencia para que llegara este momento. Por sus consejos y por su amistad, les debo mi gratitud. Por otra parte, quiero agradecer a mis compañeros asesores militares de Nextel Engineering sus explicaciones y sabios consejos, que sin duda han sido muy oportunos para escribir el capítulo cuarto de este trabajo. Del mismo modo, agradecer a Pepe Hevia, arquitecto de software de Alhambra Eidos, los buenos ratos compartidos alrrededor de nuestros viejos proyectos sobre XMPP y que encendieron prodigiosamente la mecha de este proyecto. A Jaime y a Bernardo, del Ministerio de Defensa, por haberme hecho descubrir las bondades de XMPP. -
Abstract Introduction Methodology
Kajetan Hinner (2000): Statistics of major IRC networks: methods and summary of user count. M/C: A Journal of Media and Culture 3(4). <originally: http://www.api-network.com/mc/0008/count.html> now: http://www.media-culture.org.au/0008/count.html - Actual figures and updates: www.hinner.com/ircstat/ Abstract The article explains a successful approach to monitor the major worldwide Internet Relay Chat (IRC) networks. It introduces a new research tool capable of producing detailed and accurate statistics of IRC network’s user count. Several obsolete methods are discussed before the still ongoing Socip.perl program is explained. Finally some IRC statistics are described, like development of user figures, their maximum count, IRC channel figures, etc. Introduction Internet Relay Chat (IRC) is a text-based service, where people can meet online and chat. All chat is organized in channels which a specific topic, like #usa or #linux. A user can be taking part in several channels when connected to an IRC network. For a long time the only IRC network has been EFnet (Eris-Free Network, named after its server eris.berkeley.edu), founded in 1990. The other three major IRC networks are Undernet (1993), DALnet (1994) and IRCnet, which split off EFnet in June 1996. All persons connecting to an IRC network at one time create that IRC network’s user space. People are constantly signing on and off, the total number of users ever been to a specific IRC network could be called social space of that IRC network. It is obvious, that the IRC network’s social space by far outnumbers its user space. -
Attacker Chatbots for Randomised and Interactive Security Labs, Using Secgen and Ovirt
Hackerbot: Attacker Chatbots for Randomised and Interactive Security Labs, Using SecGen and oVirt Z. Cliffe Schreuders, Thomas Shaw, Aimée Mac Muireadhaigh, Paul Staniforth, Leeds Beckett University Abstract challenges, rewarding correct solutions with flags. We deployed an oVirt infrastructure to host the VMs, and Capture the flag (CTF) has been applied with success in leveraged the SecGen framework [6] to generate lab cybersecurity education, and works particularly well sheets, provision VMs, and provide randomisation when learning offensive techniques. However, between students. defensive security and incident response do not always naturally fit the existing approaches to CTF. We present 2. Related Literature Hackerbot, a unique approach for teaching computer Capture the flag (CTF) is a type of cyber security game security: students interact with a malicious attacker which involves collecting flags by solving security chatbot, who challenges them to complete a variety of challenges. CTF events give professionals, students, security tasks, including defensive and investigatory and enthusiasts an opportunity to test their security challenges. Challenges are randomised using SecGen, skills in competition. CTFs emerged out of the and deployed onto an oVirt infrastructure. DEFCON hacker conference [7] and remain common Evaluation data included system performance, mixed activities at cybersecurity conferences and online [8]. methods questionnaires (including the Instructional Some events target students with the goal of Materials Motivation Survey (IMMS) and the System encouraging interest in the field: for example, PicoCTF Usability Scale (SUS)), and group interviews/focus is an annual high school competition [9], and CSAW groups. Results were encouraging, finding the approach CTF is an annual competition for students in Higher convenient, engaging, fun, and interactive; while Education (HE) [10]. -
Users As Co-Designers of Software-Based Media: the Co-Construction of Internet Relay Chat
Users as Co-Designers of Software-Based Media: The Co-Construction of Internet Relay Chat Guillaume Latzko-Toth Université Laval AbsTrAcT While it has become commonplace to present users as co-creators or “produsers” of digital media, their participation is generally considered in terms of content production. The case of Internet Relay Chat (IRC) shows that users can be fully involved in the design process, a co-construction in the sense of Science and Technology Studies (STS): a collective, simultaneous, and mutual construction of actors and artifacts. A case study of the early de - velopment of two IRC networks sheds light on that process and shows that “ordinary users” managed to invite themselves as co-designers of the socio-technical device. The article con - cludes by suggesting that IRC openness to user agency is not an intrinsic property of software- based media and has more to do with its architecture and governance structure. Keywords Digital media; Communication technology; Co-construction; Design process; Ordinary user résumé Il est devenu banal de présenter l’usager comme cocréateur ou « produtilisateur » des médias numériques, mais sa participation est généralement envisagée comme une production de contenus. Le cas d’IRC (Internet Relay Chat) montre que les usagers des médias à support logiciel peuvent s’engager pleinement dans le processus de conception, une co-construction au sens des Science and Technology Studies : une construction collective, simultanée et mutuelle des acteurs et des artefacts. Une étude de cas portant sur le développement de deux réseaux IRC éclaire ce processus et montre que les « usagers ordinaires » sont parvenus à s’inviter comme co-concepteurs du dispositif. -
Tomenet-Guide.Pdf
.==========================================================================+−−. | TomeNET Guide | +==========================================================================+− | Latest update: 17. September 2021 − written by C. Blue ([email protected]) | | for TomeNET version v4.7.4b − official websites are: : | https://www.tomenet.eu/ (official main site, formerly www.tomenet.net) | https://muuttuja.org/tomenet/ (Mikael’s TomeNET site) | Runes & Runemastery sections by Kurzel ([email protected]) | | You should always keep this guide up to date: Either go to www.tomenet.eu | to obtain the latest copy or simply run the TomeNET−Updater.exe in your | TomeNET installation folder (desktop shortcut should also be available) | to update it. | | If your text editor cannot display the guide properly (needs fixed−width | font like for example Courier), simply open it in any web browser instead. +−−− | Welcome to this guide! | Although I’m trying, I give no guarantee that this guide | a) contains really every detail/issue about TomeNET and | b) is all the time 100% accurate on every occasion. | Don’t blame me if something differs or is missing; it shouldn’t though. | | If you have any suggestions about the guide or the game, please use the | /rfe command in the game or write to the official forum on www.tomenet.eu. : \ Contents −−−−−−−− (0) Quickstart (If you don’t like to read much :) (0.1) Start & play, character validation, character timeout (0.1a) Colours and colour blindness (0.1b) Photosensitivity / Epilepsy issues (0.2) Command reference -
System Log Files Kernel Ring Buffer Viewing Log Files the Log Files
System Log Files Most log files are found in /var/log Checking logs are critical to see if things are working correctly Checking logs is critical to see if things are working correctly. Take a look at all the log files on scratch ls /var/log Kernel Ring Buffer The kernel ring buffer is something like a log file for the kernel; however, unlike other log files, it’s stored in memory rather than in a disk file. You can use the dmesg command to view it. Many times it is logged to /var/log/dmesg as well. It requires sudo privileges to read the /var/log/dmesg file, but not to run the dmesg command. Viewing log files There are a number of commands to view log files. cat less head tail Anytime a new entry is added to a log file, it is appended to the end of the file. This is one of those times where tail is particularly useful. Usually when we want to look at log files we want to look at the most recent entries. When organizing our viewing command - order matters. Most of the following commands produce different results. And all are useful depending on what type of results you want. Go through the thought process and figure out what each command does. Can you figure out which three produce identical results? cat /var/log/syslog cat /var/log/syslog | grep daemon cat /var/log/syslog | grep daemon | tail -n 10 cat /var/log/syslog | tail -n 10 cat /var/log/syslog | tail -n 10 | grep daemon less /var/log/syslog less /var/log/syslog | tail -n 10 | grep daemon head -n 10 /var/log/syslog head -n 10 /var/log/syslog | grep daemon tail -n 10 /var/log/syslog tail -n 10 /var/log/syslog | grep daemon If you add the -f option to the tail command it provides a live watch of the log file. -
Multi-Phase IRC Botnet and Botnet Behavior Detection Model
International Journal of Computer Applications (0975 – 8887) Volume 66– No.15, March 2013 Multi-phase IRC Botnet and Botnet Behavior Detection Model Aymen Hasan Rashid Al Awadi Bahari Belaton Information Technology Research Development School of Computer Sciences Universiti Sains Center, University of Kufa, Najaf, Iraq Malaysia 11800 USM, Penang, Malaysia School of Computer Sciences Universiti Sains Malaysia 11800 USM, Penang, Malaysia ABSTRACT schools, banks and any of governmental institutes making use of system vulnerabilities and software bugs to separate and Botnets are considered one of the most dangerous and serious execute a lot of malicious activities. Recently, bots can be the security threats facing the networks and the Internet. major one of the major sources for distributing or performing Comparing with the other security threats, botnet members many kinds of scanning related attacks (Distributed Denial-of- have the ability to be directed and controlled via C&C Service DoS) [1], spamming [2], click fraud [3], identity messages from the botmaster over common protocols such as fraud, sniffing traffic and key logging [4] etc. The nature of IRC and HTTP, or even over covert and unknown the bots activities is to respond to the botmaster's control applications. As for IRC botnets, general security instances command simultaneously. This responding will enable the like firewalls and IDSes do not provide by themselves a viable botmaster to get the full benefit from the infected hosts to solution to prevent them completely. These devices could not attack another target like in DDoS [5]. From what stated differentiate well between the legitimate and malicious traffic earlier, the botnet can be defined as a group of connected of the IRC protocol. -
IRC:N Käyttö Yritysten Viestinnässä
Maarit Klami IRC:n käyttö yritysten viestinnässä Metropolia Ammattikorkeakoulu Insinööri (AMK) Tietotekniikan koulutusohjelma Insinöörityö 3.5.2012 Tiivistelmä Tekijä Maarit Klami Otsikko IRC:n käyttö yritysten viestinnässä Sivumäärä 34 sivua + 2 liitettä Aika 3.5.2012 Tutkinto insinööri (AMK) Koulutusohjelma tietotekniikka Suuntautumisvaihtoehto ohjelmistotekniikka Ohjaaja lehtori Peter Hjort Tässä työssä perehdyttiin IRC:n käyttömahdollisuuksiin yritysten sisäisessä ja ulkoisessa viestinnässä. Työssä verrattiin IRC:tä muihin pikaviestimiin sekä sosiaalisen median tarjoamiin viestintä- ja markkinointimahdollisuuksiin. Eräs tarkastelluista seikoista on yrityksen koon vaikutus siihen, kuinka mittava operaatio IRC:n käyttöönotto yrityksessä on. Työssä on esitetty myös mahdollisuuksia muiden pikaviestimien integroimiseksi IRC:hen. Työ on tehty mielenkiinnosta käyttää IRC:tä viestinnän apuna yrityksissä. Tietoturva on keskeisessä roolissa yritysten toiminnassa, josta johtuen asia on huomioitava myös viestintäjärjestelmien käytöönotossa ja päivittäisessä käytössä. Työssä tietoturvaa on käsitelty sekä tekniseltä että inhimilliseltä kannalta. Työssä yhtenä keskeisenä osana on tutustuttu IRC-bottien hyödyntämiseen yritysten toiminnassa. Osana työtä tehtiin Eggdropilla IRC-botti, jolle perusominaisuuksien laajentamiseksi ohjelmoitiin TCL-ohjelmointikieltä käyttäen halutun toiminnalisuuden toteuttava skripti. Työssä esitellään botin käyttöönoton vaatimat toimenpiteet sekä skriptin totetuttaminen. Lisäksi työssä on esitelty erilaisia bottien toiminnallisuuksia -
Introduction to Online Sexual Exploitation Curriculum 1 Safe Online Outreach Project Learning Objectives
Introduction To Online Sexual Exploitation: Curriculum February 2003 Safe OnLine Outreach Project © M. Horton 2003 Safe OnLine Outreach Project Acknowledgements This document is the result of many hours of hard work and dedication. I'd like to thank Renata Karrys, Jaynne Aster, Nikki O'Halloran, Charlaine Avery, Lisa Ingvallsen and Elizabeth Nethery for their support and assistance in producing this document. Additionally the SOLO Advisory Committee, the Canadian National Crime Prevention - Community Mobilization Program, the Vancouver Foundation, Athabasca University/MediaCan and Parents Against Sexual Abuse have all been instrumental in turning this idea into a Project. Lastly, David and Conor have been silent contributors to this project since it began. Their support is woven into each page. Merlyn Horton SOLO Project Coordinator February 2003 Safe OnLine Outreach Project February 3, 2003 Introduction to the Curriculum Dear Reader, The curriculum you hold in your hands was produced in a former pottery studio in the middle of a coastal rain forest in British Columbia, Canada; the physical launch pad for this examination of a virtual issue. It is the result of three years of research. This introduction is intended to outline a context for the curriculum and to give you an overview of how to use this curriculum, who should present this curriculum and how appropriate audiences might be chosen. Context The philosophical foundation for this curriculum, and indeed for the Safe OnLine Outreach Project, is the United Nations Convention on the Rights of the Child (CRC)1. The CRC is one of four Conventions created by the United Nations to further the goal of recognizing the inherent dignity and rights of all members of the human family2.