Unix Computer Forensics
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Faster File Matching Using Gpgpus
Faster File Matching Using GPGPUs Deephan Mohan and John Cavazos Department of Computer and Information Sciences, University of Delaware Abstract We address the problem of file matching by modifying the MD6 algorithm that is best suited to take advantage of GPU computing. MD6 is a cryptographic hash function that is tree-based and highly parallelizable. When the message M is available initially, the hashing operations can be initiated at different starting points within the message and their results can be aggregated as the final step. In the parallel implementation, the MD6 program was partitioned and effectively parallelized across the GPU using CUDA. To demonstrate the performance of the CUDA version of MD6, we performed various experiments with inputs of different MD6 buffer sizes and varying file sizes. CUDA MD6 achieves real time speedup of more than 250X over the sequential version when executed on larger files. CUDA MD6 is a fast and effective solution for identifying similar files. Keywords: CUDA, MD6, GPU computing, File matching 1. Introduction compression level with the root being the final hash computed by the algorithm. The structure of a Merkle tree is shown in File matching is an important task in the field of forensics and Figure 1. In the MD6 implementation, the MD6 buffer size information security. Every file matching application is driven determines the number of levels in the tree and the bounds for by employing a particular hash generating algorithm. The crux parallelism. of the file matching application relies on the robustness and integrity of the hash generating algorithm. Various checksum generation algorithms like MD5[1], SHA-1[2], SHA-256[3], Tiger[4], Whirlpool[5], rolling hash have been utilized for file matching [6, 7]. -
Ocaml Standard Library Ocaml V
OCaml Standard Library OCaml v. 3.12.0 | June 7, 2011 | Copyright c 2011 OCamlPro SAS | http://www.ocamlpro.com/ Standard Modules module List module Array Basic Data Types let len = List.length l let t = Array.create len v let t = Array.init len (fun pos -> v_at_pos) Pervasives All basic functions List.iter (fun ele -> ... ) l; let v = t.(pos) String Functions on Strings let l' = List.map(fun ele -> ... ) l t.(pos) <- v; Array Functions on Polymorphic Arrays let l' = List.rev l1 let len = Array.length t List Functions on Polymorphic Lists let acc' = List.fold left (fun acc ele -> ...) acc l let t' = Array.sub t pos len Char Functions on Characters let acc' = List.fold right (fun ele acc -> ...) l acc let t = Array.of_list list Int32 Functions on 32 bits Integers if List.mem ele l then ... let list = Array.to_list t Int64 Functions on 64 bits Integers if List.for all (fun ele -> ele >= 0) l then ... Array.iter (fun v -> ... ) t; Nativeint Functions on Native Integers if List.exists (fun ele -> ele < 0) l then ... let neg = List.find (fun x -> x < 0) ints Array.iteri (fun pos v -> ... ) t; Advanced Data Types let negs = List.find all (fun x -> x < 0) ints let t' = Array.map (fun v -> ... ) t let t' = Array.mapi (fun pos v -> ... ) t Buffer Automatically resizable strings let (negs,pos) = List.partition (fun x -> x < 0) ints let ele = List.nth 2 list let concat = Array.append prefix suffix Complex Complex Numbers Array.sort compare t; Digest MD5 Checksums let head = List.hd list Hashtbl Polymorphic Hash Tables let tail = List.tl list let value = List.assoc key assocs Queue Polymorphic FIFO module Char Stack Polymorphic LIFO if List.mem assoc key assocs then .. -
Software Developement Tools Introduction to Software Building
Software Developement Tools Introduction to software building SED & friends Outline 1. an example 2. what is software building? 3. tools 4. about CMake SED & friends – Introduction to software building 2 1 an example SED & friends – Introduction to software building 3 - non-portability: works only on a Unix systems, with mpicc shortcut and MPI libraries and headers installed in standard directories - every build, we compile all files - 0 level: hit the following line: mpicc -Wall -o heat par heat par.c heat.c mat utils.c -lm - 0.1 level: write a script (bash, csh, Zsh, ...) • drawbacks: Example • we want to build the parallel program solving heat equation: SED & friends – Introduction to software building 4 - non-portability: works only on a Unix systems, with mpicc shortcut and MPI libraries and headers installed in standard directories - every build, we compile all files - 0.1 level: write a script (bash, csh, Zsh, ...) • drawbacks: Example • we want to build the parallel program solving heat equation: - 0 level: hit the following line: mpicc -Wall -o heat par heat par.c heat.c mat utils.c -lm SED & friends – Introduction to software building 4 - non-portability: works only on a Unix systems, with mpicc shortcut and MPI libraries and headers installed in standard directories - every build, we compile all files • drawbacks: Example • we want to build the parallel program solving heat equation: - 0 level: hit the following line: mpicc -Wall -o heat par heat par.c heat.c mat utils.c -lm - 0.1 level: write a script (bash, csh, Zsh, ...) SED -
2017 W5.2 Fixity Integrity
FIXITY & DATA INTEGRITY DATA INTEGRITY DATA INTEGRITY PRESERVATION CONSIDERATIONS ▸ Data that can be rendered ▸ Data that is properly formed and can be validated ▸ DROID, JHOVE, etc. DATA DEGRADATION HOW DO FILES LOSE INTEGRITY? DATA DEGRADATION HOW DO FILES LOSE INTEGRITY? Storage: hardware issues ▸ Physical damage, improper orientation, magnets, dust particles, mold, disasters Storage: software issues ▸ "bit rot", "flipped" bits, small electronic charge, solar flares, radiation DATA DEGRADATION HOW DO FILES LOSE INTEGRITY? Transfer/Retrieval ‣ Transfer from one operating system or file system to another, transfer across network protocols, ▸ Metadata loss: example – Linux has no "Creation Date" (usually "file system" metadata) Mismanagement ▸ Permissions issues (read/write allowed), human error DATA PROTECTION VERIFICATION DATA PROTECTION VERIFICATION ▸ Material proof or evidence that data is unchanged ▸ Material proof or evidence that data is well-formed and should be renderable ▸ Example: Different vendors write code for standard formats in different ways DATA PROTECTION VERIFICATION Verify that data is well-formed using... DATA PROTECTION VERIFICATION Verify that data is well-formed using... ▸ JHOVE ▸ DROID ▸ XML Validator ▸ DVAnalyzer ▸ NARA File Analyzer ▸ BWF MetaEdit WHOLE-FILE CONSISTENCY FIXITY FIXITY BASIC METHODS Manual checks of file metadata such as... FIXITY BASIC METHODS Manual checks of file metadata such as... ▸ File name ▸ File size ▸ Creation date ▸ Modified date ▸ Duration (time-based media) FIXITY ADVANCED METHODS FIXITY -
Security 1 Lab
Security 1 Lab Installing Command-Line Hash Generators and Comparing Hashes In this project, you download different command-line hash generators to compare hash values. 1. Use your Web browser to go to https://kevincurran.org/com320/labs/md5deep.zip 2. Download this zip archive. 3. Using Windows Explorer, navigate to the location of the downloaded file. Right-click the file and then click Extract All to extract the files. 4. Create a Microsoft Word document with the line below: Now is the time for all good men to come to the aid of their country. 5. Save the document as Country1.docx in the directory containing files and then close the document. 6. Start a command prompt by clicking Start, entering cmd, and then pressing Enter. 7. Navigate to the location of the downloaded files. 8. Enter MD5DEEP64 Country1.docx to start the application that creates an MD5 hash of Country1.docx and then press Enter. What is the length of this hash? (note: If you are not working on a 64 bit machine, then simply run the MD5deep.exe 32 bit version). 9. Now enter MD5DEEP64 MD5DEEP.TXT to start the application that creates an MD5 hash of the accompanying documentation file MD5DEEP.TXT and then press Enter. What is the length of this hash? Compare it to the hash of Country1.docx. What does this tell you about the strength of the MD5 hash? 10. Start Microsoft Word and then open Country1.docx. 11. Remove the period at the end of the sentence so it says Now is the time for all good men to come to the aid of their country and then save the document as Country2.docx in the directory that contains the files. -
Netcat and Trojans/Backdoors
Netcat and Trojans/Backdoors ECE4883 – Internetwork Security 1 Agenda Overview • Netcat • Trojans/Backdoors ECE 4883 - Internetwork Security 2 Agenda Netcat • Netcat ! Overview ! Major Features ! Installation and Configuration ! Possible Uses • Netcat Defenses • Summary ECE 4883 - Internetwork Security 3 Netcat – TCP/IP Swiss Army Knife • Reads and Writes data across the network using TCP/UDP connections • Feature-rich network debugging and exploration tool • Part of the Red Hat Power Tools collection and comes standard on SuSE Linux, Debian Linux, NetBSD and OpenBSD distributions. • UNIX and Windows versions available at: http://www.atstake.com/research/tools/network_utilities/ ECE 4883 - Internetwork Security 4 Netcat • Designed to be a reliable “back-end” tool – to be used directly or easily driven by other programs/scripts • Very powerful in combination with scripting languages (eg. Perl) “If you were on a desert island, Netcat would be your tool of choice!” - Ed Skoudis ECE 4883 - Internetwork Security 5 Netcat – Major Features • Outbound or inbound connections • TCP or UDP, to or from any ports • Full DNS forward/reverse checking, with appropriate warnings • Ability to use any local source port • Ability to use any locally-configured network source address • Built-in port-scanning capabilities, with randomizer ECE 4883 - Internetwork Security 6 Netcat – Major Features (contd) • Built-in loose source-routing capability • Can read command line arguments from standard input • Slow-send mode, one line every N seconds • Hex dump of transmitted and received data • Optional ability to let another program service established connections • Optional telnet-options responder ECE 4883 - Internetwork Security 7 Netcat (called ‘nc’) • Can run in client/server mode • Default mode – client • Same executable for both modes • client mode nc [dest] [port_no_to_connect_to] • listen mode (-l option) nc –l –p [port_no_to_connect_to] ECE 4883 - Internetwork Security 8 Netcat – Client mode Computer with netcat in Client mode 1. -
Unix Tools 2
Unix Tools 2 Jeff Freymueller Outline § Variables as a collec9on of words § Making basic output and input § A sampler of unix tools: remote login, text processing, slicing and dicing § ssh (secure shell – use a remote computer!) § grep (“get regular expression”) § awk (a text processing language) § sed (“stream editor”) § tr (translator) § A smorgasbord of examples Variables as a collec9on of words § The shell treats variables as a collec9on of words § set files = ( file1 file2 file3 ) § This sets the variable files to have 3 “words” § If you want the whole variable, access it with $files § If you want just the second word, use $files[2] § The shell doesn’t count characters, only words Basic output: echo and cat § echo string § Writes a line to standard output containing the text string. This can be one or more words, and can include references to variables. § echo “Opening files” § echo “working on week $week” § echo –n “no carriage return at the end of this” § cat file § Sends the contents of a file to standard output Input, Output, Pipes § Output to file, vs. append to file. § > filename creates or overwrites the file filename § >> filename appends output to file filename § Take input from file, or from “inline input” § < filename take all input from file filename § <<STRING take input from the current file, using all lines un9l you get to the label STRING (see next slide for example) § Use a pipe § date | awk '{print $1}' Example of “inline input” gpsdisp << END • Many programs, especially Okmok2002-2010.disp Okmok2002-2010.vec older ones, interac9vely y prompt you to enter input Okmok2002-2010.gmtvec • You can automate (or self- y Okmok2002-2010.newdisp document) this by using << Okmok2002_mod.stacov • Standard input is set to the 5 contents of this file 5 Okmok2010.stacov between << END and END 5 • You can use any label, not 5 just “END”. -
High Performance Multi-Node File Copies and Checksums for Clustered File Systems∗
High Performance Multi-Node File Copies and Checksums for Clustered File Systems∗ Paul Z. Kolano, Robert B. Ciotti NASA Advanced Supercomputing Division NASA Ames Research Center, M/S 258-6 Moffett Field, CA 94035 U.S.A. {paul.kolano,bob.ciotti}@nasa.gov Abstract To achieve peak performance from such systems, it is Mcp and msum are drop-in replacements for the stan- typically necessary to utilize multiple concurrent read- dard cp and md5sum programs that utilize multiple types ers/writers from multiple systems to overcome various of parallelism and other optimizations to achieve maxi- single-system limitations such as number of processors mum copy and checksum performance on clustered file and network bandwidth. The standard cp and md5sum systems. Multi-threading is used to ensure that nodes are tools of GNU coreutils [11] found on every modern kept as busy as possible. Read/write parallelism allows Unix/Linux system, however, utilize a single execution individual operations of a single copy to be overlapped thread on a single CPU core of a single system, hence using asynchronous I/O. Multi-node cooperation allows cannot take full advantage of the increased performance different nodes to take part in the same copy/checksum. of clustered file system. Split file processing allows multiple threads to operate This paper describes mcp and msum, which are drop- concurrently on the same file. Finally, hash trees allow in replacements for cp and md5sum that utilize multi- inherently serial checksums to be performed in parallel. ple types of parallelism to achieve maximum copy and This paper presents the design of mcp and msum and de- checksum performance on clustered file systems. -
NCIRC Security Tools NIAPC Submission Summary “HELIX Live CD”
NATO UNCLASSIFIED RELEASABLE TO THE INTERNET NCIRC Security Tools NIAPC Submission Summary “HELIX Live CD” Document Reference: Security Tools Internal NIAPC Submission NIAPC Category: Computer Forensics Date Approved for Submission: 24-04-2007 Evaluation/Submission Agency: NCIRC Issue Number: Draft 0.01 NATO UNCLASSIFIED RELEASABLE TO THE INTERNET NATO UNCLASSIFIED RELEASABLE TO THE INTERNET TABLE of CONTENTS 1 Product ......................................................................................................................................3 2 Category ....................................................................................................................................3 3 Role ............................................................................................................................................3 4 Overview....................................................................................................................................3 5 Certification ..............................................................................................................................3 6 Company....................................................................................................................................3 7 Country of Origin .....................................................................................................................3 8 Web Link ...................................................................................................................................3 -
15-122: Principles of Imperative Computation, Fall 2014 Lab 11: Strings in C
15-122: Principles of Imperative Computation, Fall 2014 Lab 11: Strings in C Tom Cortina(tcortina@cs) and Rob Simmons(rjsimmon@cs) Monday, November 10, 2014 For this lab, you will show your TA your answers once you complete your activities. Autolab is not used for this lab. SETUP: Make a directory lab11 in your private 15122 directory and copy the required lab files to your lab11 directory: cd private/15122 mkdir lab11 cp /afs/andrew.cmu.edu/usr9/tcortina/public/15122-f14/*.c lab11 1 Storing and using strings using C Load the file lab11ex1.c into a text editor. Read through the file and write down what you think the output will be before you run the program. (The ASCII value of 'a' is 97.) Then compile and run the program. Be sure to use all of the required flags for the C compiler. Answer the following questions on paper: Exercise 1. When word is initially printed out character by character, why does only one character get printed? Exercise 2. Change the program so word[3] = 'd'. Recompile and rerun. Explain the change in the output. Run valgrind on your program. How do the messages from valgrind correspond to the change we made? Exercise 3. Change word[3] back. Uncomment the code that treats the four character array word as a 32-bit integer. Compile and run again. Based on the answer, how are bytes of an integer stored on the computer where you are running your code? 2 Arrays of strings Load the file lab11ex2.c into a text editor. -
Netcat Starter
www.allitebooks.com Instant Netcat Starter Learn to harness the power and versatility of Netcat, and understand why it remains an integral part of IT and Security Toolkits to this day K.C. Yerrid BIRMINGHAM - MUMBAI www.allitebooks.com Instant Netcat Starter Copyright © 2013 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: January 2013 Production Reference: 1170113 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-84951-996-0 www.packtpub.com www.allitebooks.com Credits Author Project Coordinators K.C. "K0nsp1racy" Yerrid Shraddha Bagadia Esha Thakker Reviewer Jonathan Craton Proofreader Kelly Hutchison IT Content and Commissioning Editor Graphics Grant Mizen Aditi Gajjar Commissioning Editor Production Coordinator Priyanka Shah Melwyn D'sa Technical Editor Cover Work Ameya Sawant Melwyn D'sa Copy Editor Cover Image Alfida Paiva Conidon Miranda www.allitebooks.com About the author K.C. -
NOMADS User Guide V1.0
NOMADS User Guide V1.0 Table of Contents • Introduction o Explanation of "Online" and "Offline" Data o Obtaining Offline Data o Offline Order Limitations • Distributed Data Access o Functional Overview o Getting Started with OPeNDAP o Desktop Services ("Clients") o OPeNDAP Servers • Quick Start to Retrieve or Plot Data o Getting Started o Using FTP to Retrieve Data o Using HTTP to Retrieve Data o Using wget to Retrieve Data o Using GDS Clients to Plot Data o Using the NOMADS Server to Create Plots • Advanced Data Access Methods o Subsetting Parameters and Levels Using FTP4u o Obtain ASCII Data Using GDS o Scripting Wget in a Time Loop o Mass GRIB Subsetting: Utilizing Partial-file HTTP Transfers • General Data Structure o Organization of Data Directories o Organization of Data Files o Use of Templates in GrADS Data Descriptor Files • MD5 Hash Files o Purpose of MD5 Files o Identifying MD5 Files o Using md5sum on MD5 Files (Linux) o Using gmd5sum on MD5 Files (FreeBSD) o External Links • Miscellaneous Information o Building Dataset Collections o Working with Timeseries Data o Creating Plots with GrADS o Scheduled Downtime Notification Introduction The model data repository at NCEI contains both deep archived (offline) and online model data. We provide a variety of ways to access our weather and climate model data. You can access the online data using traditional access methods (web-based or FTP), or you can use open and distributed access methods promoted under the collaborative approach called the NOAA National Operational Model Archive and Distribution System (NOMADS). On the Data Products page you are presented with a table that contains basic information about each dataset, as well as links to the various services available for each dataset.