SSC1: I/O
Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Software Systems Components 1 Standard Streams I/O Redirection Input/Output Cmd Line Input Formatted I/O Formatted Output Formatted Input Volker Sorge Buffered Streams Memory Mapping http://www.cs.bham.ac.uk/~vxs/teaching/ssc1 File System Manipulations
Polling Topic Overview SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams 1. Simple Input/Output: streams for various types, Standard Streams I/O Redirection redirection. . . java.io.* Cmd Line Input 2. Formatted Input/Output: dealing with mixed input, Formatted I/O Formatted Output formatting output. . . java.util.Scanner, Formatted Input java.util.Formatter Buffered Streams 3. Advanced Techniques: buffering, file system Memory Mapping File System manipulations, memory mapping, polling, Manipulations ... java.nio.* Polling Simple Input/Output Techniques SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams I/O Redirection I Streams: the basis of all data input/output Cmd Line Input Redirecting Standard Input/Output: turning Formatted I/O I Formatted Output System.in, System.out into streams Formatted Input Buffered Streams Console: some specialities dealing with direct input and I Memory Mapping
passwords File System Manipulations
Polling What is a stream SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams I Streams are an abstraction over the actual low-level Data Streams operations of data handling. Standard Streams I/O Redirection I Streams enable a programming language to implement Cmd Line Input a uniform interface to the communication infrastructure Formatted I/O Formatted Output of diverse devices (files, sockets, hardware devices, etc.) Formatted Input Buffered Streams I A stream can be viewed as a sequence of data elements Memory Mapping
I A stream represent either a data source (input) or data File System sink (output) Manipulations Polling I There are different type of streams depending on what data s communicated and how. Types of streams SSC1: I/O Volker Sorge
Overview
I/O Streams There are streams of various types. We will have a look at Basic Streams Byte Streams The most basic of all streams, they handle Data Streams Standard Streams single bytes. I/O Redirection Character Streams Handle input/output of characters. Cmd Line Input Formatted I/O Data Streams Handle primitive data types and Strings. Formatted Output Formatted Input Object Streams Handle all kinds of objects. Buffered Streams Obviously each type of streams can be sub-divided into two Memory Mapping File System basic directions: Manipulations Input Stream A source that makes data available over time. Polling Output Stream A sink into which data can be written/sent over time. Basic operations on Input streams SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams read() Reads data (e.g. a byte) from a stream. Standard Streams I/O Redirection close() Closes the stream. Cmd Line Input
Formatted I/O Although streams are generally of a sequential nature, there Formatted Output are some streams that support revisiting their data. Formatted Input Buffered Streams
mark(int n) Marks a position on the stream. n specifies Memory Mapping
“how long” this position will be remembered. File System Manipulations reset() Resets the stream to the position where mark Polling was last called. Basic operations on Output streams SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams write(x) Writes some x to the stream. Data Streams flush() Forces all output to be actually written. Standard Streams I/O Redirection close() Closes the stream. Cmd Line Input Formatted I/O Never forget to flush and close streams. Formatted Output Formatted Input I Flushing can be done more than once. It is Buffered Streams recommended to flush at all critical points in a program. Memory Mapping
File System I Closing an output stream will automatically flush all Manipulations
output. Polling
I Not closing a stream does not cause an error. Too many open streams do! Byte Streams SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams I Byte Streams are implemented in the InputStream and Standard Streams I/O Redirection classes in the package. OutputStream java.io Cmd Line Input I They represent the most basic stream class in Java and Formatted I/O Formatted Output can be used for various purposes. Formatted Input Buffered Streams I A mentioned before we will concentrate on file input/output which is implemented in Memory Mapping File System FileInputStream and FileOutputStream Manipulations
Polling I They enable to read and write ASCII files byte-by-byte. A simple example program SSC1: I/O Volker Sorge //A simple Byte stream example. // Call with: // java io1 infile outfile Overview // Copies infile to outfile and prints the intermediate bytes. // Source: Sun Java Tutorials I/O Streams Basic Streams import java.io.∗; Data Streams Standard Streams public class io1 { I/O Redirection public static void main(String[] args) throws IOException { Cmd Line Input
FileInputStream in = null; Formatted I/O FileOutputStream out = null; Formatted Output try { Formatted Input in = new FileInputStream(args[0]); out = new FileOutputStream(args[1]); Buffered Streams intc ; Memory Mapping
while ((c = in.read()) != −1) { File System System.out.println(c); Manipulations out.write(c); } Polling
} finally { if (in != null) { in.close(); } if (out != null) { out.close(); } } } } A simple example program SSC1: I/O Volker Sorge
Let’s take a closer look at the program: Overview 1. io1 takes two command line arguments: I/O Streams Basic Streams I the input file (args[0]), which should exist; Data Streams Standard Streams I the output file (args[1]), which will be created or I/O Redirection overwritten. Cmd Line Input
2. It reads bytes of input until it reaches the end of file. Formatted I/O Formatted Output This is checked by testing for -1. Formatted Input 3. Byte by byte is read, printed out, and written to the Buffered Streams output file. Memory Mapping File System 4. Finally we close both output and input stream (if they Manipulations had been opened). This automatically flushes the Polling output stream. Observe that we have to explicitly state that there could be an IOException in the function, otherwise the Java compiler gets confused. Character Streams SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams I Character Streams are very similar to Byte Streams, I/O Redirection however work on 16 bit characters instead of 8 bit bytes. Cmd Line Input Formatted I/O I Their constructors are Formatted Output Formatted Input I FileReader instead of FileInputStream, and Buffered Streams I FileWriter instead of FileOutputStream. Memory Mapping
I They also enable us to read and write ASCII files File System byte-by-byte. Manipulations Polling A simple Character Stream IO class SSC1: I/O Volker Sorge
public class io3 { Overview
public static List
The full listing is in the handouts. File System The program performs the following tasks: Manipulations Polling 1. reads characters from input file, appends them to list, (observe that characters are integers!) 2. prints the list of characters to standard output, and 3. writes all characters again to the output file. I There are read/write methods for each supported type, e.g.: Type Reader method Writer method boolean readBoolean() writeBoolean(boolean v) char readChar() writeChar(int v) int readInt() writeInt(int v) double readDouble() writeDouble(double v) float readFloat() writeFloat(float v) String readLine() writeChars(String s)
Data Streams SSC1: I/O Volker Sorge I Data Streams support I/O of primitive data type values: Overview boolean, char, byte, short, int, long, float, and double I/O Streams Basic Streams I In addition they work on String values. Data Streams
I Data streams read and write binary files. Standard Streams I/O Redirection I The constructors are wrappers for byte streams. E.g.: Cmd Line Input DataInputStream in = new DataInputStream(new FileInputStream(filenameFormatted)); I/O Formatted Output DataOutputStream out = new DataOutputStream(new FileOutputStream(filenameFormatted Input)); Buffered Streams
Memory Mapping
File System Manipulations
Polling Data Streams SSC1: I/O Volker Sorge I Data Streams support I/O of primitive data type values: Overview boolean, char, byte, short, int, long, float, and double I/O Streams Basic Streams I In addition they work on String values. Data Streams
I Data streams read and write binary files. Standard Streams I/O Redirection I The constructors are wrappers for byte streams. Cmd Line Input There are read/write methods for each supported Formatted I/O I Formatted Output type, e.g.: Formatted Input Type Reader method Writer method Buffered Streams boolean readBoolean() writeBoolean(boolean v) Memory Mapping File System char readChar() writeChar(int v) Manipulations
int readInt() writeInt(int v) Polling double readDouble() writeDouble(double v) float readFloat() writeFloat(float v) String readLine() writeChars(String s) Using Data Streams SSC1: I/O Volker Sorge public class io4 {
public static List
public static void write_to_file(String filename, List
System.out.println("Writing file..."); while(iter.hasNext()) { We have to use the read/write out.writeInt(iter.next()); out.flush(); methods for integers. } out.close(); } ... Object Streams SSC1: I/O Volker Sorge Object streams are very similar to ordinary data streams. Overview While the latter support I/O of primitive data types, the I/O Streams former support I/O of objects. Basic Streams Data Streams I The constructors Standard Streams ObjectInputStream/ObjectOutputStream are I/O Redirection Cmd Line Input wrappers for byte streams. Formatted I/O Formatted Output I Object streams read and write binary files. Formatted Input I Reading and writing is done with Buffered Streams readObject()/writeObject(obj) methods. Memory Mapping File System I You can mix different objects on the same stream. Manipulations Polling Object Streams SSC1: I/O Volker Sorge Object streams are very similar to ordinary data streams. Overview While the latter support I/O of primitive data types, the I/O Streams former support I/O of objects. There are some peculiarities Basic Streams Data Streams to observe: Standard Streams 1. Objects can have many components and therefore need I/O Redirection Cmd Line Input to be “serializable”: They need to have an Formatted I/O implementation of the Serializable interface. Formatted Output Formatted Input 2. Many library classes have a Serializable interface: Buffered Streams BigDecimals, LinkeLists, Hashtables, ArrayLists, etc. Memory Mapping File System 3. readObject() needs to be type cast to the correct Manipulations
class, otherwise a ClassNotFoundException is Polling thrown. E.g.: BigDecimal obj1 = (BigDecimal) in.readObject(); 4. You can input and output primitive data types using readObject()/writeObject(obj). SSC1: I/O
Volker Sorge
Overview Data streams raise a couple of question: I/O Streams Basic Streams 1. How do we get binary files suitable for data streams in Data Streams Standard Streams the first place? I/O Redirection 2. How can we transform ASCII files containing data Cmd Line Input suitable for simple data types into binary format? Formatted I/O Formatted Output 3. If a file contains a mix of types or objects, how can we Formatted Input Buffered Streams check ahead what is read and avoid exceptions? Memory Mapping
We will answer the third question later when we discuss File System formatted input/output. For now we concentrate on Manipulations question one and two. Polling Three Standard Streams SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Input (stdin) The input from the console. Standard Streams Its corresponding Java object is System.in I/O Redirection Cmd Line Input Standard Output (stdout) Printing output to the console. Formatted I/O Its corresponding Java object is System.out Formatted Output Formatted Input Standard Error (stderr) Print errors messages to the console. Buffered Streams Its corresponding Java object is System.err Memory Mapping File System Both stdout and stderr print to the console. However, Manipulations separating them enables us to separate intended output from Polling unintended error messages. Example with System.in SSC1: I/O Volker Sorge
The following program fragment turns System.in into an Overview input stream: I/O Streams Basic Streams Data Streams import java.util.∗; import java.io.∗; Standard Streams I/O Redirection public class makeIntFile { Cmd Line Input public static List
Overview
I/O Streams I makeIntFile reads integers from stdin. Basic Streams Data Streams I Suppose it writes those to a binary file using a data Standard Streams stream. E.g. the command line I/O Redirection Cmd Line Input java makeIntFile outfile Formatted I/O writes all integer input to outfile. Formatted Output Formatted Input I We now exploit this class to translate an ASCII file Buffered Streams
infile containing integers into its corresponding Memory Mapping
binary data file using the command File System Manipulations java makeIntFile outfile < infile Polling I This binds System.in to the infile as a stream. It essentially parses integers from the file using whitespace as tokens. Redirecting stdout SSC1: I/O Volker Sorge
Overview
I/O Streams I Similar to stdin we can redirect stdout Basic Streams Data Streams java makeIntFile outfile > outfile2 Standard Streams I/O Redirection I This command will Cmd Line Input I read integers from the console, Formatted I/O I write them to the file outfile, Formatted Output Formatted Input I saves everything that would be printed to the console to outfile2 Buffered Streams Memory Mapping E.g., everything that is printed using I File System System.out.println ends up in outfile2 Manipulations Polling I Observe that outfile and outfile2 will be different. In particular the former is a binary file, while the latter is ASCII. Redirecting both stdin and stdout SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams We can now simply combine the redirection in a single shell Data Streams command: Standard Streams I/O Redirection java makeIntFile outfile < infile > outfile2 Cmd Line Input
Formatted I/O I No input will be taken from the console. Formatted Output No output will be printed to the console. Formatted Input I Buffered Streams I Error messages will still be printed to the console, the Memory Mapping do not go into outfile2. File System Manipulations
I One can also redirect stderr in some shells (e.g., with Polling 2> in bash). Command Line Input SSC1: I/O Volker Sorge
Finally we have a look at two special functions to allow for Overview
direct console input of regular arguments and passwords. I/O Streams Basic Streams System.console() constructs a console that has the Data Streams Standard Streams functionality of the standard streams. E.g. I/O Redirection Consolec = System.console(); Cmd Line Input readLine(String prompt) prints a prompt and reads a Formatted I/O Formatted Output string from the console. E.g. Formatted Input Buffered Streams String login = c.readLine("Enter your login: "); Memory Mapping
c.readPassword prints a prompt and reads a character File System array from the console, but without echoing the input. Manipulations E.g. Polling char [] password = c.readPassword("Enter your password: "); Observe that the character array contains the plain characters. For security you should always immediately process them and blank the array! Formatted Input and Output SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams I Previously we have seen how to do I/O with essentially Data Streams homogeneous data. Standard Streams I/O Redirection I However, only very rarely will we be dealing with such a Cmd Line Input situation, e.g. files containing only integers. Formatted I/O Formatted Output Formatted Input I For more sophisticated applications the ability to format output and reading it back is essential. Buffered Streams Memory Mapping
I Reading formatted input enables us to effectively parse File System input. Manipulations Polling I Formatting output gives us the means to generate output with respect to some patterns. Formatted Input and Output SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams I/O Redirection Cmd Line Input I Formatted Output: formatting to files with Formatted I/O PrintWriter and to strings with Formatter. Formatted Output Formatted Input I Formatted Input: simple parsing of input using the Buffered Streams Scanner class. Memory Mapping File System Manipulations
Polling The PrintWriter Class SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams I We have seen previously how we can use System.out I/O Redirection as an output stream. Cmd Line Input Formatted I/O I Now we use the methods available to System.out for Formatted Output outputting data to general streams. Formatted Input Buffered Streams
I We will concentrate on print, println, and printf. Memory Mapping I These methods are implemented in the PrintWriter File System Manipulations class of the java.io package. Polling Constructors SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams I The PrintWriter class is essentially a wrapper for Standard Streams output streams. I/O Redirection Cmd Line Input Consequently it inherits the main methods like I Formatted I/O flush(),close(), etc. Formatted Output Formatted Input I The standard constructor is PrintWriter(out), where Buffered Streams out is either an output stream, a file, or a string. If it is Memory Mapping a string a file of that name will be created. File System Manipulations E.g. the following creates a PrintWriter stream to the file Polling output.txt. PrintWriter out = new PrintWriter("output.txt"); Methods SSC1: I/O Volker Sorge
Overview
print(x) Prints a string representation of x. I/O Streams If works on primitive data types as well as objects using Basic Streams Data Streams the toString method. Standard Streams If no toString method is implemented it prints I/O Redirection general object information. Cmd Line Input Formatted I/O println(x) Same as print(x) with an additional newline. Formatted Output Formatted Input printf(format,args) Prints formatted output according Buffered Streams to the format string. Args is a comma separated list of Memory Mapping arguments for the format string. Example: File System Manipulations . ( ,3446,3446,3446); out printf "Dec: %d,\nHex: %04x,\nOct: %o" Polling yields Dec: 3446 Hex: 0D76 Oct: 6566 Printf SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Let’s have a closer look at the printf and its formatting Data Streams argument: Standard Streams I/O Redirection Cmd Line Input I The format string is a mix of ordinary string, conversion Formatted I/O characters and control characters. Formatted Output Control characters start with a backslash. Formatted Input I Buffered Streams I Conversion characters start with a percentage sign. Memory Mapping They “consume” arguments from the given list. File System Manipulations
I Conversions can have flags for additional formatting Polling purposes. Printf SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams For example consider the format string Data Streams "Dec: %d,\nHex: %04x,\nOct: %o" Standard Streams I/O Redirection I It needs three arguments which should be integers. Cmd Line Input I %d prints the first integer argument in decimal format. Formatted I/O Formatted Output I \n prints a newline. Formatted Input Buffered Streams %04x prints the second argument in hexadecimal format. I Memory Mapping
It has two flags 0 and 4, which means that the number File System will be printed with four digits and, if necessary filled Manipulations with leading zeros. Polling
I %o prints the third argument as octal number. Some Conversions for Printf SSC1: I/O Volker Sorge
Overview Here is a selected list of conversion characters for printf I/O Streams Basic Streams Data Streams Conversion Argument Result Standard Streams I/O Redirection s any a string similar to print Cmd Line Input
c char a unicode character Formatted I/O Formatted Output d int a decimal integer Formatted Input o int an octal integer Buffered Streams x,X int a hexadecimal integer (small or capital)Memory Mapping f float a decimal number File System Manipulations
a,A float a hexadecimal floating-point number Polling Many of the conversion accept several flags. In addition there are conversion characters for time and date. For a comprehensive list see the API documentation. The Formatter Class SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams I Formatter uses the same facilities as printf. Data Streams Standard Streams I It enables output both to files and strings. I/O Redirection I The Formatter class is implemented in the java.util Cmd Line Input package. Formatted I/O Formatted Output Formatted Input I The main constructors are: Buffered Streams
Formatter() constructs a string, Memory Mapping
Formatter(out) opens a file. File System Manipulations I In addition to the methods on stream, we have Polling format(string,args) which works similar to printf. toString() which returns a formatted string. Formating to File SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams I/O Redirection This is very similar to using printf with PrintWriter: Cmd Line Input 1. Create a file output stream with Formatted I/O Formatted Output Formatter out = new Formatter(filename); Formatted Input 2. Write formatted output to it using format, e.g.: Buffered Streams Memory Mapping out.format("% 4d: %#010X %#012o\n",output,output,output); File System Manipulations
Polling Formating to String SSC1: I/O Volker Sorge
This is slightly more interesting. The basic idea is as follows: Overview
I/O Streams Basic Streams 1. Create an empty string as data sink with Formatter(). Data Streams 2. Every call to appends the format result to the Standard Streams format I/O Redirection stream. Cmd Line Input 3. The toString() method returns the collected string. Formatted I/O Formatted Output Formatted Input public static void outputList(List
Overview
I/O Streams Basic Streams Data Streams I Is a sophisticated way to Standard Streams I It offers primarily two types of functionality I/O Redirection “Look ahead parsing” A simple mechanism to query Cmd Line Input Formatted I/O the next token on the input stream and handle it Formatted Output according to its type. Formatted Input Buffered Streams Regular Expression Matching A more sophisticated way Memory Mapping of handling input tokens with respect to pattern File System matching. Manipulations We will exclusively concentrate on the former of the two Polling methods. Constructors SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams The Scanner class is essentially an iterator operating on Data Streams Standard Streams I Streams, e.g. I/O Redirection Scanner sc = new Scanner(System.in); Cmd Line Input Files, e.g. Formatted I/O I Formatted Output Scanner sc = new Scanner(new File("Infile")); Formatted Input Buffered Streams I Strings, e.g. Memory Mapping String input = "1 fish 2 fish red fish blue fish"; File System Scanner sc = new Scanner(input); Manipulations All these constructors parse the given input considering all Polling non-whitespace characters as tokens. Methods SSC1: I/O Volker Sorge
Overview Like other iterators Scanner offers iteration control with I/O Streams Basic Streams look ahead using the two methods: Data Streams Standard Streams hasNext() True if there is a next token. I/O Redirection next() Returns the next token. Cmd Line Input Formatted I/O In addition Scanner implements a number of similar Formatted Output methods for special data types: Formatted Input Buffered Streams
hasNext () True if the next token is of a particular type. Memory Mapping
Examples: File System hasNextInt(), hasNextFloat(),hasNextLine(),. . . Manipulations Polling next () Returns the next token of the specified type. Example: nextInt(), nextFloat(),nextLine(),. . . nextLine() returns a string containing one input line. Controlling Formatted Input SSC1: I/O Volker Sorge
Overview
I/O Streams The following program fragment demonstrates how Scanner Basic Streams can be used for formatted input: Data Streams Standard Streams public static List
{ Cmd Line Input Scanner in = new Scanner(new File(filename)); List
Overview
I/O Streams Basic Streams Data Streams Standard Streams I/O Redirection I Scanner enables to directly check for certain keywords Cmd Line Input on the input stream by using string matching. Formatted I/O Formatted Output I Example: in.hasNext("accept") Formatted Input Buffered Streams This method call returns true if the next token indeed I Memory Mapping
corresponds to the string "accept". File System Manipulations
Polling Outlook: RegExp Matching SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Scanner offers a number of elaborate methods to exploit Data Streams Standard Streams regular expression matching: I/O Redirection Cmd Line Input I Search for tokens matching a given pattern in the input. Formatted I/O I Query or skip the next token if it matches a given Formatted Output Formatted Input pattern. Buffered Streams I Changing delimiters: Use patterns other than Memory Mapping whitespace to delimit the tokens on the input. File System Manipulations
You will learn about regular expressions (and their syntax) in Polling your models of computation lecture in the second term. Advanced I/O Topics SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams In the following we will look at I/O Redirection Cmd Line Input I Buffered Streams: Make I/O more efficient. Formatted I/O Memory Mapping: How to deal with really large files. Formatted Output I Formatted Input I File System Manipulations: Doing things with files and Buffered Streams directories. Memory Mapping File System I Polling: Watching out for file changes in the future. Manipulations Polling Buffered Streams SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams I Using standard I/O streams for file communication is Data Streams generally inefficient. Standard Streams I/O Redirection I After each operation I/O handling is done by Operating Cmd Line Input
System routines. Formatted I/O Formatted Output I Buffered streams allow to simulate streams in memory: Formatted Input
I Input streams are pre-loaded before being accessed. Buffered Streams
I Output streams are assembled in memory until explicitly Memory Mapping flushed. File System Manipulations
I This is generally faster in particular on large files. Polling Observe, that while for standard streams flushing is not always necessary, for buffered streams it is absolutely critical! Using Buffered Streams SSC1: I/O Volker Sorge
Overview
Buffered streams are simply wrappers around byte or I/O Streams Basic Streams character streams and therefore extremely easy to use. Data Streams 1. Byte Streams: use BufferedInputStream and Standard Streams I/O Redirection BufferedOutputStream. Cmd Line Input
BufferedInputStream in = Formatted I/O new BufferedInputStream(new FileInputStream(filename)); Formatted Output Formatted Input 2. Character Streams: use BufferedReader and Buffered Streams BufferedWriter. Memory Mapping File System BufferedReader in = new BufferedReader(new FileReader(filename)); Manipulations 3. Data Streams and Object Streams can be wrapped Polling around Buffered Streams as usual: DataInputStream in = new DataInputStream(new BufferedInputStream (new FileInputStream(filename))); Memory Mapping SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams
I Memory Mapping is a technique to work with files even Standard Streams I/O Redirection more efficiently. Cmd Line Input I It also helps to work with very large files that could not Formatted I/O Formatted Output be buffered entirely. Formatted Input Buffered Streams I So far our streams have worked in a sequential manner, i.e., we read one byte, character or datum at a time. Memory Mapping File System Manipulations I However, for effective memory mapping we need random access files. Polling Memory Mapped Files SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams I The basic idea is to establish a mapping between parts Standard Streams of a file to a memory buffer. I/O Redirection Cmd Line Input I Files are divided into virtual pages. Formatted I/O I A page is then mapped into a memory buffer and Formatted Output processed there. Formatted Input Buffered Streams I Whenever a page boundary is crossed the old page is written and a new one loaded. Memory Mapping File System I Double mapping for files to different memory buffers is Manipulations generally avoided by locking. Polling The FileChannel Class SSC1: I/O Volker Sorge
Overview
In Java memory mapping is primarily achieved using the I/O Streams Basic Streams FileChannel class in the java.nio.FileChannel.*. Data Streams 1. Create a random access file with an existing (e.g. Standard Streams I/O Redirection created with FileReader): Cmd Line Input
RandomAccessFile file = new RandomAccessFile(file, mode); Formatted I/O Formatted Output Mode is a string specifying the access right, e.g. "rw" Formatted Input is read-write. Buffered Streams 2. Retrieve the FileChannel from file: Memory Mapping = . (); File System FileChannel channel file getChannel Manipulations 3. Establish a memory map of a given size at a specified Polling file position: MappedByteBuffer map = channel.map(mode, position, size); Mode here is something like READ WRITE or READ ONLY. Operations on MappedByteBuffers SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams I ByteBuffers are structured as byte arrays and Data Streams Standard Streams therefore can be accessed using array indices and put() I/O Redirection and get() methods. Cmd Line Input
I Entire blocks of data can be written at once. Formatted I/O Formatted Output Formatted Input I Specialist operations of primitive data type exist, e.g., Buffered Streams putInt() or getChar() Memory Mapping
I In addition MappedByteBuffers offers methods File System load() and force() to load a file page into memory Manipulations and to force a memory page to be written to the file, Polling respectively. Possible Problems SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams I Using Memory mapping for only small changes is often Standard Streams I/O Redirection inefficient. Cmd Line Input I One has to be careful when choosing file page size. Formatted I/O Formatted Output I If pages are too large memory mapping can be Formatted Input sometimes slower than ordinary file transfer. Buffered Streams Memory Mapping Also when we create a new file and map buffers to I File System them, it is possible to leave “holes”, i.e. file pages that Manipulations are empty. Polling Accessing the File System SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams I/O Redirection I So far we were only concerned with file input/output. Cmd Line Input
I This always assumed that input file exists. Formatted I/O Formatted Output Formatted Input I Or that output files could indeed be written. Buffered Streams However, sometimes it is necessary to I Memory Mapping I This can be done using methods of the File class File System Manipulations
Polling Accessing the File System SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams
I Observe that there is a File class both in java.io and Standard Streams I/O Redirection java.nio. We will discuss the former, but the concepts Cmd Line Input
carry over to the latter. Formatted I/O Formatted Output I We will only discuss some of the most important Formatted Input methods. Buffered Streams Memory Mapping I For plenty more see the corresponding API File System documentation, e.g. Manipulations http://java.sun.com/javase/6/docs/api/java/io/File.html Polling Creating Abstract Pathnames SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams I The easiest way to create an abstract pathname is by Standard Streams I/O Redirection passing a string to the constructor: File Cmd Line Input
File file = new File(pathname); Formatted I/O Formatted Output I file is so far only an abstract pathname, it has not yet Formatted Input been connected to the actual file system. Buffered Streams Memory Mapping I Only once we use it to create an input/output stream File System can we get problems if the abstract pathname is invalid. Manipulations
Polling I Until then we can use it for file system manipulations. Accessor Methods SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams getName() Returns the name of the file or directory I/O Redirection Cmd Line Input denoted by this abstract pathname. Formatted I/O Formatted Output getPath() Converts this abstract pathname into a Formatted Input pathname string. Buffered Streams getParent() Returns the pathname string of this abstract Memory Mapping File System pathname’s parent. Manipulations
Polling Query Methods SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams exists() Tests whether the file or directory denoted by this I/O Redirection Cmd Line Input abstract pathname exists. Formatted I/O Formatted Output isDirectory() Tests whether the file denoted by this Formatted Input abstract pathname is a directory. Buffered Streams isFile() Tests whether the file denoted by this abstract Memory Mapping File System pathname is a normal file. Manipulations
Polling Creator Methods SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams Standard Streams createNewFile() Creates a new, empty file with the I/O Redirection abstract pathname. Cmd Line Input Formatted I/O Deletes the file with the abstract pathname. Formatted Output delete() Formatted Input rename(dest) Renames the file with the abstract Buffered Streams pathname to dest string. Memory Mapping File System mkdir() Make a new directory with the abstract pathname. Manipulations
Polling Listing Methods SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams Data Streams list() Lists all files and directories for the abstract Standard Streams pathname. I/O Redirection Cmd Line Input
listFiles() Lists all files and directories with their full Formatted I/O pathnames. Formatted Output Formatted Input Buffered Streams
I In both cases the abstract pathname has to denote an Memory Mapping
actual directory. File System Manipulations I Both methods can be used with a filename filter as Polling argument. A Small Example SSC1: I/O Volker Sorge
I The following paths class creates a new directory in the Overview
tmp directory and a new file within this directory. I/O Streams Basic Streams I Observe that it will only create either if they do not yet Data Streams exist. Standard Streams I/O Redirection I In particular, nothing will be created if new exists Cmd Line Input already as ordinary file. Formatted I/O Formatted Output Formatted Input import java.io.∗; Buffered Streams public class paths { Memory Mapping ( [] ) public static void main String args throws IOException File System { Manipulations File file = new File("/tmp/new/file"); File dir = new File(file.getParent()); Polling
if (! dir.exists() && !dir.isDirectory()) { dir.mkdir(); } if (dir.isDirectory() && !file.exists() && !file.isFile()) { file.createNewFile(); }
} } Outlook: Asynchronous Polling SSC1: I/O Volker Sorge
Overview
I/O Streams Basic Streams I Polling is a technique that continually interrogates a Data Streams Standard Streams peripheral device to see if it has data to transfer. I/O Redirection I Synchronous polling the leads to lengthy interruptions Cmd Line Input in a client program Formatted I/O Formatted Output Formatted Input I Asynchronous polling allows record events on an input device without delaying or interrupting the client Buffered Streams Memory Mapping program. File System Manipulations I This is particularly useful to check, for example, whether a file has been written or whether a directory Polling has been changed. Watch Services in Java 7 SSC1: I/O Volker Sorge
Overview
I/O Streams Java 7 will contain async polling in its java.nio package. Basic Streams Data Streams I The Watch Service API enables one to register a Standard Streams directories with a watch service. I/O Redirection Cmd Line Input
I The service registers specified types of events (e.g. file Formatted I/O creation). Formatted Output Formatted Input I These events can then be handled in a predefined way Buffered Streams (e.g. update an editor buffer). Memory Mapping File System I For example, this allows to watch changes on a great Manipulations number of open files. Polling As long as Java 7 is not yet released, this is subject to change of course!