File IO serialization
HOM HVD
Streams and Java-I/O Streams in Java File IO serialization java.nio.file Object Streams and serialization Pieter van den Hombergh Serialisation formats Richard van den Ham WARNING Javascript Object Notation
Fontys Hogeschool voor Techniek en Logistiek
March 13, 2018
HOMHVD/FHTenL File IO serialization March 13, 2018 1/23 File IO Topics serialization HOM HVD
Streams and Streams and Java-I/O Java-I/O Streams in Java Streams in Java java.nio.file java.nio.file Object Streams and serialization
Serialisation formats WARNING Object Streams and serialization Javascript Object Notation
Serialisation formats WARNING Javascript Object Notation
HOMHVD/FHTenL File IO serialization March 13, 2018 2/23 File IO Streams in Java serialization HOM HVD
Streams and Java-I/O Streams in Java java.nio.file
Object Streams and serialization
Serialisation formats WARNING Javascript Object Notation
Figure: Taken from the Oracle/Sun Java tutorial
Read or write information from different sources and types. sources: network, files, devices types: text, picture, sound Streams are FIFOs, uni-directional
HOMHVD/FHTenL File IO serialization March 13, 2018 3/23 File IO Two basic stream types serialization HOM Byte Streams HVD java.io.InputStream, java.io.OutputStream Streams and Java-I/O read and write pdf, mp3, raw... Streams in Java java.nio.file Character Streams (16-bit Unicode) Object Streams java.io.Reader, java.io.Writer and serialization simplifies reading and writing characters, character-arrays or Strings Serialisation formats WARNING Javascript Object Notation
HOMHVD/FHTenL File IO serialization March 13, 2018 4/23 File IO Input Stream Overview serialization HOM HVD
ByteStreamClass for Input CharStreamClass for Input Streams and Java-I/O InputStream Reader Streams in Java BufferedInputStream BufferedReader java.nio.file Object Streams LineNumberInputStream LineNumberReader and serialization Serialisation ByteArrayInputStream CharArrayReader formats WARNING (none) InputStreamReader Javascript Object Notation DataInputStream (none) FilterInputStream FilterReader PushbackInputStream PushbackReader PipedInputStream PipedReader StringBufferInputStream (deprecated under 1.1) StringReader SequenceInputStream (none)
HOMHVD/FHTenL File IO serialization March 13, 2018 5/23 File IO Output Stream Overview serialization HOM HVD
ByteStreamClass for Output CharStreamClass for Output Streams and OutputStream Writer Java-I/O Streams in Java BufferedOutputStream BufferedWriter java.nio.file Object Streams ByteArrayOutputStream CharArrayWriter and serialization DataOutputStream (none) Serialisation formats (none) OutputStreamWriter WARNING Javascript Object Notation FileOutputStream FileWriter PrintStream PrintWriter PipedOutputStream PipedWriter (none) StringWriter
HOMHVD/FHTenL File IO serialization March 13, 2018 6/23 File IO ByteStreams - File Input and Output serialization HOM «interface» java.lang.AutoClosable used to enable HVD java 7 "try with +close(): void resources" Streams and Java-I/O «interface» «interface» Streams in Java java.io.Closable java.io.Flushable java.nio.file +close(): void +flush(): void Object Streams and serialization
java.io.InputStream java.io.OutputStream Serialisation formats + avialable(): int + write(b : int): void WARNING + mark(readlimit:int): void + write(b: byte[]): void + markSupported: boolean + write(b: byte[],off:int, len:int): void Javascript Object Notation + read(): int + reade(buffer: byte[]): int + read(b: byte[],off: int, len:int): int + reset(): void + skip(n: long): long
java.io.FileInputStream java.io.FileOutputStream + FileInputStream(file: File) + FileOutputStream(file: File) + FileInputStream(name: String) + FileOutputStream(name: String) + FileInputStream(fdObj: FileDescription) + FileOutputStream(file: File, append:boolean) # finalize(): void + FileOutputStream(name: String, append:boolean) + getChannel(): FileChannel + FileOutputStream(fdObj: FileDescription) + getFD(): FileDescriptor # finalize(): void + getChannel(): FileChannel + getFD(): FileDescriptor Both Input and Output are streams of bytes.
HOMHVD/FHTenL File IO serialization March 13, 2018 7/23 File IO CharacterStreams - Writer and Reader serialization HOM HVD «interface» used to enable java.lang.AutoClosable java 7 "try with Streams and + close(): void resources" Java-I/O «interface» Streams in Java java.lang.Appendable java.nio.file «interface» «interface» «interface» + append( char c): Appendable java.io.Flushable java.io.Closable java.lang.Readable + append(csq: CharSequence): Appendable Object Streams + append(csq: CharSequence csq, start:int,+ flush(): end: int):void Appendable+ close(): void + read(cb: CharBuffer): int and serialization Serialisation formats WARNING Javascript Object Notation java.io.Writer java.io.Reader +Writer() + Reader() +Writer(lock: Object) + Reader(lock:Object) + write(cbuf: char[]): Writer + mark(readAheadLimit:int): void + write(csq : CharSequence ): Writer + markSupported: boolean + write(csq : CharSequence, start: int, end: int ): Writer + read(): int + read(buf: char[]) : int + read(buf: char[], off: int , len: int): int + read(target: CharBuffer): int + ready(): boolean + reset(): void + skip(n: long): long
Reader reads character data (text) Writer writes character data
HOMHVD/FHTenL File IO serialization March 13, 2018 8/23 File IO Audio Stream in Servlets serialization HOM HVD Idea: Use Servlet Output Stream to stream MP3s via HTTP Streams and set ContentType to “audio/mpeg” Java-I/O Streams in Java read MP3 files via InputStream java.nio.file Object Streams write the bytes into the Servlet Output Stream and serialization
Serialisation formats WARNING Javascript Object Notation
HOMHVD/FHTenL File IO serialization March 13, 2018 9/23 File IO Wrapping Streams serialization HOM HVD Streams can be combined Streams and Depending on source and type Java-I/O Streams in Java useful additions java.nio.file buffering Object Streams and serialization encryption Serialisation filtering formats ... WARNING Javascript Object Notation
HOMHVD/FHTenL File IO serialization March 13, 2018 10/23 File IO Wrapping Streams - Buffering serialization HOM HVD Buffered streams speed up input and output by reducing the number of Streams and reads and writes. Java-I/O Streams in Java They employ a buffered array of bytes or characters that act as a cache java.nio.file
Object Streams If no buffersize is specified, the default size is 512 bytes or characters. and serialization A buffered output stream calls the write method only when its buffer fills up Serialisation formats or when the flush() method is called WARNING Javascript Object Notation FileReader reader = new FileReader ( "/home/hom/test.dat" ); BufferedReader bufferedIn = new BufferedReader ( reader );
Useful additional function:
String line = bufferedIn . readLine ();
HOMHVD/FHTenL File IO serialization March 13, 2018 11/23 File IO Wrapping Streams - On-the-fly encryption serialization HOM HVD Java IO streams implement the decorator design pattern Streams and new/additional behaviour can be added to an existing stream dynamically Java-I/O Streams in Java might remind you of the ISO/OSI reference model for network protocols java.nio.file Object Streams and serialization
Serialisation formats WARNING Javascript Object Notation
HOMHVD/FHTenL File IO serialization March 13, 2018 12/23 File IO TIP: The java.nio.file package serialization HOM HVD Since Java 1.4 there is a new java.nio package, which adds extra Streams and buffering capabilities. Java-I/O Streams in Java Since Java7 the package java.nio.file has been added. The you can java.nio.file find some very useful classes to help you with your daily file hassling code. Object Streams and serialization In particular java.nio.file.Files has many useful methods. Serialisation List
public static void main(String[] args) throws IOException { Path path = Paths.get("/etc/passwd"); Files.lines(path).forEach(System.out::println); }
HOMHVD/FHTenL File IO serialization March 13, 2018 13/23 File IO Serialization serialization HOM HVD
Streams and Java-I/O Streams in Java java.nio.file
Object Streams and serialization
Serialisation formats Serialization is the process of converting an object into a sequence of bits. WARNING Javascript Object Notation This is necessary to transport objects via streams. Serialization respectively Deserialization is also known as Mashalling/Unmarshalling (broader scope).
HOMHVD/FHTenL File IO serialization March 13, 2018 14/23 File IO What makes an Object, data wise serialization HOM Quiz: HVD
Streams and Java-I/O Streams in Java java.nio.file
Object Streams and serialization
Serialisation If you imagine some objects you’ve encountered, what is the closest data formats WARNING structure that describes the structure of all objects? Javascript Object Notation
HOMHVD/FHTenL File IO serialization March 13, 2018 15/23 File IO Serialization in Java serialization HOM HVD Not every object is serializable (e.g. Thread, or Socket). Quiz: Why not? (Think transparency, security). Streams and Java-I/O Streams in Java In Java, the Serializable-Interface “marks” serializable objects java.nio.file The stream implementations ObjectInputStream and Object Streams ObjectOutputStream can be used to serialize/deserialize serializable and serialization Serialisation objects formats final void writeObject( Object obj ) throws IOException WARNING Javascript Object Notation final Object readObject() throws ClassNotFoundException, IOException To exclude non-serializable attributes of serializable objects, transient can be used to mark these fields. All transient fields are excluded from the serialization.
HOMHVD/FHTenL File IO serialization March 13, 2018 16/23 File IO Serialisation/Marshalling serialization HOM 20 p r i v a t e static final String OBJECT_STORE HVD 21 = "/tmp/objects.ser" ; 22 Streams and 23 p r i v a t e void m a r s h a l () { Java-I/O 24 t r y( ObjectOutputStream out Streams in Java 25 = new ObjectOutputStream ( java.nio.file 26 new FileOutputStream ( OBJECT_STORE ));) { 27 UserBean user1 = new U s e r B e a n ( "Uwe" , Object Streams 28 "UwePass" ); and serialization 29 UserBean user2 = new U s e r B e a n ( "Peter" , 30 "PeterPass" ); Serialisation 31 UserBean user3 = new U s e r B e a n ( "Paula" , formats 32 "PaulaPass" ); WARNING 33 user3 . s e t S o c k e t ( new S o c k e t ()); Javascript Object Notation 34 S y s t e m . out . p r i n t l n ( "BeanCounter=" 35 + U s e r B e a n . getBeanCounter ()); 36 37 out . writeObject ( user1 ); 38 out . writeObject ( user2 ); 39 out . writeObject ( user3 ); 40 } catch( IOException ex ) { 41 l o g g e r . log ( Level . SEVERE , null, ex ); 42 } 43 }
HOMHVD/FHTenL File IO serialization March 13, 2018 17/23 File IO DeSerialisation/UnMarshalling serialization HOM 45 p r i v a t e void u n m a r s h a l () { HVD 46 t r y( ObjectInputStream in 47 = new ObjectInputStream ( Streams and 48 new FileInputStream ( OBJECT_STORE ));) { Java-I/O 49 Streams in Java 50 f o r( int i = 0 ; i < 3 ; i++ ) { java.nio.file 51 UserBean user = ( U s e r B e a n ) in . r e a d O b j e c t (); 52 S y s t e m . out . p r i n t l n ( "user=" + user ); Object Streams 53 } and serialization 54 55 //UserBean newUser= new UserBean("Richard","Richardpw"); Serialisation 56 //System.out.println("BeanCounter= "+ newUser.getBeanCounter ←- formats ()); WARNING 57 } catch( ClassNotFoundException | IOException ex ) { Javascript Object Notation 58 l o g g e r . log ( Level . SEVERE , null, ex ); 59 } 60 }
HOMHVD/FHTenL File IO serialization March 13, 2018 18/23 File IO Serialisation formats serialization HOM HVD Binary: from Java to Java, (between JVMS even across the network) Streams and Text based (modern: always UTF-8). Can be produced and consumed by Java-I/O different technologies (e.g. Java at server, Javascript in browser), like PHP Streams in Java or C# at one end, Java at the other java.nio.file Object Streams XML often via the SOAP protocol, typically HTTP based. and serialization JSON, also typically HTTP based. Serialisation formats Proto Buffers. Binary. PROTBUF is a modern (de)serialization framework WARNING Javascript Object Notation that has a schema.
HOMHVD/FHTenL File IO serialization March 13, 2018 19/23 File IO WARNING serialization HOM HVD
Streams and Java-I/O NEVER DESERIALIZE A JAVA Streams in Java java.nio.file
Object Streams OBJECT STREAM FROM AN and serialization Serialisation formats WARNING UNKNOWN SOURCE. Javascript Object Notation
Deserialization without a schema to validate the input is dangerous at best. Efective Java 3rd ed item 85. It is very easy to create a denial of service attack this way.
HOMHVD/FHTenL File IO serialization March 13, 2018 20/23 File IO Javascript Object Notation serialization HOM Javascript Object Notation (JSON) is very popular at the moment. HVD It is less verbose and heavy on the wire. Streams and Java-I/O It is easily generated and parsed (read) by programs and humans(although Streams in Java only for developers during development). java.nio.file Object Streams Nowadays the format of choice for communication between apps and backend. and serialization
Can be stored, but also generated from traditional tables in PostgreSQL. Serialisation Has APIs that help generating (which sounds trivial) and parsing (less so) the formats WARNING wire format. e.g. javax.json defines some interfaces. Javascript Object Notation This will be the transport format in PRJ2 this year (2017).
HOMHVD/FHTenL File IO serialization March 13, 2018 21/23 File IO Person (de)serialization demo with GSON serialization HOM HVD Look ma, No hands Streams and p u b l i c static void main ( String [] args ) { Java-I/O Gson gson= new Gson (); Streams in Java java.nio.file Person max = new Person ( "Max" , LocalDate . of ( 1960, 04, 30 ) )←- Object Streams ; and serialization String jo = gson . toJson ( max ); System . out . println ( "jo=" + jo ); Serialisation formats Person otherMax = gson . fromJson ( jo , Person .class); WARNING System . out . println ( "fromJson=" + otherMax ); Javascript Object Notation assert max . equals ( otherMax ); System . out . println ( "all done" ); }
HOMHVD/FHTenL File IO serialization March 13, 2018 22/23 File IO Questions and links serialization HOM Not all understood? HVD
Study Streams and Java-I/O http://docs.oracle.com/javase/tutorial/essential/io/index.html Basic I/O Part Streams in Java I/O Streams. java.nio.file Object Streams Questions? and serialization Serialisation Questions or remarks? formats WARNING Javascript Object Notation
HOMHVD/FHTenL File IO serialization March 13, 2018 23/23