The MD6 hash function A proposal to NIST for SHA-3 Ronald L. Rivest Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology Cambridge, MA 02139 [email protected] Benjamin Agre Daniel V. Bailey Christopher Crutchfield Yevgeniy Dodis Kermin Elliott Fleming Asif Khan Jayant Krishnamurthy Yuncheng Lin Leo Reyzin Emily Shen Jim Sukha Drew Sutherland Eran Tromer Yiqun Lisa Yin October 27, 2008 Abstract This report describes and analyzes the MD6 hash function and is part of our submission package for MD6 as an entry in the NIST SHA-3 hash function competition1. Significant features of MD6 include: • Accepts input messages of any length up to 264 − 1 bits, and produces message digests of any desired size from 1 to 512 bits, inclusive, including the SHA-3 required sizes of 224, 256, 384, and 512 bits. • Security|MD6 is by design very conservative. We aim for provable security whenever possible; we provide reduction proofs for the security of the MD6 mode of operation, and prove that standard differential attacks against the compression function are less efficient than birthday attacks for find- ing collisions. We also show that when used as a MAC within NIST recommendedations, the keyed version of MD6 is not vulnerable to linear cryptanalysis. The compression function and the mode of operation are each shown to be indifferentiable from a random oracle under reasonable assumptions. • MD6 has good efficiency: 22.4{44.1M bytes/second on a 2.4GHz Core 2 Duo laptop with 32-bit code compiled with Microsoft Visual Studio 2005 for digest sizes in the range 160{512 bits. When compiled for 64-bit operation, it runs at 61.8{120.8M bytes/second, compiled with MS VS, running on a 3.0GHz E6850 Core Duo processor. • MD6 works extremely well for multicore and parallel processors; we have demonstrated hash rates of over 1GB/second on one 16-core system, and over 427MB/sec on an 8-core system, both for 256-bit digests. We have also demonstrated MD6 hashing rates of 375 MB/second on a typical desktop GPU (graphics processing unit) card. We also show that MD6 runs very well on special-purpose hardware. • MD6 uses a single compression function, no matter what the desired digest size, to map input data blocks of 4096 bits to output blocks of 1024 bits| a fourfold reduction. (The number of rounds does, however, increase for larger digest sizes.) The compression function has auxiliary inputs: a \key" (K), a \number of rounds" (r), a \control word" (V ), and a \unique ID" word (U). • The standard mode of operation is tree-based: the data enters at the leaves of a 4-ary tree, and the hash value is computed at the root. See Figure 2.1. This standard mode of operation is highly parallelizable. 1http://www.csrc.nist.gov/pki/HashWorkshop/index.html 2 • Since the standard MD6 mode requires storage proportional to the height of the tree, there is an alternative low-storage variant mode obtained by adjusting the optional parameter L that decreases both the storage re- quirements and the parallelizability; setting L = 0 results in a Merkle- Damg˚ard-like sequential mode of operation. • All intermediate \chaining values" passed up the tree are 1024 bits in length; the final output value is obtained by truncating the final 1024- bit compression function output to the desired length. This \wide-pipe" design makes \internal collisions" extremely unlikely. • MD6 automatically permits the computation of message authentication codes (MAC's), since the auxiliary 512-bit key input (K) to the compres- sion function may be secret. The key may alternatively be set to a random value, for randomized hashing applications. • MD6 is defined for 64-bit machines, but is very easy to implement on machines of other word sizes (e.g. 32-bit or 8-bit). • The only data operations used are XOR, AND, and SHIFT (right and left shifts by fixed amounts); all operating on 64-bit words. There are no data-dependent table lookups or other similar data-dependent operations. In hardware, each round of the compression function can be executed in constant time{only a few gate delays. • The compression function can be viewed as encryption with a fixed key (or equivalently, as applying a fixed random permutation of the message space) followed by truncation. The inner loop can be represented as an invertible non-linear feedback shift register (NLFSR). Security can be ad- justed by adjusting the number of compression function rounds. • Simplicity|the MD6 mode of operation and compression function are very simple: see Figure 2.1 for the mode of operation and Figure 2.10 for the compression operation (each Figure is one page). • Flexibility|MD6 is easily adapted for applications or analysis needing non-default parameter values, such as reduced-round versions. (Some of the detailed analyses are in our companion papers.) Contents 1 Introduction 7 1.1 NIST SHA-3 competition . .8 1.2 Overview . .8 2 MD6 Specification 9 2.1 Notation . .9 2.2 MD6 Inputs . 10 2.2.1 Message M to be hashed . 11 2.2.2 Message digest length d ................... 11 2.2.3 Key K (optional) . 11 2.2.4 Mode control L (optional) . 12 2.2.5 Number of rounds r (optional) . 13 2.2.6 Other MD6 parameters . 13 2.2.7 Naming versions of MD6 . 13 2.3 MD6 Output . 14 2.4 MD6 Mode of Operation . 14 2.4.1 A hierarchical mode of operation . 15 2.4.2 Compression function input . 16 2.4.2.1 Unique Node ID U ................. 18 2.4.2.2 Control Word V .................. 18 2.5 MD6 Compression Function . 23 2.5.1 Steps, rounds and rotations . 27 2.5.2 Intra-word Diffusion via xorshifts . 28 2.5.3 Shift amounts . 28 2.5.4 Round Constants . 28 2.5.5 Alternative representations of the compression function . 29 2.6 Summary . 29 3 Design Rationale 31 3.1 Compression function inputs . 31 3.1.1 Main inputs: message and chaining variable . 32 3.1.2 Auxiliary inputs: key, unique nodeID, control word . 33 3.2 Provable security . 33 3.3 Memory usage is less of a constraint . 34 1 CONTENTS 2 3.3.1 Larger block size . 34 3.3.2 Enabling parallelism . 35 3.4 Parallelism . 36 3.4.1 Hierarchical mode of operation . 37 3.4.2 Branching factor of four . 38 3.5 A keyed hash function . 38 3.6 Pervasive auxiliary inputs . 39 3.6.1 Pervasive key . 39 3.6.2 Pervasive location information: \position-awareness" . 39 3.6.3 Pervasive control word information . 40 3.7 Restricted instruction set . 40 3.7.1 No operations with input-dependent timing . 40 3.7.2 Few operations . 41 3.7.3 Efficiency . 41 3.8 Wide-pipe strategy . 42 3.9 Nonlinear feedback shift register . 42 3.9.1 Tap positions . 42 3.9.2 Round constants . 43 3.9.3 Intra-word diffusion operation g ............... 44 3.9.4 Constant Vector Q ...................... 44 3.10 Input symmetry . 45 3.11 Output symmetry . 45 3.12 Relation to encryption . 45 3.13 Truncation . 46 3.14 Summary . 46 4 Software Implementations 47 4.1 Software implementation strategies . 47 4.1.1 Mode of operation . 47 4.1.1.1 Layer-by-layer . 47 4.1.1.2 Data-driven tree-building . 48 4.1.2 Compression function . 49 4.2 \Standard" MD6 implementation(s) . 50 4.2.1 Reference Implementation . 50 4.2.2 Optimized Implementations . 50 4.2.2.1 Optimized 32-bit version . 50 4.2.2.2 Optimized 64-bit version . 50 4.2.3 Clean-room implementation . 51 4.3 MD6 Software Efficiency Measurement Approach . 51 4.3.1 Platforms . 51 4.3.1.1 32-bit . 51 4.3.1.2 64-bit . 52 4.3.1.3 8-bit . 52 4.4 MD6 Setup and Initialization Efficiency . 52 4.5 MD6 speed in software . 53 4.5.1 32-bit processors . 53 CONTENTS 3 4.5.2 64-bit processors . 54 4.5.3 8-bit . 55 4.5.3.0.1 Whither small processors? . 59 4.6 MD6 Memory Usage . 59 4.7 Parallel Implementations . 60 4.7.1 CILK Implementation . 60 4.7.2 GPU Implementation . 61 4.8 Summary . 67 5 Hardware Implementations 68 5.1 Hardware Implementation . 68 5.1.1 Compression Function . 69 5.1.2 Memory Control Logic . 70 5.2 FPGA . 71 5.3 ASIC/Custom . 75 5.4 Multi-core . 76 5.5 Summary . 77 6 Compression Function Security 81 6.1 Theoretical foundations . 82 6.1.1 Blockcipher-based Hash Functions . 83 6.1.2 Permutation-based hash functions and indifferentiability from random oracles . 84 6.1.3 Can MD6 generate any even permutation? . 89 6.1.4 Keyed permutation-based hash functions . 89 6.1.5 m-bit to n-bit truncation . 90 6.2 Choice of compression function constants . 90 6.2.1 Constant Q .......................... 90 6.2.2 Tap positions . 91 6.2.3 Shift amounts and properties of g .............. 92 6.2.4 Avalanche properties . 92 6.2.5 Absence of trapdoors . 92 6.3 Collision resistance . 93 6.4 Preimage Resistance . 93 6.5 Second Preimage Resistance . 94 6.6 Pseudorandomness (PRF) . 94 6.6.1 Standard statistical tests . 94 6.6.1.1 NIST Statistical Test Suite . 95 6.6.1.2 TestU01 . 95 6.6.2 Other statistical tests . 97 6.7 Unpredictability (MAC) . 99 6.8 Key-Blinding . 99 6.9 Differential cryptanalysis . 101 6.9.1 Basic definitions and assumptions .