SpeculativeSpeculative EncryptionEncryption onon GPUGPU AppliedApplied toto CryptographicCryptographic FileFile SystemsSystems Vandeir Eduardo1,2, Wagner M. Nunan Zola1, and Luis C. Erpen de Bona1 1 Federal University of Paraná 2 University of Blumenau AgendaAgenda ➢ Introduction and motivation ➢ Rationale: Cryptographic File Systems (CFSs), CBC and CTR encryption mode, file system EncFS (user space) and GPU library WAESlib ➢ CTR encryption mode applied to CFSs (in file system EncFS++ ) Generation and storage of nonces ➢ Spawning parallel encryption tasks in EncFS++ (Challenges in organization and management of encryption contexts) ➢ Experimental Performance Analysis EncFS++ ➢ Conclusions 2/31 IntroductionIntroduction andand motivationmotivation ➢ Security in data storage: especially in the era of computing in the cloud. ➢ Natural evolution: integration of encryption in File Systems: FSs → CFSs ➢ Use of symmetric block ciphers (good security/speed ratio) ➢ Problems: Larger data volumes + faster media + alternative ciphers + larger keys = increase in CPU utilization 3/31 MotivationMotivation (cont.) ➢ Wanted: Using parallel processors for the task (e.g with GPUs) (or with multicore processors) ➢ Previous study of acceleration of AES in GPU GPU kernel WAES and WAESlib: exploring CTR mode → defines priorities for generation of encryption masks ➢ Current work: “Explore advantages of CTR mode in the context of CFSs”, with parallel multicore or manycore processors: - using GPU cryptographic functions (current work) - get higher throughput with more efficient CPU usage - extend to other accelerators, multicore or heterogeneous (future work) 4/31 Cryptographic FILE Systems ➢ Integrated at different system levels: ①11 User Space: FUSE-based CFSs ②22 Kernel Space: CFSs ↔ VFS ③33 Kernel Space: Cryptographic Systems ↔ I / O Blocks 22 User space 33 Kernel space User space eCryptfseCryptfs dm-cryptdm-crypt Kernel space ApplicationApplication Storage VFSVFS FileFile system system DeviceDevice mapper mapper BlockBlock I/O I/O Storage devicedevice EncFSEncFS FUSEFUSE 11 libfuselibfuse 5/31 usually: CBC mode of operation EncryptionEncryption Clear text 1 Clear text 2 Clear text N Clear text 1 Clear text 2 Clear text N IV ➢ IV Detailed in NIST Cipher textN-1 Cipher textN-1 Key Key Key document SP 800-38A Key Key Key Encrypt Encrypt Encrypt Encrypt Encrypt ... Encrypt (AES) (AES) ... (AES) (AES) (AES) (AES) Cipher text 1 Cipher text 2 Cipher text N ➢ Sequential encryption Cipher text 1 Cipher text 2 Cipher text N (data dependency) DecryptionDecryption Cipher text 1 Cipher text 2 Cipher text N Cipher text 1 Cipher text 2 Cipher text N Key Key Key Key Key Key ➢ Decrypt Decrypt Decrypt Security requirement: Decrypt Decrypt Decrypt (AES) (AES) ... (AES) (AES) (AES) ... (AES) IV IV Cipher textN-1 necessary to use an Cipher textN-1 “unpredictable” Clear text 1 Clear text 2 Clear text N Clear text 1 Clear text 2 Clear text N Initialization Vector (IV) 6/31 Wanted: work with CTR Mode ➢ EncryptionEncryption Parallelizable Counter: 1 Counter: 2 Counter: N Counter: 1 Counter: 2 Counter: N ➢ Key Key Key Possibility of encryption Key Key Key Encrypt Encrypt Encrypt Encrypt Encrypt ... Encrypt (AES) (AES) ... (AES) Anticipation (of (AES) (AES) (AES) encryption masks) Clear text 1 Clear text 2 Clear text N Clear text 1 Clear text 2 Clear text N Cipher text 1 Cipher text 2 Cipher text N Cipher text 1 Cipher text 2 Cipher text N ➢ Security requirement: (uniqueness requirement) DecryptionDecryption Counter: 1 Counter: 2 Counter: N Counter: 1 Counter: 2 Counter: N Key Key Key necessary to use a given Key Key Key Encrypt Encrypt Encrypt Encrypt Encrypt ... Encrypt (AES) (AES) ... (AES) (key, IV) pair only once (AES) (AES) (AES) at any encryption Cipher text 1 Cipher text 2 Cipher text N Cipher text 1 Cipher text 2 Cipher text N Clear text 1 Clear text 2 Clear text N ➢ Clear text 1 Clear text 2 Clear text N IV is called “Nonce” 7/31 EncFS file system: (some Features) ➢based on FUSE → works in user space → facilitates development / testing → allows easier GPU library Integration in EncFS++ ➢ CUDA API and libfuse are in user space ➢ IF using kernel space FS module: needed an intermediate process to use CUDA API (+ complexity, + latency) 8/31 EncFS Features → based on FUSE / space user facilitates development / testing → uses OpenSSL (CPU) FileFile formatformat → file content encrypted in data blocks → uses CBC for each data block IVAIVA DataData DataData ...... DataData Header Block 0 Block 1 Block n vK = volume Key Header Block 0 Block 1 Block n IV use unpredictability requirement: IVV = Volume IV IVV = Volume IV * data block IV calculated dynamically with IVA = File IV encryption hash (no need to store) * reusable in block rewriting IVB = data Block IV data Block IV (IVB) = HMAC_CTX (vK, IVV || (NumBlock ⊕ IVA)) 9/31 GPUGPU EncryptionEncryption AccelerationAcceleration ➢ Extensively studied: for varous symmetric ciphers such as AES, Blowfish, IDEA, Camellia, etc. ➢ Related work: acceleration of cryptographic functions in some applications: → User space: Engine-CUDA, CrystalGPU, CRSFS → kernel space: OCF, Gdev, GPUStore ➢ Usually: using CBC+GPU → usually only compensates for larger requests (> 16 KiB) ➢ Applied to CFSs: no previous work have exploited the benefits of CTR mode 10/31 WhyWhy CTR?CTR? ➢ CTR Mode: → parallelizable → allows speculative encryption (creation encryption masks ahead of time) → XOR on CPU (avoids CPU → GPU data transfer) → As safe as CBC ➢ Previous library available in previous work: WAESlib → Reduces GPU processing complexity → Aggregation of small (4 KiB) contexts : ∙ fewer WAES kernel activations ∙ higher throughput (GPU → CPU) ∙ more control in the order of production of masks (with priorities) 11/31 ChallengesChallenges ofof usingusing CTRCTR inin CFSsCFSs ➢ Each recording and rewriting of a block requires a new nonce (due to: the uniqueness requirement) ➢ Problem: Necessary to store a nonce per block (same unique nonce used in encryption is necessary in decryption) ➢ Overhead of nonce storage could negatively impact CFS performance Nonce storage format AND Access mechanism AND granularity are important for performance 12/31 Nnodes: how nonce Nodes are stored in EncFS++ 11 22 Nonce Nonce Nonce CFS Global Number of Occupation map ... Counter nnodes used Node 0 Node 1 Node n Nonce nodes file format (loaded when CFS 128 bits 32 bits 524.128 bits 260 260 is mounted) bytes bytes 64 KiB Nonce Nonce node format 11 22 Value obtained from CFS CTR Inode Nonce Nonce Nonce internal ... Global Counter counter number 0 1 15 128 bits 32 bits 16 bytes 16 bytes bits reserved for CTR internal counter Nonce Nonce Nonce Exclusive nonces file format ... Group 0 Group 1 Group N Only for files > 64 KiB (16 * 4 KiB) (loaded when file is opened) 4096 bytes 4096 bytes (256 nonces) (256 nonces) 13/31 Challenges in using speculative encryption in CFSs ➢ Managing encryption contexts → How to organize the encryption contexts within the FS application? → How to use these contexts in the different CFS operations? ➢ When is the best time to trigger the generation of encryption masks (define contexts)? ➢ How to take advantage of the priority feature? 14/31 Write Context pool: maintained for encryption+writing CFS Global Counter value: 0 CFS Global Counter value: 256 0 256 512 768 1024 1280 1536 1792 256 512 768 1024 1280 1536 1792 2048 0 1 2 3 4 5 6 7 1 2 3 4 5 6 7 0 Nonces used Pool beginning Nonce used in Pool beginning indicator in ahead of indicator production of a new (next mask to be consumed) time masks (next mask to be mask after a mask production Context indexes consumed) consumption Before a block encryption After aa blockblock encryptionencryption encrypt+write op ➢ Used for sequential and random writing (only one write context POOL needed per CFS) ➢ Contexts initially defined at CFS mount operation ➢ Contexts in this POOL are redefined as masks are consumed (uses lower priority) ➢ Implemented as a virtual circular queue (no storage → performance) 15/31 Context pool for decryption / read (seq.) 0 1 2 3 4 5 6 7 8 9 n File blocks Contexts 0 256 512 768 1024 1280 1536 1792 Context indexes n Contexts (“virtua”l) with nonces 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 8 9 New masks Masks being 256 512 768 1024 1280 1536 1792 2048 being consumed 1 2 3 4 5 6 7 0 produced 0 1 2 3 4 5 6 7 8 9 512 768 1024 1280 1536 1792 2048 2304 2 3 4 5 6 7 0 1 Window move direction rotation on “indexes” (modulo window size) ➢ Used for sequential and random reading (1 per file) ➢ Contexts initially defined in each file open operation (decreasing priority according to position) ➢ Contexts redefined as masks are consumed (uses lower priority) 16/31 Context pool for decryption / read (random) → TotalTotal windowwindow displacementdisplacement restarart all contexts in pool (hygher speculation overhead) (x-y)>z z New window Old window 9472 9728 9984 10240 10496 10752 11008 11264 12032 12288 12544 12800 13056 13312 13568 13824 5 6 7 0 1 2 3 4 7 0 1 2 3 4 5 6 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 y New start position x Old start position Window move direction Reused masks New masks produced PartialPartial WindowWindow ShiftShift (y-x)<=z 10496 10752 11008 11264 11520 11776 12032 12288 New window 1 2 3 4 5 6 7 0 Old window 9728 9984 10240 10496 10752 11008 11264 11520 6 7 0 1 2 3 4 5 Use of 36 37 38 39 40 41 42 43 44 45 46 47 48 49 Use of x y priorities!priorities! Old start position New start position Window move direction 17/31 PerformancePerformance AnalysisAnalysis ➢Performance