JFFS and JFFS2
Total Page:16
File Type:pdf, Size:1020Kb
EE382N-4 Advanced Microcontroller Systems Flash Memory Technology and Flash Based File Systems Mark McDermott Spring 2021 EE382N-4 Class Notes File Systems for Embedded Linux EE382N-4 Class Notes Embedded file systems § RAMDISKS – Provide a mechanism for Linux to emulates a hard disk using volatile memory • Need a ramdisk when you do not have traditional storage device such as a hard disk or flash – Provide a mechanism by which you can load an actual file system into memory and use it as root file system – ‘initrd’ provides a mechanism by which a boot loader loads kernel image along with a root file system into memory 3/23/21 EE382N-4 Class Notes 3 Embedded file systems (cont) § RAMFS (RAM File System) – Typically, an embedded system will have files that are not useful after a reboot. These file would be normally stored in /tmp directory – Storing these files in memory is a better option than storing them in non- volatile memory because writes are expensive and wear out the flash. 3/23/21 EE382N-4 Class Notes 4 Embedded file systems (cont) § CRAMFS (Compressed RAM File System) – Useful file system for flash storage – Introduced in the 2.4 kernel – High-compression read only “regular” file system • Accessed via buffer cache using block device holding actual data – Enabled using MTD block device driver mode – CRAMFS uses ‘zlib’ routines for compression and it does compression for every 4 KB block – Using a program called mkcramfs, we can create CRAMFS image that needs to be burned in flash 3/23/21 EE382N-4 Class Notes 5 Embedded file systems (cont) § Journaling Flash File System – JFFS and JFFS2 – A write to a file creates a log that records the offset in the file to where the data is written and size of data written along with actual data – When a file needs to be read, the logs are played back and using the data size and offset, the file gets recreated – Requires garbage collection. As time proceeds, certain logs become obsolete partially and/or totally and must be cleaned. 3/23/21 EE382N-4 Class Notes 6 Embedded file systems (cont) § Main features of the JFFS2 file system – Management of erase blocks • Erase blocks are placed in three list: clean, dirty and free. Clean list contains only valid logs. Dirty list contains obsolete logs to be purged when garbage collection is called. Free list contains no logs and will be used for storing new logs. – Garbage collection • Work as a separate thread when gets started when we mount a JFFS2 file system • For every 99 out of 100 times, this thread will pick up a block from dirty list and for the other 1 out of 100 will pick up a block from clean list to ensure wear leveling • JFFS2 reserves 5 blocks for doing garbage collection – Compression • JFFS2 uses various compression routines including zlib and rubin – JFSS and JFSS2 file system image is created using mkfs.jffs and mkfs.jffs2 3/23/21 EE382N-4 Class Notes 7 Embedded file systems (cont) § NFS (Network File System) – Popular desktop file system EXT2 and EXT3 are exportable to an embedded system using NFS • Access it as a root file system on embedded system – During debugging stage, often developers can make changes to root file system. In such a case writing to flash can be costly (because flash has limited write cycles) and time consuming. NFS can be a good choice provided the network driver is ready – With NFS you do not have any size restrictions because all the storage is done on the remote server. – Linux kernel provides a mechanism for automatically mounting a file system using NFS at boot time using following steps • config options CONFIG_NFS_FS, which allow you to mount a file system from a remote server and CONFIG_ROOT_NFS, which allows you to use NFS as root file system needs to be turned on at build time 3/23/21 EE382N-4 Class Notes 8 Embedded file systems (cont) § PROC file system – A logical file system used by kernel to export its information to external world – Proc file system can be used for system monitoring, debugging and tuning – Proc files are created on the fly when they get opened. – Proc files can be read-only or read-write depending on the information exported by files – Read-only proc files can be used to get some kernel status and cannot be modified at runtime. (ex) process status information – Read-write proc files contain information that can be changed at runtime. (ex) TCP keep-alive time. 3/23/21 EE382N-4 Class Notes 9 The EXT Family § EXT was written by Remy Card in 1992 and was loosely based on UFS (Unix File System). – It had a 2GB file size max. Th e original Linu x file syste m wa s § EXT2 replaced EXT in 1993. th e m inix file syste m . § EXT3 emerged in 2001 § EXT4 was available in 2006 3/23/21 EE382N-4 Class Notes 10 The venerable EXT2 § The file size is dependent on the block size. A 1KB block size allows a 16GB file size § By increasing the block size you can have files up to 2TB § Directories are not indexed Inode # Access Extended Control List Attribute § File compression can be added to the file system Direct/Indirect Time stamp(3) Number of Disk Blocks Access,change, blocks § Files and directories are modification stored in the Inodes. File Deletion File generation File Size Time number § Time stamps are 32-bits which mean they go negative File Type Group Number of links 1/18/2038 Owner Permissions Status flags 3/23/21 EE382N-4 Class Notes 11 EXT3 adds journaling to EXT2 § Stephen Tweedie introduced EXT3 in 1998 and was included in 2.4.15 kernel in 2001. § EXT3 is an in-place upgrade to EXT2. § This is essentially still an EXT2 file system. § The big benefit is that a journal is maintained in metadata for subsequent FSCKs 3/23/21 EE382N-4 Class Notes 12 EXT4 – major upgrade to EXT2/3 § EXT4 was originally part of the Cluster File System § Ted Tso brought EXT4 into the 2.6.28 kernel for development in 2008. § Google used ext4 for its storage systems in 2010, and enabled it on Android later that year. 3/23/21 EE382N-4 Class Notes 13 EXT4 – New features § Large file systems: – Volumes up to 1 exbibyte (EiB) File sizes up to 16 tebibytes (TiB) – Note that Red Hat recommends using XFS for volumes larger than 100TB. System uses 48 bit addressing – Time stamp has a date limitation of April 25, 2514. – Online defragmentation § Additional features – Inode is 256KB where EXT3 was 128KB – Persistent pre-allocation. Extra space is reserved. – Backwards compatibility. An EXT2 or EXT3 volume can be mounted as an EXT4 volume. This will slightly improve performance. – Extent-based storage 3/23/21 EE382N-4 Class Notes 14 EXT4 – Performance Features § EXT4 has a number of performance- related features – Multi-block allocation where multiple blocks can be written at once – Delayed Allocation – This allows multiple block writes, reduced fragmentation, reduced processor time for files that are short-term or temporary. 3/23/21 EE382N-4 Class Notes 15 BTRFS1 – The Linux file system of the future § BTRFS is a new copy on write (CoW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration. Jointly developed at multiple companies, BTRFS is licensed under the GPL and open for contribution from anyone. § BTRFS was initially developed by Chris Mason who had experience working on the Reiser File System § Currently default in OpenSuse 13.2. Will be the default in Fedora 23 1 BTRFS, an abbreviation for b-tree file system, is a file system based on the copy-on-write principle, initially designed at Oracle Corporation for use in Linux. EE382N-4 Class Notes 16 BTRFS Details § Max. file size: 16 EiB § File system permissions: POSIX and ACL § Max. number of files: 264 § File allocation: Extents § Supported operating systems: Linux, ReactOS § Max. volume size: 16 EiB 3/23/21 EE382N-4 Class Notes 17 BTRFS - Features § BTRFS has the following features – Extent-based storage – 16 Exibyte maximum file size (2^64) Space efficient packing for small files. Space efficient indexed directories Dynamic inode allocation – Writeable and read-only snapshots Subvolumes § Checksums on data and metadata Compression § Integrated multiple device support – Striping – Mirroring – Striping + mirroring – Striping with single and dual parity § SSD awareness § Efficient incremental backup EE382N-4 Class Notes 18 BTRFS - Features § Conversion of EXT3 or EXT4 file systems § Seed devices – essentially a template § Sub-volume aware quota support § Send/receive sub volume changes – Efficient incremental filesystem mirroring § Batch deduplication (happens after writing) EE382N-4 Class Notes 19 Flash Memory Technology EE382N-4 Class Notes 10/15/193/23/21 EE382N-4 Class Notes 21191921 Semiconductor Memory Types Non-Volatile Volatile Semiconductor Memory RAM DRAM SRAM Floating Gate Nitride Emerging ROM & Fuse “NV Ram” Ferro-electric Magnetic Phase Change Polymer “Unified Mem” Courtesy Intel 3/23/21 EE382N-4 Class Notes 22 Non-Volatile Memory History MRAM, Phase Change, ?? Polymer 2000’s Ferro-electric 1988 Nitride Storage 2002 MLC NOR Flash 1995 NOR Flash 1988 AND, DiNOR Flash 1990’s EPROM 1971 MLC NAND Flash 1996 NAND Flash 1985 EEPROM Bipolar ROMS/PROMS 1980 àLate 60’s 1970 1980 1990 2000 MLC = Multi-Level Cell Courtesy Intel 3/23/21 EE382N-4 Class Notes 23 Non-Volatile Memory Terminology § Program: Store charge on a floating gate § Erase: Remove charge from the floating gate § Data Retention: The longest time for the NVM to keep the stored information § Endurance: The number of Program/Erase Cycles the memory can withstand § Disturbs: A mechanism that corrupts the stored data in the memory cell § Memory Cell: Device that contains the memory storage element § Memory Array: Arrangement of Memory cells Courtesy Intel 3/23/21 EE382N-4 Class Notes 24 Flash memory § Extension of EEPROM – Same floating gate principle – Same write ability and storage permanence § Fast erase – Large blocks of memory erased at once, rather than one word at a time – Blocks typically several thousand bytes large § Flash memory wears out during writing.