DOCSLIB.ORG

Administering the Recoverable File System on UNIX

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Administering the Recoverable File System on UNIX C:\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\RFS\ARFSTITL.fm March 8, 2010 4:20 pm Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta UniData Administering the Recoverable File System on UNIX UDT-720-RFSU-1 C:\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\RFS\ARFSTITL.fm March 8, 2010 4:20 pm Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Notices Edition Publication date: June 2008 Book number: UDT-720-RFSU-1 Product version: UniData 7.2 Copyright © Rocket Software, Inc. 1988-2008. All Rights Reserved. Trademarks The following trademarks appear in this publication: Trademark Trademark Owner Rocket Software™ Rocket Software, Inc. Dynamic Connect® Rocket Software, Inc. RedBack® Rocket Software, Inc. SystemBuilder™ Rocket Software, Inc. UniData® Rocket Software, Inc. UniVerse™ Rocket Software, Inc. U2™ Rocket Software, Inc. U2.NET™ Rocket Software, Inc. U2 Web Development Environment™ Rocket Software, Inc. wIntegrate® Rocket Software, Inc. Microsoft® .NET Microsoft Corporation Microsoft® Office Excel®, Outlook®, Word Microsoft Corporation Windows® Microsoft Corporation Windows® 7 Microsoft Corporation Windows Vista® Microsoft Corporation Java™ and all Java-based trademarks and logos Sun Microsystems, Inc. UNIX® X/Open Company Limited ii Administering the Recoverable File System on UNIX The above trademarks are property of the specified companies in the United States, other countries, or both. All other products or services mentioned in this document may be covered by the trademarks, service marks, or product names as designated by the companies who own or market them. License agreement This software and the associated documentation are proprietary and confidential to Rocket Software, Inc., are furnished under license, and may be used and copied only in accordance with the terms of such license and with the inclusion of the copyright notice. This software and any copies thereof may not be provided or otherwise made available to any other person. No title to or ownership of the software and associated documentation is hereby transferred. Any unauthorized use or reproduction of this software or documentation may be subject to civil or criminal liability. The information in the software and documentation is subject to change and should not be construed as a commitment by Rocket Software, Inc. Restricted rights notice for license to the U.S. Government: Use, reproduction, or disclosure is subject to restrictions as stated in the “Rights in Technical Data- General” clause (alternate III), in FAR section 52.222-14. All title and ownership in this computer software remain with Rocket Software, Inc. Note This product may contain encryption technology. Many countries prohibit or restrict the use, import, or export of encryption technologies, and current use, import, and export regulations should be followed when exporting this product. Please be aware: Any images or indications reflecting ownership or branding of the product(s) documented herein may or may not reflect the current legal ownership of the intellectual property rights associated with such product(s). All right and title to the product(s) documented herein belong solely to Rocket Software, Inc. and its subsidiaries, notwithstanding any notices (including screen captures) or any other indications to the contrary. Contact information Rocket Software 275 Grove Street Suite 3-410 Newton, MA 02466-2272 USA Tel: (617) 614-4321 Fax: (617) 630-7100 Web Site: www.rocketsoftware.com Administering the Recoverable File System on UNIX iii Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Table of Contents Table of Contents Chapter 1 Introduction to the Recoverable File System (RFS) RFS System Requirements . 1-3 Disk Space . 1-3 Memory . 1-4 Recommended Knowledge Base . 1-6 Overview of RFS . 1-7 ACID Qualities . 1-7 System Failures . 1-8 Media Failures . 1-9 Enabling RFS. 1-9 RFS Architecture . 1-10 RFS Components . 1-11 Logging . 1-12 Log Files . 1-12 Log File Overflow . 1-13 Archiving . 1-15 Archive Files . 1-15 Automatic Backup of Archive Files . 1-15 Synchronizing Backups with Archive Files . 1-16 Archive Configuration Table . 1-16 Creating and Converting Recoverable Files . 1-17 Creating a Recoverable File . 1-17 Converting to a Recoverable File . 1-18 Crash Recovery. 1-19 Recovering from a Media Crash . 1-19 Monitoring and Tuning RFS. 1-21 Chapter 2 RFS Commands and Daemons cntl_install Command. 2-3 Creating a Recoverable File . 2-6 :\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\RFS\ARFSTOC.fm (bookTOC.template) March 8 2010 4:16 pm Disabling a Recoverable File. 2-7 forcecp Command . 2-8 mediarec Command . 2-9 startud Command . 2-12 Example . 2-13 udfile Command . 2-14 Daemons for the Recoverable File System . 2-15 sm . 2-15 tm . 2-16 bimglog. 2-16 aimglog. 2-16 archive . 2-16 ar_backupd . 2-16 sync . 2-17 Chapter 3 Configuration Steps for Logging How Logging Works . 3-3 About Log Files . 3-3 About Checkpoints . 3-4 How to Turn On and Configure Logging . 3-6 Chapter 4 Configuration Steps for Archiving How Archiving Works. 4-3 How to Turn On and Configure Archiving . 4-4 Managing Archive Backup . 4-16 Backing Up Archives Manually . 4-16 Backing Up Archives Automatically . 4-17 Chapter 5 Creating and Configuring Recoverable Files Converting Nonrecoverable Files to Recoverable Files . 5-3 Creating New Recoverable Files . 5-6 Creating a List of Recoverable Files . 5-8 Special Considerations for Recoverable Files . 5-9 Chapter 6 System Crash Recovery System Crash Recovery . 6-3 Chapter 7 Media Crash Recovery Media Crash Recovery . 7-3 Data Lost, Logs, and Archives Unaffected . 7-4 Table of Contents v Data and Archive Files Unaffected, Logs Lost. 7-11 Data and Log Files Unaffected, Archives Lost. 7-13 Data and Logs Lost, Archives Unaffected . 7-15 Data and Archives Lost, Logs Unaffected . 7-22 Logs and Archives Lost, Data Unaffected . 7-29 Disk Containing /usr/ud72/include Lost . 7-30 Chapter 8 RFS Configuration Parameters UniData Configuration Parameters . 8-3 Modifying udtconfig Parameters . 8-7 RFS Configuration Parameters . 8-10 Chapter 9 Monitoring and Tuning The sysmon Utility . 9-3 sysmon Fields and Values . 9-4 Performance Tips . 9-11 Tuning N_PUT and N_BIG . 9-12 Adjusting the Log Files . 9-13 Adjusting the Archive Files . 9-13 Tuning CM_SLEEP . 9-14 RFS File Open Performance . 9-15 How RFS Tracks Open Files. 9-15 Tuning RFS Open File Performance . 9-15 Chapter 10 Troubleshooting RFS Possible Errors . 10-4 Failure of UniData to Start . 10-4 File Log Size Too Small . 10-4 Inadequate Number of Message Queues Defined . 10-5 Value of SYS_PV Changed in udtconfig . 10-6 Process Errors . 10-7 Values of N_TMQ and N_PGQ Are Zero. 10-7 UniData Daemon Killed . 10-8 Errors During Processing . 10-9 Archive Files Are Full. 10-9 aimglog and bimglog Executables Have Been Removed . 10-9 Parameter Limits Exceeded. 10-10 MAX_OPEN_FILE . 10-10 N_AFT . 10-10 BPF_NFILES . 10-10 vi Administering the Recoverable File System on UNIX Files Are Not Being Treated as Recoverable . 10-11 File is Not Defined as Recoverable . 10-11 SB_FLAG Turned Off . 10-11 Recoverable File System Not Licensed . 10-12 Warning Messages . 10-13 Log Files Are Too Small . 10-13 Table of Contents vii Chapter Introduction to the Recoverable File System 1 (RFS) RFS System Requirements. 1-3 Disk Space . 1-3 Memory . 1-4 Recommended Knowledge Base . 1-6 Overview of RFS. 1-7 ACID Qualities . 1-7 System Failures . 1-8 Media Failures . 1-9 Enabling RFS . 1-9 RFS Architecture. 1-10 RFS Components . 1-11 Logging . 1-12 Log Files . 1-12 Log File Overflow . 1-13 Archiving . 1-15 Archive Files . 1-15 Automatic Backup of Archive Files . 1-16 Synchronizing Backups with Archive Files . 1-16 Archive Configuration Table. 1-16 Creating and Converting Recoverable Files . 1-17 Creating a Recoverable File . 1-17 Converting to a Recoverable File . 1-18 Crash Recovery . 1-20 Recovering from a Media Crash . 1-20 Monitoring and Tuning RFS . 1-22 This document describes the Recoverable File System (RFS). RFS functions and utilities are designed to protect UniData files against system or media failures. RFS also supports UniData Transaction Processing semantics to provide the ACID properties (atomicity, consistency, isolation, and durability). This document does not cover the use of Transaction Processing semantics. For information on Transaction Processing, see Developing UniBasic Applications. For step-by-step information on configuration and use of RFS, see the chapters following this introduction. The purpose of this chapter is to describe the concepts associated with RFS. This chapter contains the following: RFS Requirements Overview of RFS RFS Architecture (graphic) RFS Components 1-2 RFS System Requirements If you plan to run the Recoverable File System, you need additional space and memory. The amount of additional space and memory depends on the type of platform you are running. UniData can only provide initial recommendations. The initial recommendations are based on baseline tests on a variety of platforms, with 20 users, record sizes ranging from 70 to 2,500 bytes, modulo 200, block size multiplier 3, and hash type 0. The recommendations discussed in the following sections are based on the test results. Check your installation media and make sure it matches your UNIX version. Refer to Installing and Licensing UniData Products for detailed information about installing UniData. Disk Space On most systems, you must have at least 125 MB of free disk space under one mount point for the UniData installation to run. Depending on the type of installation, some space—as much as 10 MB —may be released after installation.
Load more

Recommended publications
  • Administering Unidata on UNIX Platforms
    C:\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\ADMINUNIX\ADMINUNIXTITLE.fm March 5, 2010 1:34 pm Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta UniData Administering UniData on UNIX Platforms UDT-720-ADMU-1 C:\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\ADMINUNIX\ADMINUNIXTITLE.fm March 5, 2010 1:34 pm Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Notices Edition Publication date: July, 2008 Book number: UDT-720-ADMU-1 Product version: UniData 7.2 Copyright © Rocket Software, Inc. 1988-2010. All Rights Reserved. Trademarks The following trademarks appear in this publication: Trademark Trademark Owner Rocket Software™ Rocket Software, Inc. Dynamic Connect® Rocket Software, Inc. RedBack® Rocket Software, Inc. SystemBuilder™ Rocket Software, Inc. UniData® Rocket Software, Inc. UniVerse™ Rocket Software, Inc. U2™ Rocket Software, Inc. U2.NET™ Rocket Software, Inc. U2 Web Development Environment™ Rocket Software, Inc. wIntegrate® Rocket Software, Inc. Microsoft® .NET Microsoft Corporation Microsoft® Office Excel®, Outlook®, Word Microsoft Corporation Windows® Microsoft Corporation Windows® 7 Microsoft Corporation Windows Vista® Microsoft Corporation Java™ and all Java-based trademarks and logos Sun Microsystems, Inc. UNIX® X/Open Company Limited ii SB/XA Getting Started The above trademarks are property of the specified companies in the United States, other countries, or both. All other products or services mentioned in this document may be covered by the trademarks, service marks, or product names as designated by the companies who own or market them. License agreement This software and the associated documentation are proprietary and confidential to Rocket Software, Inc., are furnished under license, and may be used and copied only in accordance with the terms of such license and with the inclusion of the copyright notice.
    [Show full text]
  • DC Console Using DC Console Application Design Software
    DC Console Using DC Console Application Design Software DC Console is easy-to-use, application design software developed specifically to work in conjunction with AML’s DC Suite. Create. Distribute. Collect. Every LDX10 handheld computer comes with DC Suite, which includes seven (7) pre-developed applications for common data collection tasks. Now LDX10 users can use DC Console to modify these applications, or create their own from scratch. AML 800.648.4452 Made in USA www.amltd.com Introduction This document briefly covers how to use DC Console and the features and settings. Be sure to read this document in its entirety before attempting to use AML’s DC Console with a DC Suite compatible device. What is the difference between an “App” and a “Suite”? “Apps” are single applications running on the device used to collect and store data. In most cases, multiple apps would be utilized to handle various operations. For example, the ‘Item_Quantity’ app is one of the most widely used apps and the most direct means to take a basic inventory count, it produces a data file showing what items are in stock, the relative quantities, and requires minimal input from the mobile worker(s). Other operations will require additional input, for example, if you also need to know the specific location for each item in inventory, the ‘Item_Lot_Quantity’ app would be a better fit. Apps can be used in a variety of ways and provide the LDX10 the flexibility to handle virtually any data collection operation. “Suite” files are simply collections of individual apps. Suite files allow you to easily manage and edit multiple apps from within a single ‘store-house’ file and provide an effortless means for device deployment.
    [Show full text]
  • The Ifplatform Package
    The ifplatform package Original code by Johannes Große Package by Will Robertson http://github.com/wspr/ifplatform v0.4a∗ 2017/10/13 1 Main features and usage This package provides the three following conditionals to test which operating system is being used to run TEX: \ifwindows \iflinux \ifmacosx \ifcygwin If you only wish to detect \ifwindows, then it does not matter how you load this package. Note then that use of (Linux or Mac OS X or Cygwin) can then be detected with \ifwindows\else. If you also wish to determine the difference between which Unix-variant you are using (i.e., also detect \iflinux, \ifmacosx, and \ifcygwin) then shell escape must be enabled. This is achieved by using the -shell-escape command line option when executing LATEX. If shell escape is not enabled, \iflinux, \ifmacosx, and \ifcygwin will all return false. A warning will be printed in the console output to remind you in this case. ∗Thanks to Ken Brown, Joseph Wright, Zebb Prime, and others for testing this package. 1 2 Auxiliary features \ifshellescape is provided as a conditional to test whether shell escape is active or not. (Note: new versions of pdfTEX allow you to query shell escape with \ifnum\pdfshellescape>0 , and the pdftexcmds package provides the wrapper \pdf@shellescape which works with X TE EX, pdfTEX, and LuaTEX.) Also, the \platformname command is defined to expand to a macro that represents the operating system. Default definitions are (respectively): \windowsname ! ‘Windows’ \notwindowsname ! ‘*NIX’ (when shell escape is disabled) \linuxname ! ‘Linux’ \macosxname ! ‘Mac OS X’ \cygwinname ! ‘Cygwin’ \unknownplatform ! whatever is returned by uname E.g., if \ifwindows is true then \platformname expands to \windowsname, which expands to ‘Windows’.
    [Show full text]
  • A First Course to Openfoam
    Basic Shell Scripting Slides from Wei Feinstein HPC User Services LSU HPC & LON [email protected] September 2018 Outline • Introduction to Linux Shell • Shell Scripting Basics • Variables/Special Characters • Arithmetic Operations • Arrays • Beyond Basic Shell Scripting – Flow Control – Functions • Advanced Text Processing Commands (grep, sed, awk) Basic Shell Scripting 2 Linux System Architecture Basic Shell Scripting 3 Linux Shell What is a Shell ▪ An application running on top of the kernel and provides a command line interface to the system ▪ Process user’s commands, gather input from user and execute programs ▪ Types of shell with varied features o sh o csh o ksh o bash o tcsh Basic Shell Scripting 4 Shell Comparison Software sh csh ksh bash tcsh Programming language y y y y y Shell variables y y y y y Command alias n y y y y Command history n y y y y Filename autocompletion n y* y* y y Command line editing n n y* y y Job control n y y y y *: not by default http://www.cis.rit.edu/class/simg211/unixintro/Shell.html Basic Shell Scripting 5 What can you do with a shell? ▪ Check the current shell ▪ echo $SHELL ▪ List available shells on the system ▪ cat /etc/shells ▪ Change to another shell ▪ csh ▪ Date ▪ date ▪ wget: get online files ▪ wget https://ftp.gnu.org/gnu/gcc/gcc-7.1.0/gcc-7.1.0.tar.gz ▪ Compile and run applications ▪ gcc hello.c –o hello ▪ ./hello ▪ What we need to learn today? o Automation of an entire script of commands! o Use the shell script to run jobs – Write job scripts Basic Shell Scripting 6 Shell Scripting ▪ Script: a program written for a software environment to automate execution of tasks ▪ A series of shell commands put together in a file ▪ When the script is executed, those commands will be executed one line at a time automatically ▪ Shell script is interpreted, not compiled.
    [Show full text]
  • Linking + Libraries
    LinkingLinking ● Last stage in building a program PRE- COMPILATION ASSEMBLY LINKING PROCESSING ● Combining separate code into one executable ● Linking done by the Linker ● ld in Unix ● a.k.a. “link-editor” or “loader” ● Often transparent (gcc can do it all for you) 1 LinkingLinking involves...involves... ● Combining several object modules (the .o files corresponding to .c files) into one file ● Resolving external references to variables and functions ● Producing an executable file (if no errors) file1.c file1.o file2.c gcc file2.o Linker Executable fileN.c fileN.o Header files External references 2 LinkingLinking withwith ExternalExternal ReferencesReferences file1.c file2.c int count; #include <stdio.h> void display(void); Compiler extern int count; int main(void) void display(void) { file1.o file2.o { count = 10; with placeholders printf(“%d”,count); display(); } return 0; Linker } ● file1.o has placeholder for display() ● file2.o has placeholder for count ● object modules are relocatable ● addresses are relative offsets from top of file 3 LibrariesLibraries ● Definition: ● a file containing functions that can be referenced externally by a C program ● Purpose: ● easy access to functions used repeatedly ● promote code modularity and re-use ● reduce source and executable file size 4 LibrariesLibraries ● Static (Archive) ● libname.a on Unix; name.lib on DOS/Windows ● Only modules with referenced code linked when compiling ● unlike .o files ● Linker copies function from library into executable file ● Update to library requires recompiling program 5 LibrariesLibraries ● Dynamic (Shared Object or Dynamic Link Library) ● libname.so on Unix; name.dll on DOS/Windows ● Referenced code not copied into executable ● Loaded in memory at run time ● Smaller executable size ● Can update library without recompiling program ● Drawback: slightly slower program startup 6 LibrariesLibraries ● Linking a static library libpepsi.a /* crave source file */ … gcc ..
    [Show full text]
  • “Log” File in Stata
    Updated July 2018 Creating a “Log” File in Stata This set of notes describes how to create a log file within the computer program Stata. It assumes that you have set Stata up on your computer (see the “Getting Started with Stata” handout), and that you have read in the set of data that you want to analyze (see the “Reading in Stata Format (.dta) Data Files” handout). A log file records all your Stata commands and output in a given session, with the exception of graphs. It is usually wise to retain a copy of the work that you’ve done on a given project, to refer to while you are writing up your findings, or later on when you are revising a paper. A log file is a separate file that has either a “.log” or “.smcl” extension. Saving the log as a .smcl file (“Stata Markup and Control Language file”) keeps the formatting from the Results window. It is recommended to save the log as a .log file. Although saving it as a .log file removes the formatting and saves the output in plain text format, it can be opened in most text editing programs. A .smcl file can only be opened in Stata. To create a log file: You may create a log file by typing log using ”filepath & filename” in the Stata Command box. On a PC: If one wanted to save a log file (.log) for a set of analyses on hard disk C:, in the folder “LOGS”, one would type log using "C:\LOGS\analysis_log.log" On a Mac: If one wanted to save a log file (.log) for a set of analyses in user1’s folder on the hard drive, in the folder “logs”, one would type log using "/Users/user1/logs/analysis_log.log" If you would like to replace an existing log file with a newer version add “replace” after the file name (Note: PC file path) log using "C:\LOGS\analysis_log.log", replace Alternately, you can use the menu: click on File, then on Log, then on Begin.
    [Show full text]
  • Encryption Introduction to Using 7-Zip
    IT Services Training Guide Encryption Introduction to using 7-Zip It Services Training Team The University of Manchester email: [email protected] www.itservices.manchester.ac.uk/trainingcourses/coursesforstaff Version: 5.3 Training Guide Introduction to Using 7-Zip Page 2 IT Services Training Introduction to Using 7-Zip Table of Contents Contents Introduction ......................................................................................................................... 4 Compress/encrypt individual files ....................................................................................... 5 Email compressed/encrypted files ....................................................................................... 8 Decrypt an encrypted file ..................................................................................................... 9 Create a self-extracting encrypted file .............................................................................. 10 Decrypt/un-zip a file .......................................................................................................... 14 APPENDIX A Downloading and installing 7-Zip ................................................................. 15 Help and Further Reference ............................................................................................... 18 Page 3 Training Guide Introduction to Using 7-Zip Introduction 7-Zip is an application that allows you to: Compress a file – for example a file that is 5MB can be compressed to 3MB Secure the
    [Show full text]
  • Red Hat Jboss Data Grid 7.2 Data Grid for Openshift
    Red Hat JBoss Data Grid 7.2 Data Grid for OpenShift Developing and deploying Red Hat JBoss Data Grid for OpenShift Last Updated: 2019-06-10 Red Hat JBoss Data Grid 7.2 Data Grid for OpenShift Developing and deploying Red Hat JBoss Data Grid for OpenShift Legal Notice Copyright © 2019 Red Hat, Inc. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/ . In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries. Node.js ® is an official trademark of Joyent.
    [Show full text]
  • 21Files2.Pdf
    Here is a portion of a Unix directory tree. The ovals represent files, the rectangles represent directories (which are really just special cases of files). A simple implementation of a directory consists of an array of pairs of component name and inode number, where the latter identifies the target file’s inode to the operating system (an inode is data structure maintained by the operating system that represents a file). Note that every directory contains two special entries, “.” and “..”. The former refers to the directory itself, the latter to the directory’s parent (in the case of the slide, the directory is the root directory and has no parent, thus its “..” entry is a special case that refers to the directory itself). While this implementation of a directory was used in early file systems for Unix, it suffers from a number of practical problems (for example, it doesn’t scale well for large directories). It provides a good model for the semantics of directory operations, but directory implementations on modern systems are more complicated than this (and are beyond the scope of this course). Here are two directory entries referring to the same file. This is done, via the shell, through the ln command which creates a (hard) link to its first argument, giving it the name specified by its second argument. The shell’s “ln” command is implemented using the link system call. Here are the (abbreviated) contents of both the root (/) and /etc directories, showing how /unix and /etc/image are the same file. Note that if the directory entry /unix is deleted (via the shell’s “rm” command), the file (represented by inode 117) continues to exist, since there is still a directory entry referring to it.
    [Show full text]
  • Steganography and Vulnerabilities in Popular Archives Formats.| Nyxengine Nyx.Reversinglabs.Com
    Hiding in the Familiar: Steganography and Vulnerabilities in Popular Archives Formats.| NyxEngine nyx.reversinglabs.com Contents Introduction to NyxEngine ............................................................................................................................ 3 Introduction to ZIP file format ...................................................................................................................... 4 Introduction to steganography in ZIP archives ............................................................................................. 5 Steganography and file malformation security impacts ............................................................................... 8 References and tools .................................................................................................................................... 9 2 Introduction to NyxEngine Steganography1 is the art and science of writing hidden messages in such a way that no one, apart from the sender and intended recipient, suspects the existence of the message, a form of security through obscurity. When it comes to digital steganography no stone should be left unturned in the search for viable hidden data. Although digital steganography is commonly used to hide data inside multimedia files, a similar approach can be used to hide data in archives as well. Steganography imposes the following data hiding rule: Data must be hidden in such a fashion that the user has no clue about the hidden message or file's existence. This can be achieved by
    [Show full text]
  • Chapter 11. Media Formats for Data Submission and Archive 11-1
    Chapter 11. Media Formats for Data Submission and Archive 11-1 Chapter 11. Media Formats for Data Submission and Archive This standard identifies the physical media formats to be used for data submission or delivery to the PDS or its science nodes. The PDS expects flight projects to deliver all archive products on magnetic or optical media. Electronic delivery of modest volumes of special science data products may be negotiated with the science nodes. Archive Planning - During archive planning, the data producer and PDS will determine the medium (or media) to use for data submission and archiving. This standard lists the media that are most commonly used for submitting data to and subsequently archiving data with the PDS. Delivery of data on media other than those listed here may be negotiated with the PDS on a case- by-case basis. Physical Media for Archive - For archival products only media that conform to the appropriate International Standards Organization (ISO) standard for physical and logical recording formats may be used. 1. The preferred data delivery medium is the Compact Disk (CD-ROM or CD-Recordable) produced in ISO 9660 format, using Interchange Level 1, subject to the restrictions listed in Section 10.1.1. 2. Compact Disks may be produced in ISO 9660 format using Interchange Level 2, subject to the restrictions listed in Section 10.1.2. 3. Digital Versatile Disk (DVD-ROM or DVD-R) should be produced in UDF-Bridge format (Universal Disc Format) with ISO 9660 Level 1 or Level 2 compatibility. Because of hardware compatibility and long-term stability issues, the use of 12-inch Write Once Read Many (WORM) disk, 8-mm Exabyte tape, 4-mm DAT tape, Bernoulli Disks, Zip disks, Syquest disks and Jaz disks is not recommended for archival use.
    [Show full text]
  • User Commands GZIP ( 1 ) Gzip, Gunzip, Gzcat – Compress Or Expand Files Gzip [ –Acdfhllnnrtvv19 ] [–S Suffix] [ Name ... ]
    User Commands GZIP ( 1 ) NAME gzip, gunzip, gzcat – compress or expand files SYNOPSIS gzip [–acdfhlLnNrtvV19 ] [– S suffix] [ name ... ] gunzip [–acfhlLnNrtvV ] [– S suffix] [ name ... ] gzcat [–fhLV ] [ name ... ] DESCRIPTION Gzip reduces the size of the named files using Lempel-Ziv coding (LZ77). Whenever possible, each file is replaced by one with the extension .gz, while keeping the same ownership modes, access and modification times. (The default extension is – gz for VMS, z for MSDOS, OS/2 FAT, Windows NT FAT and Atari.) If no files are specified, or if a file name is "-", the standard input is compressed to the standard output. Gzip will only attempt to compress regular files. In particular, it will ignore symbolic links. If the compressed file name is too long for its file system, gzip truncates it. Gzip attempts to truncate only the parts of the file name longer than 3 characters. (A part is delimited by dots.) If the name con- sists of small parts only, the longest parts are truncated. For example, if file names are limited to 14 characters, gzip.msdos.exe is compressed to gzi.msd.exe.gz. Names are not truncated on systems which do not have a limit on file name length. By default, gzip keeps the original file name and timestamp in the compressed file. These are used when decompressing the file with the – N option. This is useful when the compressed file name was truncated or when the time stamp was not preserved after a file transfer. Compressed files can be restored to their original form using gzip -d or gunzip or gzcat.
    [Show full text]