DominoDefrag v3.0 Release Notes Sep. 12 Overview DominoDefrag is an open source C API Lotus Domino server task I developed using Visual Studio C++. It uses "contig.exe" (v1.55) to defrag Domino databases on all 32bit and 64bit Windows server versions and will also defrag a full-text index associated with a Notes database and the Domino server’s transaction log and DAOS files. The output is recorded to the Domino server console and miscellaneous events log. DominoDefrag has the added functionality of being able to compact a database prior to defragging and supports multi-processing (can load multiple times to run concurrently) and use of an indirect file (.ind) to have the same compact batch functionality. Performance checks can also be tested using generated document collections to determine the before and after defrag millisecond read performance of databases and their associated full-text indexes. DominoDefrag now incorporates “jkdefragcmd.exe” (v3.36) for Domino data volume free space consolidation and “fast sort” file placement optimization. In conjunction with “contig.exe” and the compact task (ncompact.exe) they provide the functionality outlined below: Order Utility Description 1 jkdefragcmd.exe Consolidates free space (-a5), alphabetically sorts all file placements (-a7) and performs a complete defrag and fast sort optimization (-a3) of the Domino data volume. It runs as a separate function and can take some time to perform. Runs when DominoDefrag_OptimizeDataVolume=1 2 ncompact.exe Compacts a database when DominoDefrag_Compact=1. If DominoDefrag_FreeSpace=x also used only compacts the database that has > x% unused space 3 contig.exe Optimizes a database to have x% unused space (free space) = 100%-x used space when DominoDefrag_FreeSpace=x 4 contig.exe Defrags a database. When DominoDefrag_FreeSpace=x ensures future Domino writes will be made into the free space residing on one contiguous fragment. It will defrag a database’s full-text index when DominoDefrag_FTIndex=1, the Domino server’s transaction log when DominoDefrag_TransactionLog=1 and the Domino server’s DAOS files when DominoDefrag_DAOS=1 5 jkdefragcmd.exe Performs a “fast sort” file placement optimization of database, full-text, transaction log or DAOS files when DominoDefrag_OptimizeFiles=1 1 | P a g e DominoDefrag v3.0 Release Notes Sep. 12 Usage and Copyright Disclaimer This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Please read more about the details of the GNU Affero General Public License @ http://www.gnu.org/licenses/ and in the distribution’s “GNU AFFERO GENERAL PUBLIC LICENSE.txt” file. Copyright © 12 September 2010 Andrew Luder ([email protected]) http://notestools.com and http://www.linkedin.com/in/andrewluder) DominoDefrag Enterprise Edition The "DominoDefrag Enterprise Edition" product comprises a mail-in database with file name "DominoDefragEE.nsf" and business support for it and the current "DominoDefrag.exe" (provided on OpenNTF) under an Enterprise Support Agreement (ESA) upon full fee payment. A fully functioning “DominoDefragEE.nsf” mail-in database can be downloaded from ftp://notestools.com/DominoDefragEE.zip , but has limited support and can only process one Server Defrag Profile and run on one Domino server within your domain. A new “DominoDefrag.nsf” mail- in database will be provided configured with owner license details upon full fee payment. The new DominoDefrag_EnterpriseSupport INI parameter specifies that the “DD_ServerLog.csv” and “DD_DatabaseLog.csv” files should be created and attached to the summary email for use by the DominoDefrag Enterprise Edition mail-in database. Only works when the mail-in database e-mail address is specified using the DominoDefrag_SummaryEmailAddress parameter. If you would like to get an obligation free quote to purchase the DominoDefrag Enterprise Edition (price includes one year business support and software maintenance), please complete the Contact form (also providing the number of Domino server licenses you require) at https://notestools.com/index.php?p=1_16 . Note: Other Information such as your Domino Domain name and number of “DominoDefragEE.nsf” instances will be requested if you decide to purchase the "DominoDefrag Enterprise Edition" product. 2 | P a g e DominoDefrag v3.0 Release Notes Sep. 12 Domino Data Volume Free Space Consolidation and Optimization A question that’s been posed to me several times is why does “contig.exe” only do a partial defrag of some large Domino databases? The answer is "contig.exe" simply looks to move the fragmented database to the first contiguous gap (defragmented free space) on the disk volume that is big enough. If there is no gap big enough then "contig.exe" will reduce the number of fragments in the file by moving as much of the file as possible into the largest gaps available. The DominoDefrag server task now uses the "JkDefragcmd.exe” utility to consolidate and optimize the free space on the Domino data volume in preparation, so that these partial database defrags will be complete database defrags next time “contig.exe” processes them. It has been implemented to operate as a separate function and I’d recommend something you would perform once a month. Figure 1 below shows a Windows 2000 Domino data volume which “contig.exe” has been running on for a while. Figure 1 - Windows 2000 Domino Data Volume using “contig.exe” Figure 2 below shows the result of using the DominoDefrag_OptimizeDataVolume=1 parameter to tell DominoDefrag to use "JkDefragcmd.exe” to consolidate free space (-a5), alphabetically sort all file placements (-a7) and perform a complete defrag and “fast sort” optimization (-a3) of the Windows 2000 Domino data volume. A similar strategy is recommended on the http://kessels.com/jkdefrag/ page. 3 | P a g e DominoDefrag v3.0 Release Notes Sep. 12 Figure 2- Windows 2000 Domino Data Volume using “JkDefragcmd.exe” The remaining issue is that some of the free space can be fragmented and I am looking at other free command line tools like Defraggler (http://www.piriform.com/defraggler) that have the functionality to defragment this free space available in a future DominoDefrag version. Optimizing Database Free Space A Notes database is proportioned into used and unused space (also called free space). Free space is only recovered when it’s compacted; otherwise Domino will continue writing data into the free space until it’s full and has to allocate more. Defragging a database after optimizing the free space (unused space) ensures future writes will be made into the free space that resides within the database on one contiguous fragment. The new DominoDefrag_FreeSpace Notes INI parameter attempts to optimize the free space (unused space) to the percentage specified and will only allow a compact to occur if there's a greater percentage free space. The new DominoDefrag_FreeSpaceMaxMB ensures that the amount of unused space optimized (new free space that can be pre-allocated) is limited to the specified megabyte amount (when not specified it defaults to 1MB). For example, if DominoDefrag_FreeSpace=10 and set to DominoDefrag_FreeSpaceMaxMB=5, only 5MB of unused space is optimized for a 1GB database and not 100MB. This is vital because it allows you to estimate increased disk usage with certainty, because in the worst case, 100% used space would mean at most the database size would only increase by 5MB. 4 | P a g e DominoDefrag v3.0 Release Notes Sep. 12 Optimizing File Placement Performs a “fast sort” file placement optimization of database, full-text, transaction log or DAOS files when DominoDefrag_OptimizeFiles=1 after defragmentation has occurred. The “fast sort” file placement optimization attempts to keep Domino files in the same zone – that is together, closer to the beginning of the hard disk and therefore means the hard disk head doesn’t have to travel as far to get the Domino data. Therefore, faster I/O! Note: Optimizing file placement is discussed more on the http://kessels.com/jkdefrag/ page. Donations Please click here to make donation if you feel DominoDefrag is worth supporting. No amount is too small - it all helps :-) 5 | P a g e DominoDefrag v3.0 Release Notes Sep. 12 Version History Version Changes 3.0 DominoDefrag_OptimizeDataVolume Notes INI parameter specifies that “jkdefragcmd.exe” is used to consolidate free space (-a5), alphabetically sort all file placements (-a7) and perform a complete defrag and fast sort optimization (- a3) of the Domino data volume. It runs as a separate function and can take some time to perform. Does not work on NT4. DominoDefrag_OptimizeFiles Notes INI parameter specifies that “jkdefragcmd.exe” is used after defragmentation to "fast" sort file placement on the Domino data volume. Does not work on NT4. DominoDefrag_EnterpriseSupport INI parameter specifies that the “DD_ServerLog.csv” and “DD_DatabaseLog.csv” files should be created and attached to the summary email for use by the DominoDefrag Enterprise Edition mail-in database Fixed bug NOSS-87ED49 to check database quota size limit and skip adding more free space if exceeds Added summary information counters to report both before and after fragments averages for