Comparison: Perforce and CVS Perforce& CVS Overview This document compares Perforce (version 2009.1) with CVS (CVSNT version 2.5.04.3510). Rather than compare every available feature in the two products, it focuses on their most significant differences, such as performance, atomic transactions, and distributed development. Executive Summary Attribute CVS Perforce Branching and CVS offers branching, but requires manual Perforce branching automatically tracks the merging tracking of merge history. history of all branching operations. Distributed No performance solutions for remote The Perforce Proxy offers a caching solution development development are available. for remote users, with minimal administrative overhead, and at no extra cost. Atomic transactions Doesn’t have an atomic change mechanism Natively supported as changelists, enabling and cannot group changes to related files. users to track file versions associated with a feature addition or issue resolution. Scalability and Metadata is appended and maintained in the Centralized metadata records all user and performance content of each versioned file. file activity. Deployed in 10,000+ user environments. Defect tracking Not part of CVS; need to use a separate Perforce provides a basic defect tracking defect tracking solution. system called jobs, and integrates with leading third-party defect tracking systems. Integration with Integrations are available via the open source Many third-party tools are designed related tools community. specifically to work with Perforce. Support Available at additional cost from a third party. Relied upon by over 350,000 users. 1 Branching and Merging branches automatically. This enables a variety of development scenarios, such as client- CVS branching doesn’t keep a history of specific versions, experimental branches, operations, such as integrations between files personal or task branches, and the classic or branches. Users must track integrations release branching patterns. A built-in graphing manually. No automated conflict resolution is tool (see Figure 1) displays the branching available when merging files. CVS branches history of each file. files with the same mechanism it employs to label files. Because all metadata is maintained in the physical versioned file, this branching Distributed Development method requires that each versioned file be opened and updated to include branch CVS doesn’t offer a scalable remote information. Locks are required for all targeted development solution. Users must remotely files during the branch operation, which can access a central server. This solution performs impose long wait times on other participating when users have a fast, reliable network developers. connection, but there are no efficiencies for groups of users at remote locations. The Perforce branching model is powerful and Replication solutions are commercially flexible. It’s capable of branching thousands of available from third-party providers. files, and tracks all merges between branches. Merge tracking is an important characteristic Perforce’s distributed architecture is for synchronizing branches. Perforce keeps supported by Perforce Proxies at remote track of all incremental changes as they are locations (see Figure 2). introduced to original and branched versions of files. Instead of manually tracking changes The Perforce Proxy caches and serves files to across branches, users can rely upon Perforce users at remote locations, thereby reducing to integrate file changes across multiple traffic across slower WAN links. Figure 1: The Perforce Revision Graph shows the branching history of a file, together with its merge history. Comparison: Perforce and CVS 2 Figure 2: The Perforce Proxy provides a simple architecture for distributed development. All users, local or remote, connect to the same the set of changes (adds, deletes, and edits) central depot, and hence look at the exact involved in the given changelist. same project files. Distributed development with Perforce doesn’t create any additional process overhead, and the Perforce Proxy Scalability and Performance requires minimal administrative attention. The CVS architecture doesn’t support a centralized database for tracking branch, label, Atomic Transactions and change history. Therefore, all change history must be stored in each versioned file CVS doesn’t have an atomic change for the duration of its lifetime in RCS. This mechanism and cannot group changes to includes all branch and label information. related files. Relationships between files at User updates of client file information require check-in are lost, unless artificial methods a time-consuming comparison of the client such as check-in time or common comments environment to the repository. As the number are used. If some of the files involved in a and size of files increase in the repository, check-in operation are rejected, the codebase all associated software configuration is left in an inconsistent state. management (SCM) operations take more time to execute. Concurrent users are often Perforce organizes the changes made placed in a wait state as requested files are to multiple files into units of work called updated with branch and label information. changelists. Typically, changelists represent Performance for these operations can only be features or bug fixes that are implemented by improved by removing stale file history from modifying multiple files. Because changelists the physical files. are atomic, they ensure the integrity of each check-in, and avoid the corruption Perforce is architected first and foremost for introduced by partial file submissions. Perforce speed. Perforce scales linearly, so there’s no guarantees that whenever a changelist is performance penalty imposed by the number submitted, the state of the entire system of revisions of any given file or the size of the before and after the changelist varies only by file. Perforce’s built-in relational database 3 maintains all metadata and segregates it from Test Procedure the associated files. Labeling, branching, Testing began with both source control and reporting are accomplished directly in systems in a newly installed state. Tests were the database, instead of updates and scans run on each system alternately. All developer of all the files in a particular branch tree. and source import tests were automated and The Perforce Server has been deployed timed using the DOS time command to a successfully in environments that support granularity of .01 second. Software installation more than 10,000 users. was timed externally. Performance Tests Where tests would invoke the file system To test CVS performance against Perforce, cache or database cache, the tests were run comparative benchmarking tests were three times, and average times for second constructed to reflect real-life steps and third iterations were recorded. This developers would take in carrying out their simulates an environment where many files day-to-day work. In some cases, individual are accessed repeatedly through builds and commands were used. In other cases, scripts source code searches. orchestrating multiple commands were employed. In all cases, functionally equivalent A common file structure was used. Each sets of commands were used and executed product created, deleted, and modified the using the command line utilities. same files during testing. File copies were This comparison provides a look at these two used to simulate file creation, and automated products as they might be used in a real- edits were used to simulate file modifications. world environment. When a sequence of commands was required Test Environment to complete a single test for either system, a The test environment was selected to provide DOS batch file was used to execute the test a common development environment contem- for both systems. porary to 2009. The configuration exceeded the minimum recommendations for both CVS No performance-oriented techniques were and Perforce. The environment consisted of used to optimize the use of either product, the following: other than those used to remove interactive effects. • Network: Both server and client on same segment of 1 Gbps network The tests were designed to compare the two • Server OS: Windows Server 2008 products during normal developer usage. For (Standard) SP1 (32 bit) all processing, automated merges were used, as the tests were constructed to avoid file • Client OS: Windows XP Professional SP2 conflicts. The following tasks were tested: (64 bit) • Server Hardware: Dell PowerEdge 1850 • Server installation with dual 3.3 GHz Xeon processor X5470, • Client installation 132GB hard disk and 16GB RAM • Importing source from a local file system • Client Hardware: Dell Precision 490 with • Populating the workspace Intel Xeon 5130 2.0 GHz processor and • Updating the workspace (with no changes 2GB RAM on the server) • Labeling all files • Deleting files on the server • Undeleting files on the server Comparison: Perforce and CVS 4 • Branching files separate trees: 33,920 files totaling 417MB. • Merging back to the parent branch Tests for the large project performed the • Branching binary files same operations in each tree. Test data for binary files consisted of 217 files (MPG, EXE, Test Data and TAR) totalling 1GB. Test data was a combination of two open Test Results source software packages: Apache httpd Installing CVS was a larger task than install- 2.2.14 and Apache Tomcat 6.0.20. A “small” ing Perforce, but not drastically. Perforce project contained