Helix Versioning Engine. Perforce technical description

perforce_helix

 

Helix Versioning Engine

Collaborate Without Limits

Helix is an enterprise-grade SCM and content collaboration platform that helps teams build better products faster.
It offers a hybrid approach to version control—use it as a distributed version control system (DVCS) or take a centralized approach.

Architecture

It works in client/server mode. The server manages a central data base that contains one or more repositories with file´s versions. Clients can work in local mode if they import files to their workspace. Then modified files can be grouped by changes list being ready to be submitted. The TCP connection uses RPC and streaming.

Database

 depot-metadata
 

Perforce has a proprietary data base, pre-configured and pre-installed that stores file´s metadata of the repository (status, history of the branches and reunions, change lists, descriptions of changes, users, groups, tags …). Only file´s hash (MD5) is stored in the data base, it is in the repository so it is possible to check it´s integrity.

Tables of the database are stored in binary format.The checkpoints and logs are stored in text format, which can be compressed and downloaded. In this way it is possible recover a corrupted data base due to a hardware problem or another calamity. Administrators can fix it by programming logs and configuring the periodicity of control points.

Repository

depot-workspace

The different files and their versions are stored in a directory structure called repository. Revisions of text files are stored like deltas RCS, while binary are stored integrally. Text files are codified in ASCII or Unicode, depending on the server configuration. The repository files are not encrypted. Branched revisions or copied ones maintain a virtual copy inside the repository. All revisions are saved by default, but you can limit the number of versions to save in the repository.

Obsolete versions of files can be purged by the administrator. It has to back up files from the repository.

Customers

Perforce customers are grouped into four categories: commands, GUI, web and plug-ins.

  • The command line basic interface is called P4. It can be used in the command window of every   operating system or script. Always produces a text output. That can include labels, as well of  scripts Python o Ruby. They are also available APIs natives in C++ and Java, as well as Lua, Perl Python, PHP, Objective-C and Rubi, all of them are implemented on the API C++. By using commands it is possible to run every customer´s functionality. You can use file´s syntax proper of the operating system or Perforce´s neutral syntax.
  • The P4V Qt based interface supports the majority of client-server operations. P4Admin is the adminstration graphic tool which is based on Gt and is portable between different platforms. P4V and P4Admin can be extended with scripts in JavaScript and HTML.
  • P4Web provides a Web interface and works as a Perforce client and also as a HTTP independent server. P4Web can be run like a shared Web server and gives access to the repository files and metadata. It can be implemented on a client machine, so that allows a navigator to access Perforce.
  • Plug-ins are transparent programs for user that integrate Perforce functionality´s client with other third applications. There are available environments and development tools, code revision systems, defects paths, compilation commands, office automation tools, clients SQL and FTP.

Features

  •  Full story metadata and file
  •  History of revisions, branched, renamed, moved, copied and deleted files.
  • Three-way text file merging. File remerging control and ancestor file detection.Graph differences presentations and merging and reconciliation tools.
  • Historic and branched file graphic view.
  • Administrative Graphics interface.
  • Controlled distributed version support.
  • Icon images
  • Changes list: modified files can group and be managed as logic units.
  • Atomic modifications: change list are updated in indivisible way by the server.
  • Parking: work in course can be stored temporally to change task.
  • ASCII files support, Unicode, binary, symbolic links, Mac and UTF-16 specifics.
  • Internationalization and location supported.
  • Keyword RCS style expansion.
  • File compression for facilitate transmission and store.
  • Unix and Windows clients support any operating system.
  • Events triggers on the server.
  • External systems integrations with SDK.
  • Change list notifications via RSS or e-mail.
  • Metadata and file replication.
  • Local policy manager implementation, command restrictions or alternative server redirection.
  • Archive files to free disk space.

Concurrency model

The Perforce system offers a hybrid of merge and lock concurrency models. As with similar systems, users do not have to lock files in order to work on them and are required to resolve concurrent, committed changes before submitting their work. Users may optionally lock files to ensure that they won't have to resolve concurrent changes.

However, the Perforce model is slightly different from those of similar systems in that users are expected to let the system know in advance which files they intend to change, even if they don't mean to lock them. Giving advance notice puts files in a pending changelist that can be submitted to the server. It also enables the system to alert other users working on the same files. Thus users can tell when they are working in parallel and can take the opportunity to coordinate with one another before making changes that could otherwise be difficult to merge.

Perforce enforces this advanced notification requirement loosely by setting read-only permission on workspace files as it fetches them from the repository. Users can bypass the requirement, by choice or by necessity (when working offline, for example), simply by hijacking file permissions and modifying files as they see fit. It is up to the user, in these cases, to remember to use Perforce to reconcile offline work and put hijacked files in a pending changelist so they can be submitted. (It is also up to users to leave hijacked files writable after changing them. A read-only file that is not in a pending changelist is assumed by Perforce to be a candidate for update by replacement.)

Branches and merging

Aiquely id file is unentified by its complete filename, e.g., //depot/trunk/src/item.cpp. Any non-deleted revision of a file can be branched. Perforce uses inter-file branching,wherein branching creates a new file with a new name. For example, my/index.php may be branched into your/index.php and each file may then evolve independently. Repository paths are typically designated as containers for branched sets of files. For example, files in the //depot/trunk path may be branched as a set into a new //depot/rel1.0 path, resulting in two sets of files evolving independently and between which changes can be merged.

In Perforce, the operation that merges changes from one branch to another is called integration. Integration propagates changes from a set of donor files into a set of corresponding target files; optional branch views can store customized donor–target mappings. By default, integration propagates all outstanding donor changes. Donor changes can be limited or cherry-picked by changelist, date, label, filename, or filename pattern-matching. The system records all integrations, uses them to select common ancestors for file merging, and does not by default perform redundant or unnecessary integrations.

Merging is actually only one of three possible outcomes of an integration. The others are ignoring (aka "blocking") and copying (aka "promoting"). Merging is used to keep one set of files up to date with another. For example, a development branch may be kept up to date with its trunk through repeated merging. Ignoring disqualifies changes in one set of files from future integration into another. It is often used when a development branch must be up to date with, and yet divergent from, its trunk. Copying is typically used to promote the content of an up-to-date development branch into a trunk.

Branching also supports renamed and moved files. The 'move' command branches originals to new files and deletes the originals. A branched file is no different from an added file; branched files are peers, not offshoots, of their originals. The system keeps track of file origins, however, and refers to them when displaying the history of renamed files.

Perforce streamsoffer a way to capture more information about a branch, including ancestry and included paths. This information is used to provide guidelines for branching and merging operations.

Streams

In 2011.1 and newer Perforce, a group o packed files were made corresponding to a version. Streams model allows to create a new branch –and merge each file compounding them- in a single branching-merging operation as described above.

The functionality of streams offers an innovative way to manage concurrent development, dependencies and other common sports activities and publishing branch versions. Streams provide a framework for projects in establishing workflows based on best practices from years of development. Streams are flexible enough to accommodate many models of branching in development, including mainline and development of appropriate promotional Web sites.

The migration of an existing project to streams requires some planning. Although the mechanics of moving the data to a stream is direct deposit, other points should be considered, as the impact on users and tools. Switching to streams provides a productive workflow and solves traditional bottlenecks.

Distributed revision control

Perforce has two mechanisms for a distributed revision control, these mechanisms can be used independently or combined. The first is a proxy server that caches frequently read versions, to reduce the access time for remote users. This mechanism accommodates closed development organizations in which a centrally controlled repository and universally database access are desirable.

The second mechanism (the so-called remote repository) allow to server connected users to access at the file version saved by others. By using remote repository, organizations manage their own servers and make it visible partially –or totally- to other servers. This mechanism can be used by disperse developing organizations, wherever is desired a peer-to-peer approximation.

Access control and security

The Perforce server stores file content in a master repository that, when properly installed, is inaccessible to users. User access to files is controlled by one or more Perforce superusers. A range of file access protection levels can be granted. Protections can be set for repository file paths, users, groups, and IP Address subnet. The server can maintain an audit log of client access events for SOX and other compliance requirements.

User authentication is controlled by the Perforce system administrator. Pasword is configurable; ticket-based authentication can be configured as well. Triggers (custom scripts or programs that run at predefined events) can be set on many but not all Perforce user commands and used to extend user authentication (with LDAP or SSO, for example), to block or allow user commands, and to constrain or normalize file modifications. Triggers are run by the Perforce server and do not have access to client machines or workspaces.

Perforce, like most version control systems, does not encrypt file content in the master repository or on user machines. Nor does it encrypt file content sent over the network; a tunneling-protocol (like VPN or SSH) must be used to secure network transfers. Full use of file protection, authentication, and tunneling is required to secure Perforce-managed content.

The Perforce client completely trusts the server, including writing arbitrary files anywhere in the local filesystem, and therefore running arbitrary code from the server.That means the server has complete control over the client user's account, including reading and writing and sending all non-source code files of the user. In environments where the Perforce server is managed by a third party, this poses a significant threat to the client's security and privacy.

Availability

Client´s Perforce server evaluation downloads, are available, as well as plug-ins on the Perforce´s website. Usually, two new versions of server´s programs and clients are published like executables each year. They´re available for Microsoft Windows, Mac OS X, Linux, Solaris, FreeBSD and other operating systems.

Integration with the following IDEs is available:

  • Eclipse
  • WebSphere
  • ColdFusion
  • Dreamweaver
  • CodeWright
  • IAR Embedded Workbench
  • Visual Studio
  • Primalscript
  • Zeus
  • Komodo
  • Articy:draft
  • BBEdit
  • Jbuilder
  • Embarcadero Delphi
  • Delphi
  • IntelliJ IDE
  • LabVIEW
  • PythonWorks
  • SlickEdit

Licenses and terms of use

Perforce server is free to twenty users and twenty workspaces or for an unlimited number of users and one thousand files. License can be purchased via subscription. Free licenses are distrubuted for academic or scholar projects and for evaluation periods. The use of clients or plug-ins is unlimited, as well as technical documentation.

Installed Base

As of June 2013, Perforce was being used by 400,000 users in 5,500 organizations.