Perforce and Jenkins

While many smaller projects often use Git or SVN due to budget constraints, larger game companies tend to prefer Perforce as their version control solution.

On the other hand, Jenkins is an open-source CI/CD tool widely used by many organizations.
As a result, it is common to see Perforce and Jenkins being used together in game development environments.

Perforce supports various APIs, allowing it to be used directly within Jenkins build pipelines. However, an official Perforce plugin is also provided to seamlessly integrate with Jenkins' SCM (Software Configuration Management) system and pipelines.
GitHub - jenkinsci/p4-plugin: Perforce plugin for Jenkins

Using this Perforce plugin, you can access features such as:

• Viewing Perforce changelists directly on the build page
• Automatic build triggers via polling
• Automatic creation of Perforce workspaces for builds

The Problem

In simpler configurations, this may not pose a significant issue. However, as the pipeline becomes more complex and the number of slots on build nodes increases, conflicts can arise.

Riot Games has a well-organized article on this topic, which is worth referencing first.
Using Perforce in a Complex Jenkins Pipeline | Riot Games Technology
This article will provide supplementary explanations and alternatives based on that content.

Understanding Perforce Workspaces

To understand the problem, it's essential to first grasp the concept of workspaces in Perforce.

Unlike Git, Perforce has a centralized structure.
To download files from Perforce, you must register a workspace, which includes a unique name along with mapping information between the Perforce path and the local directory.

For example, if a workspace named kjs104901_depot is registered as shown below, files from the Perforce server's //depot path will be downloaded to the D:\p4work\kjs104901_depot directory.

Understanding Jenkins Workspaces

Next, you need to understand the workspace structure in Jenkins.

Jenkins creates a unique directory for each build.
For instance, if Jenkins' root directory is C:\Jenkins\workspace, the build workspace for a project named blog would be:

C:\Jenkins\workspace\blog

When a single build is executed in multiple instances on a build node, Jenkins automatically appends @number to the workspace name to avoid conflicts:

C:\Jenkins\workspace\blog@2
C:\Jenkins\workspace\blog@3

Issues with the Perforce Plugin

To download files and store build artifacts, a Perforce workspace is required.
This workspace can be automatically created through the Jenkins Perforce plugin.

In this case, the local path of the Perforce workspace can simply use the Jenkins workspace directory.
The problem lies in how to name the Perforce workspace.

The official documentation for the Perforce plugin suggests the following naming convention:

jenkins-%{NODE_NAME}-%{JOB_NAME}-${EXECUTOR_NUMBER}

This name combines the node name, job name, and build execution number, which seems reasonable for ensuring the uniqueness of workspace names.

However, the issue arises because the Jenkins workspace names like @2, @3 may not match the EXECUTOR_NUMBER.
JENKINS-48882 Build Executor number is not consistent with EXECUTOR_NUMBER

These two values are managed independently, resulting in a situation where a Perforce workspace named jenkins-node-blog-2 might actually run in a Jenkins workspace with @1 or @3.

This can lead to serious issues due to the structure of Perforce.
Perforce caches information about which files have been downloaded to the local directory. For instance, if all files are downloaded in @2 and then you switch to @3, Perforce will determine that "it is already up to date" and will not download the files again.

In other words, if different workspace paths share the same Perforce workspace name, it can lead to various problems such as missing build artifacts and cache errors.

Solutions

Option 1 - Riot Games Approach

To address the mismatch issue with EXECUTOR_NUMBER, Riot Games checks the actual workspace path generated by Jenkins, parses it based on the @ symbol to extract the slot number, and uses this in the Perforce workspace name.

Additionally, they differentiate Jenkins workspaces by stage and further separate Perforce workspaces based on the slot number within that stage.
(The stage name may become too long, so they use a hash value to take only the first four characters.)

def riotP4Sync(Map config = [:]) {
  def humanReadableName = safePath("${JOB_NAME}-${STAGE_NAME}")
  def jenkinsWorkspaceName = safePath("${JOB_NAME}-") + workspaceShortname(env.STAGE_NAME)
  
  ws(jenkinsWorkspaceName) {
    echo "[INFO] [riotP4Sync] Running in ${pwd()}"
    def workspaceNumber = 1
    if (pwd().contains('@')) {
      workspaceNumber = pwd().split('@').last()
      if (!workspaceNumber.isInteger()) {
        error("${workspaceNumber} is not an integer! Something went wrong in riotP4Sync. PWD is ${pwd()}")
      }
    }
    def p4WorkspaceName = safePath("${humanReadableName}-${workspaceNumber}")
    echo "[INFO] [riotP4Sync] Using P4 Workspace ${p4WorkspaceName}"
}

Option 2 - Simplified Version

Riot's example code does not include NODE_NAME, and in many cases, there may not be a need to separate workspaces by stage.

Thus, you can create a simple function to safely extract the slot number, using it instead of the existing EXECUTOR_NUMBER.

private int getSlotNumber() {
    if (pwd().contains('@')) {
        workspaceNumber = pwd().split('@').last()
        if (!workspaceNumber.isInteger()) {
            error("${workspaceNumber} is not an integer. PWD is ${pwd()}")
        }
        return workspaceNumber as int
    }
    return 0
}

Option 3 - Limiting Concurrent Builds

If there is no need for the same build to run simultaneously in multiple places, you can maintain the uniqueness of the Perforce workspace without extracting the slot number.

If each build runs exclusively on a single node, the following configuration is possible.

How can you ensure the uniqueness of each build on a single node?
By utilizing Jenkins' Lockable Resources plugin, you can lock at the node + job level.
Lockable Resources | Jenkins plugin
For example, you can ensure that the build-A job on the agent1 node runs only one instance at a time.

If you are not distributing builds across nodes and are using a fixed node for execution, you can simplify this further.
By using the following build property to prevent concurrent builds, uniqueness is guaranteed across all Jenkins nodes.

properties properties: [ disableConcurrentBuilds() ]

The Issue of Finding Changes

Normally, you could automatically check for changes using only the Jenkins Perforce plugin, as shown below.

However, as previously explained, if workspaces are separated by stage or slot number, the Perforce plugin will perceive the workspace as newly created or changed each time.
This can lead to a situation where, even if the build is executed normally, changes may be missed.

In other words, even if it is the same job, if the workspace name changes, the Perforce plugin will determine that there is "no continuity with the previous build."

Solution: Tracking Changes Manually

If you are integrating change tracking with Slack notifications, release notes, deployment history systems, etc., you will need to track changes manually.

How to track changes:

• On a successful build, save the current changelist number as a Jenkins Artifact using archiveArtifacts.
• In the next successful build, directly retrieve the changelist between the previously saved number and the current number using Perforce commands or APIs.