Test Case Management

Automatically export Jenkins test results to TestRail, Xray, or ALM using BuildButler's JSON configuration.

BuildButler can automatically export test results from your Jenkins builds into external test case management tools. Configure one or more export rules as a JSON array — each rule matches Jenkins jobs by pattern and maps results to the target system.

TestRail

BuildButler exports test results to TestRail by creating/updating test cases, test runs, and test plans. Configure export rules as a JSON array where each object targets a set of Jenkins jobs.

TestRail configuration — credentials, job mappings, and test sync

Example configuration

[
  {
    "jobPattern": "backend/*",
    "project": "Backend",
    "testPath": "Automated Tests/API",
    "testRun": "{jobName} #{buildNumber}",
    "searchMode": "name",
    "caseType": "Automated",
    "casePriority": "Medium",
    "closeRun": true,
    "fullyQualifiedTestName": true
  },
  {
    "jobPattern": "frontend/*",
    "project": "Frontend",
    "testPath": "Automated Tests/UI",
    "testPlan": "Release {branch}",
    "milestonePath": "Releases/{branch}",
    "searchMode": "name",
    "closeRun": false,
    "closePlan": false
  }
]

Template variables

Use template variables inside string values to dynamically reference build metadata:

Variable	Description
`{jobName}`	Jenkins job name
`{buildNumber}`	Build number
`{branch}`	Git branch name
`{date}`	Build date (YYYY-MM-DD)

Configuration reference

Required fields

Field	Type	Description
`jobPattern`	string	Glob pattern to match Jenkins job names (e.g. `backend/`, `/e2e-`)
`project`	string	TestRail project name
`testPath`	string	TestRail test case path where cases are created (e.g. `Automated Tests/API`)

Test run and plan fields

Field	Type	Default	Description
`testRun`	string	—	TestRail test run name. Supports template variables.
`testPlan`	string	—	TestRail test plan name. When set, the test run is created inside this plan. Supports template variables.
`milestonePath`	string	—	TestRail milestone path (e.g. `Releases/{branch}`). Creates milestones if they don't exist.
`closeRun`	boolean	`false`	Automatically close the test run after export
`closePlan`	boolean	`false`	Automatically close the test plan after export

Search and matching fields

Field	Type	Default	Description
`searchMode`	string	`name`	How to match existing test cases: `name` (match by test name) or `path` (match by full class path + test name)
`fullyQualifiedTestName`	boolean	`false`	Use the fully qualified class name as the test case title

Test case metadata fields

Field	Type	Default	Description
`caseType`	string	—	TestRail case type (e.g. `Automated`, `Functional`, `Regression`)
`casePriority`	string	—	TestRail case priority (e.g. `Critical`, `High`, `Medium`, `Low`)
`caseFields`	object	—	Custom TestRail case field values (e.g. `{"custom_automation_tool": "Selenium"}`)
`resultFields`	object	—	Custom TestRail result field values (e.g. `{"custom_environment": "staging"}`)

Upload behaviour fields

Field	Type	Default	Description
`uploadMode`	number	`0`	Controls how test cases are created: `0` = create new cases and update existing, `1` = create new cases only, `2` = update existing only, `3` = do not create or update cases
`threadCount`	number	`4`	Number of concurrent threads for exporting data
`timeout`	number	`120`	Upload timeout in seconds

Xray

BuildButler can export test results to Xray for Jira. Xray integration maps Jenkins test results to Xray test cases, test executions, and test plans within your Jira project.

Example configuration

[
  {
    "jobPattern": "backend/*",
    "jiraProject": "BACK",
    "testPlan": "BACK-100",
    "testEnvironment": "staging",
    "searchMode": "name"
  },
  {
    "jobPattern": "frontend/*",
    "jiraProject": "FRONT",
    "testPlan": "FRONT-200",
    "testEnvironment": "production",
    "searchMode": "name"
  }
]

Configuration reference

Field	Type	Required	Description
`jobPattern`	string	Yes	Glob pattern to match Jenkins job names
`jiraProject`	string	Yes	Jira project key (e.g. `BACK`, `FRONT`)
`testPlan`	string	No	Xray test plan issue key to associate test executions with
`testExecution`	string	No	Existing test execution issue key. If omitted, a new execution is created.
`testEnvironment`	string	No	Xray test environment name (e.g. `staging`, `production`)
`searchMode`	string	No	How to match existing test cases: `name` or `path`. Default: `name`
`revision`	string	No	Source code revision to record against the test execution
`labels`	string[]	No	Jira labels to apply to created test issues

ALM

BuildButler can export test results to Micro Focus ALM (formerly HP ALM / Quality Center). Test results are mapped to ALM test sets and test instances.

Example configuration

[
  {
    "jobPattern": "regression/*",
    "domain": "DEFAULT",
    "almProject": "RegressionSuite",
    "testSetPath": "Root/Automated/{jobName}",
    "testPath": "Subject/Automated Tests",
    "searchMode": "name"
  }
]

Configuration reference

Field	Type	Required	Description
`jobPattern`	string	Yes	Glob pattern to match Jenkins job names
`domain`	string	Yes	ALM domain name
`almProject`	string	Yes	ALM project name
`testSetPath`	string	Yes	ALM test set folder path (e.g. `Root/Automated/{jobName}`). Supports template variables.
`testPath`	string	Yes	ALM test plan folder path where test cases are created
`searchMode`	string	No	How to match existing test cases: `name` or `path`. Default: `name`
`runName`	string	No	ALM test run name. Supports template variables.
`timeout`	number	No	Upload timeout in seconds. Default: `120`

On this page