BuildButler

Xray + GitHub Actions — How to Import Test Results into Xray for Jira

Learn how to automatically import JUnit, Cucumber, and TestNG test results from GitHub Actions into Xray for Jira. Compare the xray-action, REST API, and BuildButler approaches.

Your team runs tests in GitHub Actions. Your test cases live in Xray for Jira. Getting results from one into the other shouldn't require a weekend of REST API scripting.

There are three ways to connect GitHub Actions to Xray. Each has trade-offs. This guide covers all three so you can pick the right one for your team.

Option 1: The xray-action GitHub Action

The community-maintained xray-action by Mike Penz is the most popular way to import test results into Xray from GitHub Actions.

How it works

Add a step to your workflow that runs after your tests:

- name: Import results to Xray
  uses: mikepenz/xray-action@v3
  with:
    xrayCloud: true
    username: ${{ secrets.XRAY_CLIENT_ID }}
    password: ${{ secrets.XRAY_CLIENT_SECRET }}
    testFormat: "junit"
    testPaths: "**/target/surefire-reports/*.xml"
    projectKey: "BACK"

Supported formats

JUnit, TestNG, NUnit, xUnit, Cucumber, Behave, Robot, and the native Xray JSON format.

Pros

  • Free and open source
  • Direct GitHub Actions integration — no external service needed
  • Supports both Xray Cloud and Xray Server/DC
  • Active maintenance

Cons

  • GitHub Actions only — if you also use Jenkins or GitLab CI, you need separate integrations
  • Per-workflow configuration — each workflow needs its own step and credentials
  • No analytics — results go to Xray, but you get no build trends, flaky test detection, or cross-CI visibility
  • No pattern-based routing — you configure each workflow individually, not with glob patterns

Option 2: Xray REST API directly

Xray provides a REST API for importing execution results in multiple formats. You can call it from any CI system using curl.

Example

- name: Import JUnit results to Xray
  run: |
    curl -X POST \
      -H "Authorization: Bearer ${{ secrets.XRAY_TOKEN }}" \
      -H "Content-Type: text/xml" \
      -d @target/surefire-reports/TEST-results.xml \
      "https://xray.cloud.getxray.app/api/v2/import/execution/junit?projectKey=BACK"

Pros

  • Works from any CI system
  • Full control over the request
  • No dependencies

Cons

  • You're writing glue code — authentication, error handling, file globbing, multipart uploads
  • No test case creation — you need to manage test case lifecycle separately
  • Fragile — API changes, token rotation, and edge cases are your problem
  • No analytics or visibility

Option 3: BuildButler

BuildButler takes a different approach. Instead of configuring each workflow individually, you define JSON rules that pattern-match CI jobs and route results to Xray projects.

How it works

  1. Add the BuildButler CLI reporter to your GitHub Actions workflow
  2. Define mapping rules once in BuildButler settings:
[
  {
    "jobPattern": "backend/*",
    "jiraProject": "BACK",
    "testPlan": "BACK-100",
    "testEnvironment": "staging",
    "searchMode": "name"
  }
]
  1. Every matching workflow automatically exports results to Xray

Pros

  • One config for all CI systems — Jenkins, GitHub Actions, GitLab CI, TeamCity, BuildKite
  • Pattern-based routing — glob patterns match jobs automatically, no per-workflow setup
  • CI analytics included — build trends, flaky test detection, test history, Ask AI
  • Multi-TCM support — same platform exports to Xray, TestRail, ALM, or Zephyr Scale
  • JIRA integration — triage failures and sync tickets alongside Xray results

Cons

  • External service (not self-contained in the workflow)
  • Additional tool to manage

Which option should you choose?

Criteriaxray-actionREST APIBuildButler
GitHub Actions onlyGood fitWorksGood fit
Multiple CI systemsNeed separate toolsNeed separate scriptsOne config for all
Analytics & trendsNoNoYes
Flaky test detectionNoNoYes
Pattern-based routingNoNoYes
Setup complexityLowMediumLow
MaintenanceLowHighLow

If you only use GitHub Actions and only need Xray, the xray-action is the simplest choice.

If you use multiple CI systems, want analytics, or also need TestRail/ALM integration, BuildButler is the better fit.

Getting started with BuildButler

  1. Sign up for BuildButler — Xray integration is included on all plans
  2. Add the CLI reporter to your GitHub Actions workflow
  3. Configure Xray/Jira credentials in Settings
  4. Define your mapping rules
  5. Push a commit — results appear in Xray on the next run

See the full Xray integration guide for all configuration options.

On this page