Xcode test failing does not failed Github action pipeline

Question

I have a Github Action pipeline:

name: default

on: [push]

jobs:
  build:

    runs-on: macOS-latest

    steps:
    - uses: actions/checkout@v1
    - name: CocoaPod Install
      run: pod install
    - name: Force xcode 11
      run: sudo xcode-select -switch /Applications/Xcode_11.1.app
    - name: Test
      run: ./pipelines.sh test
    - name: Save report
      uses: actions/upload-artifact@v1
      with:
        name: test_report
        path: ./Test

And a script Shell:

function test
{
    xcodebuild \
       -workspace MyApp.xcworkspace \
       -scheme DEBUG\ -\ MyApp \
       -destination 'platform=iOS Simulator,name=iPhone 11' \
       test
}

My problem is when i run my pipeline with failing test, the pipeline is marked as PASSED, which is a problem...

I have also check with fastlane, failing test does not fail the pipeline.

How can I make my pipeline as FAIL when test does not pass?

Cf screenshot for fastlane:

peterevans · Accepted Answer · 2019-11-05 01:23:04Z

3

You need to return a non-zero value to fail the step. Try adding this to the xcodebuild command.

xcodebuild -... || exit 1

There are some other solutions besides this at the following question. How to get the return value of xcodebuild?

Update: Based on your comments that you would like steps afterwards to complete you can do the following.

Change your script to set an output containing the result of the xcodebuild command.

    xcodebuild \
       -workspace MyApp.xcworkspace \
       -scheme DEBUG\ -\ MyApp \
       -destination 'platform=iOS Simulator,name=iPhone 11' \
       test
    echo "::set-output name=result::$?"

Add an id to the step where this script is executed.

    - name: Test
      id: xcodebuild
      run: ./pipelines.sh test

At the end of your workflow you can check if the tests did not pass and fail the workflow.

      - name: Check Tests Passed
        if: steps.xcodebuild.outputs.result != 0
        run: exit 1

edited Nov 5, 2019 at 1:23

answered Nov 4, 2019 at 13:57

peterevans

43.4k10 gold badges96 silver badges89 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Kevin ABRIOUX Over a year ago

When i make it xcodebuild -... || exit 1, commands after this line are not executed if i have some

Kevin ABRIOUX Over a year ago

Step Save report is not executed in the pipeline

peterevans Over a year ago

@KevinABRIOUX Updated the answer with a solution to allow further steps to execute and fail afterwards.

Ben Thomas · Accepted Answer · 2023-11-28 02:30:36Z

1

If, like me, you're using xcbeautify in your GitHub action to clean up the output of xcodebuild, you can do the following to see the error from the xcodebuild step.

Prepend set -o pipefail && before your xcodebuild command.

eg:

set -o pipefail && xcodebuild -workspace <your workspace>.xcworkspace -scheme "$SCHEME" -derivedDataPath /tmp/derivedData -destination "platform=iOS Simulator,id=$DEVICE_ID" build-for-testing | xcbeautify --renderer github-actions

In GitHub you should see a red cross next to the step that failed, in my case it's the Build Package for Testing step.

answered Nov 28, 2023 at 2:30

Ben Thomas

1,5162 gold badges17 silver badges26 bronze badges

Collectives™ on Stack Overflow

Xcode test failing does not failed Github action pipeline

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related