'Mastering Custom Test Suites in Xcode: Run Only What Matters'

'Mastering Custom Test Suites in Xcode: Run Only What Matters'

• Metallica, Swiftui, Xcode, Ios • 7 min read

author: Wesley Matlock read_time: 6 —

Build smarter, not slower — from the cockpit of modular Swift apps

⚠️ 28-Minute Test Runs? Been There.

You know the drill — you tweak one line in a Swift file, open a PR, and suddenly Xcode spins up a test apocalypse: integration tests, UI tests, Swift Package tests, and that one snapshot suite that nobody’s touched since the iPhone X.

That was us at Frontier Airlines. Every branch, every commit, every merge — the whole test suite ran. Always. Even when 90% of those tests had nothing to do with what changed.

We were wasting CI cycles. Devs were losing flow. And our pipelines? Full-on bloat mode.

That’s when we grabbed the scalpel instead of the sledgehammer.

  • Modular .xctestplan configs
  • Surgical CLI runs with only-testing
  • Smart test routing in Azure DevOps, GitHub Actions, and Fastlane

No need for heavy external tools — just Xcode’s own test arsenal, used with intent.

🫶 Quick thing before we keep going: We’ll cut the fat, speed up feedback, and bring order back to your test chaos — all without bolting on another tool. If that saves you some sanity, smashing that 👏 button (up to 50 times!) helps it reach more iOS devs. Appreciate it, seriously.

🚦 Why Custom Test Suites Actually Matter

This isn’t just about speed. It’s about precision.

You don’t need a full regression run every time someone renames a property in a Swift Package. You need the right tests, in the right contexts, across the right targets.

Here’s what we saw once we cleaned up our setup:

  • PRs ran scoped unit tests only
  • Feature branches ran modular test plans
  • Nightlies ran everything — app, UI, packages, snapshots
  • CI bills went down, dev confidence went up

It’s like touring with Metallica — you don’t play St. Anger every night unless you’re asking for complaints.

🧪 What’s a Test Suite in Xcode, Really?

Here’s where devs sometimes get tripped up. In Xcode, “test suite” can refer to a few very different things:

XCTestSuite: A programmatically defined collection of test cases. You won’t use this much unless you’re rolling your own runner.

.xctestplan: A declarative file (JSON under the hood) that defines which tests to run, how often, and under what conditions — great for CI.

only-testing: A command-line flag used with xcodebuild or Fastlane to surgically run specific test cases or classes.

Still running ‘All Tests’ on every commit? You’re burning time. Let’s fix it.

🗂 Creating a .xctestplan That Spans App Targets & Swift Packages

Xcode supports .xctestplan natively — even in Swift Package-based projects. Here’s how to set one up that scopes tests cleanly:

In Xcode

  • Select your scheme → Edit Scheme
  • Under “Test”, click “Convert to Test Plan”
  • Create a new .xctestplan file
  • Add test targets: both app and Swift Packages
  • Configure selected/disabled tests

⚠️ Swift Package test targets don’t show up automatically in Xcode’s test plan UI. You’ll need to manually expose them via your scheme and ensure they’re ticked in the test plan file. Otherwise, they silently get skipped on CI.

Under the hood (JSON)Here’s a clean example:

swift { "configurations": [ { "name": "Default", "options": { "targetForDevice": true, "targetForSimulator": true } } ], "defaultOptions": { "testRepetitionMode": "retry_on_failure", "maximumTestRepetitions": 2, "onlyTestIdentifiers": [], "skipTestIdentifiers": [] }, "testTargets": [ { "target": "AppUnitTests" }, { "target": "CoreLibPackageTests" } ] } swift

Pro Tip
You can create multiple test plans:

  • UnitTests.xctestplan for fast PR runs
  • FullRegression.xctestplan for nightly/weekly CI
  • CriticalTests.xctestplan for release validation

🔄 Want even more control? Test plans let you enable fail-fast mode to stop after the first failure, repeat flaky tests automatically, or even randomize test order to catch hidden test interdependencies.

🧼 Running Specific Tests with only-testing (CLI & Fastlane)

Here’s where the CLI comes in clutch. You don’t always need a full plan — sometimes, you just want to run one class or even one method.

Run a specific test case:

swift xcodebuild \ -scheme MyApp \ -sdk iphonesimulator \ -destination 'platform=iOS Simulator,name=iPhone 15' \ -only-testing:MyAppTests/MyFeatureTests swift

Run just one method:

swift -only-testing:MyAppTests/MyFeatureTests/testFlightStatusUpdates swift

Run Swift Package tests:

swift -only-testing:CoreLibPackageTests/BookingParserTests swift

In Fastlane scan:

swift scan( scheme: "MyApp", only_testing: ["MyAppTests/MyFeatureTests"] ) swift

No fluff. Just the tests you meant to run.

🔁 Plugging This Into CI (ADO, GitHub Actions, Fastlane)

At Frontier, our ADO pipeline conditionally ran different test plans based on the branch or file changes. Here’s how we approached it:

In Azure DevOps:

```swift

  • script: |
    if git diff –name-only origin/main…HEAD | grep -q ‘^Sources/App/’; then
    TESTPLAN=AppOnly.xctestplan
    else
    TESTPLAN=UnitOnly.xctestplan
    fi
    xcodebuild -scheme MyApp -testPlan $TESTPLAN ```swift

In GitHub Actions:

```swift

  • name: Run only impacted tests
    run: |
    CHANGED=$(git diff –name-only $ $)
    if echo “$CHANGED” | grep -q ‘Sources/FeatureX’; then
    PLAN=FeatureXTests.xctestplan
    else
    PLAN=FastTests.xctestplan
    fi
    xcodebuild -scheme MyApp -testPlan $PLAN ```swift

In Fastlane:

swift if git_diff_contains("Sources/Booking") scan(testplan: "BookingTests.xctestplan") else scan(testplan: "UnitTests.xctestplan") end swift

Test plans are just files — which means you can swap ’em, trigger ’em, version ’em, and never wonder what’s running again.

💥 Gotchas & CI Landmines

Here’s what bit us — and how you can avoid stepping in the same traps:

  • Swift Package tests must be in your scheme
  • Xcode sometimes forgets selectedTests after plan changes
  • UI tests in.xctestplan don’t parallel cleanly** unless isolated by device
  • Clean CI builds need pre-resolved dependencies or test plans might not discover them
  • Tag flaky tests using a prefix like test_flaky_, then run them in a separate job or report them without breaking the build
  • Use Git-based test routing with tools like danger-swift, custom Git diffs, or impact analysis scripts to target only the changed areas
  • Name your test plans like you name tour stops: FlightDeckRegression.xctestplan, BlackBoxSmoke.xctestplan, or SeekAndDestroyUITests.swift

✅ TL;DR Comparison Summary

Test Plan

  • Works across targets and Swift Packages
  • Easily used in Xcode and CI
  • Best for structured plans like nightly/full regression

Only-Testing

  • CLI/CI-friendly and fast
  • Target specific classes or test methods
  • Best for Fastlane and fine-grained control

XCTestSuite

  • Limited cross-target support
  • Doesn’t support Swift Packages directly
  • Useful for custom runners or harnesses

🧰 Bonus: Utility Snippets

Copy-Paste Ready: CI Snippet Pack

```swift

Run a full test plan

xcodebuild -scheme MyApp -testPlan UnitTests

Run one test class

xcodebuild -scheme MyApp -only-testing:MyAppTests/MyFeatureTests

Run one test method

xcodebuild -only-testing:MyAppTests/MyFeatureTests/testWhatever

Skip flaky test

Add to your .xctestplan JSON

“skipTestIdentifiers”: [
“MyAppTests/MyCoolTests/testSometimesFails”
] ```swift

Test plan JSON base:

swift { "configurations": [{"name": "Default"}], "testTargets": [{"target": "AppUnitTests"}] } swift

🤘 Wrap-Up

Test suites aren’t all-or-nothing. With test plans and only-testing, you can control what runs, when, and where — without sacrificing safety.

We stopped treating our suite like a greatest hits album — and started using setlists.

Your CI doesn’t need to run Master of Puppets on every push. Just hit the tracks that matter for that show. Save Seek & Destroy for release day.

Got a favorite way to split tests? Or war stories from a CI meltdown? Hit me up.

🎯 Bonus: More Real-World iOS Survival Stories

If you’re hungry for more tips, tricks, and a few battle-tested stories from the trenches of native mobile development, swing by my collection of articles:
https://medium.com/@wesleymatlock

These posts are packed with real-world solutions, some laughs, and the kind of knowledge that’s saved me from a few late-night debugging sessions. Let’s keep building apps that rock — and if you’ve got questions or stories of your own, drop me a line. I’d love to hear from you.

Originally published on Medium

Originally published on Medium

← Back to Blog