Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions references/benchmark-artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Recommended outputs:
- `.build-benchmark/<timestamp>-<scheme>-clean-1.log`
- `.build-benchmark/<timestamp>-<scheme>-clean-2.log`
- `.build-benchmark/<timestamp>-<scheme>-clean-3.log`
- `.build-benchmark/<timestamp>-<scheme>-cached-clean-1.log` (when COMPILATION_CACHING is enabled)
- `.build-benchmark/<timestamp>-<scheme>-cached-clean-1.log` (when COMPILATION_CACHE_ENABLE_CACHING is enabled)
- `.build-benchmark/<timestamp>-<scheme>-cached-clean-2.log`
- `.build-benchmark/<timestamp>-<scheme>-cached-clean-3.log`
- `.build-benchmark/<timestamp>-<scheme>-incremental-1.log`
Expand Down Expand Up @@ -50,7 +50,7 @@ Each JSON artifact should include:
Do not merge different build type measurements into a single list. They answer different questions:

- **Clean builds** show full build-system, package, and module setup cost with a cold compilation cache.
- **Cached clean builds** show clean build cost when the compilation cache is warm. This is the realistic scenario for branch switching, pulling changes, or Clean Build Folder. Only present when `COMPILATION_CACHING = YES` is detected.
- **Cached clean builds** show clean build cost when the compilation cache is warm. This is the realistic scenario for branch switching, pulling changes, or Clean Build Folder. Only present when `COMPILATION_CACHE_ENABLE_CACHING = YES` is detected.
- **Incremental builds** show edit-loop productivity and script or cache invalidation problems.

## Raw Logs
Expand All @@ -64,17 +64,17 @@ Store raw `xcodebuild` output beside the JSON artifact whenever possible. That a

## Measurement Caveats

### COMPILATION_CACHING
### COMPILATION_CACHE_ENABLE_CACHING

`COMPILATION_CACHING = YES` stores compiled artifacts in a system-managed cache outside DerivedData so that repeated compilations of identical inputs are served from cache. The standard clean-build benchmark (`xcodebuild clean` between runs) may add overhead from cache population without showing the corresponding cache-hit benefit.
`COMPILATION_CACHE_ENABLE_CACHING = YES` stores compiled artifacts in a system-managed cache outside DerivedData so that repeated compilations of identical inputs are served from cache. The standard clean-build benchmark (`xcodebuild clean` between runs) may add overhead from cache population without showing the corresponding cache-hit benefit.

The benchmark script automatically detects `COMPILATION_CACHING = YES` and runs a **cached clean** benchmark phase. This phase:
The benchmark script automatically detects `COMPILATION_CACHE_ENABLE_CACHING = YES` and runs a **cached clean** benchmark phase. This phase:

1. Builds once to warm the compilation cache.
2. Deletes DerivedData (but not the compilation cache) before each measured run.
3. Rebuilds, measuring the cache-hit clean build time.

The cached clean metric captures the realistic developer experience: branch switching, pulling changes, and Clean Build Folder. Use the cached clean median as the primary comparison metric when evaluating `COMPILATION_CACHING` impact.
The cached clean metric captures the realistic developer experience: branch switching, pulling changes, and Clean Build Folder. Use the cached clean median as the primary comparison metric when evaluating `COMPILATION_CACHE_ENABLE_CACHING` impact.

To skip this phase, pass `--no-cached-clean`.

Expand Down
2 changes: 1 addition & 1 deletion references/build-optimization-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ Source:

Key takeaways:

- Granular caching is controlled by `SWIFT_ENABLE_COMPILE_CACHE` and `CLANG_ENABLE_COMPILE_CACHE`, under the umbrella `COMPILATION_CACHING` setting.
- Granular caching is controlled by `SWIFT_ENABLE_COMPILE_CACHE` and `CLANG_ENABLE_COMPILE_CACHE`, under the umbrella `COMPILATION_CACHE_ENABLE_CACHING` setting.
- Non-cacheable tasks include `CompileStoryboard`, `CompileXIB`, `CompileAssetCatalogVariant`, `PhaseScriptExecution`, `DataModelCompile`, `CopyPNGFile`, `GenerateDSYMFile`, and `Ld`.
- SPM dependencies are not yet cacheable as of Xcode 26 beta.

Expand Down
4 changes: 2 additions & 2 deletions references/build-settings-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ These settings optimize for production builds.

### Compilation Caching

- **Key:** `COMPILATION_CACHING`
- **Key:** `COMPILATION_CACHE_ENABLE_CACHING`
- **Recommended:** `YES`
- **Why:** Caches compilation results for Swift and C-family sources so repeated compilations of the same inputs are served from cache. The biggest wins come from branch switching and clean builds where source files are recompiled unchanged. This is an opt-in feature. The umbrella setting controls both `SWIFT_ENABLE_COMPILE_CACHE` and `CLANG_ENABLE_COMPILE_CACHE` under the hood; those can be toggled independently if needed.
- **Measurement:** Measured 5-14% faster clean builds across tested projects (87 to 1,991 Swift files). The benefit compounds in real developer workflows where the cache persists between builds -- branch switching, pulling changes, and CI with persistent DerivedData -- though the exact savings depend on how many files change between builds.
Expand Down Expand Up @@ -206,7 +206,7 @@ When reporting results, use this structure:
...

### General (All Configurations)
- [ ] `COMPILATION_CACHING`: `NO` (recommended: `YES`)
- [ ] `COMPILATION_CACHE_ENABLE_CACHING`: `NO` (recommended: `YES`)
...

### Cross-Target Consistency
Expand Down
8 changes: 4 additions & 4 deletions scripts/benchmark_builds.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def parse_args() -> argparse.Namespace:
parser.add_argument(
"--no-cached-clean",
action="store_true",
help="Skip cached clean builds even when COMPILATION_CACHING is detected.",
help="Skip cached clean builds even when COMPILATION_CACHE_ENABLE_CACHING is detected.",
)
parser.add_argument(
"--extra-arg",
Expand Down Expand Up @@ -142,13 +142,13 @@ def xcode_version() -> str:


def detect_compilation_caching(base_command: List[str]) -> bool:
"""Check whether COMPILATION_CACHING is enabled in the resolved build settings."""
"""Check whether COMPILATION_CACHE_ENABLE_CACHING is enabled in the resolved build settings."""
result = run_command([*base_command, "-showBuildSettings"])
if result.returncode != 0:
return False
for line in result.stdout.splitlines():
stripped = line.strip()
if stripped.startswith("COMPILATION_CACHING") and "=" in stripped:
if stripped.startswith("COMPILATION_CACHE_ENABLE_CACHING") and "=" in stripped:
value = stripped.split("=", 1)[1].strip()
return value == "YES"
return False
Expand Down Expand Up @@ -214,7 +214,7 @@ def main() -> int:
runs["clean"].append(measure_build(base_command, artifact_stem, output_dir, "clean", index))

# --- Cached clean builds ---------------------------------------------------
# When COMPILATION_CACHING is enabled, the compilation cache lives outside
# When COMPILATION_CACHE_ENABLE_CACHING is enabled, the compilation cache lives outside
# DerivedData and survives product deletion. We measure "cached clean"
# builds by pointing DerivedData at a temp directory, warming the cache with
# one build, then deleting the DerivedData directory (but not the cache)
Expand Down
6 changes: 3 additions & 3 deletions scripts/generate_optimization_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ def _parse_target_configs(pbxproj: str) -> Dict[str, Dict[str, Dict[str, str]]]:
# ---------------------------------------------------------------------------

_DEBUG_EXPECTATIONS: List[Tuple[str, str, str]] = [
("SWIFT_COMPILATION_MODE", "incremental", "Incremental recompiles only changed files"),
("SWIFT_COMPILATION_MODE", "singlefile", "Single-file mode recompiles only changed files (Xcode UI: Incremental)"),
("SWIFT_OPTIMIZATION_LEVEL", "-Onone", "Optimization passes add compile time without debug benefit"),
("GCC_OPTIMIZATION_LEVEL", "0", "C/ObjC optimization adds compile time without debug benefit"),
("ONLY_ACTIVE_ARCH", "YES", "Building all architectures multiplies compile and link time"),
Expand All @@ -99,7 +99,7 @@ def _parse_target_configs(pbxproj: str) -> Dict[str, Dict[str, Dict[str, str]]]:
]

_GENERAL_EXPECTATIONS: List[Tuple[str, str, str]] = [
("COMPILATION_CACHING", "YES", "Caches compilation results so repeat builds of unchanged inputs are served from cache. Measured 5-14% faster clean builds across tested projects; benefit compounds during branch switching and pulling changes"),
("COMPILATION_CACHE_ENABLE_CACHING", "YES", "Caches compilation results so repeat builds of unchanged inputs are served from cache. Measured 5-14% faster clean builds across tested projects; benefit compounds during branch switching and pulling changes"),
]

_RELEASE_EXPECTATIONS: List[Tuple[str, str, str]] = [
Expand Down Expand Up @@ -127,7 +127,7 @@ def _effective_value(

def _check(actual: Optional[str], expected: str) -> bool:
if actual is None:
if expected in ("incremental",):
if expected in ("singlefile",):
return True
return False
if expected == "-O" and actual in ("-O", '"-O"', '"-Osize"', "-Osize"):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ Source:

Key takeaways:

- Granular caching is controlled by `SWIFT_ENABLE_COMPILE_CACHE` and `CLANG_ENABLE_COMPILE_CACHE`, under the umbrella `COMPILATION_CACHING` setting.
- Granular caching is controlled by `SWIFT_ENABLE_COMPILE_CACHE` and `CLANG_ENABLE_COMPILE_CACHE`, under the umbrella `COMPILATION_CACHE_ENABLE_CACHING` setting.
- Non-cacheable tasks include `CompileStoryboard`, `CompileXIB`, `CompileAssetCatalogVariant`, `PhaseScriptExecution`, `DataModelCompile`, `CopyPNGFile`, `GenerateDSYMFile`, and `Ld`.
- SPM dependencies are not yet cacheable as of Xcode 26 beta.

Expand Down
4 changes: 2 additions & 2 deletions skills/xcode-build-benchmark/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ When benchmarking inside a git worktree, SPM packages with `exclude:` paths that
1. Normalize the build command and note every flag that affects caching or module reuse.
2. Run one warm-up build if needed to validate that the command succeeds.
3. Run 3 clean builds.
4. If `COMPILATION_CACHING = YES` is detected, run 3 cached clean builds. These measure clean build time with a warm compilation cache -- the realistic scenario for branch switching, pulling changes, or Clean Build Folder. The script handles this automatically by building once to warm the cache, then deleting DerivedData (but not the compilation cache) before each measured run. Pass `--no-cached-clean` to skip.
4. If `COMPILATION_CACHE_ENABLE_CACHING = YES` is detected, run 3 cached clean builds. These measure clean build time with a warm compilation cache -- the realistic scenario for branch switching, pulling changes, or Clean Build Folder. The script handles this automatically by building once to warm the cache, then deleting DerivedData (but not the compilation cache) before each measured run. Pass `--no-cached-clean` to skip.
5. Run 3 zero-change builds (build immediately after a successful build with no edits). This measures the fixed overhead floor: dependency computation, project description transfer, build description creation, script phases, codesigning, and validation. A zero-change build that takes more than a few seconds indicates avoidable per-build overhead. Use the default `benchmark_builds.py` invocation (no `--touch-file` flag).
6. Optionally run 3 incremental builds with a file touch to measure a real edit-rebuild loop. Use `--touch-file path/to/SomeFile.swift` to touch a representative source file before each build.
7. Save the raw results and summary into `.build-benchmark/`.
Expand All @@ -63,7 +63,7 @@ If you cannot use the helper script, run equivalent `xcodebuild` commands with `
Return:

- clean build median, min, max
- cached clean build median, min, max (when COMPILATION_CACHING is enabled)
- cached clean build median, min, max (when COMPILATION_CACHE_ENABLE_CACHING is enabled)
- zero-change build median, min, max (fixed overhead floor)
- incremental build median, min, max (if `--touch-file` was used)
- biggest timing-summary categories
Expand Down
12 changes: 6 additions & 6 deletions skills/xcode-build-benchmark/references/benchmark-artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Recommended outputs:
- `.build-benchmark/<timestamp>-<scheme>-clean-1.log`
- `.build-benchmark/<timestamp>-<scheme>-clean-2.log`
- `.build-benchmark/<timestamp>-<scheme>-clean-3.log`
- `.build-benchmark/<timestamp>-<scheme>-cached-clean-1.log` (when COMPILATION_CACHING is enabled)
- `.build-benchmark/<timestamp>-<scheme>-cached-clean-1.log` (when COMPILATION_CACHE_ENABLE_CACHING is enabled)
- `.build-benchmark/<timestamp>-<scheme>-cached-clean-2.log`
- `.build-benchmark/<timestamp>-<scheme>-cached-clean-3.log`
- `.build-benchmark/<timestamp>-<scheme>-incremental-1.log`
Expand Down Expand Up @@ -50,7 +50,7 @@ Each JSON artifact should include:
Do not merge different build type measurements into a single list. They answer different questions:

- **Clean builds** show full build-system, package, and module setup cost with a cold compilation cache.
- **Cached clean builds** show clean build cost when the compilation cache is warm. This is the realistic scenario for branch switching, pulling changes, or Clean Build Folder. Only present when `COMPILATION_CACHING = YES` is detected.
- **Cached clean builds** show clean build cost when the compilation cache is warm. This is the realistic scenario for branch switching, pulling changes, or Clean Build Folder. Only present when `COMPILATION_CACHE_ENABLE_CACHING = YES` is detected.
- **Incremental builds** show edit-loop productivity and script or cache invalidation problems.

## Raw Logs
Expand All @@ -64,17 +64,17 @@ Store raw `xcodebuild` output beside the JSON artifact whenever possible. That a

## Measurement Caveats

### COMPILATION_CACHING
### COMPILATION_CACHE_ENABLE_CACHING

`COMPILATION_CACHING = YES` stores compiled artifacts in a system-managed cache outside DerivedData so that repeated compilations of identical inputs are served from cache. The standard clean-build benchmark (`xcodebuild clean` between runs) may add overhead from cache population without showing the corresponding cache-hit benefit.
`COMPILATION_CACHE_ENABLE_CACHING = YES` stores compiled artifacts in a system-managed cache outside DerivedData so that repeated compilations of identical inputs are served from cache. The standard clean-build benchmark (`xcodebuild clean` between runs) may add overhead from cache population without showing the corresponding cache-hit benefit.

The benchmark script automatically detects `COMPILATION_CACHING = YES` and runs a **cached clean** benchmark phase. This phase:
The benchmark script automatically detects `COMPILATION_CACHE_ENABLE_CACHING = YES` and runs a **cached clean** benchmark phase. This phase:

1. Builds once to warm the compilation cache.
2. Deletes DerivedData (but not the compilation cache) before each measured run.
3. Rebuilds, measuring the cache-hit clean build time.

The cached clean metric captures the realistic developer experience: branch switching, pulling changes, and Clean Build Folder. Use the cached clean median as the primary comparison metric when evaluating `COMPILATION_CACHING` impact.
The cached clean metric captures the realistic developer experience: branch switching, pulling changes, and Clean Build Folder. Use the cached clean median as the primary comparison metric when evaluating `COMPILATION_CACHE_ENABLE_CACHING` impact.

To skip this phase, pass `--no-cached-clean`.

Expand Down
8 changes: 4 additions & 4 deletions skills/xcode-build-benchmark/scripts/benchmark_builds.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def parse_args() -> argparse.Namespace:
parser.add_argument(
"--no-cached-clean",
action="store_true",
help="Skip cached clean builds even when COMPILATION_CACHING is detected.",
help="Skip cached clean builds even when COMPILATION_CACHE_ENABLE_CACHING is detected.",
)
parser.add_argument(
"--extra-arg",
Expand Down Expand Up @@ -142,13 +142,13 @@ def xcode_version() -> str:


def detect_compilation_caching(base_command: List[str]) -> bool:
"""Check whether COMPILATION_CACHING is enabled in the resolved build settings."""
"""Check whether COMPILATION_CACHE_ENABLE_CACHING is enabled in the resolved build settings."""
result = run_command([*base_command, "-showBuildSettings"])
if result.returncode != 0:
return False
for line in result.stdout.splitlines():
stripped = line.strip()
if stripped.startswith("COMPILATION_CACHING") and "=" in stripped:
if stripped.startswith("COMPILATION_CACHE_ENABLE_CACHING") and "=" in stripped:
value = stripped.split("=", 1)[1].strip()
return value == "YES"
return False
Expand Down Expand Up @@ -214,7 +214,7 @@ def main() -> int:
runs["clean"].append(measure_build(base_command, artifact_stem, output_dir, "clean", index))

# --- Cached clean builds ---------------------------------------------------
# When COMPILATION_CACHING is enabled, the compilation cache lives outside
# When COMPILATION_CACHE_ENABLE_CACHING is enabled, the compilation cache lives outside
# DerivedData and survives product deletion. We measure "cached clean"
# builds by pointing DerivedData at a temp directory, warming the cache with
# one build, then deleting the DerivedData directory (but not the cache)
Expand Down
8 changes: 4 additions & 4 deletions skills/xcode-build-fixer/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Typical fixes:

- Set `DEBUG_INFORMATION_FORMAT = dwarf` for Debug
- Set `SWIFT_COMPILATION_MODE = singlefile` for Debug
- Enable `COMPILATION_CACHING = YES`
- Enable `COMPILATION_CACHE_ENABLE_CACHING = YES`
- Enable `EAGER_LINKING = YES` for Debug
- Align cross-target settings to eliminate module variants

Expand Down Expand Up @@ -106,7 +106,7 @@ Not every slower number is a true regression. The fixer must evaluate the full p

### Compilation caching trade-off

A change like `COMPILATION_CACHING = YES` may make a standard clean build slightly slower (cache population overhead) while making cached clean builds significantly faster. Since cached clean builds reflect the realistic developer workflow (branch switching, pulling changes, Clean Build Folder with a warm cache), a slower standard clean build paired with a faster cached clean build is a net improvement, not a regression. The same logic applies to any change where the first-time cost is higher but subsequent builds benefit.
A change like `COMPILATION_CACHE_ENABLE_CACHING = YES` may make a standard clean build slightly slower (cache population overhead) while making cached clean builds significantly faster. Since cached clean builds reflect the realistic developer workflow (branch switching, pulling changes, Clean Build Folder with a warm cache), a slower standard clean build paired with a faster cached clean build is a net improvement, not a regression. The same logic applies to any change where the first-time cost is higher but subsequent builds benefit.

### Compare all build types

Expand All @@ -122,7 +122,7 @@ Some build settings are Apple's recommended modern defaults. These should be app

Best-practice settings that should always be kept once applied:

- `COMPILATION_CACHING = YES` -- Apple is actively investing in this; the cache improves with each Xcode release and compounds across real workflows
- `COMPILATION_CACHE_ENABLE_CACHING = YES` -- Apple is actively investing in this; the cache improves with each Xcode release and compounds across real workflows
- `EAGER_LINKING = YES` (Debug) -- allows the linker to overlap with compilation
- `SWIFT_USE_INTEGRATED_DRIVER = YES` -- eliminates inter-process scheduling overhead
- `DEBUG_INFORMATION_FORMAT = dwarf` (Debug) -- avoids unnecessary dSYM generation
Expand Down Expand Up @@ -162,7 +162,7 @@ If a fix produced no measurable wall-time improvement, note `No measurable wall-

For changes valuable for non-benchmark reasons (deterministic package resolution, branch-switch caching), label them: "No wait-time improvement expected from this change. The benefit is [deterministic builds / faster branch switching / reduced CI cost]."

Note: `COMPILATION_CACHING` has been measured at 5-14% faster clean builds across tested projects (87 to 1,991 Swift files). The benefit compounds in real developer workflows where the cache persists between builds -- branch switching, pulling changes, and CI with persistent DerivedData. The benchmark script auto-detects this setting and runs a cached clean phase for validation.
Note: `COMPILATION_CACHE_ENABLE_CACHING` has been measured at 5-14% faster clean builds across tested projects (87 to 1,991 Swift files). The benefit compounds in real developer workflows where the cache persists between builds -- branch switching, pulling changes, and CI with persistent DerivedData. The benchmark script auto-detects this setting and runs a cached clean phase for validation.

## Execution Report

Expand Down
Loading
Loading