Skip to content

Commit 8efc119

Browse files
VahilaVahila
authored
Add Azure parallel strategy example
* Add azure parallel startegy example * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Update file * Address review comments * Address review comments * Address review comments * Update buildtool task * Update README * Update README * Update file * Update file * Address review comments * Address review comments
1 parent 7d386ae commit 8efc119

File tree

4 files changed

+182
-1
lines changed

4 files changed

+182
-1
lines changed

‎AzureDevOps/DistributeTests.ps1‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
<#
2+
.SYNOPSIS
3+
Distribute tests across multiple agents in VSTS pipeline
4+
.DESCRIPTION
5+
This script divides test files across multiple agents for faster execution. It searches for files matching a specific pattern (for example, `test*`) and assigns them based on the agent number.
6+
For example, if there are multiple files [test1..test10] and 2 agents:
7+
- Agent 1 runs tests from odd-numbered files.
8+
- Agent 2 runs tests from even-numbered files.
9+
For detailed slicing information, see https://learn.microsoft.com/en-us/azure/devops/pipelines/test/parallel-testing-any-test-runner?view=azure-devops
10+
#>
11+
12+
$tests = Get-ChildItem .\tests\ -Filter *.m -File # Search for test files matching the specified pattern
13+
$totalAgents = [int]$Env:SYSTEM_TOTALJOBSINPHASE # Standard VSTS variable containing the number of parallel jobs
14+
$agentNumber = [int]$Env:SYSTEM_JOBPOSITIONINPHASE # Current job position
15+
$testCount = $tests.Count
16+
17+
# Handle cases where the pipeline runs without parallel configuration (single agent)
18+
if ($totalAgents -eq 0) {
19+
$totalAgents = 1
20+
}
21+
if (!$agentNumber -or $agentNumber -eq 0) {
22+
$agentNumber = 1
23+
}
24+
25+
Write-Host "Total agents: $totalAgents"
26+
Write-Host "Agent number: $agentNumber"
27+
Write-Host "Total tests: $testCount"
28+
29+
$testsToRun= @()
30+
# Slice test files so each agent gets a unique set of files to execute
31+
For ($i=$agentNumber; $i -le $testCount;) {
32+
$file = $tests[$i-1]
33+
34+
$fileName = [System.IO.Path]::GetFileNameWithoutExtension($file.Name)
35+
$testsToRun += "$fileName/*"
36+
$i = $i + $totalAgents
37+
}
38+
39+
# Join all test files into a space-separated string
40+
$testFiles = $testsToRun -Join " "
41+
Write-Host "Tests to run $testFiles"
42+
# Write files into a variable for execution in a subsequent task
43+
Write-Host "##vso[task.setvariable variable=MATLABTestFiles;]$testFiles"

‎AzureDevOps/DistributeTests.sh‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/bin/bash
2+
#===============================================================================
3+
#
4+
# FILE: distribute_tests.sh
5+
#
6+
# USAGE: ./distribute_tests.sh
7+
#
8+
# DESCRIPTION: This script divides test files across multiple agents for faster execution. It searches for files matching a specific pattern (for example, `test*`) and assigns them based on the agent number.
9+
# For example, if there are multiple files [test1..test10] and 2 agents:
10+
# - Agent 1 runs tests from odd-numbered files.
11+
# - Agent 2 runs tests from even-numbered files.
12+
# For detailed slicing information, see https://learn.microsoft.com/en-us/azure/devops/pipelines/test/parallel-testing-any-test-runner?view=azure-devops
13+
#
14+
#===============================================================================
15+
16+
tests=()
17+
# Search for test files matching the specified pattern
18+
while IFS= read -r file; do
19+
tests+=("$file")
20+
done < <(find ./tests -type f -name "*.m" | sort)
21+
22+
totalAgents=${SYSTEM_TOTALJOBSINPHASE} # Standard VSTS variable containing the number of parallel jobs
23+
agentNumber=${SYSTEM_JOBPOSITIONINPHASE} # Current job position
24+
testCount=${#tests[@]}
25+
26+
# Handle cases where the pipeline runs without parallel configuration (single agent)
27+
if [ $totalAgents -eq 0 ]; then
28+
totalAgents=1
29+
fi
30+
if [ -z $agentNumber ]; then
31+
agentNumber=1
32+
fi
33+
34+
echo "Total agents: $totalAgents"
35+
echo "Agent number: $agentNumber"
36+
echo "Total tests: $testCount"
37+
38+
testsToRun=()
39+
# Slice test files so each agent gets a unique set of files
40+
for (( i=agentNumber; i<=testCount; i+=totalAgents )); do
41+
file="${tests[i-1]}"
42+
43+
fileName=$(basename "$file" .m)
44+
testsToRun+=("${fileName}/*")
45+
done
46+
47+
# Join all test files into a space-separated string
48+
testFiles="${testsToRun[*]}"
49+
echo "Tests to run $testFiles"
50+
51+
# Write files into a variable for execution in a subsequent task
52+
echo "##vso[task.setvariable variable=MATLABTestFiles;]$testFiles"

‎AzureDevOps/ParallelStrategy.yml‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
jobs:
2+
- job: ParallelWindows
3+
# Specify 'parallel' strategy to run tests in parallel
4+
strategy:
5+
parallel: 2
6+
pool:
7+
vmImage: windows-latest
8+
steps:
9+
# Install MATLAB and required products
10+
- task: InstallMATLAB@1
11+
inputs:
12+
products: MATLAB_Compiler_SDK MATLAB_Test
13+
14+
- task: RunMATLABBuild@1
15+
inputs:
16+
tasks: mex buildPythonPackage
17+
env:
18+
MLM_LICENSE_TOKEN: $(MLM_LICENSE_TOKEN)
19+
20+
- powershell: .\AzureDevOps\DistributeTests.ps1
21+
displayName: 'PowerShell script to distribute tests'
22+
23+
- task: RunMATLABTests@1
24+
inputs:
25+
selectByName: $(MATLABTestFiles)
26+
sourceFolder: src
27+
env:
28+
MLM_LICENSE_TOKEN: $(MLM_LICENSE_TOKEN)
29+
30+
- job: ParallelLinux
31+
# Specify 'parallel' strategy to run tests in parallel
32+
strategy:
33+
parallel: 2
34+
pool:
35+
vmImage: ubuntu-latest
36+
steps:
37+
# Install MATLAB and required products
38+
- task: InstallMATLAB@1
39+
inputs:
40+
products: MATLAB_Compiler_SDK MATLAB_Test
41+
42+
- task: RunMATLABBuild@1
43+
inputs:
44+
tasks: mex buildPythonPackage
45+
env:
46+
MLM_LICENSE_TOKEN: $(MLM_LICENSE_TOKEN)
47+
48+
- bash: chmod +x ./AzureDevOps/DistributeTests.sh && ./AzureDevOps/DistributeTests.sh
49+
displayName: 'Bash script to distribute tests'
50+
51+
- task: RunMATLABTests@1
52+
inputs:
53+
selectByName: $(MATLABTestFiles)
54+
sourceFolder: src
55+
env:
56+
MLM_LICENSE_TOKEN: $(MLM_LICENSE_TOKEN)
57+
58+
- job: ParallelMac
59+
# Specify 'parallel' strategy to run tests in parallel
60+
strategy:
61+
parallel: 2
62+
pool:
63+
vmImage: macOS-latest
64+
steps:
65+
# Install MATLAB and required products
66+
- task: InstallMATLAB@1
67+
inputs:
68+
products: MATLAB_Compiler_SDK MATLAB_Test
69+
70+
- task: RunMATLABBuild@1
71+
inputs:
72+
tasks: mex buildPythonPackage
73+
env:
74+
MLM_LICENSE_TOKEN: $(MLM_LICENSE_TOKEN)
75+
76+
- bash: chmod +x ./AzureDevOps/DistributeTests.sh && ./AzureDevOps/DistributeTests.sh
77+
displayName: 'Bash script to distribute tests'
78+
79+
- task: RunMATLABTests@1
80+
inputs:
81+
selectByName: $(MATLABTestFiles)
82+
sourceFolder: src
83+
env:
84+
MLM_LICENSE_TOKEN: $(MLM_LICENSE_TOKEN)

‎README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ For starter workflows, use the [`ci-configuration-examples`](https://github.com/
1414

1515
# Workflows
1616

17-
The repository contains examples for packaging and distributing a toolbox, as well as building and uploading Python<sup>&reg;</sup> packages.
17+
The repository contains examples for packaging and distributing a toolbox, building and uploading Python<sup>&reg;</sup> packages, and running tests across multiple build agents.
1818

1919
- **Package and Distribute Toolbox**: Using a matrix build across multiple platforms, compile, link, and test your C source files to produce a binary MEX file per operating system. Then, bundle the resulting binaries into a toolbox and distribute it as a GitHub release.
2020

@@ -36,6 +36,8 @@ The repository contains examples for packaging and distributing a toolbox, as we
3636
| GitHub Actions| [`.github/workflows/CrossPlatformBuilder.yml`](https://github.com/mathworks/advanced-ci-configuration-examples/blob/main/.github/workflows/CrossPlatformBuilder.yml) |
3737
| Jenkins | [`Jenkins/CrossPlatformBuilder/Jenkinsfile`](https://github.com/mathworks/advanced-ci-configuration-examples/blob/main/Jenkins/CrossPlatformBuilder/Jenkinsfile) |
3838

39+
- **Run Tests Across Multiple Agents**: Use the [parallel strategy](https://learn.microsoft.com/en-us/azure/devops/pipelines/test/parallel-testing-any-test-runner?view=azure-devops) in Azure DevOps to run tests across multiple agents and speed up the testing process. For configuration details, see the example in [`AzureDevOps/ParallelStrategy.yml`](https://github.com/mathworks/advanced-ci-configuration-examples/blob/main/AzureDevOps/ParallelStrategy.yml).
40+
3941
<br>
4042

4143
## Get Started

0 commit comments

Comments
 (0)