-
Notifications
You must be signed in to change notification settings - Fork 0
Add BenchmarkDotNet benchmarks and manual workflow trigger #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
0feefe7
feat: Add BenchmarkDotNet benchmarks for performance evaluation and C…
koenbeuk 4c4a30c
feat: Enable manual triggering of benchmark workflow with workflow_di…
koenbeuk a1fe7c4
feat: Improve benchmark workflow by adding error handling and updatin…
koenbeuk 5c12a51
Update benchmarks/ExpressiveSharp.Benchmarks/ExpressiveSharp.Benchmar…
koenbeuk ca55511
feat: Remove redundant benchmark for WithExpressives_BlockBody
koenbeuk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| name: Benchmarks | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| pull_request: | ||
| branches: [main] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| benchmark: | ||
| name: Run Benchmarks | ||
| runs-on: ubuntu-latest | ||
|
|
||
| env: | ||
| CI: true | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup .NET SDK | ||
| uses: actions/setup-dotnet@v4 | ||
| with: | ||
| dotnet-version: 10.0.x | ||
|
|
||
| - name: Cache NuGet packages | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: ~/.nuget/packages | ||
| key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj', 'Directory.Packages.props') }} | ||
| restore-keys: | | ||
| nuget-${{ runner.os }}- | ||
|
|
||
| - name: Restore | ||
| run: dotnet restore | ||
|
|
||
| - name: Build | ||
| run: dotnet build --no-restore -c Release | ||
|
|
||
| - name: Run benchmarks | ||
| run: >- | ||
| dotnet run -c Release | ||
| --project benchmarks/ExpressiveSharp.Benchmarks/ExpressiveSharp.Benchmarks.csproj | ||
| -- | ||
| --filter "*" | ||
| --exporters json | ||
| --join | ||
| --job short | ||
| --iterationCount 3 | ||
| --warmupCount 1 | ||
|
|
||
| - name: Find benchmark result | ||
| id: find-result | ||
| run: | | ||
| file=$(find BenchmarkDotNet.Artifacts -name '*-report-full-compressed.json' | head -1) | ||
| if [ -z "$file" ]; then | ||
| echo "No benchmark result file found" >&2 | ||
| exit 1 | ||
| fi | ||
| echo "file=$file" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Store benchmark result | ||
| uses: benchmark-action/github-action-benchmark@v1 | ||
| with: | ||
| name: ExpressiveSharp Benchmarks | ||
| tool: benchmarkdotnet | ||
| output-file-path: ${{ steps.find-result.outputs.file }} | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||
| auto-push: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }} | ||
| comment-on-alert: true | ||
| alert-threshold: '120%' | ||
| fail-on-alert: false | ||
| gh-pages-branch: gh-pages | ||
| benchmark-data-dir-path: dev/bench | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| <Project> | ||
| <Import Project="$([MSBuild]::GetPathOfFileAbove('Directory.Build.props', '$(MSBuildThisFileDirectory)../'))" /> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFrameworks>net10.0</TargetFrameworks> | ||
| </PropertyGroup> | ||
| </Project> |
56 changes: 56 additions & 0 deletions
56
benchmarks/ExpressiveSharp.Benchmarks/EFCoreQueryOverheadBenchmarks.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| using BenchmarkDotNet.Attributes; | ||
| using ExpressiveSharp.Benchmarks.Helpers; | ||
| using Microsoft.EntityFrameworkCore; | ||
|
|
||
| namespace ExpressiveSharp.Benchmarks; | ||
|
|
||
| [MemoryDiagnoser] | ||
| public class EFCoreQueryOverheadBenchmarks | ||
| { | ||
| private TestDbContext _baselineCtx = null!; | ||
| private TestDbContext _expressiveCtx = null!; | ||
|
|
||
| [GlobalSetup] | ||
| public void Setup() | ||
| { | ||
| _baselineCtx = new TestDbContext(useExpressives: false); | ||
| _expressiveCtx = new TestDbContext(useExpressives: true); | ||
| } | ||
|
|
||
| [GlobalCleanup] | ||
| public void Cleanup() | ||
| { | ||
| _baselineCtx.Dispose(); | ||
| _expressiveCtx.Dispose(); | ||
| } | ||
|
|
||
| [Benchmark(Baseline = true)] | ||
| public string Baseline() | ||
| => _baselineCtx.Entities.Select(x => x.Id).ToQueryString(); | ||
|
|
||
| [Benchmark] | ||
| public string WithExpressives_Property() | ||
| => _expressiveCtx.Entities.Select(x => x.IdPlus1).ToQueryString(); | ||
|
|
||
| [Benchmark] | ||
| public string WithExpressives_Method() | ||
| => _expressiveCtx.Entities.Select(x => x.IdPlus1Method()).ToQueryString(); | ||
|
|
||
| [Benchmark] | ||
| public string WithExpressives_NullConditional() | ||
| => _expressiveCtx.Entities.Select(x => x.EmailLength).ToQueryString(); | ||
|
|
||
| [Benchmark] | ||
| public string ColdStart_WithExpressives() | ||
| { | ||
| using var ctx = new TestDbContext(useExpressives: true); | ||
| return ctx.Entities.Select(x => x.IdPlus1).ToQueryString(); | ||
| } | ||
|
|
||
| [Benchmark] | ||
| public string ColdStart_Baseline() | ||
| { | ||
| using var ctx = new TestDbContext(useExpressives: false); | ||
| return ctx.Entities.Select(x => x.Id).ToQueryString(); | ||
| } | ||
| } |
59 changes: 59 additions & 0 deletions
59
benchmarks/ExpressiveSharp.Benchmarks/ExpressionReplacerBenchmarks.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| using System.Linq.Expressions; | ||
| using BenchmarkDotNet.Attributes; | ||
| using ExpressiveSharp.Benchmarks.Helpers; | ||
| using ExpressiveSharp.Services; | ||
|
|
||
| namespace ExpressiveSharp.Benchmarks; | ||
|
|
||
| [MemoryDiagnoser] | ||
| public class ExpressionReplacerBenchmarks | ||
| { | ||
| private Expression _propertyExpression = null!; | ||
| private Expression _methodExpression = null!; | ||
| private Expression _nullConditionalExpression = null!; | ||
| private Expression _blockBodyExpression = null!; | ||
| private Expression _deepChainExpression = null!; | ||
|
|
||
| [GlobalSetup] | ||
| public void Setup() | ||
| { | ||
| // Build expression trees that reference [Expressive] members on TestEntity. | ||
| // When Replace is called, the replacer will resolve and inline them. | ||
| Expression<Func<TestEntity, int>> propExpr = e => e.IdPlus1; | ||
| _propertyExpression = propExpr; | ||
|
|
||
| Expression<Func<TestEntity, int>> methodExpr = e => e.IdPlus1Method(); | ||
| _methodExpression = methodExpr; | ||
|
|
||
| Expression<Func<TestEntity, int?>> nullCondExpr = e => e.EmailLength; | ||
| _nullConditionalExpression = nullCondExpr; | ||
|
|
||
| Expression<Func<TestEntity, string>> blockExpr = e => e.GetCategory(); | ||
| _blockBodyExpression = blockExpr; | ||
|
|
||
| // Deep chain: multiple [Expressive] member accesses in one tree | ||
| Expression<Func<TestEntity, string>> deepExpr = e => | ||
| $"{e.FullName} ({e.GetCategory()}) #{e.IdPlus1Method()}"; | ||
| _deepChainExpression = deepExpr; | ||
| } | ||
|
|
||
| [Benchmark(Baseline = true)] | ||
| public Expression Replace_Property() | ||
| => new ExpressiveReplacer(new ExpressiveResolver()).Replace(_propertyExpression); | ||
|
|
||
| [Benchmark] | ||
| public Expression Replace_Method() | ||
| => new ExpressiveReplacer(new ExpressiveResolver()).Replace(_methodExpression); | ||
|
|
||
| [Benchmark] | ||
| public Expression Replace_NullConditional() | ||
| => new ExpressiveReplacer(new ExpressiveResolver()).Replace(_nullConditionalExpression); | ||
|
|
||
| [Benchmark] | ||
| public Expression Replace_BlockBody() | ||
| => new ExpressiveReplacer(new ExpressiveResolver()).Replace(_blockBodyExpression); | ||
|
|
||
| [Benchmark] | ||
| public Expression Replace_DeepChain() | ||
| => new ExpressiveReplacer(new ExpressiveResolver()).Replace(_deepChainExpression); | ||
| } |
66 changes: 66 additions & 0 deletions
66
benchmarks/ExpressiveSharp.Benchmarks/ExpressionResolverBenchmarks.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| using System.Reflection; | ||
| using BenchmarkDotNet.Attributes; | ||
| using ExpressiveSharp.Benchmarks.Helpers; | ||
| using ExpressiveSharp.Services; | ||
|
|
||
| namespace ExpressiveSharp.Benchmarks; | ||
|
|
||
| [MemoryDiagnoser] | ||
| public class ExpressionResolverBenchmarks | ||
| { | ||
| private ExpressiveResolver _resolver = null!; | ||
| private MemberInfo _propertyMember = null!; | ||
| private MemberInfo _methodMember = null!; | ||
| private MemberInfo _methodWithParamsMember = null!; | ||
| private MemberInfo _constructorMember = null!; | ||
|
|
||
| [GlobalSetup] | ||
| public void Setup() | ||
| { | ||
| _resolver = new ExpressiveResolver(); | ||
|
|
||
| var type = typeof(TestEntity); | ||
| _propertyMember = type.GetProperty(nameof(TestEntity.IdPlus1))!; | ||
| _methodMember = type.GetMethod(nameof(TestEntity.IdPlus1Method))!; | ||
| _methodWithParamsMember = type.GetMethod(nameof(TestEntity.IdPlusDelta))!; | ||
| _constructorMember = type.GetConstructor(new[] { typeof(TestEntity) })!; | ||
|
|
||
| // Warm up the caches so we measure steady-state | ||
| _resolver.FindGeneratedExpression(_propertyMember); | ||
| _resolver.FindGeneratedExpression(_methodMember); | ||
| _resolver.FindGeneratedExpression(_methodWithParamsMember); | ||
| _resolver.FindGeneratedExpression(_constructorMember); | ||
| } | ||
|
|
||
| [Benchmark(Baseline = true)] | ||
| public object Resolve_Property() | ||
| => _resolver.FindGeneratedExpression(_propertyMember); | ||
|
|
||
| [Benchmark] | ||
| public object Resolve_Method() | ||
| => _resolver.FindGeneratedExpression(_methodMember); | ||
|
|
||
| [Benchmark] | ||
| public object Resolve_MethodWithParams() | ||
| => _resolver.FindGeneratedExpression(_methodWithParamsMember); | ||
|
|
||
| [Benchmark] | ||
| public object Resolve_Constructor() | ||
| => _resolver.FindGeneratedExpression(_constructorMember); | ||
|
|
||
| [Benchmark] | ||
| public object? ResolveViaReflection_Property() | ||
| => ExpressiveResolver.FindGeneratedExpressionViaReflection(_propertyMember); | ||
|
|
||
| [Benchmark] | ||
| public object? ResolveViaReflection_Method() | ||
| => ExpressiveResolver.FindGeneratedExpressionViaReflection(_methodMember); | ||
|
|
||
| [Benchmark] | ||
| public object? ResolveViaReflection_MethodWithParams() | ||
| => ExpressiveResolver.FindGeneratedExpressionViaReflection(_methodWithParamsMember); | ||
|
|
||
| [Benchmark] | ||
| public object? ResolveViaReflection_Constructor() | ||
| => ExpressiveResolver.FindGeneratedExpressionViaReflection(_constructorMember); | ||
| } |
22 changes: 22 additions & 0 deletions
22
benchmarks/ExpressiveSharp.Benchmarks/ExpressiveSharp.Benchmarks.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <OutputType>Exe</OutputType> | ||
| <IsPackable>false</IsPackable> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="BenchmarkDotNet" /> | ||
| <PackageReference Include="Microsoft.CodeAnalysis.CSharp" /> | ||
| <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" VersionOverride="10.0.5" /> | ||
| <PackageReference Include="Basic.Reference.Assemblies.Net100" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\..\src\ExpressiveSharp\ExpressiveSharp.csproj" /> | ||
| <ProjectReference Include="..\..\src\ExpressiveSharp.EntityFrameworkCore\ExpressiveSharp.EntityFrameworkCore.csproj" /> | ||
| <ProjectReference Include="..\..\src\ExpressiveSharp.Generator\ExpressiveSharp.Generator.csproj" | ||
| OutputItemType="Analyzer" ReferenceOutputAssembly="true" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This workflow grants
contents: writeandpull-requests: writefor all events, includingpull_request. For least privilege (and to reduce the blast radius if an action dependency is compromised), consider scoping write permissions to only thepush/maincase that actually needs to push togh-pages, and keep PR runs read-only (or split into separate jobs with different permissions).