From eed54c8db22a06d2769550fdd4b52620882fcd9d Mon Sep 17 00:00:00 2001 From: Cheena Malhotra Date: Thu, 13 Aug 2026 22:41:39 -0700 Subject: [PATCH 1/4] Add async counterparts to the enclave provider hierarchy (Phase 3, partial) Implements the enclave-provider portion of Phase 3 of the async Always Encrypted spec (specs/002-async-always-encrypted/spec.md). SqlColumnEncryptionEnclaveProvider gains four `virtual` async counterparts whose default implementations defer to the existing sync overloads, mirroring the pattern already established for SqlColumnEncryptionKeyStoreProvider in Phase 1. Because C# forbids `out` parameters on async methods, the two members that report multiple values return tuples instead (spec Design Decision 4). The two providers that perform real network I/O explicitly override the defaults rather than inheriting the blocking fallback, which removes both sync-over-async blocking calls in this hierarchy: * AzureAttestationBasedEnclaveProvider now awaits ConfigurationManager.GetConfigurationAsync instead of blocking on .Result. * HostGuardianServiceEnclaveProvider now awaits GetStreamAsync, JsonSerializer.DeserializeAsync and the retry backoff instead of blocking on .GetAwaiter().GetResult() and Thread.Sleep. FR-015: the attestation gate (AutoResetEvent) has no awaitable wait, so the async path gets its own SemaphoreSlim gate via GetEnclaveSessionHelperAsync. The gates are deliberately independent so that a synchronous caller can never block a thread for the duration of an awaited attestation round trip. The five `lock` statements called out in the spec are left as-is: their bodies only touch MemoryCache and flags, and the awaited attestation happens before session storage rather than inside those regions. Unlike the sync path, the async gate is also released when the caller cancels, since a cancelled caller never goes on to create the session. FR-010: no existing sync code path is modified. Every change is an insertion. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: cf57fdc0-1c77-40f1-a1d5-7c0bd98c1a89 --- .../SqlColumnEncryptionEnclaveProvider.xml | 131 +++- .../AzureAttestationBasedEnclaveProvider.cs | 193 ++++++ .../Data/SqlClient/EnclaveProviderBase.cs | 157 +++++ .../SqlColumnEncryptionEnclaveProvider.cs | 112 +++ .../VirtualSecureModeEnclaveProvider.cs | 46 ++ .../VirtualSecureModeEnclaveProviderBase.cs | 182 +++++ ...umnEncryptionEnclaveProviderAsyncShould.cs | 650 ++++++++++++++++++ 7 files changed, 1470 insertions(+), 1 deletion(-) create mode 100644 src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs diff --git a/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml b/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml index 4276c98b9f..4e6881a746 100644 --- a/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml +++ b/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml @@ -5,7 +5,15 @@ The base class that defines the interface for enclave providers for Always Encrypted. - An enclave is a protected region of memory inside SQL Server, used for computations on encrypted columns. An enclave provider encapsulates the client-side implementation details of the enclave attestation protocol as well as the logic for creating and caching enclave sessions. + + An enclave is a protected region of memory inside SQL Server, used for computations on encrypted columns. An enclave provider encapsulates the client-side implementation details of the enclave attestation protocol as well as the logic for creating and caching enclave sessions. + + + Attestation is a three-call sequence: GetEnclaveSession, GetAttestationParameters, then CreateEnclaveSession. The first call may take an attestation gate that is only released by the matching CreateEnclaveSession call. + + + Callers must not mix the synchronous and asynchronous members within a single attestation sequence. The base implementation maintains separate gates for the two paths, so pairing GetEnclaveSessionAsync with the synchronous CreateEnclaveSession (or vice versa) leaves a gate held until its timeout expires and stalls other callers. + @@ -39,6 +47,43 @@ When overridden in a derived class, performs enclave attestation, generates a symmetric key for the session, creates an enclave session and stores the session information in the cache. + + + The information the provider uses to attest the enclave and generate a symmetric key for the session. The format of this information is specific to the enclave attestation protocol. + + + A Diffie-Hellman algorithm object that encapsulates a client-side key pair. + + + The set of parameters required for an enclave session. + + + The set of extra data needed for attesting the enclave. + + + The length of the extra data needed for attesting the enclave. + + + A token to cancel the asynchronous operation. + + + Asynchronously performs enclave attestation, generates a symmetric key for the session, creates an enclave session and stores the session information in the cache. + + + A task that returns a tuple on completion. SqlEnclaveSession is the created enclave session, or if the provider doesn't implement session caching. Counter is a counter that the enclave provider is expected to increment each time SqlClient retrieves the session from the cache, which prevents replay attacks. + + + + Asynchronous methods cannot declare parameters, so the values reported through the parameters of CreateEnclaveSession are returned as a tuple. + + + The default implementation first checks the and returns a canceled task if cancellation has been requested. Otherwise, it calls the synchronous CreateEnclaveSession method and wraps the result in a completed task. If the synchronous method throws, the exception is caught and a faulted task is returned rather than throwing synchronously. + + + Providers that perform network I/O during attestation must override this method with a genuinely asynchronous implementation so that the calling thread is not blocked for the duration of the attestation round trip. + + + The endpoint of an attestation service for attesting the enclave. @@ -56,6 +101,34 @@ The information SqlClient subsequently uses to initiate the process of attesting the enclave and to establish a secure session with the enclave. + + + The endpoint of an attestation service for attesting the enclave. + + + A set of extra data needed for attesting the enclave. + + + The length of the extra data needed for attesting the enclave. + + + A token to cancel the asynchronous operation. + + + Asynchronously gets the information that SqlClient subsequently uses to initiate the process of attesting the enclave and to establish a secure session with the enclave. + + + A task that returns, on completion, the information SqlClient subsequently uses to initiate the process of attesting the enclave and to establish a secure session with the enclave. + + + + The default implementation first checks the and returns a canceled task if cancellation has been requested. Otherwise, it calls the synchronous GetAttestationParameters method and wraps the result in a completed task. If the synchronous method throws, the exception is caught and a faulted task is returned rather than throwing synchronously. + + + This operation is CPU bound for all in-box providers, so overriding it is only necessary if a provider performs I/O here. + + + The set of parameters required for enclave session. @@ -82,6 +155,37 @@ When overridden in a derived class, looks up an existing enclave session information in the enclave session cache. If the enclave provider doesn't implement enclave session caching, this method is expected to return in the parameter. + + + The set of parameters required for enclave session. + + + to indicate that a set of extra data needs to be generated for attestation; otherwise, . + + + Indicates if this is a retry from a failed call. + + + A token to cancel the asynchronous operation. + + + Asynchronously looks up an existing enclave session information in the enclave session cache. If the enclave provider doesn't implement enclave session caching, this method is expected to return for the enclave session. + + + A task that returns a tuple on completion. SqlEnclaveSession is the requested enclave session, or if the provider doesn't implement session caching. Counter is a counter that the enclave provider is expected to increment each time SqlClient retrieves the session from the cache, which prevents replay attacks. CustomData is a set of extra data needed for attesting the enclave, and CustomDataLength is its length. + + + + Asynchronous methods cannot declare parameters, so the values reported through the parameters of GetEnclaveSession are returned as a tuple. + + + The default implementation first checks the and returns a canceled task if cancellation has been requested. Otherwise, it calls the synchronous GetEnclaveSession method and wraps the result in a completed task. If the synchronous method throws, the exception is caught and a faulted task is returned rather than throwing synchronously. + + + Providers that perform network I/O during attestation must override this method with a genuinely asynchronous implementation so that the calling thread is not blocked while waiting for an in-flight attestation to complete. + + + The set of parameters required for enclave session. @@ -93,5 +197,30 @@ When overridden in a derived class, looks up and evicts an enclave session from the enclave session cache, if the provider implements session caching. + + + The set of parameters required for enclave session. + + + The session to be invalidated. + + + A token to cancel the asynchronous operation. + + + Asynchronously looks up and evicts an enclave session from the enclave session cache, if the provider implements session caching. + + + A task that completes when the enclave session has been invalidated. + + + + The default implementation first checks the and returns a canceled task if cancellation has been requested. Otherwise, it calls the synchronous InvalidateEnclaveSession method and returns a completed task. If the synchronous method throws, the exception is caught and a faulted task is returned rather than throwing synchronously. + + + Session invalidation is an in-memory cache operation for all in-box providers. + + + diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs index 6d26429122..c6b3080696 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs @@ -10,6 +10,8 @@ using System.Security.Cryptography; using System.Text; using System.Threading; +using System.Threading.Tasks; +using Microsoft.Data.Common; using Microsoft.Extensions.Caching.Memory; using Microsoft.IdentityModel.JsonWebTokens; using Microsoft.IdentityModel.Logging; @@ -129,6 +131,120 @@ internal override void InvalidateEnclaveSession(EnclaveSessionParameters enclave { InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSessionToInvalidate); } + + // Asynchronous counterpart of GetEnclaveSession. Uses the async attestation gate so that + // callers never block a thread pool thread waiting for an in-flight attestation. + internal override Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionAsync( + EnclaveSessionParameters enclaveSessionParameters, + bool generateCustomData, + bool isRetry, + CancellationToken cancellationToken = default) + { + return GetEnclaveSessionHelperAsync(enclaveSessionParameters, generateCustomData, isRetry, cancellationToken); + } + + // Asynchronous counterpart of GetAttestationParameters. This operation is CPU bound (key + // generation and buffer marshalling), so it completes synchronously. + internal override Task GetAttestationParametersAsync( + string attestationUrl, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken = default) + { + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled(cancellationToken); + } + + try + { + return Task.FromResult(GetAttestationParameters(attestationUrl, customData, customDataLength)); + } + catch (Exception e) when (ADP.IsCatchableExceptionType(e)) + { + return Task.FromException(e); + } + } + + // Asynchronous counterpart of CreateEnclaveSession. Performs the attestation service round + // trip (OpenID Connect metadata download) asynchronously. + internal override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( + byte[] attestationInfo, + ECDiffieHellman clientDHKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken = default) + { + SqlEnclaveSession sqlEnclaveSession = null; + long counter = 0; + try + { + ThreadRetryCache.Remove(Thread.CurrentThread.ManagedThreadId.ToString()); + sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out counter); + if (sqlEnclaveSession == null) + { + if (!string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl) && customData != null && customDataLength > 0) + { + byte[] nonce = customData; + + IdentityModelEventSource.ShowPII = true; + + // Deserialize the payload + AzureAttestationInfo attestInfo = new AzureAttestationInfo(attestationInfo); + + // Validate the attestation info + await VerifyAzureAttestationInfoAsync( + enclaveSessionParameters.AttestationUrl, + attestInfo.EnclaveType, + attestInfo.AttestationToken.AttestationToken, + attestInfo.Identity, + nonce, + cancellationToken).ConfigureAwait(false); + + // Set up shared secret and validate signature + byte[] sharedSecret = GetSharedSecret(attestInfo.Identity, nonce, attestInfo.EnclaveType, attestInfo.EnclaveDHInfo, clientDHKey); + + // add session to cache + sqlEnclaveSession = AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, attestInfo.SessionId, out counter); + } + else + { + throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); + } + } + } + finally + { + // See UpdateEnclaveSessionLockStatus for the rationale; this releases the async gate. + UpdateAsyncEnclaveSessionLockStatus(sqlEnclaveSession, cancellationToken); + } + + return (sqlEnclaveSession, counter); + } + + // Asynchronous counterpart of InvalidateEnclaveSession. Session eviction is an in-memory + // cache operation, so it completes synchronously. + internal override Task InvalidateEnclaveSessionAsync( + EnclaveSessionParameters enclaveSessionParameters, + SqlEnclaveSession enclaveSessionToInvalidate, + CancellationToken cancellationToken = default) + { + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled(cancellationToken); + } + + try + { + InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSessionToInvalidate); + return Task.CompletedTask; + } + catch (Exception e) when (ADP.IsCatchableExceptionType(e)) + { + return Task.FromException(e); + } + } #endregion #region Internal Class @@ -317,6 +433,83 @@ private void VerifyAzureAttestationInfo(string attestationUrl, EnclaveType encla ValidateAttestationClaims(enclaveType, attestationToken, enclavePublicKey, nonce); } + // Performs Attestation per the protocol used by Azure Attestation Service. + // Asynchronous counterpart of VerifyAzureAttestationInfo. + private async Task VerifyAzureAttestationInfoAsync( + string attestationUrl, + EnclaveType enclaveType, + string attestationToken, + EnclavePublicKey enclavePublicKey, + byte[] nonce, + CancellationToken cancellationToken) + { + bool shouldForceUpdateSigningKeys = false; + string attestationInstanceUrl = GetAttestationInstanceUrl(attestationUrl); + + bool shouldRetryValidation; + bool isSignatureValid; + string exceptionMessage = string.Empty; + do + { + shouldRetryValidation = false; + + // Get the OpenId config object for the signing keys + OpenIdConnectConfiguration openIdConfig = + await GetOpenIdConfigForSigningKeysAsync(attestationInstanceUrl, shouldForceUpdateSigningKeys, cancellationToken) + .ConfigureAwait(false); + + // Verify the token signature against the signing keys downloaded from meta data end point + bool isKeySigningExpired; + isSignatureValid = VerifyTokenSignature(attestationToken, attestationInstanceUrl, openIdConfig.SigningKeys, out isKeySigningExpired, out exceptionMessage); + + // In cases if we fail to validate the token, since we are using the old signing keys + // let's re-download the signing keys again and re-validate the token signature + if (!isSignatureValid && isKeySigningExpired && !shouldForceUpdateSigningKeys) + { + shouldForceUpdateSigningKeys = true; + shouldRetryValidation = true; + } + } + while (shouldRetryValidation); + + if (!isSignatureValid) + { + throw SQL.AttestationFailed(string.Format(Strings.AttestationTokenSignatureValidationFailed, exceptionMessage)); + } + + // Validate claims in the token + ValidateAttestationClaims(enclaveType, attestationToken, enclavePublicKey, nonce); + } + + // For the given attestation url it downloads the token signing keys from the well-known openid configuration end point. + // It also caches that information for 1 day to avoid DDOS attacks. + // Asynchronous counterpart of GetOpenIdConfigForSigningKeys: the metadata download is awaited + // instead of being blocked on with Task.Result. + private async Task GetOpenIdConfigForSigningKeysAsync(string url, bool forceUpdate, CancellationToken cancellationToken) + { + OpenIdConnectConfiguration openIdConnectConfig = OpenIdConnectConfigurationCache.Get(url); + if (forceUpdate || openIdConnectConfig == null) + { + // Compute the meta data endpoint + string openIdMetadataEndpoint = url + AttestationUrlSuffix; + + try + { + IConfigurationManager configurationManager = + new ConfigurationManager(openIdMetadataEndpoint, new OpenIdConnectConfigurationRetriever()); + openIdConnectConfig = await configurationManager.GetConfigurationAsync(cancellationToken).ConfigureAwait(false); + } + catch (Exception exception) when (!(exception is OperationCanceledException && cancellationToken.IsCancellationRequested)) + { + throw SQL.AttestationFailed(string.Format(Strings.GetAttestationTokenSigningKeysFailed, GetInnerMostExceptionMessage(exception)), exception); + } + + OpenIdConnectConfigurationCache.Set(url, openIdConnectConfig, absoluteExpirationRelativeToNow: s_openIdConnectConfigurationCacheTimeout); + } + + return openIdConnectConfig; + } + // Returns the innermost exception value private static string GetInnerMostExceptionMessage(Exception exception) { diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs index c81f04471c..504f2f95f1 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs @@ -5,6 +5,7 @@ using System; using System.Security.Cryptography; using System.Threading; +using System.Threading.Tasks; using Microsoft.Extensions.Caching.Memory; // Enclave session locking model @@ -82,6 +83,24 @@ internal abstract class EnclaveProviderBase : SqlColumnEncryptionEnclaveProvider private static readonly Object lockUpdateSessionLock = new Object(); + // Asynchronous counterparts of the three fields above. + // + // The async attestation path deliberately uses its own gate rather than sharing + // 'sessionLockEvent' with the synchronous path. AutoResetEvent has no awaitable wait, and + // sharing a single SemaphoreSlim across both paths would let a synchronous caller block a + // thread for the whole duration of an awaited attestation HTTP round trip (and vice versa), + // which is exactly the thread-pool starvation this work is meant to remove. The two gates are + // therefore independent: at worst a concurrent sync and async cold start performs two + // attestations, which the existing design already tolerates (see the lock-timeout cases + // described above), and the session cache makes the outcome idempotent. + private static readonly SemaphoreSlim s_asyncSessionLockEvent = new SemaphoreSlim(1, 1); + + private static int s_asyncLockTimeoutInMilliseconds = LockTimeoutMaxInMilliseconds; + + private static bool s_isAsyncSessionLockAcquired = false; + + private static readonly object s_asyncLockUpdateSessionLock = new object(); + // It is used to save the attestation url and nonce value across API calls protected static readonly MemoryCache ThreadRetryCache = new MemoryCache(new MemoryCacheOptions()); private static readonly TimeSpan s_threadRetryCacheTimeout = TimeSpan.FromMinutes(10); @@ -173,6 +192,144 @@ protected void GetEnclaveSessionHelper(EnclaveSessionParameters enclaveSessionPa } } + // Helper method to get the enclave session from the cache if present. + // Asynchronous counterpart of GetEnclaveSessionHelper. + // + // Because C# async methods cannot declare 'out' parameters, the four values reported through + // 'out' parameters by the synchronous helper are returned as a tuple. + // + // This method intentionally duplicates rather than shares the synchronous helper's body: the + // synchronous path must remain byte-for-byte unchanged (FR-010), and the two paths use + // independent attestation gates (see s_asyncSessionLockEvent). + protected async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionHelperAsync( + EnclaveSessionParameters enclaveSessionParameters, + bool shouldGenerateNonce, + bool isRetry, + CancellationToken cancellationToken = default) + { + cancellationToken.ThrowIfCancellationRequested(); + + byte[] customData = null; + int customDataLength = 0; + SqlEnclaveSession sqlEnclaveSession = SessionCache.GetEnclaveSession(enclaveSessionParameters, out long counter); + + if (sqlEnclaveSession == null) + { + bool sessionCacheLockTaken = false; + bool sameThreadRetry = false; + string currentThreadId = Thread.CurrentThread.ManagedThreadId.ToString(); + + // In case if on some thread we are running SQL workload which don't require attestation, then in those cases we don't want same thread to wait for the gate to be released. + // hence skipping it + string retryThreadID = ThreadRetryCache.Get(currentThreadId); + if (!string.IsNullOrEmpty(retryThreadID)) + { + sameThreadRetry = true; + } + else if (!isRetry) + { + // We are explicitly not releasing the gate here, as we want to hold it until the driver calls CreateEnclaveSessionAsync. + // If we release it now, then multiple callers end up calling GetAttestationParameters which triggers the attestation workflow. + sessionCacheLockTaken = await s_asyncSessionLockEvent + .WaitAsync(Volatile.Read(ref s_asyncLockTimeoutInMilliseconds), cancellationToken) + .ConfigureAwait(false); + + if (sessionCacheLockTaken) + { + lock (s_asyncLockUpdateSessionLock) + { + s_isAsyncSessionLockAcquired = true; + } + } + } + + // In case of a multi-threaded application, the first caller takes the gate and all the subsequent callers wait here either until the enclave + // session is created or the timeout happens. + if (sessionCacheLockTaken || sameThreadRetry || isRetry) + { + // While the current caller was waiting for the gate we may already have completed the attestation elsewhere, + // in which case we need to release the gate here. + sqlEnclaveSession = SessionCache.GetEnclaveSession(enclaveSessionParameters, out counter); + if (sqlEnclaveSession != null && !sameThreadRetry) + { + ReleaseAsyncSessionLock(restoreLockTimeout: false); + } + } + else + { + // In case we are unable to take the gate, then it represents either + // 1. Another caller has an ongoing attestation request which is taking more time, may be due to a slow network, or + // 2. The current workload doesn't require enclave computation, hence the driver is not invoking CreateEnclaveSessionAsync and sqlEnclaveSession is never set. + // In both cases we need to reduce the timeout to 0 so that subsequent requests should not wait. + Interlocked.Exchange(ref s_asyncLockTimeoutInMilliseconds, 0); + } + + if (sqlEnclaveSession == null) + { + if (shouldGenerateNonce) + { + using (RandomNumberGenerator rng = RandomNumberGenerator.Create()) + { + // Client decides to initiate the process of attesting the enclave and to establish a secure session with the enclave. + // To ensure that server send new attestation request instead of replaying / re-sending the old token, we will create a nonce for current attestation request. + byte[] nonce = new byte[NonceSize]; + rng.GetBytes(nonce); + customData = nonce; + customDataLength = nonce.Length; + } + } + + if (!sameThreadRetry) + { + retryThreadID = currentThreadId; + } + + ThreadRetryCache.Set(currentThreadId, retryThreadID, + absoluteExpirationRelativeToNow: s_threadRetryCacheTimeout); + } + } + + return (sqlEnclaveSession, counter, customData, customDataLength); + } + + // Reset the async session gate. Asynchronous counterpart of UpdateEnclaveSessionLockStatus. + // This method performs no I/O, so a plain lock remains the correct (and cheapest) primitive. + // + // Unlike the synchronous counterpart, the gate is also released when the caller cancelled. A + // cancelled caller will never go on to create the session, so keeping the gate would stall + // every other async caller until the lock timeout expires. The synchronous path has no + // equivalent case because it has no cancellation. + protected void UpdateAsyncEnclaveSessionLockStatus(SqlEnclaveSession sqlEnclaveSession, CancellationToken cancellationToken = default) + { + if (sqlEnclaveSession != null || cancellationToken.IsCancellationRequested) + { + ReleaseAsyncSessionLock(restoreLockTimeout: true); + } + } + + // Releases the async attestation gate if it is currently held. + // + // Unlike AutoResetEvent.Set(), SemaphoreSlim.Release() throws when the semaphore is already + // full, so the release is guarded by s_isAsyncSessionLockAcquired. The guard preserves the + // synchronous design's "any caller may signal" semantics while keeping the release idempotent. + private static void ReleaseAsyncSessionLock(bool restoreLockTimeout) + { + lock (s_asyncLockUpdateSessionLock) + { + if (s_isAsyncSessionLockAcquired) + { + s_isAsyncSessionLockAcquired = false; + + if (restoreLockTimeout) + { + Interlocked.Exchange(ref s_asyncLockTimeoutInMilliseconds, LockTimeoutMaxInMilliseconds); + } + + s_asyncSessionLockEvent.Release(); + } + } + } + // Reset the session lock status protected void UpdateEnclaveSessionLockStatus(SqlEnclaveSession sqlEnclaveSession) { diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlColumnEncryptionEnclaveProvider.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlColumnEncryptionEnclaveProvider.cs index 21a62a80c3..444f7c043d 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlColumnEncryptionEnclaveProvider.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlColumnEncryptionEnclaveProvider.cs @@ -2,7 +2,11 @@ // The .NET Foundation licenses this file to you under the MIT license. // See the LICENSE file in the project root for more information. +using System; using System.Security.Cryptography; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Data.Common; namespace Microsoft.Data.SqlClient { @@ -20,5 +24,113 @@ internal abstract class SqlColumnEncryptionEnclaveProvider /// internal abstract void InvalidateEnclaveSession(EnclaveSessionParameters enclaveSessionParameters, SqlEnclaveSession enclaveSession); + + /// + internal virtual Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionAsync( + EnclaveSessionParameters enclaveSessionParameters, + bool generateCustomData, + bool isRetry, + CancellationToken cancellationToken = default) + { + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled<(SqlEnclaveSession, long, byte[], int)>(cancellationToken); + } + + try + { + GetEnclaveSession( + enclaveSessionParameters, + generateCustomData, + isRetry, + out SqlEnclaveSession sqlEnclaveSession, + out long counter, + out byte[] customData, + out int customDataLength); + + return Task.FromResult((sqlEnclaveSession, counter, customData, customDataLength)); + } + catch (Exception e) when (ADP.IsCatchableExceptionType(e)) + { + return Task.FromException<(SqlEnclaveSession, long, byte[], int)>(e); + } + } + + /// + internal virtual Task GetAttestationParametersAsync( + string attestationUrl, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken = default) + { + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled(cancellationToken); + } + + try + { + return Task.FromResult(GetAttestationParameters(attestationUrl, customData, customDataLength)); + } + catch (Exception e) when (ADP.IsCatchableExceptionType(e)) + { + return Task.FromException(e); + } + } + + /// + internal virtual Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( + byte[] enclaveAttestationInfo, + ECDiffieHellman clientDiffieHellmanKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken = default) + { + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled<(SqlEnclaveSession, long)>(cancellationToken); + } + + try + { + CreateEnclaveSession( + enclaveAttestationInfo, + clientDiffieHellmanKey, + enclaveSessionParameters, + customData, + customDataLength, + out SqlEnclaveSession sqlEnclaveSession, + out long counter); + + return Task.FromResult((sqlEnclaveSession, counter)); + } + catch (Exception e) when (ADP.IsCatchableExceptionType(e)) + { + return Task.FromException<(SqlEnclaveSession, long)>(e); + } + } + + /// + internal virtual Task InvalidateEnclaveSessionAsync( + EnclaveSessionParameters enclaveSessionParameters, + SqlEnclaveSession enclaveSession, + CancellationToken cancellationToken = default) + { + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled(cancellationToken); + } + + try + { + InvalidateEnclaveSession(enclaveSessionParameters, enclaveSession); + return Task.CompletedTask; + } + catch (Exception e) when (ADP.IsCatchableExceptionType(e)) + { + return Task.FromException(e); + } + } } } diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProvider.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProvider.cs index aa7e17be03..02815f4870 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProvider.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProvider.cs @@ -10,6 +10,7 @@ using System.Security.Cryptography.X509Certificates; using System.Text.Json; using System.Threading; +using System.Threading.Tasks; namespace Microsoft.Data.SqlClient { @@ -84,6 +85,51 @@ protected override byte[] MakeRequest(string url) throw SQL.AttestationFailed(string.Format(Strings.GetAttestationSigningCertificateRequestFailedFormat, url, exception.Message), exception); } + // Makes a web request to the provided url and returns the response as a byte[]. + // Asynchronous counterpart of MakeRequest: the HTTP round trip, the retry backoff and the + // JSON deserialization are all awaited rather than blocked on. + protected override async Task MakeRequestAsync(string url, CancellationToken cancellationToken) + { + Exception exception = null; + + for (int n = 0; n < MaxNumRetries + 1 /* Initial attempt + numRetries */; n++) + { + cancellationToken.ThrowIfCancellationRequested(); + + try + { + if (n != 0) + { + await Task.Delay(EnclaveRetrySleepInSeconds * 1000, cancellationToken).ConfigureAwait(false); + } + +#if NET + using (Stream stream = await s_client.GetStreamAsync(url, cancellationToken).ConfigureAwait(false)) +#else + using (Stream stream = await s_client.GetStreamAsync(url).ConfigureAwait(false)) +#endif + { + List payload = await JsonSerializer + .DeserializeAsync>(stream, cancellationToken: cancellationToken) + .ConfigureAwait(false); + + return payload?.ToArray(); + } + } + catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested) + { + // Cancellation is not an attestation failure; surface it to the caller unchanged. + throw; + } + catch (Exception e) + { + exception = e; + } + } + + throw SQL.AttestationFailed(string.Format(Strings.GetAttestationSigningCertificateRequestFailedFormat, url, exception.Message), exception); + } + #endregion } diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs index a3b4545d03..20090f7c55 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs @@ -8,6 +8,8 @@ using System.Security.Cryptography.X509Certificates; using System.Security.Cryptography.Pkcs; using System.Threading; +using System.Threading.Tasks; +using Microsoft.Data.Common; using Microsoft.Extensions.Caching.Memory; namespace Microsoft.Data.SqlClient @@ -146,6 +148,116 @@ internal override void InvalidateEnclaveSession(EnclaveSessionParameters enclave InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSessionToInvalidate); } + // Asynchronous counterpart of GetEnclaveSession. Uses the async attestation gate so that + // callers never block a thread pool thread waiting for an in-flight attestation. + internal override Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionAsync( + EnclaveSessionParameters enclaveSessionParameters, + bool generateCustomData, + bool isRetry, + CancellationToken cancellationToken = default) + { + return GetEnclaveSessionHelperAsync(enclaveSessionParameters, false, isRetry, cancellationToken); + } + + // Asynchronous counterpart of GetAttestationParameters. This operation is CPU bound (key + // generation), so it completes synchronously. + internal override Task GetAttestationParametersAsync( + string attestationUrl, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken = default) + { + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled(cancellationToken); + } + + try + { + return Task.FromResult(GetAttestationParameters(attestationUrl, customData, customDataLength)); + } + catch (Exception e) when (ADP.IsCatchableExceptionType(e)) + { + return Task.FromException(e); + } + } + + // Asynchronous counterpart of CreateEnclaveSession. Performs the attestation service round + // trip (signing certificate download) asynchronously. + internal override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( + byte[] attestationInfo, + ECDiffieHellman clientDHKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken = default) + { + SqlEnclaveSession sqlEnclaveSession = null; + long counter = 0; + try + { + ThreadRetryCache.Remove(Thread.CurrentThread.ManagedThreadId.ToString()); + sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out counter); + if (sqlEnclaveSession == null) + { + if (!string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl)) + { + // Deserialize the payload + AttestationInfo info = new AttestationInfo(attestationInfo); + + // Verify enclave policy matches expected policy + VerifyEnclavePolicy(info.EnclaveReportPackage); + + // Perform Attestation per VSM protocol + await VerifyAttestationInfoAsync( + enclaveSessionParameters.AttestationUrl, + info.HealthReport, + info.EnclaveReportPackage, + cancellationToken).ConfigureAwait(false); + + // Set up shared secret and validate signature + byte[] sharedSecret = GetSharedSecret(info.Identity, info.EnclaveDHInfo, clientDHKey); + + // add session to cache + sqlEnclaveSession = AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, info.SessionId, out counter); + } + else + { + throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); + } + } + } + finally + { + UpdateAsyncEnclaveSessionLockStatus(sqlEnclaveSession, cancellationToken); + } + + return (sqlEnclaveSession, counter); + } + + // Asynchronous counterpart of InvalidateEnclaveSession. Session eviction is an in-memory + // cache operation, so it completes synchronously. + internal override Task InvalidateEnclaveSessionAsync( + EnclaveSessionParameters enclaveSessionParameters, + SqlEnclaveSession enclaveSessionToInvalidate, + CancellationToken cancellationToken = default) + { + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled(cancellationToken); + } + + try + { + InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSessionToInvalidate); + return Task.CompletedTask; + } + catch (Exception e) when (ADP.IsCatchableExceptionType(e)) + { + return Task.FromException(e); + } + } + #endregion #region Private helpers @@ -187,6 +299,76 @@ private void VerifyAttestationInfo(string attestationUrl, HealthReport healthRep // Makes a web request to the provided url and returns the response as a byte[] protected abstract byte[] MakeRequest(string url); + // Performs Attestation per the protocol used by Virtual Secure Modules. + // Asynchronous counterpart of VerifyAttestationInfo. + private async Task VerifyAttestationInfoAsync(string attestationUrl, HealthReport healthReport, EnclaveReportPackage enclaveReportPackage, CancellationToken cancellationToken) + { + bool shouldRetryValidation; + bool shouldForceUpdateSigningKeys = false; + do + { + shouldRetryValidation = false; + + // Get HGS Root signing certs from HGS + X509Certificate2Collection signingCerts = + await GetSigningCertificateAsync(attestationUrl, shouldForceUpdateSigningKeys, cancellationToken).ConfigureAwait(false); + + // Verify SQL Health report root chain of trust is the HGS root signing cert + if (!VerifyHealthReportAgainstRootCertificate(signingCerts, healthReport.Certificate, out X509ChainStatusFlags chainStatus) || + chainStatus != X509ChainStatusFlags.NoError) + { + // In cases if we fail to validate the health report, it might be possible that we are using old signing keys + // let's re-download the signing keys again and re-validate the health report + if (!shouldForceUpdateSigningKeys) + { + shouldForceUpdateSigningKeys = true; + shouldRetryValidation = true; + } + else + { + throw SQL.AttestationFailed(string.Format(Strings.VerifyHealthCertificateChainFormat, attestationUrl, chainStatus)); + } + } + } while (shouldRetryValidation); + + // Verify enclave report is signed by IDK_S from health report + VerifyEnclaveReportSignature(enclaveReportPackage, healthReport.Certificate); + } + + // Gets the root signing certificate for the provided attestation service. + // Asynchronous counterpart of GetSigningCertificate. + private async Task GetSigningCertificateAsync(string attestationUrl, bool forceUpdate, CancellationToken cancellationToken) + { + attestationUrl = GetAttestationUrl(attestationUrl); + X509Certificate2Collection signingCertificates = rootSigningCertificateCache.Get(attestationUrl); + if (forceUpdate || signingCertificates == null || AnyCertificatesExpired(signingCertificates)) + { + byte[] data = await MakeRequestAsync(attestationUrl, cancellationToken).ConfigureAwait(false); + var certificateCollection = new X509Certificate2Collection(); + + try + { + SignedCms s = new SignedCms(); + s.Decode(data); + certificateCollection.AddRange(s.Certificates); + } + catch (CryptographicException exception) + { + throw SQL.AttestationFailed(string.Format(Strings.GetAttestationSigningCertificateFailedInvalidCertificate, attestationUrl), exception); + } + + rootSigningCertificateCache.Set(attestationUrl, certificateCollection, + absoluteExpirationRelativeToNow: s_rootSigningCertificateCacheTimeout); + } + + return rootSigningCertificateCache.Get(attestationUrl); + } + + // Makes a web request to the provided url and returns the response as a byte[]. + // Asynchronous counterpart of MakeRequest. This member is abstract rather than virtual so + // that derived providers cannot silently inherit a blocking implementation. + protected abstract Task MakeRequestAsync(string url, CancellationToken cancellationToken); + // Gets the root signing certificate for the provided attestation service. // If the certificate does not exist in the cache, this will make a call to the // attestation service's "/signingCertificates" endpoint. This endpoint can diff --git a/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs b/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs new file mode 100644 index 0000000000..5260af146b --- /dev/null +++ b/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs @@ -0,0 +1,650 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +// The enclave provider hierarchy is internal and nullable-oblivious, and several of its members +// legitimately accept or produce nulls (for example, an absent enclave session). Nullable analysis +// is disabled for this file so the tests can mirror those signatures exactly. +#nullable disable + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Security.Cryptography; +using System.Threading; +using System.Threading.Tasks; +using Xunit; + +namespace Microsoft.Data.SqlClient.UnitTests.AlwaysEncrypted +{ + /// + /// Tests for the async counterparts declared on and + /// implemented by , covering the default sync fallbacks, + /// cancellation, and concurrent enclave session creation on both the sync and async paths. + /// + public class SqlColumnEncryptionEnclaveProviderAsyncShould + { + private static readonly byte[] SharedSecret = new byte[] { 1, 2, 3, 4, 5, 6, 7, 8 }; + + private static int s_uniqueSuffix; + + /// + /// Produces enclave session parameters with a cache key that no other test can collide with. + /// The enclave session cache is process-wide static state shared by every provider instance. + /// + private static EnclaveSessionParameters NewSessionParameters() + { + int id = Interlocked.Increment(ref s_uniqueSuffix); + return new EnclaveSessionParameters( + serverName: $"unit-test-server-{id}", + attestationUrl: $"https://unit.test.invalid/{id}", + database: $"unit-test-db-{id}"); + } + + #region Default fallback behavior + + /// + /// Verifies that the default async implementations defer to the synchronous overloads. + /// + [Fact] + public async Task AsyncMethods_DefaultFallback_DeferToSyncOverloads() + { + RecordingEnclaveProvider provider = new RecordingEnclaveProvider(); + EnclaveSessionParameters parameters = NewSessionParameters(); + + (SqlEnclaveSession session, long counter, byte[] customData, int customDataLength) = + await provider.GetEnclaveSessionAsync(parameters, generateCustomData: true, isRetry: false); + + Assert.Same(provider.Session, session); + Assert.Equal(RecordingEnclaveProvider.ExpectedCounter, counter); + Assert.Equal(RecordingEnclaveProvider.ExpectedCustomData, customData); + Assert.Equal(RecordingEnclaveProvider.ExpectedCustomData.Length, customDataLength); + Assert.Equal(1, provider.GetEnclaveSessionCallCount); + + SqlEnclaveAttestationParameters attestationParameters = + await provider.GetAttestationParametersAsync("https://unit.test.invalid", Array.Empty(), 0); + Assert.NotNull(attestationParameters); + Assert.Equal(1, provider.GetAttestationParametersCallCount); + + (SqlEnclaveSession createdSession, long createdCounter) = + await provider.CreateEnclaveSessionAsync(Array.Empty(), null, parameters, Array.Empty(), 0); + Assert.Same(provider.Session, createdSession); + Assert.Equal(RecordingEnclaveProvider.ExpectedCounter, createdCounter); + Assert.Equal(1, provider.CreateEnclaveSessionCallCount); + + await provider.InvalidateEnclaveSessionAsync(parameters, provider.Session); + Assert.Equal(1, provider.InvalidateEnclaveSessionCallCount); + } + + /// + /// Verifies that a throwing sync implementation surfaces as a faulted Task rather than a + /// synchronous throw from the async default. + /// + [Fact] + public async Task AsyncMethods_WhenSyncThrows_ReturnFaultedTask() + { + ThrowingEnclaveProvider provider = new ThrowingEnclaveProvider(); + EnclaveSessionParameters parameters = NewSessionParameters(); + + Task getSessionTask = provider.GetEnclaveSessionAsync(parameters, generateCustomData: false, isRetry: false); + Task getParametersTask = provider.GetAttestationParametersAsync("https://unit.test.invalid", Array.Empty(), 0); + Task createTask = provider.CreateEnclaveSessionAsync(Array.Empty(), null, parameters, Array.Empty(), 0); + Task invalidateTask = provider.InvalidateEnclaveSessionAsync(parameters, null); + + Assert.True(getSessionTask.IsFaulted); + Assert.True(getParametersTask.IsFaulted); + Assert.True(createTask.IsFaulted); + Assert.True(invalidateTask.IsFaulted); + + await Assert.ThrowsAsync(() => getSessionTask); + await Assert.ThrowsAsync(() => getParametersTask); + await Assert.ThrowsAsync(() => createTask); + await Assert.ThrowsAsync(() => invalidateTask); + } + + #endregion + + #region Cancellation + + /// + /// Verifies that every async default observes an already-cancelled token before doing any work. + /// + [Fact] + public async Task AsyncMethods_WithCancelledToken_ReturnCancelledTaskWithoutInvokingSyncOverload() + { + RecordingEnclaveProvider provider = new RecordingEnclaveProvider(); + EnclaveSessionParameters parameters = NewSessionParameters(); + CancellationToken cancelled = new CancellationToken(canceled: true); + + await Assert.ThrowsAnyAsync( + () => provider.GetEnclaveSessionAsync(parameters, false, false, cancelled)); + await Assert.ThrowsAnyAsync( + () => provider.GetAttestationParametersAsync("https://unit.test.invalid", Array.Empty(), 0, cancelled)); + await Assert.ThrowsAnyAsync( + () => provider.CreateEnclaveSessionAsync(Array.Empty(), null, parameters, Array.Empty(), 0, cancelled)); + await Assert.ThrowsAnyAsync( + () => provider.InvalidateEnclaveSessionAsync(parameters, provider.Session, cancelled)); + + Assert.Equal(0, provider.GetEnclaveSessionCallCount); + Assert.Equal(0, provider.GetAttestationParametersCallCount); + Assert.Equal(0, provider.CreateEnclaveSessionCallCount); + Assert.Equal(0, provider.InvalidateEnclaveSessionCallCount); + } + + /// + /// Verifies that 's CPU-bound async overrides + /// honour cancellation. + /// + [Fact] + public async Task AzureAttestationProvider_CpuBoundAsyncOverrides_HonourCancellation() + { + AzureAttestationEnclaveProvider provider = new AzureAttestationEnclaveProvider(); + EnclaveSessionParameters parameters = NewSessionParameters(); + CancellationToken cancelled = new CancellationToken(canceled: true); + + await Assert.ThrowsAnyAsync( + () => provider.GetAttestationParametersAsync(parameters.AttestationUrl, new byte[] { 1 }, 1, cancelled)); + await Assert.ThrowsAnyAsync( + () => provider.InvalidateEnclaveSessionAsync(parameters, null, cancelled)); + } + + /// + /// Verifies that the async session helper propagates cancellation while waiting for an + /// in-flight attestation instead of blocking. + /// + [Fact] + public async Task GetEnclaveSessionAsync_WhenTokenIsCancelled_PropagatesCancellation() + { + FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.Zero); + EnclaveSessionParameters parameters = NewSessionParameters(); + CancellationToken cancelled = new CancellationToken(canceled: true); + + await Assert.ThrowsAnyAsync( + () => provider.CreateEnclaveSessionAsync(Array.Empty(), null, parameters, Array.Empty(), 0, cancelled)); + + // The cancelled attempt must not have left an enclave session behind. + (SqlEnclaveSession session, _, _, _) = + await provider.GetEnclaveSessionAsync(parameters, generateCustomData: false, isRetry: true); + Assert.Null(session); + } + + /// + /// Verifies that cancelling an in-flight attestation releases the async gate. If the gate were + /// leaked, the next async caller would stall for the 15 second lock timeout before proceeding. + /// + [Fact] + public async Task CreateEnclaveSessionAsync_WhenCancelled_ReleasesTheAsyncGate() + { + FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.FromMilliseconds(50)); + EnclaveSessionParameters cancelledParameters = NewSessionParameters(); + + // Take the gate, then cancel before the session can be created. + (SqlEnclaveSession session, _, byte[] customData, int customDataLength) = + await provider.GetEnclaveSessionAsync(cancelledParameters, generateCustomData: true, isRetry: false); + Assert.Null(session); + + using (CancellationTokenSource cts = new CancellationTokenSource()) + { + cts.Cancel(); + await Assert.ThrowsAnyAsync( + () => provider.CreateEnclaveSessionAsync( + Array.Empty(), null, cancelledParameters, customData, customDataLength, cts.Token)); + } + + // A subsequent, unrelated attestation must not wait on the abandoned gate. + System.Diagnostics.Stopwatch stopwatch = System.Diagnostics.Stopwatch.StartNew(); + SqlEnclaveSession next = await AttestAsync(provider, NewSessionParameters()); + stopwatch.Stop(); + + Assert.NotNull(next); + Assert.True( + stopwatch.Elapsed < TimeSpan.FromSeconds(5), + $"Attestation took {stopwatch.Elapsed}, which suggests the async gate was not released on cancellation."); + } + + #endregion + + #region Concurrency — async path + /// + /// Verifies that once an enclave session is cached, concurrent async callers all observe the + /// same session and no further attestation is performed. + /// + [Fact] + public async Task GetEnclaveSessionAsync_AfterAttestation_ReturnsCachedSessionWithoutReattesting() + { + FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.FromMilliseconds(20)); + EnclaveSessionParameters parameters = NewSessionParameters(); + + SqlEnclaveSession created = await AttestAsync(provider, parameters); + Assert.NotNull(created); + Assert.Equal(1, provider.AttestationCount); + + SqlEnclaveSession[] sessions = await Task.WhenAll( + Enumerable.Range(0, 8).Select(async index => + { + (SqlEnclaveSession session, _, _, _) = + await provider.GetEnclaveSessionAsync(parameters, generateCustomData: false, isRetry: false) + .ConfigureAwait(false); + return session; + })); + + Assert.All(sessions, session => Assert.NotNull(session)); + Assert.All(sessions, session => Assert.Equal(created.SessionId, session.SessionId)); + Assert.Equal(1, provider.AttestationCount); + } + + /// + /// Verifies that many concurrent cold-start async attestations complete without deadlock or a + /// semaphore release imbalance, and that the async gate remains usable afterwards. + /// + [Fact] + public async Task CreateEnclaveSessionAsync_ConcurrentColdStart_CompletesWithoutDeadlock() + { + FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.FromMilliseconds(20)); + EnclaveSessionParameters parameters = NewSessionParameters(); + + SqlEnclaveSession[] sessions = await Task.WhenAll( + Enumerable.Range(0, 8).Select(index => Task.Run(() => AttestAsync(provider, parameters)))); + + Assert.All(sessions, session => Assert.NotNull(session)); + Assert.InRange(provider.AttestationCount, 1, sessions.Length); + + // The gate must still be usable for a subsequent, unrelated attestation. + SqlEnclaveSession next = await AttestAsync(provider, NewSessionParameters()); + Assert.NotNull(next); + } + + /// + /// Verifies that sync and async callers can attest concurrently. The two paths use independent + /// gates, so neither may block or starve the other. + /// + [Fact] + public async Task Attestation_MixedSyncAndAsyncCallers_AllObtainSessions() + { + FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.FromMilliseconds(20)); + EnclaveSessionParameters parameters = NewSessionParameters(); + + List> tasks = new List>(); + for (int i = 0; i < 4; i++) + { + tasks.Add(Task.Run(() => AttestAsync(provider, parameters))); + tasks.Add(Task.Run(() => Attest(provider, parameters))); + } + + SqlEnclaveSession[] sessions = await Task.WhenAll(tasks); + + Assert.All(sessions, session => Assert.NotNull(session)); + } + + #endregion + + #region Concurrency — sync path (regression guard) + + /// + /// Verifies the synchronous equivalent of + /// , + /// guarding against regressions in the untouched sync path. + /// + [Fact] + public void GetEnclaveSession_AfterAttestation_ReturnsCachedSessionWithoutReattesting() + { + FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.FromMilliseconds(20)); + EnclaveSessionParameters parameters = NewSessionParameters(); + + SqlEnclaveSession created = Attest(provider, parameters); + Assert.NotNull(created); + Assert.Equal(1, provider.AttestationCount); + + for (int i = 0; i < 8; i++) + { + provider.GetEnclaveSession(parameters, false, false, out SqlEnclaveSession session, out _, out _, out _); + Assert.NotNull(session); + Assert.Equal(created.SessionId, session.SessionId); + } + + Assert.Equal(1, provider.AttestationCount); + } + + #endregion + + #region HGS provider — truly async HTTP + + /// + /// Verifies that the HGS attestation request observes an already-cancelled token before + /// issuing any HTTP request. + /// + [Fact] + public async Task HgsMakeRequestAsync_WithCancelledToken_Throws() + { + TestHostGuardianServiceEnclaveProvider provider = new TestHostGuardianServiceEnclaveProvider(); + + await Assert.ThrowsAnyAsync( + () => provider.InvokeMakeRequestAsync("http://localhost:1/signingCertificates", new CancellationToken(canceled: true))); + } + + /// + /// Verifies that a failed HGS attestation request surfaces as an attestation failure, matching + /// the synchronous behaviour. + /// + [Fact] + public async Task HgsMakeRequestAsync_WhenRequestFails_ThrowsAttestationFailure() + { + TestHostGuardianServiceEnclaveProvider provider = new TestHostGuardianServiceEnclaveProvider { MaxNumRetries = 0 }; + + SqlException exception = await Assert.ThrowsAsync( + () => provider.InvokeMakeRequestAsync("http://localhost:1/signingCertificates", CancellationToken.None)); + + Assert.NotNull(exception.InnerException); + } + + #endregion + + #region Helpers + + /// + /// Drives the async GetEnclaveSession -> CreateEnclaveSession attestation sequence. + /// + private static async Task AttestAsync( + SqlColumnEncryptionEnclaveProvider provider, + EnclaveSessionParameters parameters) + { + (SqlEnclaveSession session, _, byte[] customData, int customDataLength) = + await provider.GetEnclaveSessionAsync(parameters, generateCustomData: true, isRetry: false) + .ConfigureAwait(false); + + if (session != null) + { + return session; + } + + (SqlEnclaveSession created, _) = await provider + .CreateEnclaveSessionAsync(Array.Empty(), null, parameters, customData, customDataLength) + .ConfigureAwait(false); + + return created; + } + + /// + /// Drives the sync GetEnclaveSession -> CreateEnclaveSession attestation sequence. + /// + private static SqlEnclaveSession Attest( + SqlColumnEncryptionEnclaveProvider provider, + EnclaveSessionParameters parameters) + { + provider.GetEnclaveSession( + parameters, + generateCustomData: true, + isRetry: false, + out SqlEnclaveSession session, + out _, + out byte[] customData, + out int customDataLength); + + if (session != null) + { + return session; + } + + provider.CreateEnclaveSession( + Array.Empty(), + null, + parameters, + customData, + customDataLength, + out SqlEnclaveSession created, + out _); + + return created; + } + + /// + /// A provider whose synchronous members record their invocations, used to prove that the async + /// defaults on the abstract base type defer to them. + /// + private sealed class RecordingEnclaveProvider : SqlColumnEncryptionEnclaveProvider + { + internal const long ExpectedCounter = 42; + + internal static readonly byte[] ExpectedCustomData = new byte[] { 9, 8, 7 }; + + internal SqlEnclaveSession Session { get; } = new SqlEnclaveSession(SharedSecret, sessionId: 1); + + internal int GetEnclaveSessionCallCount { get; private set; } + + internal int GetAttestationParametersCallCount { get; private set; } + + internal int CreateEnclaveSessionCallCount { get; private set; } + + internal int InvalidateEnclaveSessionCallCount { get; private set; } + + internal override void GetEnclaveSession( + EnclaveSessionParameters enclaveSessionParameters, + bool generateCustomData, + bool isRetry, + out SqlEnclaveSession sqlEnclaveSession, + out long counter, + out byte[] customData, + out int customDataLength) + { + GetEnclaveSessionCallCount++; + sqlEnclaveSession = Session; + counter = ExpectedCounter; + customData = ExpectedCustomData; + customDataLength = ExpectedCustomData.Length; + } + + internal override SqlEnclaveAttestationParameters GetAttestationParameters( + string attestationUrl, + byte[] customData, + int customDataLength) + { + GetAttestationParametersCallCount++; + return new SqlEnclaveAttestationParameters( + protocol: 1, + input: Array.Empty(), + clientDiffieHellmanKey: ECDiffieHellman.Create()); + } + + internal override void CreateEnclaveSession( + byte[] enclaveAttestationInfo, + ECDiffieHellman clientDiffieHellmanKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + out SqlEnclaveSession sqlEnclaveSession, + out long counter) + { + CreateEnclaveSessionCallCount++; + sqlEnclaveSession = Session; + counter = ExpectedCounter; + } + + internal override void InvalidateEnclaveSession( + EnclaveSessionParameters enclaveSessionParameters, + SqlEnclaveSession enclaveSession) + { + InvalidateEnclaveSessionCallCount++; + } + } + + /// + /// A provider whose synchronous members always throw, used to prove that the async defaults + /// return faulted Tasks instead of throwing synchronously. + /// + private sealed class ThrowingEnclaveProvider : SqlColumnEncryptionEnclaveProvider + { + internal override void GetEnclaveSession( + EnclaveSessionParameters enclaveSessionParameters, + bool generateCustomData, + bool isRetry, + out SqlEnclaveSession sqlEnclaveSession, + out long counter, + out byte[] customData, + out int customDataLength) + => throw new InvalidOperationException(); + + internal override SqlEnclaveAttestationParameters GetAttestationParameters( + string attestationUrl, + byte[] customData, + int customDataLength) + => throw new InvalidOperationException(); + + internal override void CreateEnclaveSession( + byte[] enclaveAttestationInfo, + ECDiffieHellman clientDiffieHellmanKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + out SqlEnclaveSession sqlEnclaveSession, + out long counter) + => throw new InvalidOperationException(); + + internal override void InvalidateEnclaveSession( + EnclaveSessionParameters enclaveSessionParameters, + SqlEnclaveSession enclaveSession) + => throw new InvalidOperationException(); + } + + /// + /// An whose "attestation" is a delay, so that the sync and + /// async session gates can be exercised without a network dependency. + /// + private sealed class FakeAttestationEnclaveProvider : EnclaveProviderBase + { + private static long s_nextSessionId; + + private readonly TimeSpan _attestationDelay; + + private int _attestationCount; + + internal FakeAttestationEnclaveProvider(TimeSpan attestationDelay) + { + _attestationDelay = attestationDelay; + } + + internal int AttestationCount => Volatile.Read(ref _attestationCount); + + internal override void GetEnclaveSession( + EnclaveSessionParameters enclaveSessionParameters, + bool generateCustomData, + bool isRetry, + out SqlEnclaveSession sqlEnclaveSession, + out long counter, + out byte[] customData, + out int customDataLength) + { + GetEnclaveSessionHelper( + enclaveSessionParameters, + generateCustomData, + isRetry, + out sqlEnclaveSession, + out counter, + out customData, + out customDataLength); + } + + internal override SqlEnclaveAttestationParameters GetAttestationParameters( + string attestationUrl, + byte[] customData, + int customDataLength) + { + return new SqlEnclaveAttestationParameters( + protocol: 1, + input: Array.Empty(), + clientDiffieHellmanKey: ECDiffieHellman.Create()); + } + + internal override void CreateEnclaveSession( + byte[] enclaveAttestationInfo, + ECDiffieHellman clientDiffieHellmanKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + out SqlEnclaveSession sqlEnclaveSession, + out long counter) + { + sqlEnclaveSession = null; + counter = 0; + try + { + ThreadRetryCache.Remove(Thread.CurrentThread.ManagedThreadId.ToString()); + sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out counter); + if (sqlEnclaveSession == null) + { + Interlocked.Increment(ref _attestationCount); + Thread.Sleep(_attestationDelay); + sqlEnclaveSession = AddEnclaveSessionToCache( + enclaveSessionParameters, + SharedSecret, + Interlocked.Increment(ref s_nextSessionId), + out counter); + } + } + finally + { + UpdateEnclaveSessionLockStatus(sqlEnclaveSession); + } + } + + internal override void InvalidateEnclaveSession( + EnclaveSessionParameters enclaveSessionParameters, + SqlEnclaveSession enclaveSession) + { + InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSession); + } + + internal override Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionAsync( + EnclaveSessionParameters enclaveSessionParameters, + bool generateCustomData, + bool isRetry, + CancellationToken cancellationToken = default) + { + return GetEnclaveSessionHelperAsync(enclaveSessionParameters, generateCustomData, isRetry, cancellationToken); + } + + internal override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( + byte[] enclaveAttestationInfo, + ECDiffieHellman clientDiffieHellmanKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken = default) + { + SqlEnclaveSession sqlEnclaveSession = null; + long counter = 0; + try + { + ThreadRetryCache.Remove(Thread.CurrentThread.ManagedThreadId.ToString()); + sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out counter); + if (sqlEnclaveSession == null) + { + Interlocked.Increment(ref _attestationCount); + await Task.Delay(_attestationDelay, cancellationToken).ConfigureAwait(false); + sqlEnclaveSession = AddEnclaveSessionToCache( + enclaveSessionParameters, + SharedSecret, + Interlocked.Increment(ref s_nextSessionId), + out counter); + } + } + finally + { + UpdateAsyncEnclaveSessionLockStatus(sqlEnclaveSession, cancellationToken); + } + + return (sqlEnclaveSession, counter); + } + } + + /// + /// Exposes the HGS provider's protected request members to the test. + /// + private sealed class TestHostGuardianServiceEnclaveProvider : HostGuardianServiceEnclaveProvider + { + internal Task InvokeMakeRequestAsync(string url, CancellationToken cancellationToken) + => MakeRequestAsync(url, cancellationToken); + } + + #endregion + } +} From bcfe2c478e4c8bad6acdddb5e7bb8f7cf96dc531 Mon Sep 17 00:00:00 2001 From: Cheena Malhotra Date: Mon, 17 Aug 2026 13:47:07 -0700 Subject: [PATCH 2/4] Reference Microsoft.Extensions.Caching.Memory explicitly in unit tests The enclave provider async tests derive from EnclaveProviderBase and call ThreadRetryCache.Remove, where ThreadRetryCache is a MemoryCache. That type lives in Microsoft.Extensions.Caching.Memory, so the test assembly needs a direct compile reference to it. In Project mode the reference flows transitively through the SqlClient project reference, which is why local builds passed. In Package mode the SqlClient package reference sets ExcludeAssets="compile" so that the compiler binds against the implementation assembly rather than the ref assembly, and that exclusion also suppresses the transitive compile asset. The result was: SqlColumnEncryptionEnclaveProviderAsyncShould.cs(570,21): error CS0012: The type 'MemoryCache' is defined in an assembly that is not referenced. Declaring the PackageReference explicitly fixes Package mode and is accurate in both modes, since the test code really does compile against that type. The version resolves through central package management, which already selects 8.0.1 or 9.0.18 based on the target framework. Verified by reproducing the failure locally in Package mode without this change and confirming it builds and passes with it, on net8.0, net9.0 and net10.0. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: cf57fdc0-1c77-40f1-a1d5-7c0bd98c1a89 --- .../UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj b/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj index 04afcf3910..06d0783c22 100644 --- a/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj +++ b/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj @@ -89,6 +89,13 @@ + + + + From 575a2a2b69c70462a40015e2218ff8608f76b4d7 Mon Sep 17 00:00:00 2001 From: Cheena Malhotra Date: Thu, 20 Aug 2026 11:46:38 -0700 Subject: [PATCH 3/4] Rework async enclave attestation gate to remove thread-affinity Addresses review feedback on the async enclave provider hierarchy: - The async attestation gate is now taken and released entirely inside CreateEnclaveSessionAsync via a disposable lease, so the semaphore is released in a finally on every exit path (success, failure, cancellation) and ownership never depends on which thread a continuation resumes on. - The async path no longer reads or writes the static, thread-id keyed ThreadRetryCache, so it can no longer leave stale entries that would make a later synchronous caller skip the sync gate. - Concurrent cold starts still collapse into a single attestation because CreateEnclaveSessionAsync re-checks the session cache after taking the gate. - GetEnclaveSessionHelperAsync now early-returns a cached session and performs no gating, so it never waits on another caller's in-flight attestation. - The signing key retry backoff is awaited (Task.Delay) on the async path instead of blocking a thread pool thread with Thread.Sleep; the synchronous path keeps its existing Thread.Sleep behaviour. - Documented the net462 cancellation granularity limit in MakeRequestAsync. - Tests: renamed the mismatched cancellation test, added coverage for gate release on attestation failure, and tightened the concurrent cold-start assertion to exactly one attestation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../SqlColumnEncryptionEnclaveProvider.xml | 9 +- .../AzureAttestationBasedEnclaveProvider.cs | 103 +++++---- .../Data/SqlClient/EnclaveProviderBase.cs | 196 +++++++----------- .../VirtualSecureModeEnclaveProvider.cs | 6 + .../VirtualSecureModeEnclaveProviderBase.cs | 68 +++--- ...umnEncryptionEnclaveProviderAsyncShould.cs | 72 +++++-- 6 files changed, 238 insertions(+), 216 deletions(-) diff --git a/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml b/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml index 4e6881a746..0c2a1204d4 100644 --- a/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml +++ b/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml @@ -9,10 +9,13 @@ An enclave is a protected region of memory inside SQL Server, used for computations on encrypted columns. An enclave provider encapsulates the client-side implementation details of the enclave attestation protocol as well as the logic for creating and caching enclave sessions. - Attestation is a three-call sequence: GetEnclaveSession, GetAttestationParameters, then CreateEnclaveSession. The first call may take an attestation gate that is only released by the matching CreateEnclaveSession call. + Attestation is a three-call sequence: GetEnclaveSession, GetAttestationParameters, then CreateEnclaveSession. On the synchronous path, the first call may take an attestation gate that is only released by the matching CreateEnclaveSession call, so the synchronous members of a single attestation sequence must all be invoked on the same thread. - Callers must not mix the synchronous and asynchronous members within a single attestation sequence. The base implementation maintains separate gates for the two paths, so pairing GetEnclaveSessionAsync with the synchronous CreateEnclaveSession (or vice versa) leaves a gate held until its timeout expires and stalls other callers. + The asynchronous members carry no such requirement. Their attestation gate is taken and released entirely within CreateEnclaveSessionAsync, so no state is held between calls and continuations may resume on any thread. Concurrent cold starts are still collapsed into a single attestation because CreateEnclaveSessionAsync re-checks the session cache after taking the gate. + + + Callers must not mix the synchronous and asynchronous members within a single attestation sequence, because the two paths use independent gates. @@ -182,7 +185,7 @@ The default implementation first checks the and returns a canceled task if cancellation has been requested. Otherwise, it calls the synchronous GetEnclaveSession method and wraps the result in a completed task. If the synchronous method throws, the exception is caught and a faulted task is returned rather than throwing synchronously. - Providers that perform network I/O during attestation must override this method with a genuinely asynchronous implementation so that the calling thread is not blocked while waiting for an in-flight attestation to complete. + This method performs no network I/O and never waits on another caller's in-flight attestation, so it always completes promptly. The parameter exists for parity with GetEnclaveSession and does not affect the asynchronous path. diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs index c6b3080696..ce380989d5 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs @@ -168,6 +168,10 @@ internal override Task GetAttestationParameters // Asynchronous counterpart of CreateEnclaveSession. Performs the attestation service round // trip (OpenID Connect metadata download) asynchronously. + // + // The async attestation gate is taken for the duration of this method only, so it is released + // on every exit path — success, failure and cancellation alike — without depending on which + // thread a continuation resumes on. internal override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( byte[] attestationInfo, ECDiffieHellman clientDHKey, @@ -176,51 +180,44 @@ internal override Task GetAttestationParameters int customDataLength, CancellationToken cancellationToken = default) { - SqlEnclaveSession sqlEnclaveSession = null; - long counter = 0; - try + using (await AcquireAsyncAttestationGateAsync(cancellationToken).ConfigureAwait(false)) { - ThreadRetryCache.Remove(Thread.CurrentThread.ManagedThreadId.ToString()); - sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out counter); - if (sqlEnclaveSession == null) + // Another caller may have completed the attestation while we waited for the gate. + SqlEnclaveSession sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out long counter); + if (sqlEnclaveSession != null) { - if (!string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl) && customData != null && customDataLength > 0) - { - byte[] nonce = customData; + return (sqlEnclaveSession, counter); + } - IdentityModelEventSource.ShowPII = true; + if (string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl) || customData == null || customDataLength <= 0) + { + throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); + } - // Deserialize the payload - AzureAttestationInfo attestInfo = new AzureAttestationInfo(attestationInfo); + byte[] nonce = customData; - // Validate the attestation info - await VerifyAzureAttestationInfoAsync( - enclaveSessionParameters.AttestationUrl, - attestInfo.EnclaveType, - attestInfo.AttestationToken.AttestationToken, - attestInfo.Identity, - nonce, - cancellationToken).ConfigureAwait(false); + IdentityModelEventSource.ShowPII = true; - // Set up shared secret and validate signature - byte[] sharedSecret = GetSharedSecret(attestInfo.Identity, nonce, attestInfo.EnclaveType, attestInfo.EnclaveDHInfo, clientDHKey); + // Deserialize the payload + AzureAttestationInfo attestInfo = new AzureAttestationInfo(attestationInfo); - // add session to cache - sqlEnclaveSession = AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, attestInfo.SessionId, out counter); - } - else - { - throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); - } - } - } - finally - { - // See UpdateEnclaveSessionLockStatus for the rationale; this releases the async gate. - UpdateAsyncEnclaveSessionLockStatus(sqlEnclaveSession, cancellationToken); - } + // Validate the attestation info + await VerifyAzureAttestationInfoAsync( + enclaveSessionParameters.AttestationUrl, + attestInfo.EnclaveType, + attestInfo.AttestationToken.AttestationToken, + attestInfo.Identity, + nonce, + cancellationToken).ConfigureAwait(false); + + // Set up shared secret and validate signature + byte[] sharedSecret = GetSharedSecret(attestInfo.Identity, nonce, attestInfo.EnclaveType, attestInfo.EnclaveDHInfo, clientDHKey); + + // add session to cache + sqlEnclaveSession = AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, attestInfo.SessionId, out counter); - return (sqlEnclaveSession, counter); + return (sqlEnclaveSession, counter); + } } // Asynchronous counterpart of InvalidateEnclaveSession. Session eviction is an in-memory @@ -460,7 +457,14 @@ await GetOpenIdConfigForSigningKeysAsync(attestationInstanceUrl, shouldForceUpda // Verify the token signature against the signing keys downloaded from meta data end point bool isKeySigningExpired; - isSignatureValid = VerifyTokenSignature(attestationToken, attestationInstanceUrl, openIdConfig.SigningKeys, out isKeySigningExpired, out exceptionMessage); + isSignatureValid = VerifyTokenSignatureCore(attestationToken, attestationInstanceUrl, openIdConfig.SigningKeys, out isKeySigningExpired, out exceptionMessage); + + if (isKeySigningExpired) + { + // Wait for SigningKeyRetryInSec sec before retrying to download the signing keys again. + // The synchronous path blocks the calling thread here; the async path must not. + await Task.Delay(SigningKeyRetryInSec * 1000, cancellationToken).ConfigureAwait(false); + } // In cases if we fail to validate the token, since we are using the old signing keys // let's re-download the signing keys again and re-validate the token signature @@ -578,7 +582,27 @@ private static ICollection GenerateListOfIssuers(string tokenIssuerUrl) } // Verifies the attestation token is signed by correct signing keys. + // + // On the SecurityTokenValidationException retry path this blocks the calling thread for + // SigningKeyRetryInSec seconds. The delay is applied here (rather than in VerifyTokenSignatureCore) + // so that the asynchronous path can await the same backoff instead of blocking a thread pool thread. private bool VerifyTokenSignature(string attestationToken, string tokenIssuerUrl, ICollection issuerSigningKeys, out bool isKeySigningExpired, out string exceptionMessage) + { + bool isSignatureValid = VerifyTokenSignatureCore(attestationToken, tokenIssuerUrl, issuerSigningKeys, out isKeySigningExpired, out exceptionMessage); + + if (isKeySigningExpired) + { + // Sleep for SigningKeyRetryInSec sec before retrying to download the signing keys again. + Thread.Sleep(SigningKeyRetryInSec * 1000); + } + + return isSignatureValid; + } + + // Verifies the attestation token is signed by correct signing keys, without applying the + // signing key retry backoff. Callers are responsible for the backoff so that synchronous and + // asynchronous callers can each wait in the manner appropriate to them. + private bool VerifyTokenSignatureCore(string attestationToken, string tokenIssuerUrl, ICollection issuerSigningKeys, out bool isKeySigningExpired, out string exceptionMessage) { exceptionMessage = string.Empty; bool isSignatureValid = false; @@ -610,9 +634,6 @@ private bool VerifyTokenSignature(string attestationToken, string tokenIssuerUrl catch (SecurityTokenValidationException securityTokenException) { isKeySigningExpired = true; - - // Sleep for SigningKeyRetryInSec sec before retrying to download the signing keys again. - Thread.Sleep(SigningKeyRetryInSec * 1000); exceptionMessage = GetInnerMostExceptionMessage(securityTokenException); } catch (Exception exception) diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs index 504f2f95f1..ce16e29225 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs @@ -83,23 +83,26 @@ internal abstract class EnclaveProviderBase : SqlColumnEncryptionEnclaveProvider private static readonly Object lockUpdateSessionLock = new Object(); - // Asynchronous counterparts of the three fields above. + // Attestation gate used by the asynchronous path. // - // The async attestation path deliberately uses its own gate rather than sharing - // 'sessionLockEvent' with the synchronous path. AutoResetEvent has no awaitable wait, and - // sharing a single SemaphoreSlim across both paths would let a synchronous caller block a - // thread for the whole duration of an awaited attestation HTTP round trip (and vice versa), - // which is exactly the thread-pool starvation this work is meant to remove. The two gates are - // therefore independent: at worst a concurrent sync and async cold start performs two - // attestations, which the existing design already tolerates (see the lock-timeout cases - // described above), and the session cache makes the outcome idempotent. - private static readonly SemaphoreSlim s_asyncSessionLockEvent = new SemaphoreSlim(1, 1); - - private static int s_asyncLockTimeoutInMilliseconds = LockTimeoutMaxInMilliseconds; - - private static bool s_isAsyncSessionLockAcquired = false; - - private static readonly object s_asyncLockUpdateSessionLock = new object(); + // The async path deliberately uses its own gate rather than sharing 'sessionLockEvent' with + // the synchronous path. AutoResetEvent has no awaitable wait, and sharing a single + // SemaphoreSlim across both paths would let a synchronous caller block a thread for the whole + // duration of an awaited attestation round trip (and vice versa), which is exactly the + // thread-pool starvation this work is meant to remove. + // + // Unlike the synchronous gate, the async gate is never held across API calls: it is taken and + // released inside CreateEnclaveSessionAsync, which is the only member that talks to the + // attestation service. That keeps acquisition and release in a single try/finally scope and + // removes any need to track which thread (or which continuation) owns the gate — an ownership + // model that is unsound in async code, where ConfigureAwait(false) continuations routinely + // resume on a different thread pool thread. + // + // Collapsing concurrent attestations is preserved by re-checking the session cache after the + // gate is taken: the first caller performs the attestation and caches the session, and every + // caller queued behind it observes that session and returns without contacting the + // attestation service. + private static readonly SemaphoreSlim s_asyncAttestationGate = new SemaphoreSlim(1, 1); // It is used to save the attestation url and nonce value across API calls protected static readonly MemoryCache ThreadRetryCache = new MemoryCache(new MemoryCacheOptions()); @@ -198,134 +201,87 @@ protected void GetEnclaveSessionHelper(EnclaveSessionParameters enclaveSessionPa // Because C# async methods cannot declare 'out' parameters, the four values reported through // 'out' parameters by the synchronous helper are returned as a tuple. // - // This method intentionally duplicates rather than shares the synchronous helper's body: the - // synchronous path must remain byte-for-byte unchanged (FR-010), and the two paths use - // independent attestation gates (see s_asyncSessionLockEvent). - protected async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionHelperAsync( + // Unlike the synchronous helper, this method takes no lock: the async attestation gate is + // taken and released entirely within CreateEnclaveSessionAsync (see s_asyncAttestationGate), + // so a caller that only needs to probe the session cache never waits on another caller's + // in-flight attestation. 'isRetry' is accepted for signature parity with the synchronous + // helper and is unused for the same reason. + protected Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionHelperAsync( EnclaveSessionParameters enclaveSessionParameters, bool shouldGenerateNonce, bool isRetry, CancellationToken cancellationToken = default) { - cancellationToken.ThrowIfCancellationRequested(); + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled<(SqlEnclaveSession, long, byte[], int)>(cancellationToken); + } - byte[] customData = null; - int customDataLength = 0; SqlEnclaveSession sqlEnclaveSession = SessionCache.GetEnclaveSession(enclaveSessionParameters, out long counter); - if (sqlEnclaveSession == null) + if (sqlEnclaveSession != null) { - bool sessionCacheLockTaken = false; - bool sameThreadRetry = false; - string currentThreadId = Thread.CurrentThread.ManagedThreadId.ToString(); - - // In case if on some thread we are running SQL workload which don't require attestation, then in those cases we don't want same thread to wait for the gate to be released. - // hence skipping it - string retryThreadID = ThreadRetryCache.Get(currentThreadId); - if (!string.IsNullOrEmpty(retryThreadID)) - { - sameThreadRetry = true; - } - else if (!isRetry) - { - // We are explicitly not releasing the gate here, as we want to hold it until the driver calls CreateEnclaveSessionAsync. - // If we release it now, then multiple callers end up calling GetAttestationParameters which triggers the attestation workflow. - sessionCacheLockTaken = await s_asyncSessionLockEvent - .WaitAsync(Volatile.Read(ref s_asyncLockTimeoutInMilliseconds), cancellationToken) - .ConfigureAwait(false); - - if (sessionCacheLockTaken) - { - lock (s_asyncLockUpdateSessionLock) - { - s_isAsyncSessionLockAcquired = true; - } - } - } + return Task.FromResult((sqlEnclaveSession, counter, (byte[])null, 0)); + } - // In case of a multi-threaded application, the first caller takes the gate and all the subsequent callers wait here either until the enclave - // session is created or the timeout happens. - if (sessionCacheLockTaken || sameThreadRetry || isRetry) - { - // While the current caller was waiting for the gate we may already have completed the attestation elsewhere, - // in which case we need to release the gate here. - sqlEnclaveSession = SessionCache.GetEnclaveSession(enclaveSessionParameters, out counter); - if (sqlEnclaveSession != null && !sameThreadRetry) - { - ReleaseAsyncSessionLock(restoreLockTimeout: false); - } - } - else - { - // In case we are unable to take the gate, then it represents either - // 1. Another caller has an ongoing attestation request which is taking more time, may be due to a slow network, or - // 2. The current workload doesn't require enclave computation, hence the driver is not invoking CreateEnclaveSessionAsync and sqlEnclaveSession is never set. - // In both cases we need to reduce the timeout to 0 so that subsequent requests should not wait. - Interlocked.Exchange(ref s_asyncLockTimeoutInMilliseconds, 0); - } + byte[] customData = null; + int customDataLength = 0; - if (sqlEnclaveSession == null) + if (shouldGenerateNonce) + { + using (RandomNumberGenerator rng = RandomNumberGenerator.Create()) { - if (shouldGenerateNonce) - { - using (RandomNumberGenerator rng = RandomNumberGenerator.Create()) - { - // Client decides to initiate the process of attesting the enclave and to establish a secure session with the enclave. - // To ensure that server send new attestation request instead of replaying / re-sending the old token, we will create a nonce for current attestation request. - byte[] nonce = new byte[NonceSize]; - rng.GetBytes(nonce); - customData = nonce; - customDataLength = nonce.Length; - } - } - - if (!sameThreadRetry) - { - retryThreadID = currentThreadId; - } - - ThreadRetryCache.Set(currentThreadId, retryThreadID, - absoluteExpirationRelativeToNow: s_threadRetryCacheTimeout); + // Client decides to initiate the process of attesting the enclave and to establish a secure session with the enclave. + // To ensure that server send new attestation request instead of replaying / re-sending the old token, we will create a nonce for current attestation request. + byte[] nonce = new byte[NonceSize]; + rng.GetBytes(nonce); + customData = nonce; + customDataLength = nonce.Length; } } - return (sqlEnclaveSession, counter, customData, customDataLength); + return Task.FromResult((sqlEnclaveSession, counter, customData, customDataLength)); } - // Reset the async session gate. Asynchronous counterpart of UpdateEnclaveSessionLockStatus. - // This method performs no I/O, so a plain lock remains the correct (and cheapest) primitive. + // Takes the async attestation gate, returning a lease that releases it when disposed. + // + // Callers are expected to use the lease with a 'using' statement so that the gate is released + // on every exit path, including failures and cancellation: // - // Unlike the synchronous counterpart, the gate is also released when the caller cancelled. A - // cancelled caller will never go on to create the session, so keeping the gate would stall - // every other async caller until the lock timeout expires. The synchronous path has no - // equivalent case because it has no cancellation. - protected void UpdateAsyncEnclaveSessionLockStatus(SqlEnclaveSession sqlEnclaveSession, CancellationToken cancellationToken = default) + // using (await AcquireAsyncAttestationGateAsync(cancellationToken).ConfigureAwait(false)) + // { + // // re-check the session cache, then attest + // } + // + // If the gate cannot be taken within the lock timeout the returned lease is empty and the + // caller proceeds with its own attestation. This mirrors the synchronous design's deliberate + // choice to favour progress over strict collapsing when the gate holder is unusually slow. + protected static async Task AcquireAsyncAttestationGateAsync(CancellationToken cancellationToken) { - if (sqlEnclaveSession != null || cancellationToken.IsCancellationRequested) - { - ReleaseAsyncSessionLock(restoreLockTimeout: true); - } + bool acquired = await s_asyncAttestationGate + .WaitAsync(LockTimeoutMaxInMilliseconds, cancellationToken) + .ConfigureAwait(false); + + return new AsyncAttestationGateLease(acquired); } - // Releases the async attestation gate if it is currently held. - // - // Unlike AutoResetEvent.Set(), SemaphoreSlim.Release() throws when the semaphore is already - // full, so the release is guarded by s_isAsyncSessionLockAcquired. The guard preserves the - // synchronous design's "any caller may signal" semantics while keeping the release idempotent. - private static void ReleaseAsyncSessionLock(bool restoreLockTimeout) + // Represents ownership of the async attestation gate. Disposal releases the gate only when it + // was actually taken, so a 'using' statement is safe even when the wait timed out. The lease + // must be disposed exactly once, which 'using' guarantees. + protected readonly struct AsyncAttestationGateLease : IDisposable { - lock (s_asyncLockUpdateSessionLock) - { - if (s_isAsyncSessionLockAcquired) - { - s_isAsyncSessionLockAcquired = false; + private readonly bool _acquired; - if (restoreLockTimeout) - { - Interlocked.Exchange(ref s_asyncLockTimeoutInMilliseconds, LockTimeoutMaxInMilliseconds); - } + internal AsyncAttestationGateLease(bool acquired) + { + _acquired = acquired; + } - s_asyncSessionLockEvent.Release(); + public void Dispose() + { + if (_acquired) + { + s_asyncAttestationGate.Release(); } } } diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProvider.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProvider.cs index 02815f4870..ab3b3c7d30 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProvider.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProvider.cs @@ -88,6 +88,12 @@ protected override byte[] MakeRequest(string url) // Makes a web request to the provided url and returns the response as a byte[]. // Asynchronous counterpart of MakeRequest: the HTTP round trip, the retry backoff and the // JSON deserialization are all awaited rather than blocked on. + // + // Cancellation granularity differs by target framework. On .NET the token is passed to + // HttpClient, so an in-flight request is cancelled promptly. On .NET Framework there is no + // token-accepting GetStreamAsync overload, so an in-flight request runs to completion and + // cancellation is only observed between attempts. Callers must not assume uniform + // cancellation latency across target frameworks. protected override async Task MakeRequestAsync(string url, CancellationToken cancellationToken) { Exception exception = null; diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs index 20090f7c55..d198fec5ac 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs @@ -184,6 +184,10 @@ internal override Task GetAttestationParameters // Asynchronous counterpart of CreateEnclaveSession. Performs the attestation service round // trip (signing certificate download) asynchronously. + // + // The async attestation gate is taken for the duration of this method only, so it is released + // on every exit path — success, failure and cancellation alike — without depending on which + // thread a continuation resumes on. internal override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( byte[] attestationInfo, ECDiffieHellman clientDHKey, @@ -192,47 +196,41 @@ internal override Task GetAttestationParameters int customDataLength, CancellationToken cancellationToken = default) { - SqlEnclaveSession sqlEnclaveSession = null; - long counter = 0; - try + using (await AcquireAsyncAttestationGateAsync(cancellationToken).ConfigureAwait(false)) { - ThreadRetryCache.Remove(Thread.CurrentThread.ManagedThreadId.ToString()); - sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out counter); - if (sqlEnclaveSession == null) + // Another caller may have completed the attestation while we waited for the gate. + SqlEnclaveSession sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out long counter); + if (sqlEnclaveSession != null) { - if (!string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl)) - { - // Deserialize the payload - AttestationInfo info = new AttestationInfo(attestationInfo); + return (sqlEnclaveSession, counter); + } - // Verify enclave policy matches expected policy - VerifyEnclavePolicy(info.EnclaveReportPackage); + if (string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl)) + { + throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); + } - // Perform Attestation per VSM protocol - await VerifyAttestationInfoAsync( - enclaveSessionParameters.AttestationUrl, - info.HealthReport, - info.EnclaveReportPackage, - cancellationToken).ConfigureAwait(false); + // Deserialize the payload + AttestationInfo info = new AttestationInfo(attestationInfo); - // Set up shared secret and validate signature - byte[] sharedSecret = GetSharedSecret(info.Identity, info.EnclaveDHInfo, clientDHKey); + // Verify enclave policy matches expected policy + VerifyEnclavePolicy(info.EnclaveReportPackage); - // add session to cache - sqlEnclaveSession = AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, info.SessionId, out counter); - } - else - { - throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); - } - } - } - finally - { - UpdateAsyncEnclaveSessionLockStatus(sqlEnclaveSession, cancellationToken); - } + // Perform Attestation per VSM protocol + await VerifyAttestationInfoAsync( + enclaveSessionParameters.AttestationUrl, + info.HealthReport, + info.EnclaveReportPackage, + cancellationToken).ConfigureAwait(false); - return (sqlEnclaveSession, counter); + // Set up shared secret and validate signature + byte[] sharedSecret = GetSharedSecret(info.Identity, info.EnclaveDHInfo, clientDHKey); + + // add session to cache + sqlEnclaveSession = AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, info.SessionId, out counter); + + return (sqlEnclaveSession, counter); + } } // Asynchronous counterpart of InvalidateEnclaveSession. Session eviction is an in-memory @@ -367,6 +365,8 @@ private async Task GetSigningCertificateAsync(string // Makes a web request to the provided url and returns the response as a byte[]. // Asynchronous counterpart of MakeRequest. This member is abstract rather than virtual so // that derived providers cannot silently inherit a blocking implementation. + // Implementations should honour the cancellation token as closely as their target framework + // allows, and document any framework specific limits. protected abstract Task MakeRequestAsync(string url, CancellationToken cancellationToken); // Gets the root signing certificate for the provided attestation service. diff --git a/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs b/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs index 5260af146b..412c80dffe 100644 --- a/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs +++ b/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs @@ -149,11 +149,11 @@ await Assert.ThrowsAnyAsync( } /// - /// Verifies that the async session helper propagates cancellation while waiting for an - /// in-flight attestation instead of blocking. + /// Verifies that a cancelled CreateEnclaveSessionAsync surfaces cancellation and leaves no + /// enclave session behind in the session cache. /// [Fact] - public async Task GetEnclaveSessionAsync_WhenTokenIsCancelled_PropagatesCancellation() + public async Task CreateEnclaveSessionAsync_WhenTokenIsCancelled_CachesNoSession() { FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.Zero); EnclaveSessionParameters parameters = NewSessionParameters(); @@ -204,6 +204,35 @@ await Assert.ThrowsAnyAsync( #endregion + #region Failure handling + + /// + /// Verifies that a failed attestation releases the async gate. A caller that leaks the gate on + /// the failure path would stall every subsequent async caller for the 15 second lock timeout. + /// + [Fact] + public async Task CreateEnclaveSessionAsync_WhenAttestationFails_ReleasesTheAsyncGate() + { + FakeAttestationEnclaveProvider failing = + new FakeAttestationEnclaveProvider(TimeSpan.Zero, failAttestation: true); + + await Assert.ThrowsAsync( + () => AttestAsync(failing, NewSessionParameters())); + + // A subsequent, unrelated attestation must not wait on the abandoned gate. + FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.Zero); + System.Diagnostics.Stopwatch stopwatch = System.Diagnostics.Stopwatch.StartNew(); + SqlEnclaveSession next = await AttestAsync(provider, NewSessionParameters()); + stopwatch.Stop(); + + Assert.NotNull(next); + Assert.True( + stopwatch.Elapsed < TimeSpan.FromSeconds(5), + $"Attestation took {stopwatch.Elapsed}, which suggests the async gate was not released on failure."); + } + + #endregion + #region Concurrency — async path /// /// Verifies that once an enclave session is cached, concurrent async callers all observe the @@ -234,8 +263,9 @@ await provider.GetEnclaveSessionAsync(parameters, generateCustomData: false, isR } /// - /// Verifies that many concurrent cold-start async attestations complete without deadlock or a - /// semaphore release imbalance, and that the async gate remains usable afterwards. + /// Verifies that many concurrent cold-start async attestations collapse into a single + /// attestation, complete without deadlock or a semaphore release imbalance, and leave the + /// async gate usable afterwards. /// [Fact] public async Task CreateEnclaveSessionAsync_ConcurrentColdStart_CompletesWithoutDeadlock() @@ -247,7 +277,11 @@ public async Task CreateEnclaveSessionAsync_ConcurrentColdStart_CompletesWithout Enumerable.Range(0, 8).Select(index => Task.Run(() => AttestAsync(provider, parameters)))); Assert.All(sessions, session => Assert.NotNull(session)); - Assert.InRange(provider.AttestationCount, 1, sessions.Length); + + // The gate serializes the cold start and the post-gate cache re-check makes every queued + // caller reuse the session created by the winner, so exactly one attestation is performed. + Assert.Equal(1, provider.AttestationCount); + Assert.All(sessions, session => Assert.Equal(sessions[0].SessionId, session.SessionId)); // The gate must still be usable for a subsequent, unrelated attestation. SqlEnclaveSession next = await AttestAsync(provider, NewSessionParameters()); @@ -515,11 +549,14 @@ private sealed class FakeAttestationEnclaveProvider : EnclaveProviderBase private readonly TimeSpan _attestationDelay; + private readonly bool _failAttestation; + private int _attestationCount; - internal FakeAttestationEnclaveProvider(TimeSpan attestationDelay) + internal FakeAttestationEnclaveProvider(TimeSpan attestationDelay, bool failAttestation = false) { _attestationDelay = attestationDelay; + _failAttestation = failAttestation; } internal int AttestationCount => Volatile.Read(ref _attestationCount); @@ -610,29 +647,28 @@ internal override void InvalidateEnclaveSession( int customDataLength, CancellationToken cancellationToken = default) { - SqlEnclaveSession sqlEnclaveSession = null; - long counter = 0; - try + using (await AcquireAsyncAttestationGateAsync(cancellationToken).ConfigureAwait(false)) { - ThreadRetryCache.Remove(Thread.CurrentThread.ManagedThreadId.ToString()); - sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out counter); + SqlEnclaveSession sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out long counter); if (sqlEnclaveSession == null) { Interlocked.Increment(ref _attestationCount); await Task.Delay(_attestationDelay, cancellationToken).ConfigureAwait(false); + + if (_failAttestation) + { + throw new InvalidOperationException("Simulated attestation failure."); + } + sqlEnclaveSession = AddEnclaveSessionToCache( enclaveSessionParameters, SharedSecret, Interlocked.Increment(ref s_nextSessionId), out counter); } - } - finally - { - UpdateAsyncEnclaveSessionLockStatus(sqlEnclaveSession, cancellationToken); - } - return (sqlEnclaveSession, counter); + return (sqlEnclaveSession, counter); + } } } From c390d8afe1d121dfeeb1912f68ec776fdef5e0a2 Mon Sep 17 00:00:00 2001 From: Cheena Malhotra Date: Fri, 21 Aug 2026 15:17:38 -0700 Subject: [PATCH 4/4] Enforce the async attestation gate protocol in EnclaveProviderBase Follow-up to review feedback on the async enclave provider hierarchy. - EnclaveProviderBase now seals CreateEnclaveSessionAsync. It acquires the async gate, re-checks the session cache once held, and releases in a finally; providers supply only protocol-specific logic via the new protected abstract CreateEnclaveSessionCoreAsync. The gate can no longer be bypassed or leaked by a derived provider, which also removes the publicly-reachable lease type and its disposal contract. - EnclaveProviderBase also seals GetEnclaveSessionAsync, routing through GetEnclaveSessionHelperAsync via the new GeneratesNonceForAttestation hook. This fixes a latent bug: NoneAttestationEnclaveProvider had no async overrides, so it inherited the default that calls the *synchronous* GetEnclaveSession, taking the sync gate that only a later synchronous CreateEnclaveSession would release. An async caller never makes that call, so the sync gate was stranded for its full 15s timeout. Covered by a new regression test that fails (15s stall) without the seal. - NoneAttestationEnclaveProvider: extracted the session-setup parsing into a shared helper used by both the sync and async paths. - Removed the GetAttestationParametersAsync/InvalidateEnclaveSessionAsync overrides from the Azure and VSM providers; they were byte-for-byte identical to the inherited defaults. - Documented that the async path collapses the attestation service call but deliberately does not collapse the per-caller work before it, in the source comment, the doc snippet, and a new assertion on AttestationParametersCount. - Tests: drive the real three-call sequence (including GetAttestationParametersAsync, which produces the client ECDH key) in the Attest/AttestAsync helpers; reuse one provider instance in the gate-release-on-failure test so it does not depend on the gate being static; assert the mixed sync/async race converges on a single cached session. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../SqlColumnEncryptionEnclaveProvider.xml | 5 +- .../AzureAttestationBasedEnclaveProvider.cs | 118 +++--------- .../Data/SqlClient/EnclaveProviderBase.cs | 114 ++++++++---- .../NoneAttestationEnclaveProvider.cs | 124 ++++++++----- .../VirtualSecureModeEnclaveProviderBase.cs | 114 +++--------- ...umnEncryptionEnclaveProviderAsyncShould.cs | 175 +++++++++++++----- 6 files changed, 348 insertions(+), 302 deletions(-) diff --git a/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml b/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml index 0c2a1204d4..f04d26efc8 100644 --- a/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml +++ b/doc/snippets/Microsoft.Data.SqlClient/SqlColumnEncryptionEnclaveProvider.xml @@ -12,7 +12,10 @@ Attestation is a three-call sequence: GetEnclaveSession, GetAttestationParameters, then CreateEnclaveSession. On the synchronous path, the first call may take an attestation gate that is only released by the matching CreateEnclaveSession call, so the synchronous members of a single attestation sequence must all be invoked on the same thread. - The asynchronous members carry no such requirement. Their attestation gate is taken and released entirely within CreateEnclaveSessionAsync, so no state is held between calls and continuations may resume on any thread. Concurrent cold starts are still collapsed into a single attestation because CreateEnclaveSessionAsync re-checks the session cache after taking the gate. + The asynchronous members carry no such requirement. Their attestation gate is taken and released entirely within CreateEnclaveSessionAsync, so no state is held between calls and continuations may resume on any thread. + + + Because that gate is not held across GetEnclaveSessionAsync, concurrent cold-start callers are collapsed later than on the synchronous path. Each caller generates its own attestation parameters and receives its own enclave attestation info from the server; they converge at CreateEnclaveSessionAsync, which re-checks the session cache once it holds the gate, so only one call to the attestation service is made. This trades a little redundant per-caller work for an ownership model that is sound when continuations resume on arbitrary threads. Callers must not mix the synchronous and asynchronous members within a single attestation sequence, because the two paths use independent gates. diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs index ce380989d5..0965e96518 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/AzureAttestationBasedEnclaveProvider.cs @@ -132,115 +132,51 @@ internal override void InvalidateEnclaveSession(EnclaveSessionParameters enclave InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSessionToInvalidate); } - // Asynchronous counterpart of GetEnclaveSession. Uses the async attestation gate so that - // callers never block a thread pool thread waiting for an in-flight attestation. - internal override Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionAsync( - EnclaveSessionParameters enclaveSessionParameters, - bool generateCustomData, - bool isRetry, - CancellationToken cancellationToken = default) - { - return GetEnclaveSessionHelperAsync(enclaveSessionParameters, generateCustomData, isRetry, cancellationToken); - } - - // Asynchronous counterpart of GetAttestationParameters. This operation is CPU bound (key - // generation and buffer marshalling), so it completes synchronously. - internal override Task GetAttestationParametersAsync( - string attestationUrl, - byte[] customData, - int customDataLength, - CancellationToken cancellationToken = default) - { - if (cancellationToken.IsCancellationRequested) - { - return Task.FromCanceled(cancellationToken); - } - - try - { - return Task.FromResult(GetAttestationParameters(attestationUrl, customData, customDataLength)); - } - catch (Exception e) when (ADP.IsCatchableExceptionType(e)) - { - return Task.FromException(e); - } - } + // The Azure Attestation protocol uses a client-generated nonce to prevent token replay. + protected override bool GeneratesNonceForAttestation => true; // Asynchronous counterpart of CreateEnclaveSession. Performs the attestation service round // trip (OpenID Connect metadata download) asynchronously. // - // The async attestation gate is taken for the duration of this method only, so it is released - // on every exit path — success, failure and cancellation alike — without depending on which - // thread a continuation resumes on. - internal override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( + // The async attestation gate is taken and released by the sealed CreateEnclaveSessionAsync in + // EnclaveProviderBase, which also re-checks the session cache before calling this method. + protected override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionCoreAsync( byte[] attestationInfo, ECDiffieHellman clientDHKey, EnclaveSessionParameters enclaveSessionParameters, byte[] customData, int customDataLength, - CancellationToken cancellationToken = default) + CancellationToken cancellationToken) { - using (await AcquireAsyncAttestationGateAsync(cancellationToken).ConfigureAwait(false)) + if (string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl) || customData == null || customDataLength <= 0) { - // Another caller may have completed the attestation while we waited for the gate. - SqlEnclaveSession sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out long counter); - if (sqlEnclaveSession != null) - { - return (sqlEnclaveSession, counter); - } - - if (string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl) || customData == null || customDataLength <= 0) - { - throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); - } - - byte[] nonce = customData; + throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); + } - IdentityModelEventSource.ShowPII = true; + byte[] nonce = customData; - // Deserialize the payload - AzureAttestationInfo attestInfo = new AzureAttestationInfo(attestationInfo); + IdentityModelEventSource.ShowPII = true; - // Validate the attestation info - await VerifyAzureAttestationInfoAsync( - enclaveSessionParameters.AttestationUrl, - attestInfo.EnclaveType, - attestInfo.AttestationToken.AttestationToken, - attestInfo.Identity, - nonce, - cancellationToken).ConfigureAwait(false); + // Deserialize the payload + AzureAttestationInfo attestInfo = new AzureAttestationInfo(attestationInfo); - // Set up shared secret and validate signature - byte[] sharedSecret = GetSharedSecret(attestInfo.Identity, nonce, attestInfo.EnclaveType, attestInfo.EnclaveDHInfo, clientDHKey); + // Validate the attestation info + await VerifyAzureAttestationInfoAsync( + enclaveSessionParameters.AttestationUrl, + attestInfo.EnclaveType, + attestInfo.AttestationToken.AttestationToken, + attestInfo.Identity, + nonce, + cancellationToken).ConfigureAwait(false); - // add session to cache - sqlEnclaveSession = AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, attestInfo.SessionId, out counter); + // Set up shared secret and validate signature + byte[] sharedSecret = GetSharedSecret(attestInfo.Identity, nonce, attestInfo.EnclaveType, attestInfo.EnclaveDHInfo, clientDHKey); - return (sqlEnclaveSession, counter); - } - } + // add session to cache + SqlEnclaveSession sqlEnclaveSession = + AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, attestInfo.SessionId, out long counter); - // Asynchronous counterpart of InvalidateEnclaveSession. Session eviction is an in-memory - // cache operation, so it completes synchronously. - internal override Task InvalidateEnclaveSessionAsync( - EnclaveSessionParameters enclaveSessionParameters, - SqlEnclaveSession enclaveSessionToInvalidate, - CancellationToken cancellationToken = default) - { - if (cancellationToken.IsCancellationRequested) - { - return Task.FromCanceled(cancellationToken); - } - - try - { - InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSessionToInvalidate); - return Task.CompletedTask; - } - catch (Exception e) when (ADP.IsCatchableExceptionType(e)) - { - return Task.FromException(e); - } + return (sqlEnclaveSession, counter); } #endregion diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs index ce16e29225..e0e039210d 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveProviderBase.cs @@ -98,10 +98,17 @@ internal abstract class EnclaveProviderBase : SqlColumnEncryptionEnclaveProvider // model that is unsound in async code, where ConfigureAwait(false) continuations routinely // resume on a different thread pool thread. // - // Collapsing concurrent attestations is preserved by re-checking the session cache after the - // gate is taken: the first caller performs the attestation and caches the session, and every - // caller queued behind it observes that session and returns without contacting the - // attestation service. + // Calls to the attestation service are still collapsed: the gate is re-checked against the + // session cache after it is taken, so the first caller performs the attestation and caches the + // session, and every caller queued behind it observes that session and returns without + // contacting the attestation service. + // + // Unlike the synchronous path, the work *before* the attestation is no longer collapsed. + // Because this gate is not held across GetEnclaveSessionAsync, concurrent cold-start callers + // each generate a Diffie-Hellman key in GetAttestationParametersAsync and each receive enclave + // attestation info from the server; only the attestation service call itself is shared. This + // is a deliberate trade: it buys an ownership model that is sound under ConfigureAwait(false) + // at the cost of some redundant per-caller work during a cold start. private static readonly SemaphoreSlim s_asyncAttestationGate = new SemaphoreSlim(1, 1); // It is used to save the attestation url and nonce value across API calls @@ -243,49 +250,92 @@ protected void GetEnclaveSessionHelper(EnclaveSessionParameters enclaveSessionPa return Task.FromResult((sqlEnclaveSession, counter, customData, customDataLength)); } - // Takes the async attestation gate, returning a lease that releases it when disposed. - // - // Callers are expected to use the lease with a 'using' statement so that the gate is released - // on every exit path, including failures and cancellation: - // - // using (await AcquireAsyncAttestationGateAsync(cancellationToken).ConfigureAwait(false)) - // { - // // re-check the session cache, then attest - // } + // Indicates whether this provider's attestation protocol uses a client-generated nonce. + // Mirrors the value each provider passes to GetEnclaveSessionHelper on the synchronous path. + protected abstract bool GeneratesNonceForAttestation { get; } + + // Looks up an existing enclave session in the session cache. // - // If the gate cannot be taken within the lock timeout the returned lease is empty and the - // caller proceeds with its own attestation. This mirrors the synchronous design's deliberate - // choice to favour progress over strict collapsing when the gate holder is unusually slow. - protected static async Task AcquireAsyncAttestationGateAsync(CancellationToken cancellationToken) + // This member is sealed because the inherited default would call the *synchronous* + // GetEnclaveSession, which takes the synchronous attestation gate and relies on a later + // synchronous CreateEnclaveSession to release it. An async caller never makes that call, so the + // sync gate would be held until its timeout expired and would stall unrelated sync callers. + // Routing through GetEnclaveSessionHelperAsync keeps async callers off the sync gate entirely. + internal sealed override Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionAsync( + EnclaveSessionParameters enclaveSessionParameters, + bool generateCustomData, + bool isRetry, + CancellationToken cancellationToken = default) { - bool acquired = await s_asyncAttestationGate - .WaitAsync(LockTimeoutMaxInMilliseconds, cancellationToken) - .ConfigureAwait(false); - - return new AsyncAttestationGateLease(acquired); + return GetEnclaveSessionHelperAsync( + enclaveSessionParameters, + GeneratesNonceForAttestation && generateCustomData, + isRetry, + cancellationToken); } - // Represents ownership of the async attestation gate. Disposal releases the gate only when it - // was actually taken, so a 'using' statement is safe even when the wait timed out. The lease - // must be disposed exactly once, which 'using' guarantees. - protected readonly struct AsyncAttestationGateLease : IDisposable + // Creates a new enclave session, serializing concurrent attestations behind the async gate. + // + // This member is sealed so that the gate protocol lives in exactly one place and cannot be + // bypassed (or leaked) by a derived provider. Providers supply only the protocol-specific + // attestation logic by overriding CreateEnclaveSessionCoreAsync; acquiring the gate, + // re-checking the session cache once it is held, and releasing the gate on every exit path + // are all handled here. + internal sealed override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( + byte[] enclaveAttestationInfo, + ECDiffieHellman clientDiffieHellmanKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken = default) { - private readonly bool _acquired; + // If the gate cannot be taken within the lock timeout we proceed with our own attestation + // rather than failing. This mirrors the synchronous design's deliberate choice to favour + // progress over strict collapsing when the gate holder is unusually slow. + bool gateAcquired = await s_asyncAttestationGate + .WaitAsync(LockTimeoutMaxInMilliseconds, cancellationToken) + .ConfigureAwait(false); - internal AsyncAttestationGateLease(bool acquired) + try { - _acquired = acquired; - } + // Another caller may have completed the attestation while we waited for the gate. + SqlEnclaveSession sqlEnclaveSession = SessionCache.GetEnclaveSession(enclaveSessionParameters, out long counter); + if (sqlEnclaveSession != null) + { + return (sqlEnclaveSession, counter); + } - public void Dispose() + return await CreateEnclaveSessionCoreAsync( + enclaveAttestationInfo, + clientDiffieHellmanKey, + enclaveSessionParameters, + customData, + customDataLength, + cancellationToken).ConfigureAwait(false); + } + finally { - if (_acquired) + if (gateAcquired) { s_asyncAttestationGate.Release(); } } } + // Performs the provider-specific enclave attestation and adds the resulting session to the + // session cache. Asynchronous counterpart of the body of CreateEnclaveSession. + // + // Called by CreateEnclaveSessionAsync with the async attestation gate already held and the + // session cache already re-checked, so implementations only need to attest and cache. They + // must not take the gate themselves. + protected abstract Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionCoreAsync( + byte[] enclaveAttestationInfo, + ECDiffieHellman clientDiffieHellmanKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken); + // Reset the session lock status protected void UpdateEnclaveSessionLockStatus(SqlEnclaveSession sqlEnclaveSession) { diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/NoneAttestationEnclaveProvider.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/NoneAttestationEnclaveProvider.cs index fabd69c976..1a85c48c61 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/NoneAttestationEnclaveProvider.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/NoneAttestationEnclaveProvider.cs @@ -6,6 +6,7 @@ using System.Diagnostics; using System.Security.Cryptography; using System.Threading; +using System.Threading.Tasks; namespace Microsoft.Data.SqlClient { @@ -30,12 +31,12 @@ internal override SqlEnclaveAttestationParameters GetAttestationParameters(strin return new SqlEnclaveAttestationParameters(NoneAttestationProtocolId, Array.Empty(), clientDHKey); } + // The None attestation protocol does not use a client-generated nonce. + protected override bool GeneratesNonceForAttestation => false; + // When overridden in a derived class, performs enclave attestation, generates a symmetric key for the session, creates an enclave session and stores the session information in the cache. internal override void CreateEnclaveSession(byte[] attestationInfo, ECDiffieHellman clientDHKey, EnclaveSessionParameters enclaveSessionParameters, byte[] customData, int customDataLength, out SqlEnclaveSession sqlEnclaveSession, out long counter) { - // for None attestation: enclave does not send public key, and sends an empty attestation info - // The only non-trivial content it sends is the session setup info (DH pubkey of enclave) - sqlEnclaveSession = null; counter = 0; try @@ -45,46 +46,7 @@ internal override void CreateEnclaveSession(byte[] attestationInfo, ECDiffieHell if (sqlEnclaveSession == null) { - // Read AttestationInfo - int attestationInfoOffset = 0; - uint sizeOfTrustedModuleAttestationInfoBuffer = BitConverter.ToUInt32(attestationInfo, attestationInfoOffset); - attestationInfoOffset += sizeof(UInt32); - int sizeOfTrustedModuleAttestationInfoBufferInt = checked((int)sizeOfTrustedModuleAttestationInfoBuffer); - Debug.Assert(sizeOfTrustedModuleAttestationInfoBuffer == 0); - - // read secure session info - uint sizeOfSecureSessionInfoResponse = BitConverter.ToUInt32(attestationInfo, attestationInfoOffset); - attestationInfoOffset += sizeof(UInt32); - - byte[] enclaveSessionHandle = new byte[EnclaveSessionHandleSize]; - Buffer.BlockCopy(attestationInfo, attestationInfoOffset, enclaveSessionHandle, 0, EnclaveSessionHandleSize); - attestationInfoOffset += EnclaveSessionHandleSize; - - uint sizeOfTrustedModuleDHPublicKeyBuffer = BitConverter.ToUInt32(attestationInfo, attestationInfoOffset); - attestationInfoOffset += sizeof(UInt32); - uint sizeOfTrustedModuleDHPublicKeySignatureBuffer = BitConverter.ToUInt32(attestationInfo, attestationInfoOffset); - attestationInfoOffset += sizeof(UInt32); - int sizeOfTrustedModuleDHPublicKeyBufferInt = checked((int)sizeOfTrustedModuleDHPublicKeyBuffer); - - byte[] trustedModuleDHPublicKey = new byte[sizeOfTrustedModuleDHPublicKeyBuffer]; - Buffer.BlockCopy(attestationInfo, attestationInfoOffset, trustedModuleDHPublicKey, 0, - sizeOfTrustedModuleDHPublicKeyBufferInt); - attestationInfoOffset += sizeOfTrustedModuleDHPublicKeyBufferInt; - - byte[] trustedModuleDHPublicKeySignature = new byte[sizeOfTrustedModuleDHPublicKeySignatureBuffer]; - Buffer.BlockCopy(attestationInfo, attestationInfoOffset, trustedModuleDHPublicKeySignature, 0, - checked((int)sizeOfTrustedModuleDHPublicKeySignatureBuffer)); - - byte[] sharedSecret; - using ECDiffieHellman ecdh = KeyConverter.CreateECDiffieHellmanFromPublicKeyBlob(trustedModuleDHPublicKey); - sharedSecret = KeyConverter.DeriveKey(clientDHKey, ecdh.PublicKey); - long sessionId = BitConverter.ToInt64(enclaveSessionHandle, 0); - sqlEnclaveSession = AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, sessionId, out counter); - - if (sqlEnclaveSession is null) - { - throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); - } + (sqlEnclaveSession, counter) = CreateEnclaveSessionCore(attestationInfo, clientDHKey, enclaveSessionParameters); } } finally @@ -93,6 +55,82 @@ internal override void CreateEnclaveSession(byte[] attestationInfo, ECDiffieHell } } + // Asynchronous counterpart of CreateEnclaveSession. + // + // None attestation performs no I/O: it only parses the session setup info returned by the + // server and derives the shared secret. The work is therefore identical to the synchronous + // path and completes synchronously. + protected override Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionCoreAsync( + byte[] attestationInfo, + ECDiffieHellman clientDHKey, + EnclaveSessionParameters enclaveSessionParameters, + byte[] customData, + int customDataLength, + CancellationToken cancellationToken) + { + if (cancellationToken.IsCancellationRequested) + { + return Task.FromCanceled<(SqlEnclaveSession, long)>(cancellationToken); + } + + return Task.FromResult(CreateEnclaveSessionCore(attestationInfo, clientDHKey, enclaveSessionParameters)); + } + + // Parses the enclave's session setup info, derives the shared secret and caches the resulting + // session. Shared by the synchronous and asynchronous paths. + private (SqlEnclaveSession SqlEnclaveSession, long Counter) CreateEnclaveSessionCore( + byte[] attestationInfo, + ECDiffieHellman clientDHKey, + EnclaveSessionParameters enclaveSessionParameters) + { + // for None attestation: enclave does not send public key, and sends an empty attestation info + // The only non-trivial content it sends is the session setup info (DH pubkey of enclave) + + // Read AttestationInfo + int attestationInfoOffset = 0; + uint sizeOfTrustedModuleAttestationInfoBuffer = BitConverter.ToUInt32(attestationInfo, attestationInfoOffset); + attestationInfoOffset += sizeof(UInt32); + int sizeOfTrustedModuleAttestationInfoBufferInt = checked((int)sizeOfTrustedModuleAttestationInfoBuffer); + Debug.Assert(sizeOfTrustedModuleAttestationInfoBuffer == 0); + + // read secure session info + uint sizeOfSecureSessionInfoResponse = BitConverter.ToUInt32(attestationInfo, attestationInfoOffset); + attestationInfoOffset += sizeof(UInt32); + + byte[] enclaveSessionHandle = new byte[EnclaveSessionHandleSize]; + Buffer.BlockCopy(attestationInfo, attestationInfoOffset, enclaveSessionHandle, 0, EnclaveSessionHandleSize); + attestationInfoOffset += EnclaveSessionHandleSize; + + uint sizeOfTrustedModuleDHPublicKeyBuffer = BitConverter.ToUInt32(attestationInfo, attestationInfoOffset); + attestationInfoOffset += sizeof(UInt32); + uint sizeOfTrustedModuleDHPublicKeySignatureBuffer = BitConverter.ToUInt32(attestationInfo, attestationInfoOffset); + attestationInfoOffset += sizeof(UInt32); + int sizeOfTrustedModuleDHPublicKeyBufferInt = checked((int)sizeOfTrustedModuleDHPublicKeyBuffer); + + byte[] trustedModuleDHPublicKey = new byte[sizeOfTrustedModuleDHPublicKeyBuffer]; + Buffer.BlockCopy(attestationInfo, attestationInfoOffset, trustedModuleDHPublicKey, 0, + sizeOfTrustedModuleDHPublicKeyBufferInt); + attestationInfoOffset += sizeOfTrustedModuleDHPublicKeyBufferInt; + + byte[] trustedModuleDHPublicKeySignature = new byte[sizeOfTrustedModuleDHPublicKeySignatureBuffer]; + Buffer.BlockCopy(attestationInfo, attestationInfoOffset, trustedModuleDHPublicKeySignature, 0, + checked((int)sizeOfTrustedModuleDHPublicKeySignatureBuffer)); + + byte[] sharedSecret; + using ECDiffieHellman ecdh = KeyConverter.CreateECDiffieHellmanFromPublicKeyBlob(trustedModuleDHPublicKey); + sharedSecret = KeyConverter.DeriveKey(clientDHKey, ecdh.PublicKey); + long sessionId = BitConverter.ToInt64(enclaveSessionHandle, 0); + SqlEnclaveSession sqlEnclaveSession = + AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, sessionId, out long counter); + + if (sqlEnclaveSession is null) + { + throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); + } + + return (sqlEnclaveSession, counter); + } + /// /// When overridden in a derived class, looks up and evicts an enclave session from the enclave session cache, if the provider implements session caching. /// diff --git a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs index d198fec5ac..1d49bb2e58 100644 --- a/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs +++ b/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/VirtualSecureModeEnclaveProviderBase.cs @@ -148,112 +148,48 @@ internal override void InvalidateEnclaveSession(EnclaveSessionParameters enclave InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSessionToInvalidate); } - // Asynchronous counterpart of GetEnclaveSession. Uses the async attestation gate so that - // callers never block a thread pool thread waiting for an in-flight attestation. - internal override Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionAsync( - EnclaveSessionParameters enclaveSessionParameters, - bool generateCustomData, - bool isRetry, - CancellationToken cancellationToken = default) - { - return GetEnclaveSessionHelperAsync(enclaveSessionParameters, false, isRetry, cancellationToken); - } - - // Asynchronous counterpart of GetAttestationParameters. This operation is CPU bound (key - // generation), so it completes synchronously. - internal override Task GetAttestationParametersAsync( - string attestationUrl, - byte[] customData, - int customDataLength, - CancellationToken cancellationToken = default) - { - if (cancellationToken.IsCancellationRequested) - { - return Task.FromCanceled(cancellationToken); - } - - try - { - return Task.FromResult(GetAttestationParameters(attestationUrl, customData, customDataLength)); - } - catch (Exception e) when (ADP.IsCatchableExceptionType(e)) - { - return Task.FromException(e); - } - } + // The VSM attestation protocol does not use a client-generated nonce. + protected override bool GeneratesNonceForAttestation => false; // Asynchronous counterpart of CreateEnclaveSession. Performs the attestation service round // trip (signing certificate download) asynchronously. // - // The async attestation gate is taken for the duration of this method only, so it is released - // on every exit path — success, failure and cancellation alike — without depending on which - // thread a continuation resumes on. - internal override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( + // The async attestation gate is taken and released by the sealed CreateEnclaveSessionAsync in + // EnclaveProviderBase, which also re-checks the session cache before calling this method. + protected override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionCoreAsync( byte[] attestationInfo, ECDiffieHellman clientDHKey, EnclaveSessionParameters enclaveSessionParameters, byte[] customData, int customDataLength, - CancellationToken cancellationToken = default) + CancellationToken cancellationToken) { - using (await AcquireAsyncAttestationGateAsync(cancellationToken).ConfigureAwait(false)) + if (string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl)) { - // Another caller may have completed the attestation while we waited for the gate. - SqlEnclaveSession sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out long counter); - if (sqlEnclaveSession != null) - { - return (sqlEnclaveSession, counter); - } - - if (string.IsNullOrEmpty(enclaveSessionParameters.AttestationUrl)) - { - throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); - } + throw SQL.AttestationFailed(Strings.FailToCreateEnclaveSession); + } - // Deserialize the payload - AttestationInfo info = new AttestationInfo(attestationInfo); + // Deserialize the payload + AttestationInfo info = new AttestationInfo(attestationInfo); - // Verify enclave policy matches expected policy - VerifyEnclavePolicy(info.EnclaveReportPackage); + // Verify enclave policy matches expected policy + VerifyEnclavePolicy(info.EnclaveReportPackage); - // Perform Attestation per VSM protocol - await VerifyAttestationInfoAsync( - enclaveSessionParameters.AttestationUrl, - info.HealthReport, - info.EnclaveReportPackage, - cancellationToken).ConfigureAwait(false); + // Perform Attestation per VSM protocol + await VerifyAttestationInfoAsync( + enclaveSessionParameters.AttestationUrl, + info.HealthReport, + info.EnclaveReportPackage, + cancellationToken).ConfigureAwait(false); - // Set up shared secret and validate signature - byte[] sharedSecret = GetSharedSecret(info.Identity, info.EnclaveDHInfo, clientDHKey); + // Set up shared secret and validate signature + byte[] sharedSecret = GetSharedSecret(info.Identity, info.EnclaveDHInfo, clientDHKey); - // add session to cache - sqlEnclaveSession = AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, info.SessionId, out counter); + // add session to cache + SqlEnclaveSession sqlEnclaveSession = + AddEnclaveSessionToCache(enclaveSessionParameters, sharedSecret, info.SessionId, out long counter); - return (sqlEnclaveSession, counter); - } - } - - // Asynchronous counterpart of InvalidateEnclaveSession. Session eviction is an in-memory - // cache operation, so it completes synchronously. - internal override Task InvalidateEnclaveSessionAsync( - EnclaveSessionParameters enclaveSessionParameters, - SqlEnclaveSession enclaveSessionToInvalidate, - CancellationToken cancellationToken = default) - { - if (cancellationToken.IsCancellationRequested) - { - return Task.FromCanceled(cancellationToken); - } - - try - { - InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSessionToInvalidate); - return Task.CompletedTask; - } - catch (Exception e) when (ADP.IsCatchableExceptionType(e)) - { - return Task.FromException(e); - } + return (sqlEnclaveSession, counter); } #endregion diff --git a/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs b/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs index 412c80dffe..e27542052c 100644 --- a/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs +++ b/src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/AlwaysEncrypted/SqlColumnEncryptionEnclaveProviderAsyncShould.cs @@ -213,14 +213,15 @@ await Assert.ThrowsAnyAsync( [Fact] public async Task CreateEnclaveSessionAsync_WhenAttestationFails_ReleasesTheAsyncGate() { - FakeAttestationEnclaveProvider failing = - new FakeAttestationEnclaveProvider(TimeSpan.Zero, failAttestation: true); + // The same provider instance is reused for both attestations so that this test keeps its + // coverage if the gate ever stops being static. + FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.Zero); + provider.FailNextAttestation = true; await Assert.ThrowsAsync( - () => AttestAsync(failing, NewSessionParameters())); + () => AttestAsync(provider, NewSessionParameters())); // A subsequent, unrelated attestation must not wait on the abandoned gate. - FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.Zero); System.Diagnostics.Stopwatch stopwatch = System.Diagnostics.Stopwatch.StartNew(); SqlEnclaveSession next = await AttestAsync(provider, NewSessionParameters()); stopwatch.Stop(); @@ -231,6 +232,45 @@ await Assert.ThrowsAsync( $"Attestation took {stopwatch.Elapsed}, which suggests the async gate was not released on failure."); } + /// + /// Verifies that an abandoned async session lookup does not hold the synchronous attestation + /// gate. + /// + /// + /// seals GetEnclaveSessionAsync precisely to prevent + /// this: the inherited default would call the synchronous GetEnclaveSession, which takes + /// the sync gate and expects a later synchronous CreateEnclaveSession to release it. An + /// async caller never makes that call, so a provider that failed to override the member would + /// strand the sync gate and stall unrelated synchronous callers for the full lock timeout. + /// + [Fact] + public async Task GetEnclaveSessionAsync_WhenAbandoned_DoesNotHoldTheSyncGate() + { + FakeAttestationEnclaveProvider provider = new FakeAttestationEnclaveProvider(TimeSpan.Zero); + + // Probe for a session that does not exist, then abandon the sequence without creating one. + (SqlEnclaveSession session, _, _, _) = await Task.Run( + () => provider.GetEnclaveSessionAsync(NewSessionParameters(), generateCustomData: true, isRetry: false)); + Assert.Null(session); + + // The synchronous attestation must run on a different thread than the abandoned probe: + // GetEnclaveSessionHelper short-circuits the gate wait for a thread that is already + // mid-attestation, which would mask a stranded gate if both ran on the same thread. A + // dedicated thread is used rather than the thread pool, which may hand back the same thread. + SqlEnclaveSession next = null; + Thread syncCaller = new Thread(() => next = Attest(provider, NewSessionParameters())); + + System.Diagnostics.Stopwatch stopwatch = System.Diagnostics.Stopwatch.StartNew(); + syncCaller.Start(); + syncCaller.Join(); + stopwatch.Stop(); + + Assert.NotNull(next); + Assert.True( + stopwatch.Elapsed < TimeSpan.FromSeconds(5), + $"Sync attestation took {stopwatch.Elapsed}, which suggests the async path took the sync gate."); + } + #endregion #region Concurrency — async path @@ -283,15 +323,30 @@ public async Task CreateEnclaveSessionAsync_ConcurrentColdStart_CompletesWithout Assert.Equal(1, provider.AttestationCount); Assert.All(sessions, session => Assert.Equal(sessions[0].SessionId, session.SessionId)); + // The gate is not held across GetEnclaveSessionAsync, so the work before the attestation is + // deliberately not collapsed: each cold-start caller generates its own attestation + // parameters. Only the attestation service call itself is shared. + Assert.Equal(sessions.Length, provider.AttestationParametersCount); + // The gate must still be usable for a subsequent, unrelated attestation. SqlEnclaveSession next = await AttestAsync(provider, NewSessionParameters()); Assert.NotNull(next); } /// - /// Verifies that sync and async callers can attest concurrently. The two paths use independent - /// gates, so neither may block or starve the other. + /// Verifies that sync and async callers can attest concurrently and that every caller ends up + /// with a usable session. /// + /// + /// The two paths use independent gates, so neither blocks or starves the other, but they do not + /// collapse against each other: a sync and an async cold start that race can both attest. That + /// is deliberate and safe. It matches the tolerance the sync design already has (when its lock + /// timeout expires, n threads perform n attestations), and it is idempotent because + /// EnclaveSessionCache.CreateSession writes under a lock and every session created for the + /// same parameters is equally valid. The invariant that matters, and the one asserted here, is + /// that no caller is starved or returns without a session; the cache converging on a single + /// entry afterwards is asserted separately. + /// [Fact] public async Task Attestation_MixedSyncAndAsyncCallers_AllObtainSessions() { @@ -308,6 +363,16 @@ public async Task Attestation_MixedSyncAndAsyncCallers_AllObtainSessions() SqlEnclaveSession[] sessions = await Task.WhenAll(tasks); Assert.All(sessions, session => Assert.NotNull(session)); + + // Whichever callers raced, the cache converges on exactly one session, and every later + // caller — sync or async — observes that same session without re-attesting. + int attestationsBefore = provider.AttestationCount; + + SqlEnclaveSession cachedAsync = await AttestAsync(provider, parameters); + SqlEnclaveSession cachedSync = Attest(provider, parameters); + + Assert.Equal(cachedAsync.SessionId, cachedSync.SessionId); + Assert.Equal(attestationsBefore, provider.AttestationCount); } #endregion @@ -376,14 +441,16 @@ public async Task HgsMakeRequestAsync_WhenRequestFails_ThrowsAttestationFailure( #region Helpers /// - /// Drives the async GetEnclaveSession -> CreateEnclaveSession attestation sequence. + /// Drives the full async attestation sequence the driver uses: + /// GetEnclaveSessionAsync -> GetAttestationParametersAsync -> CreateEnclaveSessionAsync. /// private static async Task AttestAsync( SqlColumnEncryptionEnclaveProvider provider, - EnclaveSessionParameters parameters) + EnclaveSessionParameters parameters, + CancellationToken cancellationToken = default) { (SqlEnclaveSession session, _, byte[] customData, int customDataLength) = - await provider.GetEnclaveSessionAsync(parameters, generateCustomData: true, isRetry: false) + await provider.GetEnclaveSessionAsync(parameters, generateCustomData: true, isRetry: false, cancellationToken) .ConfigureAwait(false); if (session != null) @@ -391,15 +458,27 @@ await provider.GetEnclaveSessionAsync(parameters, generateCustomData: true, isRe return session; } + // Call 2 generates the client Diffie-Hellman key that call 3 consumes. + SqlEnclaveAttestationParameters attestationParameters = await provider + .GetAttestationParametersAsync(parameters.AttestationUrl, customData, customDataLength, cancellationToken) + .ConfigureAwait(false); + (SqlEnclaveSession created, _) = await provider - .CreateEnclaveSessionAsync(Array.Empty(), null, parameters, customData, customDataLength) + .CreateEnclaveSessionAsync( + Array.Empty(), + attestationParameters.ClientDiffieHellmanKey, + parameters, + customData, + customDataLength, + cancellationToken) .ConfigureAwait(false); return created; } /// - /// Drives the sync GetEnclaveSession -> CreateEnclaveSession attestation sequence. + /// Drives the full sync attestation sequence the driver uses: + /// GetEnclaveSession -> GetAttestationParameters -> CreateEnclaveSession. /// private static SqlEnclaveSession Attest( SqlColumnEncryptionEnclaveProvider provider, @@ -419,9 +498,13 @@ private static SqlEnclaveSession Attest( return session; } + // Call 2 generates the client Diffie-Hellman key that call 3 consumes. + SqlEnclaveAttestationParameters attestationParameters = + provider.GetAttestationParameters(parameters.AttestationUrl, customData, customDataLength); + provider.CreateEnclaveSession( Array.Empty(), - null, + attestationParameters.ClientDiffieHellmanKey, parameters, customData, customDataLength, @@ -549,18 +632,32 @@ private sealed class FakeAttestationEnclaveProvider : EnclaveProviderBase private readonly TimeSpan _attestationDelay; - private readonly bool _failAttestation; - private int _attestationCount; - internal FakeAttestationEnclaveProvider(TimeSpan attestationDelay, bool failAttestation = false) + private int _attestationParametersCount; + + internal FakeAttestationEnclaveProvider(TimeSpan attestationDelay) { _attestationDelay = attestationDelay; - _failAttestation = failAttestation; } + /// + /// When set, the next attestation throws and the flag is cleared, so a single provider + /// instance can be used to exercise both a failed and a subsequent successful attestation. + /// + internal bool FailNextAttestation { get; set; } + + protected override bool GeneratesNonceForAttestation => true; + internal int AttestationCount => Volatile.Read(ref _attestationCount); + /// + /// How many times attestation parameters (including the client Diffie-Hellman key) were + /// generated. Unlike , this step sits outside the async gate, + /// so it runs once per cold-start caller. + /// + internal int AttestationParametersCount => Volatile.Read(ref _attestationParametersCount); + internal override void GetEnclaveSession( EnclaveSessionParameters enclaveSessionParameters, bool generateCustomData, @@ -585,6 +682,7 @@ internal override SqlEnclaveAttestationParameters GetAttestationParameters( byte[] customData, int customDataLength) { + Interlocked.Increment(ref _attestationParametersCount); return new SqlEnclaveAttestationParameters( protocol: 1, input: Array.Empty(), @@ -630,45 +728,30 @@ internal override void InvalidateEnclaveSession( InvalidateEnclaveSessionHelper(enclaveSessionParameters, enclaveSession); } - internal override Task<(SqlEnclaveSession SqlEnclaveSession, long Counter, byte[] CustomData, int CustomDataLength)> GetEnclaveSessionAsync( - EnclaveSessionParameters enclaveSessionParameters, - bool generateCustomData, - bool isRetry, - CancellationToken cancellationToken = default) - { - return GetEnclaveSessionHelperAsync(enclaveSessionParameters, generateCustomData, isRetry, cancellationToken); - } - - internal override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionAsync( + protected override async Task<(SqlEnclaveSession SqlEnclaveSession, long Counter)> CreateEnclaveSessionCoreAsync( byte[] enclaveAttestationInfo, ECDiffieHellman clientDiffieHellmanKey, EnclaveSessionParameters enclaveSessionParameters, byte[] customData, int customDataLength, - CancellationToken cancellationToken = default) + CancellationToken cancellationToken) { - using (await AcquireAsyncAttestationGateAsync(cancellationToken).ConfigureAwait(false)) - { - SqlEnclaveSession sqlEnclaveSession = GetEnclaveSessionFromCache(enclaveSessionParameters, out long counter); - if (sqlEnclaveSession == null) - { - Interlocked.Increment(ref _attestationCount); - await Task.Delay(_attestationDelay, cancellationToken).ConfigureAwait(false); + Interlocked.Increment(ref _attestationCount); + await Task.Delay(_attestationDelay, cancellationToken).ConfigureAwait(false); - if (_failAttestation) - { - throw new InvalidOperationException("Simulated attestation failure."); - } + if (FailNextAttestation) + { + FailNextAttestation = false; + throw new InvalidOperationException("Simulated attestation failure."); + } - sqlEnclaveSession = AddEnclaveSessionToCache( - enclaveSessionParameters, - SharedSecret, - Interlocked.Increment(ref s_nextSessionId), - out counter); - } + SqlEnclaveSession sqlEnclaveSession = AddEnclaveSessionToCache( + enclaveSessionParameters, + SharedSecret, + Interlocked.Increment(ref s_nextSessionId), + out long counter); - return (sqlEnclaveSession, counter); - } + return (sqlEnclaveSession, counter); } }