Add event environments and filtering

docs/docs/clients/dotnet/configuration.md

@@ -5,6 +5,8 @@ order: 1
 
 # Configuration
 
+Set a deployment environment once at startup, with optional per-event overrides. See [Environments](/docs/environments/) for configuration examples.
+
 There are a few ways to configure Exceptionless in your project. We'll cover them here or you can jump to app-specific examples: [Console App Example](/docs/clients/dotnet/guides/console-apps-example), [Web Server Example](/docs/clients/dotnet/guides/web-server-example).
 
 ---

docs/docs/clients/javascript/client-configuration.md

@@ -5,6 +5,8 @@ order: 1
 
 # Configuration
 
+Set a deployment environment once at startup, with optional per-event overrides. See [Environments](/docs/environments/) for configuration examples.
+
 - [Installation](#installation)
   - [Browser](#browser)
   - [Node.js](#nodejs)

docs/docs/environments.md

@@ -0,0 +1,69 @@
+---
+title: "Environments"
+---
+
+# Environments
+
+Use an environment to distinguish production, staging, development, or a custom deployment within one project. Each event can carry one optional top-level `environment` string. The same error still belongs to one stack, with one shared status and fixed version.
+
+## Configure your client
+
+Set a default during application startup. In the .NET client:
+
+```csharp
+client.Configuration.SetEnvironment("production");
+client.CreateLog("Deployment complete").SetEnvironment("staging").Submit();
+```
+
+You can also set `Exceptionless:Environment` in .NET configuration, or the `Exceptionless__Environment` environment variable. The hosting integration uses `IHostEnvironment.EnvironmentName` when no explicit environment is configured. A per-event value overrides the default.
+
+In the JavaScript client:
+
+```javascript
+await Exceptionless.startup((config) => {
+  config.apiKey = "API_KEY_HERE";
+  config.environment = "production";
+});
+
+await Exceptionless.createLog("Deployment complete")
+  .setEnvironment("staging")
+  .submit();
+```
+
+For direct API submissions:
+
+```json
+{
+  "type": "log",
+  "message": "Deployment complete",
+  "environment": "production"
+}
+```
+
+Names are trimmed, with a maximum of 64 characters. Events preserve the supplied casing: `" Production "` is stored and returned as `"Production"`. Filtering is case-insensitive, and aggregation keys are normalized to lowercase, so `Production` and `production` share one environment bucket. Custom names such as `qa-west` are supported. Empty, oversized, control-character, and non-string values are treated as unspecified. Historical events and older clients without this field remain unspecified; they are never assumed to be production.
+
+## Filter your data
+
+Choose **Manage filters → Environment** on Events, Stacks, Sessions, or Stream. Select one or more names, or **Unspecified** for events without an environment. Clear the selection to include all environments. The picker discovers names for the selected projects and time range; you can also enter a name that has no current events.
+
+Environment selections persist in URLs and saved views. Events, sessions, and stream tables have an optional Environment column. Event details show Environment in the overview; **Machine & runtime** contains the existing `data.@environment` diagnostics.
+
+Environment searches and aggregations are available on every plan:
+
+| Search | Meaning |
+| --- | --- |
+| `environment:production` | Production events |
+| `(environment:production OR environment:staging)` | Either deployment |
+| `_missing_:environment` | Historical or unspecified events |
+| `_exists_:environment` | Events with an environment |
+| `environment:"qa west"` | A custom name containing spaces |
+
+Stack dashboard counts and charts use events matching the filter. Stack status remains shared, so changing status while viewing production also changes the same stack seen in staging. Automatic session detection separates the same user's activity by environment.
+
+## Fixing stacks across deployments
+
+Use [fixed in version](/docs/versioning/) when environments run different releases. For example, if a stack is fixed in `2.4.0`, occurrences from production running `2.3.0` do not represent a regression just because staging already has the fix. Continue sending the application version with events. Plain **Fixed**, without a version, retains its existing behavior across all environments.
+
+## Rollout
+
+Deploy the server before adopting SDK versions that expose the new setting. The server adds mappings to existing event indices without rewriting historical events. Clients that do not send an environment continue to work. `data.@environment` and custom `data.environment` values retain their existing meanings.

docs/docs/filtering-and-searching.md

@@ -6,6 +6,7 @@ title: "Filtering & Searching"
 
 - [Filter by Organization \& Project](#filter-by-organization--project)
 - [Filter by Time Frame](#filter-by-time-frame)
+- [Filter by Environment](#filter-by-environment)
 - [Filter / Search by Specific Criteria](#filter--search-by-specific-criteria)
 - [Searchable Fields \& Requirements](#searchable-fields--requirements)
 - [Multiple Queries](#multiple-queries)
@@ -30,6 +31,10 @@ Click on the calendar icon in the header to select from multiple preset time fra
 
 ![Exceptionless Filter Time Frame](img/filter-by-timeframe.png)
 
+## Filter by Environment
+
+Choose **Manage filters → Environment** to select production, staging, development, or a custom name. Select **Unspecified** for events without this property, or clear the selection to include all environments. See [Environments](/docs/environments/) for client configuration and shared stack behavior.
+
 ## Filter / Search by Specific Criteria
 
 Click the magnifying glass to search by specific criteria.
@@ -53,6 +58,7 @@ View a complete list of searchable terms, examples, and FAQs below.
 | stack              | `stack:54d8315ce6bb2d0500bcc7b4`                                                                                     | true                         | Stack id                                      |
 | reference          | `reference:12345678`                                                                                                 | true                         | Reference id                                  |
 | session            | `session:12345678`                                                                                                   | true                         | Session id                                    |
+| environment        | `environment:production` or `_missing_:environment` | true | Deployment environment |
 | type               | `type:error`                                                                                                         | true                         | Event type                                    |
 | source             | `source:"my log source"` or `"my log source"`                                                                        | false                        | Event source                                  |
 | level              | `level:Error`                                                                                                        | true                         | Log level                                     |

docs/docs/versioning.md

@@ -4,6 +4,8 @@ title: "Versioning"
 
 # Versioning
 
+Stacks and their fixed version are shared across [environments](/docs/environments/). Continue sending application versions so older deployments can report occurrences without incorrectly reopening a stack fixed in a newer version.
+
 You can mark error stacks fixed and they won't show up or notify you until they regress!
 
 ## How does this work?

src/Exceptionless.Core/Extensions/PersistentEventExtensions.cs

@@ -183,6 +183,7 @@ public static PersistentEvent ToSessionStartEvent(this PersistentEvent source, I
     {
         var startEvent = new PersistentEvent
         {
+            Environment = source.Environment,
             Date = source.Date,
             Geo = source.Geo,
             OrganizationId = source.OrganizationId,

src/Exceptionless.Core/Migrations/010_AddEventEnvironment.cs

@@ -0,0 +1,36 @@
+using Exceptionless.Core.Extensions;
+using Exceptionless.Core.Models;
+using Exceptionless.Core.Repositories.Configuration;
+using Foundatio.Repositories.Elasticsearch.Extensions;
+using Foundatio.Repositories.Migrations;
+using Microsoft.Extensions.Logging;
+
+namespace Exceptionless.Core.Migrations;
+
+public sealed class AddEventEnvironment : MigrationBase
+{
+    private readonly ExceptionlessElasticConfiguration _configuration;
+
+    public AddEventEnvironment(ExceptionlessElasticConfiguration configuration, ILoggerFactory loggerFactory) : base(loggerFactory)
+    {
+        _configuration = configuration;
+        MigrationType = MigrationType.VersionedAndResumable;
+        Version = 10;
+    }
+
+    public override async Task RunAsync(MigrationContext context)
+    {
+        var response = await _configuration.Client.Indices.PutMappingAsync<PersistentEvent>(mapping => mapping
+            .Indices($"{_configuration.Events.Name}-v*-*")
+            .AllowNoIndices(true)
+            .IgnoreUnavailable(true)
+            .Properties(properties => properties.Text(ev => ev.Environment,
+                text => text.Analyzer(EventIndex.LOWER_KEYWORD_ANALYZER)
+                    .Fields(fields => fields.Keyword("keyword", keyword => keyword.Normalizer("lowercase"))))), context.CancellationToken);
+        _logger.LogRequest(response);
+        if (!response.IsValidResponse)
+        {
+            throw new InvalidOperationException("Unable to add the event environment mapping: " + response.DebugInformation);
+        }
+    }
+}

src/Exceptionless.Core/Models/Event.cs

@@ -11,6 +11,26 @@ namespace Exceptionless.Core.Models;
 [DebuggerDisplay("Type: {Type}, Date: {Date}, Message: {Message}, Value: {Value}, Count: {Count}")]
 public class Event : IData, IJsonOnDeserialized
 {
+    private string? _environment;
+
+    /// <summary>
+    /// The deployment environment, such as production, staging, or development.
+    /// Missing or invalid names remain unspecified. Machine and runtime information is stored separately in data.@environment.
+    /// </summary>
+    [StringLength(64)]
+    [JsonConverter(typeof(EventEnvironmentConverter))]
+    public string? Environment
+    {
+        get => _environment;
+        set
+        {
+            string? name = value?.Trim();
+            _environment = String.IsNullOrEmpty(name) || name.Length > 64 || name.Any(Char.IsControl)
+                ? null
+                : name;
+        }
+    }
+
     /// <summary>
     /// The event type (ie. error, log message, feature usage). Check <see cref="KnownTypes">Event.KnownTypes</see> for standard event types.
     /// Nullable in transit; the pipeline infers a default before save. Validated as required on repository save.
@@ -112,7 +132,7 @@ void IJsonOnDeserialized.OnDeserialized()
 
     protected bool Equals(Event other)
     {
-        return String.Equals(Type, other.Type) && String.Equals(Source, other.Source) && Tags.CollectionEquals(other.Tags) && String.Equals(Message, other.Message) && String.Equals(Geo, other.Geo) && Value == other.Value && Equals(Data, other.Data);
+        return String.Equals(Environment, other.Environment) && String.Equals(Type, other.Type) && String.Equals(Source, other.Source) && Tags.CollectionEquals(other.Tags) && String.Equals(Message, other.Message) && String.Equals(Geo, other.Geo) && Value == other.Value && Equals(Data, other.Data);
     }
 
     public override bool Equals(object? obj)
@@ -138,6 +158,10 @@ public override int GetHashCode()
             hashCode = (hashCode * 397) ^ (Geo?.GetHashCode() ?? 0);
             hashCode = (hashCode * 397) ^ Value.GetHashCode();
             hashCode = (hashCode * 397) ^ (Data?.GetCollectionHashCode(_exclusions) ?? 0);
+            if (Environment is not null)
+            {
+                hashCode = (hashCode * 397) ^ Environment.GetHashCode();
+            }
             return hashCode;
         }
     }

src/Exceptionless.Core/Models/EventSummaryModel.cs

@@ -2,6 +2,7 @@
 
 public record EventSummaryModel : SummaryData
 {
+    public string? Environment { get; set; }
     public DateTimeOffset Date { get; set; }
     public string ProjectId { get; set; } = null!;
     public string? ProjectName { get; set; }

src/Exceptionless.Core/Models/WebHookEvent.cs

@@ -14,6 +14,7 @@ public WebHookEvent(string baseUrl)
     public DateTimeOffset? OccurrenceDate { get; init; }
     public TagSet? Tags { get; init; }
     public string? Type { get; init; }
+    public string? Environment { get; init; }
     public string? Source { get; init; }
     public string? Message { get; init; }
     public string ProjectId { get; init; } = null!;

src/Exceptionless.Core/Plugins/EventProcessor/Default/70_SessionPlugin.cs

@@ -125,11 +125,11 @@ private async Task ProcessAutoSessionsAsync(ICollection<EventContext> contexts)
     {
         var identityGroups = contexts
             .OrderBy(c => c.Event.Date)
-            .GroupBy(c => c.Event.GetUserIdentity(_serializer, _logger)?.Identity);
+            .GroupBy(c => (c.Event.ProjectId, Environment: c.Event.Environment?.ToLowerInvariant(), Identity: c.Event.GetUserIdentity(_serializer, _logger)?.Identity));
 
         foreach (var identityGroup in identityGroups)
         {
-            if (String.IsNullOrEmpty(identityGroup.Key))
+            if (String.IsNullOrEmpty(identityGroup.Key.Identity))
                 continue;
 
             string projectId = identityGroup.First().Project.Id;
@@ -160,7 +160,7 @@ private async Task ProcessAutoSessionsAsync(ICollection<EventContext> contexts)
                     ctx.IsCancelled = true;
                 });
 
-                string? sessionId = await GetIdentitySessionIdAsync(projectId, identityGroup.Key);
+                string? sessionId = await GetIdentitySessionIdAsync(projectId, identityGroup.Key.Identity, identityGroup.Key.Environment);
 
                 // if session end, without any session events, cancel
                 if (String.IsNullOrEmpty(sessionId) && session.Count == 1 && firstSessionEvent.Event.IsSessionEnd())
@@ -192,7 +192,7 @@ private async Task ProcessAutoSessionsAsync(ICollection<EventContext> contexts)
                     }
 
                     if (!lastSessionEvent.Event.IsSessionEnd())
-                        await SetIdentitySessionIdAsync(projectId, identityGroup.Key, sessionId);
+                        await SetIdentitySessionIdAsync(projectId, identityGroup.Key.Identity, sessionId, identityGroup.Key.Environment);
                 }
                 else
                 {
@@ -217,7 +217,7 @@ public override Task EventProcessedAsync(EventContext context)
         return Task.CompletedTask;
     }
 
-    private static List<List<EventContext>> CreateSessionGroups(IGrouping<string?, EventContext> identityGroup)
+    private static List<List<EventContext>> CreateSessionGroups(IEnumerable<EventContext> identityGroup)
     {
         var sessions = new List<List<EventContext>>();
         var currentSession = new List<EventContext>();
@@ -259,14 +259,19 @@ private Task<bool> SetSessionStartEventIdAsync(string projectId, string sessionI
         return _cache.SetAsync<string>(GetSessionStartEventIdCacheKey(projectId, sessionId), eventId, TimeSpan.FromDays(1));
     }
 
-    private static string GetIdentitySessionIdCacheKey(string projectId, string identity)
+    private static string GetIdentitySessionIdCacheKey(string projectId, string identity, string? environment)
     {
+        if (environment is not null)
+        {
+            return String.Concat(projectId, ":environment:", environment.ToSHA1(), ":identity:", identity.ToSHA1());
+        }
+
         return String.Concat(projectId, ":identity:", identity.ToSHA1());
     }
 
-    private async Task<string?> GetIdentitySessionIdAsync(string projectId, string identity)
+    private async Task<string?> GetIdentitySessionIdAsync(string projectId, string identity, string? environment)
     {
-        string cacheKey = GetIdentitySessionIdCacheKey(projectId, identity);
+        string cacheKey = GetIdentitySessionIdCacheKey(projectId, identity, environment);
         string? sessionId = await _cache.GetAsync<string?>(cacheKey, null);
         if (!String.IsNullOrEmpty(sessionId))
         {
@@ -279,9 +284,9 @@ await Task.WhenAll(
         return sessionId;
     }
 
-    private Task<bool> SetIdentitySessionIdAsync(string projectId, string identity, string sessionId)
+    private Task<bool> SetIdentitySessionIdAsync(string projectId, string identity, string sessionId, string? environment)
     {
-        return _cache.SetAsync<string>(GetIdentitySessionIdCacheKey(projectId, identity), sessionId, _sessionTimeout);
+        return _cache.SetAsync<string>(GetIdentitySessionIdCacheKey(projectId, identity, environment), sessionId, _sessionTimeout);
     }
 
     private async Task<PersistentEvent> CreateSessionStartEventAsync(EventContext startContext, DateTime? lastActivityUtc, bool? isSessionEnd)

src/Exceptionless.Core/Plugins/WebHook/Default/020_VersionTwoPlugin.cs

@@ -22,6 +22,7 @@ public VersionTwoPlugin(AppOptions options, ILoggerFactory loggerFactory) : base
             Tags = ev.Tags,
             Message = ev.Message,
             Type = ev.Type,
+            Environment = ev.Environment,
             Source = ev.Source,
             ProjectId = ev.ProjectId,
             ProjectName = ctx.Project.Name,

src/Exceptionless.Core/Repositories/Configuration/Indexes/EventIndex.cs

@@ -70,6 +70,8 @@ public override void ConfigureIndexMapping(TypeMappingDescriptor<PersistentEvent
                 .Keyword(e => e.ReferenceId)
                     .FieldAlias(Alias.ReferenceId, a => a.Path(f => f.ReferenceId))
                 .Text(e => e.Type, t => t.Analyzer(LOWER_KEYWORD_ANALYZER).AddKeywordField())
+                .Text(e => e.Environment, t => t.Analyzer(LOWER_KEYWORD_ANALYZER)
+                    .Fields(fields => fields.Keyword("keyword", keyword => keyword.Normalizer("lowercase"))))
                 .Text(e => e.Source, t => t.Analyzer(STANDARDPLUS_ANALYZER).SearchAnalyzer(WHITESPACE_LOWERCASE_ANALYZER).AddKeywordField())
                 .Date(e => e.Date)
                 .Text(e => e.Message)
@@ -260,6 +262,7 @@ private void BuildAnalysis(IndexSettingsAnalysisDescriptor ad)
 
     public sealed class Alias
     {
+        public const string Environment = "environment";
         public const string OrganizationId = "organization";
         public const string ProjectId = "project";
         public const string StackId = "stack";

src/Exceptionless.Core/Repositories/Queries/EventEnvironmentFilter.cs

@@ -0,0 +1,49 @@
+using Foundatio.Parsers.LuceneQueries;
+using Foundatio.Parsers.LuceneQueries.Nodes;
+
+namespace Exceptionless.Core.Repositories.Queries;
+
+/// <summary>
+/// Preserves deployment scope when counting all project users, including users without matching errors.
+/// </summary>
+public static class EventEnvironmentFilter
+{
+    public static async Task<string?> GetAsync(string? filter)
+    {
+        if (String.IsNullOrWhiteSpace(filter))
+            return null;
+
+        return GetConstraint(await new LuceneQueryParser().ParseAsync(filter));
+    }
+
+    private static string? GetConstraint(IQueryNode? node, bool negate = false)
+    {
+        if (node is not IFieldQueryNode field)
+            return null;
+
+        if (String.Equals(field.UnescapedField, "environment", StringComparison.OrdinalIgnoreCase))
+            return negate ? $"NOT ({node})" : node.ToString();
+
+        if (field.Field is not null || node is not GroupNode group)
+            return null;
+
+        negate ^= group.IsNegated == true || group.Prefix == "-";
+        if (group.Left is null)
+            return GetConstraint(group.Right, negate);
+        if (group.Right is null)
+            return GetConstraint(group.Left, negate);
+
+        string? left = GetConstraint(group.Left, negate);
+        string? right = GetConstraint(group.Right, negate);
+        bool isOr = (group.Operator == GroupOperator.Or) ^ negate;
+        // An unrelated OR branch permits every environment. Unrelated AND clauses do not restrict it.
+        if (isOr && (left is null || right is null))
+            return null;
+        if (left is null)
+            return right;
+        if (right is null)
+            return left;
+
+        return $"({left} {(isOr ? "OR" : "AND")} {right})";
+    }
+}