From 61eea5cea5704adb8b94e7cdd67a225b3acdaf92 Mon Sep 17 00:00:00 2001 From: Ricardo Zanini Date: Wed, 6 May 2026 10:50:54 -0400 Subject: [PATCH] Fix #1149 - ADR Proposal for Serverless Workflow spec migration tool from 0.9/0.8 -> 1.0.0 definitions Signed-off-by: Ricardo Zanini --- adr/v1.0-adr-migration-tool.md | 415 +++++++++++++++++++++++++++++++++ 1 file changed, 415 insertions(+) create mode 100644 adr/v1.0-adr-migration-tool.md diff --git a/adr/v1.0-adr-migration-tool.md b/adr/v1.0-adr-migration-tool.md new file mode 100644 index 00000000..ca3878a9 --- /dev/null +++ b/adr/v1.0-adr-migration-tool.md @@ -0,0 +1,415 @@ +# ADR: Serverless Workflow Migration Tool for 0.8/0.9 to 1.0.0 + +## Status + +Proposed + +## Related Issue + +[#1149](https://github.com/serverlessworkflow/specification/issues/1149) - Proposal: Official Migration Tool for 0.8/0.9 to 1.0.0 Workflows + +## Context + +The Serverless Workflow specification has undergone a significant architectural redesign between versions 0.8/0.9 and 1.0.0. The 0.8/0.9 versions used a **state-based model** with explicit state machines and transitions, while version 1.0.0 introduces a fundamentally different **task-based model** with implicit flow control and declarative task composition. + +This architectural change creates a migration challenge for users who have existing workflows defined in 0.8 or 0.9 format and want to adopt the new 1.0.0 specification. Manual migration is: + +- **Time-consuming**: Requires understanding both specification versions deeply +- **Error-prone**: Subtle semantic differences can lead to incorrect migrations +- **Repetitive**: Many transformation patterns are mechanical and automatable +- **Blocking adoption**: The migration burden may prevent users from upgrading to 1.0.0 + +Currently, there is no official migration path or tooling to help users transition their workflows from 0.8/0.9 to 1.0.0. This ADR proposes the development of an official migration tool to address this gap. + +### Key Differences Between Versions + +| Aspect | 0.8/0.9 | 1.0.0 | +|--------|---------|-------| +| **Architecture** | State-based workflow with explicit transitions | Task-based workflow with implicit flow | +| **Flow Control** | `transition` field pointing to next state | `do` array order + `then` directive | +| **Components** | 8 state types (Event, Operation, Switch, Inject, ForEach, Parallel, Sleep, Callback) | 12+ task types with different semantics | +| **Data Flow** | Multiple filter types (stateDataFilter, actionDataFilter, eventDataFilter) | Unified `input`/`output` on tasks | +| **Expressions** | JsonPath and jq supported | jq as default expression language | +| **Document Structure** | Top-level workflow properties | `document` metadata section with namespace | + +## Decision + +We propose the development of an official **Serverless Workflow Migration Tool** written in Go to automate the migration of 0.8 and 0.9 workflow definitions to version 1.0.0. + +### Tool Characteristics + +- **Language**: Go (aligns with CNCF ecosystem, strong typing, excellent CLI support) +- **Approach**: Rule-based mapping engine with explicit transformation rules +- **Philosophy**: Best-effort automatic migration with comprehensive reporting +- **Output**: Migrated workflow + detailed migration report +- **Validation**: Validates output against 1.0.0 schema + +### Migration Strategy + +The tool will use a **best-effort approach**: + +- Automatically migrate as much as possible (target: 80%+ of common patterns) +- Generate warnings for transformations requiring manual review +- Provide detailed migration reports identifying manual steps needed +- Write output even when warnings are present +- Validate migrated workflows against 1.0.0 JSON schema + +### State-to-Task Mapping + +| 0.8/0.9 State | 1.0.0 Task | Complexity | Notes | +|---------------|------------|------------|-------| +| Event State | `listen` task | Medium | Event consumption maps well | +| Operation State | `call` task | Low-Medium | Actions map to sequential call tasks | +| Switch State | `switch` task | Medium | Data and event conditions both supported | +| Inject State | `set` task | Low | Direct data injection | +| ForEach State | `for` task | Medium | Iteration semantics align well | +| Parallel State | `fork` task | Medium | Branch execution model | +| Sleep State | `wait` task | Low | Duration/timeout mapping | +| Callback State | `listen` + conditional flow | High | Requires decomposition | + +## Architecture + +### Component Structure + +``` +swf-migrate/ +├── cmd/swf-migrate/ # CLI entry point +├── pkg/ +│ ├── parser/ # Input parsing & validation +│ ├── models/ # Go structs for 0.8, 0.9, and 1.0 schemas +│ ├── transformer/ # State-to-task transformation logic +│ │ ├── event.go +│ │ ├── operation.go +│ │ ├── switch.go +│ │ ├── inject.go +│ │ ├── foreach.go +│ │ ├── parallel.go +│ │ ├── sleep.go +│ │ └── callback.go +│ ├── expression/ # JsonPath to jq conversion +│ ├── validator/ # Output validation against 1.0.0 schema +│ ├── reporter/ # Migration report generation +│ └── writer/ # Output file writing (YAML/JSON) +├── schemas/ # Embedded JSON schemas (0.8, 0.9, 1.0) +├── testdata/ # Test workflows for validation +└── docs/ # User documentation and migration guide +``` + +### Core Components + +1. **Parser** - Reads and validates 0.8/0.9 workflow files (JSON/YAML) +2. **Migration Engine** - Orchestrates transformation and tracks migration context +3. **Transformers** - One per state type, implementing explicit mapping rules +4. **Expression Converter** - Converts JsonPath expressions to jq (best-effort) +5. **Validator** - Validates migrated output against 1.0.0 schema +6. **Report Generator** - Creates detailed JSON/Markdown migration reports +7. **Writer** - Outputs migrated workflow in YAML or JSON format + +### Key Transformations + +#### 1. Workflow Metadata + +``` +0.8/0.9: 1.0.0: + name: "my-workflow" → document: + version: "1.0.0" dsl: "1.0.0" + specVersion: "0.8" namespace: "default" + name: "my-workflow" + version: "1.0.0" +``` + +#### 2. State Transitions to Task Flow + +- 0.8/0.9: Explicit `transition` fields create state graph +- 1.0.0: Tasks in `do` array execute sequentially +- Use `then` directive for explicit flow control (branching, conditionals) +- Rebuild execution order from transition graph + +#### 3. JsonPath to jq Expression Conversion + +Critical component since expressions appear throughout workflows: + +| JsonPath Pattern | jq Equivalent | Complexity | +|------------------|---------------|------------| +| `$.field` | `.field` | Low | +| `$.array[0]` | `.array[0]` | Low | +| `$..recursive` | `.. \| .field? \| select(.)` | Medium | +| `$.array[?(@.price < 10)]` | `.array[] \| select(.price < 10)` | Medium-High | +| Complex multi-predicate filters | Custom jq logic | High (requires manual review) | + +#### 4. Data Filters + +- 0.8/0.9: `stateDataFilter`, `actionDataFilter`, `eventDataFilter` +- 1.0.0: Unified `input` and `output` on tasks +- Map filter contexts to appropriate task input/output expressions + +## Migration Report Structure + +The tool generates comprehensive reports to guide manual intervention: + +```json +{ + "migration_summary": { + "source_file": "workflow.json", + "source_version": "0.8", + "target_version": "1.0.0", + "migration_date": "2026-05-05T16:15:00Z", + "overall_status": "partial", + "statistics": { + "total_states": 8, + "migrated_states": 7, + "warnings_count": 3, + "errors_count": 0 + } + }, + "issues": [ + { + "severity": "WARNING", + "category": "expression_conversion", + "source_location": "states[2].stateDataFilter.output", + "message": "Complex JsonPath expression converted to jq", + "original": "$.store.books[?(@.price < 10)].title", + "converted": ".store.books[] | select(.price < 10) | .title", + "action_required": "Verify jq expression produces expected output" + } + ], + "manual_migration_tasks": [ + { + "task_id": 1, + "priority": "high", + "description": "Review callback state transformation", + "details": "Callback state split into listen + conditional flow", + "source_reference": "states[3]" + } + ] +} +``` + +### Issue Categories + +- `expression_conversion` - JsonPath → jq transformations +- `state_transformation` - Complex state mappings +- `data_flow` - Filter and data mapping changes +- `error_handling` - Error/retry policy adjustments +- `authentication` - Auth configuration migrations +- `unsupported_feature` - Features without direct 1.0.0 equivalent +- `extension` - Custom extensions requiring manual port + +### Severity Levels + +- **INFO**: Successfully migrated with informational notes +- **WARNING**: Migrated but requires manual review/verification +- **ERROR**: Could not migrate, manual intervention required + +## CLI Interface + +### Command Structure + +```bash +swf-migrate [flags] +``` + +### Flags + +| Flag | Short | Default | Description | +|------|-------|---------|-------------| +| `--output` | `-o` | `-migrated.yaml` | Output file path | +| `--report` | `-r` | `-report.json` | Migration report path | +| `--format` | `-f` | `yaml` | Output format: `yaml` or `json` | +| `--namespace` | `-n` | `default` | Target namespace for 1.0.0 | +| `--report-format` | | `json` | Report format: `json` or `markdown` | +| `--strict` | | `false` | Fail if any warnings generated | +| `--verbose` | `-v` | `false` | Enable verbose logging | + +### Usage Examples + +```bash +# Basic migration +swf-migrate workflow-v0.8.json + +# Custom output and namespace +swf-migrate workflow.json -o migrated/workflow.yaml -n production + +# JSON output with markdown report +swf-migrate workflow.yaml -f json --report-format markdown + +# Strict mode - fail on warnings +swf-migrate workflow.json --strict +``` + +### Exit Codes + +- `0` - Success (fully migrated or INFO only) +- `1` - Partial migration (warnings present, output written) +- `2` - Migration failed (errors present, output incomplete) +- `3` - Invalid input (parse error) + +## Implementation Effort Estimation + +| Phase | Effort | Priority | +|-------|--------|----------| +| **Phase 1: Foundation** | 2-3 weeks | Critical | +| - Project scaffolding, CLI setup | 0.5 weeks | | +| - Schema models (structs) | 1 week | | +| - Parser & validation | 0.5-1 week | | +| **Phase 2: Core Transformers** | 3-4 weeks | Critical | +| - Simple transformers (Inject, Sleep, Operation) | 1 week | | +| - Medium complexity (Event, Switch, ForEach, Parallel) | 1.5-2 weeks | | +| - Complex (Callback, nested workflows) | 1-1.5 weeks | | +| **Phase 3: Expression Conversion** | 2-3 weeks | Critical | +| - JsonPath parser | 1 week | | +| - JsonPath → jq converter | 1-1.5 weeks | | +| - Testing edge cases | 0.5-1 week | | +| **Phase 4: Validation & Reporting** | 1-2 weeks | High | +| - Schema validation integration | 0.5 week | | +| - Report generation | 0.5-1 week | | +| - Writer module | 0.5 week | | +| **Phase 5: Testing & Documentation** | 2-3 weeks | High | +| - Unit tests | 1 week | | +| - Integration tests | 1 week | | +| - Documentation | 0.5-1 week | | +| **Total Estimated Effort** | **10-15 weeks** | | + +## Risks and Mitigation Strategies + +### High-Risk Challenges + +#### 1. JsonPath → jq Conversion (HIGH RISK) + +**Challenge**: JsonPath and jq have different semantics for complex operations. + +**Mitigation**: +- Build comprehensive test suite of JsonPath patterns +- Flag complex conversions for manual review +- Provide detailed conversion guide documentation +- Use existing Go JsonPath libraries for parsing + +#### 2. Workflow State Graph Reconstruction (MEDIUM RISK) + +**Challenge**: Converting state graphs to task flow with proper `then` usage. + +**Mitigation**: +- Implement topological sort for linear flows +- Use `then` directive for non-sequential flows +- Detect cycles and handle with explicit control flow +- Warn on complex graphs requiring manual review + +#### 3. Callback State Decomposition (MEDIUM RISK) + +**Challenge**: No direct 1:1 mapping for callback states. + +**Mitigation**: +- Document pattern: listen + conditional + timeout +- Generate detailed warnings with suggested patterns +- Provide examples in migration guide + +#### 4. Data Filter Semantics (MEDIUM RISK) + +**Challenge**: Different scoping rules between filter types. + +**Mitigation**: +- Careful mapping of filter contexts to task input/output +- Generate warnings when semantics may differ +- Provide before/after data flow examples + +### Expected Limitations + +- **Custom Extensions**: Cannot auto-migrate version-specific extensions (will be flagged) +- **Complex Event Correlation**: May require manual adjustment +- **Workflow Compensation**: Different semantics may need review + +## Success Criteria + +- ✅ Migrate 80%+ of common workflow patterns automatically +- ✅ Generate actionable migration reports for remaining 20% +- ✅ Pass validation against 1.0.0 schema for migrated workflows +- ✅ Handle all 8 state types from 0.8/0.9 +- ✅ Convert common JsonPath expressions (90%+ coverage) +- ✅ Comprehensive test coverage (unit + integration) +- ✅ Clear documentation and migration guide + +## Consequences + +### Positive + +- **Lowers adoption barrier**: Users can migrate existing workflows to 1.0.0 with confidence +- **Reduces manual effort**: Automates 80%+ of mechanical transformation work +- **Provides transparency**: Detailed reports show what was migrated and what needs review +- **Builds trust**: Official tool demonstrates commitment to backward compatibility +- **Educational value**: Migration reports help users understand differences between versions +- **CNCF ecosystem alignment**: Go-based tool fits CNCF tooling ecosystem +- **Community contribution**: Opens path for community contributions and improvements + +### Trade-Offs + +- **Not 100% automatic**: Complex workflows will require manual review +- **Maintenance burden**: Tool must be maintained as specs evolve +- **Development investment**: 10-15 weeks of focused development effort +- **Expression conversion complexity**: JsonPath → jq conversion has limitations +- **Testing requirements**: Need comprehensive test coverage of edge cases + +## Licensing + +- Apache 2.0 (consistent with Serverless Workflow specification) +- All dependencies must be CNCF-compatible + +## Governance + +### Repository Ownership + +- New repository under `serverlessworkflow` GitHub organization +- Suggested name: `swf-migrate` or `workflow-migrate` + +### Maintainers + +- Initial maintainers from Serverless Workflow specification maintainers +- Open to community contributions following spec governance model + +### Release Strategy + +- Follow semantic versioning +- Align major versions with specification versions +- Support migration from 0.8/0.9 to current stable 1.x release +- Update as 1.x specification evolves + +## Milestones + +### Milestone 1 (Target: Q3 2026) - MVP CLI Tool + +- Basic CLI with core transformers for 6/8 state types +- Simple state transformers: Inject, Sleep, Operation, Switch +- Medium complexity: Event, ForEach +- Basic JsonPath → jq conversion (common patterns) +- JSON output and JSON migration reports +- Unit test coverage for core transformers +- Basic documentation + +### Milestone 2 (Target: Q4 2026) - Full Feature Set + +- All 8 state types supported (add Parallel, Callback) +- YAML output support +- Markdown migration reports +- Comprehensive JsonPath → jq conversion +- Schema validation integration +- Integration test suite with real-world workflows +- Complete CLI with all flags +- Migration guide documentation + +### Milestone 3 (Target: Q1 2027) - Production Ready + +- 1.0 release +- Extensive test coverage (unit + integration) +- Performance optimization for large workflows +- Error handling hardening +- Comprehensive documentation +- Community feedback incorporation +- CI/CD pipeline +- Release automation + +## References + +- [Serverless Workflow 0.8 Specification](https://github.com/serverlessworkflow/specification/tree/0.8.x) +- [Serverless Workflow 0.9 Specification](https://github.com/serverlessworkflow/specification/tree/0.9.x) +- [Serverless Workflow 1.0.0 Specification](https://github.com/serverlessworkflow/specification) +- [JsonPath Specification](https://goessner.net/articles/JsonPath/) +- [jq Manual](https://jqlang.github.io/jq/manual/)