Skip to content

Commit 1e0299e

Browse files
committed
Improved documentation
1 parent 686cb10 commit 1e0299e

File tree

3 files changed

+24
-8
lines changed

3 files changed

+24
-8
lines changed

README.md

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,36 @@
11
# Intro
22

3-
```Use``` [Jinja2](http://jinja.pocoo.org/docs/2.10/) ```templates to perform complex input/output stage data transformation on``` [AWS Step Functions State Machines](https://aws.amazon.com/step-functions/) and abstract away data transformation from your main state machine's logic.
3+
Happily using [AWS Step Functions State Machines](https://aws.amazon.com/step-functions/) but spending too much time matching input and output parameters and writing transformation code? Your pain is over :)
44

5-
Main Use Cases/Advantages:
6-
* To encourage the development of low-coupled reusable Lambda functions with clear input/outputs built as part of a Step Functions state machine
7-
* When a new state machine is built on top of a group of existing Lambda functions to transform the Lambda inputs/outputs without having to change any existing Lambda code
5+
Template Languages to the rescue!
86

9-
Anyone who's built a reasonably-sized AWS Step Function state machine knows that a reasonable amount of time is spent managing stage inputs and outputs, ie, transforming the JSON output structure of one stage to fit the JSON input structure of another stage. While AWS provides a way to query the JSON input and do some basic tranformation via JsonPath there are scenarios where one needs a more expresive language to perform complext transformations.
7+
## About this Project
108

11-
[Jinja2](http://jinja.pocoo.org/docs/2.10/) to the rescue!
9+
This project provides a reusable Lambda function (Connector Lambda) that can be added multiple times to a Step Functions state machine to perform input/output transformations. The Connector Lambda takes any input parameters and uses [Jinja2](http://jinja.pocoo.org/docs/2.10/) template to transform the input into a JSON output. As a results, you can focus on writing reusable Lambda functions for your state machine (or reusing existing Lambdas) without having to write transformation code as part of the Lambda's logic. Instead, transformation code will now be written using a template language!
1210

13-
This project uses Jinja2 and a Lambda function (connector) to abstract out input/output transformations from the state machine main logic so that stage outputs match the expected input of following stages. Just add the connector Lambda function to the state machine and specify the Jinja2 template to be used. The input values received by the Lambda are automatically exposed to the Jinja2 template. Use multiple connectors if needed!
11+
## Main Use Cases
12+
13+
The figure below illustrates the problem being addressed. On the left-side (A) you see a state machine with three states called ```state-1```, ```state-2```, and ```state-3```. These states could be represented by Lambda functions (state machine tasks) or EC2 workers (state machine activities). You're trying to add these Lambdas to a new state machine but their inputs/outputs don't match up. For example, ```state-1``` produces a red cicle while ```state-2``` expects a green square and ```state-3``` expects a red triangle as inputs.
14+
15+
On the right-hand side (B) you can see how the Connector Lambdas can be used to transform ```state-1```'s output to match the expected inputs of ```state-2``` and ```state-3```. Two Connector states are shown, both of which are backed up by the same Lambda function. Notice that the Connector states use different tranformation Jinja2 templates specified by you. That is, your tranformation code is not scattered through your Lambdas but instead placed into templates that can be easily understood and managed.
16+
17+
![approach-overview](/doc/aws-step-functions-connector-overview.png)
18+
19+
The main use cases for using the Connector Lambda above are:
20+
21+
* You have already built several Lambda functions that take some input parameters and produce some output values and want to put them together as part of a new state machine you're building BUT you don't want to have to modify any Lambda code to match the expected inputs and outputs in the state machine. That is, you want to ```reuse your existing Lambda functions as-is```
22+
23+
* You are building a new state machine and thus several Lambda functions. You're spending a lot of time making sure the Lambda output of a particular state matches the expected input of another state but you feel something is not right. And you're write! You're building highly-coupled Lambdas driven that only work in the context of the state machine you're building. Don't let the state machine dictate your Lambda inputs and outputs! That is, you want to ```build context-free low-coupled Lambdas with inputs and outputs that make sense regardless of the context```.
24+
25+
## Advantages of using a Template Language for Input/Output Transformation
26+
27+
* (Re)Use any of your existing Lambdas to build new state machines; handle transformations via templates
28+
* Don't worry about creating Lambda functions that fit a particular state machine (highly-coupled Lambdas); create reusable context-free Lambda functions with inputs and outputs that make sense
29+
* Don't spend hours writing and debugging input/output transformation code, instead, write simple templates and rely on the Connector Lambda to do handle the transformation logic
1430

1531
# Example
1632

17-
![approach-overview](/doc/aws-step-functions-connector.png)
33+
![approach-overview](/doc/aws-step-functions-connector-example.png)
1834

1935
# Deploying the Solution on AWS
2036

267 KB
Loading

0 commit comments

Comments
 (0)