Repository navigation
Increase Envoy dataplane efficiency by adding egress policy cache - #2200
yanavlasov wants to merge 23 commits into
Conversation
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
| ~/.cargo/registry/cache | ||
| ~/.cargo/git/db | ||
| cmd/dataplane/envoy/dynamic-modules/*/target | ||
| key: cargo-${{ runner.os }}-${{ hashFiles('cmd/dataplane/envoy/dynamic-modules/*/Cargo.lock') }} |
There was a problem hiding this comment.
just curious - why we moved it a folder?
There was a problem hiding this comment.
Each extension is in its own directory. Top level folder contains a workspace cargo files with the dynamic modules we build for Substrate. The existing egress-policy dynamic module did not move.
Lior Lieberman (LiorLieberman)
left a comment
Lior Lieberman (LiorLieberman)
left a comment
There was a problem hiding this comment.
thx yan!
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| //! The egress-policy-cache HTTP filter dynamic module. |
There was a problem hiding this comment.
why "!" at the beginning? is it a rust thing?
There was a problem hiding this comment.
Yes, it is for doc generator if we ever need it. Acts as a title.
| // TODO(yanavlasov): this is a temporary workaround of the Rust dynamic module API | ||
| // limitation that does not allow storing filter state shared with upstream. |
There was a problem hiding this comment.
not sure I understand this, what is the workaround? in other words - what would be the ideal thing that we can not do?
There was a problem hiding this comment.
Updated the comment
| An Envoy dynamic module, written in Rust, that runs as an HTTP filter on the | ||
| egress gateway's outer `CONNECT` listener and caches egress policy SNI rules | ||
| per actor certificate in a thread-local LRU cache so repeat `CONNECT` requests | ||
| from the same actor can bypass the `ext_proc` sidecar. |
There was a problem hiding this comment.
can bypass the
ext_procsidecar
I think ext-proc side car is doing more things today (like simple authn), do we need that? Or we are saying if the egress policy is cached authn already happened? If yes, is that secure?
I am not sure we need that authn but just checking
There was a problem hiding this comment.
I've updated readme with clarifications. Yes policy is cached after actor and destination port were authorized.
| per actor certificate in a thread-local LRU cache so repeat `CONNECT` requests | ||
| from the same actor can bypass the `ext_proc` sidecar. | ||
|
|
||
| ## How it works |
There was a problem hiding this comment.
shuold we store only what we care about for that stage of the module vs caching the whole egress policy? e.g a small list of https_hostnames (meaning you mitm those) and tls_hostnames (meaning you passthrough). We dont need all cred injection here etc,
There was a problem hiding this comment.
Right now policy is only SNIs and hostnames for the destination port. However I will add key injection as I will implement timed caching and local enforcement of inner policies, including key injection, in the followup. This should make the inner part more efficient as well and avoid callout per request.
| keyed by `<peer_cert_digest>;<destination_port>`, where `<peer_cert_digest>` is | ||
| the downstream peer certificate's SHA-256 digest | ||
| (`connection.sha256_peer_certificate_digest`) and `<destination_port>` is the | ||
| destination port extracted from the `:authority` request header. |
There was a problem hiding this comment.
extracted from the
:authorityrequest header.
Better to take it from the actual port, not the authority header? See #2228
There was a problem hiding this comment.
This comes from CONNECT authority, so it is the actual port. There is no other place for it to come from, since the cache is before ext_proc
| (`connection.sha256_peer_certificate_digest`) and `<destination_port>` is the | ||
| destination port extracted from the `:authority` request header. | ||
|
|
||
| ### Request path (`on_request_headers`) |
There was a problem hiding this comment.
isnt this section too detailed? can we just replace with a visual?
There was a problem hiding this comment.
I've edited it a bit. It seems like details are ok.
| `shared_with_upstream: ONCE`, making it available to the inner listener's | ||
| `egress-policy` listener filter. | ||
|
|
||
| ### Response path (`on_response_headers`) |
There was a problem hiding this comment.
same comment. too detailed and easy to get stale
There was a problem hiding this comment.
Shortened it a bit.
| if input.DisableKeepAlive { | ||
| outbound.Close = true | ||
| client.CloseIdleConnections() | ||
| } |
There was a problem hiding this comment.
reason?
There was a problem hiding this comment.
I've added a comment for this value.
| safe_regex: | ||
| google_re2: {} | ||
| regex: ".*" |
There was a problem hiding this comment.
what is the match for? i.e if this key exists in filter state, skip ext proc? why regex? cant we string match for dev.ate.policy.egress.cached ?
There was a problem hiding this comment.
I've added a dedicated filter state value to match on to avoid regex here.
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
2 participants
This PR implements egress policy cache in Envoy dataplane at CONNECT termination. It reduces number of callouts for policy retrievals from the same actor to the same destination port.
Cache for egress policy enforcement on the inner connection is in the followup PR.