Add PySpec-to-Laurel translation and method call dispatch#446
Conversation
ead7f53 to
9058bce
Compare
9058bce to
ded2598
Compare
5395cdc to
61c76bb
Compare
a51d23b to
05e1512
Compare
79657d5 to
aa3d5b3
Compare
fc02702 to
5389ecd
Compare
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
5389ecd to
6969664
Compare
There was a problem hiding this comment.
The logic to translate PySpec to Laurel (in particular, of classes to composites with instance procedures) looks good to me. The only thing that concerns me is how the translation handles static dispatch. By building a dispatch table for methods overloaded on the Python side, the Python->Laurel translator seems to be doing type checking work that it shouldn't have to do / wouldn't already be resolved by assuming maximal type annotation of the input file.
That is, I'd rather see:
PySpec -> Laurel build the typing environment not just from local variables and function parameters, but also module-level declarations: this would catch type annotations on instance variables of a class.As a result, statically resolving the type ofboto3.client(s)would just be a matter of (1) looking up the type annotated on the enclosing declaration/assignment and (2) construct the monomorphized method name as is already done in this PR.
EDIT: I see that you attempt to resolve a method call first by type annotation and then by look up in the dispatch table...I think this is fine in isolation, but it is separately a bad thing that stub names and PySpec service names don't match sometimes. It seems to be a symptom of out-of-date stubs: if that were resolved, would you still need to build a dispatch table? Or would you want to keep this functionality anyway?
|
Hi Siva, you are right. If every assignment is annotated or inferable from class definitions (for cases like Currently neither really seems to be the case, so I'd like to unblock this. I think it's better to try to get things working now then postpone for future changes. |
Agreed. It should be easy to remove this in the future if we reach a point where it's not needed. One thing I could see doing is raising a warning if we find an inconsistency (but that can be a future PR). |
Summary
Enable end-to-end verification of Python programs that call AWS SDK methods by translating PySpec type stubs into Core prelude declarations and resolving method calls to the correct prelude procedures.
Details
PySpec-to-Laurel translation (
Specs/ToLaurel.lean, new)PySpec files describe Python library APIs as type signatures in Ion format. This PR adds a translator that converts those signatures into Laurel declarations (opaque procedures and composite types) which are then lowered to Core and merged into the verification prelude. This gives the verifier knowledge of library types and method signatures that were previously invisible.
The translator also builds an overload dispatch table — a map from factory function names to their string-literal-dispatched return types (e.g.,
boto3.client("iam")→AWSIdentityManagementV20100508). This is stored as aHashMap String (HashMap String PythonIdent)for O(1) lookup.Types not yet supported in CorePrelude (e.g.,
typing.Any,bytes,Optional[float]) are mapped toTStringvia anunsupportedTypeplaceholder, with TODO comments marking them for future CorePrelude additions.Method call dispatch (
PythonToLaurel.lean)The Python-to-Laurel translator previously rendered all method calls (e.g.,
iam.get_role(...)) aspyExprToString, producing names like"iam_get_role"that never matched any prelude procedure — so every SDK call became aHole(opaque havoc). This PR adds dispatch logic that:boto3.client("iam")against the overload table and produces aNewexpression for the appropriate service type.AWSIdentityManagementV20100508_get_role), and prepends the object as theselfargument.All three steps live inside
translateCall, which takes the raw Python call expression directly. The.Callarm intranslateExpris a single delegation totranslateCall.AnnAssign dispatch fallback: Python type-stub names (e.g.,
IAMClient) don't match PySpec service names (e.g.,AWSIdentityManagementV20100508). When anAnnAssignannotation resolves toPyAnyTypeand the initializer is a dispatched factory call, the translator falls back to the dispatch table to infer the correctUserDefinedtype. This ensures subsequent method calls on the variable resolve correctly.Unrecognized type annotations in Python source are mapped to
TStringviapyUnsupportedType, allowing translation to proceed without modifying the CorePrelude.CLI integration (
StrataMain.lean)pyAnalyzeLaurel --pyspec <ion_file>translates PySpec files and merges them into the Core prelude before verification.pyAnalyzeLaurel --dispatch <ion_file>extracts only the overload dispatch table (no Laurel translation) for lightweight factory-call resolution.pySpecToLaurelcommand for standalone PySpec-to-Laurel inspection.buildPySpecPreludehandles the multi-file merge pipeline with name collision detection.PySpec infrastructure (
Specs/Decls.lean,Specs.lean,Specs/DDM.lean)Inhabitedinstance added toPythonIdent(needed for HashMap default values).TypedDictfields changed fromisTotal : Boolto per-fieldfieldRequired : Array Boolto supportRequired[T]/NotRequired[T]annotations.Unpack[TypedDict]in**kwargsexpansion.Testing
ToLaurelTest.leanadds 15#guard_msgstests covering the PySpec-to-Laurel translation: primitive and complex type mapping, optional type patterns (Union[None, T]), literal and TypedDict types, class/type definitions, void returns, error cases (unknown types, empty types, unsupported unions), overload dispatch with both extern and local class return types,extractOverloadsfiltering, and a regression test forexternTypeDecl.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.