Expect! -> Require!; SAM2 object registration and tracking#274
Open
osnr wants to merge 19 commits into
Open
Conversation
The new Expect! is useful if you want to wait until the first matching statement appears (useful in end-application code). Require! is what you want if you're asserting that the statement should already exist.
Fixes horrible issues (instability, slowness due to prompt coordinate churn) with non-stabilized pages. Also just a simpler sam2 module on the Folk side.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I wouldn't say this is a good object tracker API yet (it doesn't undistort; you don't have quads or programs or geometry queries on the contours that it provides; it can only run on one viewport at a time probably), but it works and has some other minor fixes to Folk.
need viewport, registration program, and contour visualizer:
IMG_6876.MOV
Viewport:
Registration (needs to be cleared manually):
Registration (flip to clear):
Contour visualization:
Put down a viewport (ideally make it large), put a registration page on it, put your object in the center of the registration page, and then it should track and draw the contours.
It can track multiple objects on the viewport (but you need separate registration pages for all the objects you're tracking if you use the design above)
Fixes serialization of the obj id integers in sam2 module and extends it a lot for multi-object prompts and a higher-level track api.
Changes sam2 camera predictor around a lot (I don't fully understand this, but I didn't understand the original predictor code anyway) using some of the newer sam2 changes to their video predictor that let you add objects at runtime, change state around.
Other fixes:
Expect!toRequire!; This matches the traditional meaning of Tcl expect and also communicates thatRequire!is pretty demanding and is almost an assert, your program will fail if it doesn't match exactly 1.alignGeometrytocreateFromGeometry(because it is an object constructor basically, so deserves naming as that)pointToXYthat maps from (usually camera-space) xyz point to quad-space xy point. useful for mapping between program coordinate systems, camera, camera slice, drawing!getutility, which lets you probe individual pixels on an image. Basically just the probe utility we used in the marble run and color circle demosWould be great if you can test normal Folk stuff and make sure there are no regressions or weird breakage of that. Testing SAM2 would be great but not needed. Will merge tomorrow afternoon if no objections