[Feature Request] Support for initializing seed with existing point cloud for photo relocalization

[jo-chemla]

apple/ml-sharp is very good at generating a 3DGS scene from a single input image. It would be great to be able to pass an existing pointcloud as seed/initialization of the 3Dgs training, the same way one can do it for training a 3dgs scene with eg postshot (with sparse or dense colmap ptc). The relocalization within the scene of the input 2D photo could either be user-provided (extrinsics + intrinsics in the scene coordinate-system, eg as a colmap standard/format for params), or computed on-the-go doing a rough registration of the intermediate metric depth of the 2D image.

The goal is to align/register the input photo generated 3DGS with the existing pointcloud geometry. This would help a lot generating the 3DGS pointcloud for scenes not often found in the training set, like very natural landscapes with varying depth, which currently ml-sharp is pretty bad at estimating.

Proposed Feature

Add support for providing an input seed point cloud (e.g., .ply format) that ml-sharp can use to:

1. Initialize the 3DGS positions based on existing geometry
2. Constrain/align the predicted Gaussians to match the coordinate system of the seed
3. Optionally: perform pose estimation to localize the input photo relative to the seed point cloud

Happy to provide more details about my specific use case or help test any experimental implementations!