Skip to content

Feat/improve 2d3d reproj #130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yzhou-bcom wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Feat/improve 2d3d reproj #130

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ae46322
feat(api): add method in IPCProjectOntoImage
yzhou-bcom May 12, 2026
57bb26f
feat(api): add new method in IPCSemanticFrom2D (point cloud related)
yzhou-bcom May 19, 2026
650096f
Merge branch 'master' into feat/improve-2d3d-reproj
yzhou-bcom May 26, 2026
b90d5e8
refactor(api): add struct SemanticId in IPCSemanticFrom2D
yzhou-bcom May 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions interfaces/api/pointCloud/IPCProjectOntoImage.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,17 @@ class XPCF_IGNORE IPCProjectOntoImage : virtual public org::bcom::xpcf::ICompone
const SolAR::datastructure::Transform3Df& pose,
SRef<SolAR::datastructure::Image>& image) const = 0;

/// @brief Project a list of cloud points onto an image plane
/// @param[in] points the list of cloud points
/// @param[in] cameraParameters the camera parameters
/// @param[in] pose the camera pose
/// @param[out] image output image created from the point cloud
/// @return FrameworkReturnCode::_SUCCESS (pointcloud projected onto image successfully) otherwise FrameworkReturnCode::_ERROR_ (failure)
virtual FrameworkReturnCode project(const std::vector<SRef<SolAR::datastructure::CloudPoint>>& points,
SRef<SolAR::datastructure::CameraParameters> cameraParameters,
const SolAR::datastructure::Transform3Df& pose,
SRef<SolAR::datastructure::Image>& image) const = 0;

};
}
}
Expand Down
23 changes: 23 additions & 0 deletions interfaces/api/pointCloud/IPCSemanticFrom2D.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,16 @@ namespace SolAR {
namespace api {
namespace pointCloud {

/**
* @struct SemanticId
* @brief struct used to represent semantic Id of a cloud point
*/
struct SemanticId {
SemanticId(int16_t i, float conf) : id(i), confidence(conf) {}
int16_t id;
float confidence;
};

/**
* @class IPCSemanticFrom2D
* @brief <B>Estimate semantic Ids of cloud points from 2D information.</B>
Expand All @@ -53,6 +63,19 @@ class XPCF_IGNORE IPCSemanticFrom2D : virtual public org::bcom::xpcf::IComponent
SRef<SolAR::datastructure::KeyframeCollection> keyframeCollection,
SRef<SolAR::datastructure::CameraParametersCollection> cameraParametersCollection,
SRef<SolAR::datastructure::Mask2DCollection> maskCollection) const = 0;

/// @brief Estimate point cloud semantic Ids from a keyframe collection and a mask collection
/// @param[in] points list of cloud points
/// @param[in] keyframeCollection input list of keyframes
/// @param[in] cameraParametersCollection input list of camera parameters
/// @param[in] maskCollection input list of masks
/// @param[out] semanticIds the estimated possible semantic ids for each input cloud point
/// @return FrameworkReturnCode::_SUCCESS (semantic id estimated successfully) otherwise FrameworkReturnCode::_ERROR_ (failure)
virtual FrameworkReturnCode estimate(const std::vector<SRef<SolAR::datastructure::CloudPoint>>& points,
Copy link
Copy Markdown
Contributor

@jim-bcom jim-bcom May 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some question to better understand:

  • Why has this method been added?
  • Why this one takes points as const ? In the current implementation, the other methods calls this one and uses semanticIds to update its pointCloud. Is it because the first takes a whole point cloud whereas here it can be a subset of cloud points...?
  • For the other method, aren't we interested in getting semanticIds as well ? It could be optional, for example, the argument could be added as a pointer to the vector, with nullptr as a default value?
estimate(SRef<SolAR::datastructure::PointCloud> pointCloud,
            std::vector<std::vector<std::pair<int16_t, float>>>* semanticIds = nullptr);

If it's null, you create and use it internally, otherwise, you use the given instance.

Copy link
Copy Markdown
Contributor Author

@yzhou-bcom yzhou-bcom May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this new method has been added to be able to output multiple semantic Ids for the same cloud point. previously, only the most probable semantic Id is set to cloud point via cloudPoint->setSemanticId()

the new method does not call setSemanticId() inside and the input list of points will not be modified

SRef<SolAR::datastructure::KeyframeCollection> keyframeCollection,
SRef<SolAR::datastructure::CameraParametersCollection> cameraParametersCollection,
SRef<SolAR::datastructure::Mask2DCollection> maskCollection,
std::vector<std::vector<SemanticId>>& semanticIds) const = 0;

};
}
Expand Down