From 0650fabb1a3822cf4ebe5f34b96584337cd07a89 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Fri, 20 Feb 2026 10:24:39 +0100 Subject: [PATCH 01/24] wip --- .cspell-wordlist.txt | 5 + CLAUDE.md | 100 ++++++++ .../host_objects/JsiConversions.h | 5 +- .../models/object_detection/Constants.h | 36 --- .../object_detection/ObjectDetection.cpp | 51 ++-- .../models/object_detection/ObjectDetection.h | 31 ++- .../models/object_detection/Types.h | 4 +- .../models/object_detection/Utils.cpp | 2 +- .../tests/integration/ObjectDetectionTest.cpp | 57 +++-- .../src/constants/modelUrls.ts | 13 +- .../computer_vision/useObjectDetection.ts | 113 ++++++++- packages/react-native-executorch/src/index.ts | 6 +- .../computer_vision/ObjectDetectionModule.ts | 234 +++++++++++++++--- .../src/types/objectDetection.ts | 89 ++++--- 14 files changed, 562 insertions(+), 184 deletions(-) create mode 100644 CLAUDE.md diff --git a/.cspell-wordlist.txt b/.cspell-wordlist.txt index e06bde775..2238f7142 100644 --- a/.cspell-wordlist.txt +++ b/.cspell-wordlist.txt @@ -122,3 +122,8 @@ worklet worklets BGRA RGBA +DETR +detr +metaprogramming +ktlint +lefthook diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..5e1d4d9c2 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,100 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +React Native ExecuTorch — a React Native library for running AI models on-device using Meta's ExecuTorch framework. Bridges React Native (TypeScript) with native C++/iOS/Android to run local inference for LLMs, computer vision, speech, and more. + +**Requirements**: iOS 17.0+, Android 13+, React Native 0.81+, New Architecture only. + +## Monorepo Structure + +Yarn 4.1.1 workspaces: + +- `packages/react-native-executorch/` — the library (published to npm) +- `apps/` — demo apps (computer-vision, llm, speech, text-embeddings, etc.) +- `docs/` — documentation site +- `scripts/` — build utilities + +## Commands + +```bash +# Root level +yarn lint # ESLint across all packages +yarn typecheck # TypeScript checking across all packages + +# Library (packages/react-native-executorch/) +yarn typecheck +yarn lint +yarn clean +yarn prepare # Build with react-native-builder-bob +``` + +No unit test suite — testing is done by building and running the demo apps on device. + +## Architecture + +### TypeScript → C++ Bridge + +The bridge uses JSI (JavaScript Interface) with global loader functions: + +1. **App startup**: `ETInstallerNativeModule.install()` (TurboModule, synchronous) calls C++ `RnExecutorchInstaller::injectJSIBindings()` +2. **Binding injection**: C++ template `loadModel()` registers global functions like `global.loadLLM()`, `global.loadImageSegmentation()`, etc. +3. **Model loading**: JS calls global function → C++ constructs model → returns `ModelHostObject` (JSI HostObject wrapping C++ model) +4. **Inference**: JS calls methods on the host object (e.g., `.generate()`, `.forward()`) → C++ executes ExecuTorch model → returns results as JSI TypedArrays + +Global loader declarations are in `src/index.ts`. + +### C++ Model Registration + +Models use the `REGISTER_CONSTRUCTOR` macro (in their header) to declare their constructor signature for JSI binding: + +```cpp +REGISTER_CONSTRUCTOR(models::image_segmentation::ImageSegmentation, + std::string, std::vector, std::vector, + std::shared_ptr); +``` + +The `CallInvoker` argument is always last and injected automatically — not passed from JS. + +### TypeScript Layers + +- **Modules** (`src/modules/`): Classes wrapping native host objects. Each has static factory methods or a `load()` method. Extend `BaseModule`. +- **Hooks** (`src/hooks/`): React hooks that manage module lifecycle (load, forward, delete, error/progress state). Two patterns: + - `useModule` — for constructor-based modules (`new Module()` + `load()`) + - `useModuleFactory` — for factory-based modules (async static factory) +- **Types** (`src/types/`): Shared type definitions per domain. +- **Controllers** (`src/controllers/`): Business logic layer (used by LLM). + +### C++ Side (`common/`) + +- `rnexecutorch/` — project C++ code (~16k lines) + - `models/` — model implementations, each with a `BaseModel` subclass + - `data_processing/` — image processing (OpenCV), numerical utils + - `host_objects/` — JSI interop (`ModelHostObject`, `JsiConversions`) + - `RnExecutorchInstaller.h` — JSI binding registration (template metaprogramming) + - `jsi/OwningArrayBuffer.h` — owns data returned to JS as ArrayBuffer +- `ada/` — vendored URL parser (~28k lines, don't modify) +- `runner/` — ExecuTorch runner utilities + +### Native Platform Code + +- **iOS** (`ios/`): `ETInstaller.mm` — ObjC++ TurboModule, loads frameworks, calls C++ installer +- **Android** (`android/`): `ETInstaller.kt` — Kotlin TurboModule + JNI bridge to C++ installer + +## Key Patterns + +- Models return TypedArrays (Int32Array, Float32Array) to JS, not plain arrays +- `ResourceFetcher.ts` handles model binary downloads with progress callbacks +- `OwningArrayBuffer` in C++ owns data that gets exposed as JS ArrayBuffer +- Image processing uses OpenCV (`cv::Mat` ↔ tensor conversions) +- Error handling: C++ `RnExecutorchError` → JS `RnExecutorchError` with error codes +- Cross-platform logging: `#include `, use `rnexecutorch::log(LOG_LEVEL::Info, ...)` + +## Formatting + +- **TypeScript**: Prettier (single quotes, 2-space indent, trailing commas) + ESLint with `@react-native` config +- **C++**: clang-format, C++20 standard +- **Kotlin**: ktlint +- Pre-commit hooks via lefthook enforce all of the above diff --git a/packages/react-native-executorch/common/rnexecutorch/host_objects/JsiConversions.h b/packages/react-native-executorch/common/rnexecutorch/host_objects/JsiConversions.h index 7b97108b9..187bb2792 100644 --- a/packages/react-native-executorch/common/rnexecutorch/host_objects/JsiConversions.h +++ b/packages/react-native-executorch/common/rnexecutorch/host_objects/JsiConversions.h @@ -15,7 +15,6 @@ #include #include -#include #include #include #include @@ -443,9 +442,7 @@ inline jsi::Value getJsiValue( detection.setProperty(runtime, "bbox", bbox); detection.setProperty( runtime, "label", - jsi::String::createFromAscii( - runtime, models::object_detection::constants::kCocoLablesMap.at( - detections[i].label))); + jsi::String::createFromUtf8(runtime, detections[i].label)); detection.setProperty(runtime, "score", detections[i].score); array.setValueAtIndex(runtime, i, detection); } diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Constants.h b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Constants.h index 555a5977c..9b3fd147a 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Constants.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Constants.h @@ -1,41 +1,5 @@ #pragma once -#include -#include - namespace rnexecutorch::models::object_detection::constants { inline constexpr float IOU_THRESHOLD = 0.55; - -inline const std::unordered_map kCocoLablesMap = { - {1, "PERSON"}, {2, "BICYCLE"}, {3, "CAR"}, - {4, "MOTORCYCLE"}, {5, "AIRPLANE"}, {6, "BUS"}, - {7, "TRAIN"}, {8, "TRUCK"}, {9, "BOAT"}, - {10, "TRAFFIC_LIGHT"}, {11, "FIRE_HYDRANT"}, {12, "STREET_SIGN"}, - {13, "STOP_SIGN"}, {14, "PARKING"}, {15, "BENCH"}, - {16, "BIRD"}, {17, "CAT"}, {18, "DOG"}, - {19, "HORSE"}, {20, "SHEEP"}, {21, "COW"}, - {22, "ELEPHANT"}, {23, "BEAR"}, {24, "ZEBRA"}, - {25, "GIRAFFE"}, {26, "HAT"}, {27, "BACKPACK"}, - {28, "UMBRELLA"}, {29, "SHOE"}, {30, "EYE"}, - {31, "HANDBAG"}, {32, "TIE"}, {33, "SUITCASE"}, - {34, "FRISBEE"}, {35, "SKIS"}, {36, "SNOWBOARD"}, - {37, "SPORTS"}, {38, "KITE"}, {39, "BASEBALL"}, - {40, "BASEBALL"}, {41, "SKATEBOARD"}, {42, "SURFBOARD"}, - {43, "TENNIS_RACKET"}, {44, "BOTTLE"}, {45, "PLATE"}, - {46, "WINE_GLASS"}, {47, "CUP"}, {48, "FORK"}, - {49, "KNIFE"}, {50, "SPOON"}, {51, "BOWL"}, - {52, "BANANA"}, {53, "APPLE"}, {54, "SANDWICH"}, - {55, "ORANGE"}, {56, "BROCCOLI"}, {57, "CARROT"}, - {58, "HOT_DOG"}, {59, "PIZZA"}, {60, "DONUT"}, - {61, "CAKE"}, {62, "CHAIR"}, {63, "COUCH"}, - {64, "POTTED_PLANT"}, {65, "BED"}, {66, "MIRROR"}, - {67, "DINING_TABLE"}, {68, "WINDOW"}, {69, "DESK"}, - {70, "TOILET"}, {71, "DOOR"}, {72, "TV"}, - {73, "LAPTOP"}, {74, "MOUSE"}, {75, "REMOTE"}, - {76, "KEYBOARD"}, {77, "CELL_PHONE"}, {78, "MICROWAVE"}, - {79, "OVEN"}, {80, "TOASTER"}, {81, "SINK"}, - {82, "REFRIGERATOR"}, {83, "BLENDER"}, {84, "BOOK"}, - {85, "CLOCK"}, {86, "VASE"}, {87, "SCISSORS"}, - {88, "TEDDY_BEAR"}, {89, "HAIR_DRIER"}, {90, "TOOTHBRUSH"}, - {91, "HAIR_BRUSH"}}; } // namespace rnexecutorch::models::object_detection::constants \ No newline at end of file diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp index 2670cf9dd..8eccb6ac5 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp @@ -14,7 +14,7 @@ ObjectDetection::ObjectDetection( std::shared_ptr callInvoker) : VisionModel(modelSource, callInvoker) { auto inputTensors = getAllInputShapes(); - if (inputTensors.size() == 0) { + if (inputTensors.empty()) { throw RnExecutorchError(RnExecutorchErrorCode::UnexpectedNumInputs, "Model seems to not take any input tensors."); } @@ -32,6 +32,18 @@ ObjectDetection::ObjectDetection( modelInputShape[modelInputShape.size() - 2]); } +ObjectDetection::ObjectDetection( + const std::string &modelSource, std::vector normMean, + std::vector normStd, std::shared_ptr callInvoker) + : ObjectDetection(modelSource, callInvoker) { + if (normMean.size() >= 3) { + normMean_ = cv::Scalar(normMean[0], normMean[1], normMean[2]); + } + if (normStd.size() >= 3) { + normStd_ = cv::Scalar(normStd[0], normStd[1], normStd[2]); + } +} + cv::Mat ObjectDetection::preprocessFrame(const cv::Mat &frame) const { const std::vector tensorDims = getAllInputShapes()[0]; cv::Size tensorSize = cv::Size(tensorDims[tensorDims.size() - 1], @@ -67,7 +79,8 @@ cv::Mat ObjectDetection::preprocessFrame(const cv::Mat &frame) const { std::vector ObjectDetection::postprocess(const std::vector &tensors, - cv::Size originalSize, double detectionThreshold) { + cv::Size originalSize, double detectionThreshold, + const std::vector &labelNames) { float widthRatio = static_cast(originalSize.width) / modelImageSize.width; float heightRatio = @@ -97,15 +110,18 @@ ObjectDetection::postprocess(const std::vector &tensors, float y1 = bboxes[i * 4 + 1] * heightRatio; float x2 = bboxes[i * 4 + 2] * widthRatio; float y2 = bboxes[i * 4 + 3] * heightRatio; - detections.emplace_back(x1, y1, x2, y2, static_cast(labels[i]), - scores[i]); + auto labelIdx = static_cast(labels[i]); + std::string labelName = + labelIdx < labelNames.size() ? labelNames[labelIdx] : ""; + detections.emplace_back(x1, y1, x2, y2, labelName, scores[i]); } return utils::nonMaxSuppression(detections); } std::vector -ObjectDetection::runInference(cv::Mat image, double detectionThreshold) { +ObjectDetection::runInference(cv::Mat image, double detectionThreshold, + const std::vector &labelNames) { if (detectionThreshold < 0.0 || detectionThreshold > 1.0) { throw RnExecutorchError(RnExecutorchErrorCode::InvalidUserInput, "detectionThreshold must be in range [0, 1]"); @@ -117,7 +133,10 @@ ObjectDetection::runInference(cv::Mat image, double detectionThreshold) { const std::vector tensorDims = getAllInputShapes()[0]; auto inputTensor = - image_processing::getTensorFromMatrix(tensorDims, preprocessed); + (normMean_ && normStd_) + ? image_processing::getTensorFromMatrix(tensorDims, preprocessed, + *normMean_, *normStd_) + : image_processing::getTensorFromMatrix(tensorDims, preprocessed); auto forwardResult = BaseModel::forward(inputTensor); if (!forwardResult.ok()) { @@ -126,35 +145,39 @@ ObjectDetection::runInference(cv::Mat image, double detectionThreshold) { "Ensure the model input is correct."); } - return postprocess(forwardResult.get(), originalSize, detectionThreshold); + return postprocess(forwardResult.get(), originalSize, detectionThreshold, + labelNames); } std::vector ObjectDetection::generateFromString(std::string imageSource, - double detectionThreshold) { + double detectionThreshold, + std::vector labelNames) { cv::Mat imageBGR = image_processing::readImage(imageSource); cv::Mat imageRGB; cv::cvtColor(imageBGR, imageRGB, cv::COLOR_BGR2RGB); - return runInference(imageRGB, detectionThreshold); + return runInference(imageRGB, detectionThreshold, labelNames); } std::vector ObjectDetection::generateFromFrame(jsi::Runtime &runtime, const jsi::Value &frameData, - double detectionThreshold) { + double detectionThreshold, + std::vector labelNames) { auto frameObj = frameData.asObject(runtime); cv::Mat frame = rnexecutorch::utils::extractFrame(runtime, frameObj); - return runInference(frame, detectionThreshold); + return runInference(frame, detectionThreshold, labelNames); } std::vector ObjectDetection::generateFromPixels(JSTensorViewIn pixelData, - double detectionThreshold) { + double detectionThreshold, + std::vector labelNames) { cv::Mat image = extractFromPixels(pixelData); - return runInference(image, detectionThreshold); + return runInference(image, detectionThreshold, labelNames); } -} // namespace rnexecutorch::models::object_detection \ No newline at end of file +} // namespace rnexecutorch::models::object_detection diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h index d32eea95e..800764664 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h @@ -1,6 +1,6 @@ #pragma once -#include +#include #include #include @@ -20,28 +20,39 @@ class ObjectDetection : public VisionModel { public: ObjectDetection(const std::string &modelSource, std::shared_ptr callInvoker); + ObjectDetection(const std::string &modelSource, std::vector normMean, + std::vector normStd, + std::shared_ptr callInvoker); [[nodiscard("Registered non-void function")]] std::vector - generateFromString(std::string imageSource, double detectionThreshold); + generateFromString(std::string imageSource, double detectionThreshold, + std::vector labelNames); [[nodiscard("Registered non-void function")]] std::vector generateFromFrame(jsi::Runtime &runtime, const jsi::Value &frameData, - double detectionThreshold); + double detectionThreshold, + std::vector labelNames); [[nodiscard("Registered non-void function")]] std::vector - generateFromPixels(JSTensorViewIn pixelData, double detectionThreshold); + generateFromPixels(JSTensorViewIn pixelData, double detectionThreshold, + std::vector labelNames); protected: - std::vector runInference(cv::Mat image, - double detectionThreshold); + std::vector + runInference(cv::Mat image, double detectionThreshold, + const std::vector &labelNames); cv::Mat preprocessFrame(const cv::Mat &frame) const override; private: - std::vector postprocess(const std::vector &tensors, - cv::Size originalSize, - double detectionThreshold); + std::vector + postprocess(const std::vector &tensors, cv::Size originalSize, + double detectionThreshold, + const std::vector &labelNames); cv::Size modelImageSize{0, 0}; + std::optional normMean_; + std::optional normStd_; }; } // namespace models::object_detection REGISTER_CONSTRUCTOR(models::object_detection::ObjectDetection, std::string, + std::vector, std::vector, std::shared_ptr); -} // namespace rnexecutorch \ No newline at end of file +} // namespace rnexecutorch diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Types.h b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Types.h index 58c910a5c..075ae91e0 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Types.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Types.h @@ -1,12 +1,14 @@ #pragma once +#include + namespace rnexecutorch::models::object_detection::types { struct Detection { float x1; float y1; float x2; float y2; - int label; + std::string label; float score; }; diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Utils.cpp b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Utils.cpp index 1e32dcc29..198fe1fe5 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Utils.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Utils.cpp @@ -35,7 +35,7 @@ nonMaxSuppression(std::vector detections) { std::vector result; // Apply NMS for each label for (size_t i = 0; i < detections.size();) { - float currentLabel = detections[i].label; + std::string currentLabel = detections[i].label; std::vector labelDetections; while (i < detections.size() && detections[i].label == currentLabel) { diff --git a/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp b/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp index 76c838ca1..73540e07d 100644 --- a/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp @@ -31,7 +31,7 @@ template <> struct ModelTraits { } static void callGenerate(ModelType &model) { - (void)model.generateFromString(kValidTestImagePath, 0.5); + (void)model.generateFromString(kValidTestImagePath, 0.5, {}); } }; } // namespace model_tests @@ -45,50 +45,55 @@ INSTANTIATE_TYPED_TEST_SUITE_P(ObjectDetection, CommonModelTest, // ============================================================================ TEST(ObjectDetectionGenerateTests, InvalidImagePathThrows) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - EXPECT_THROW((void)model.generateFromString("nonexistent_image.jpg", 0.5), - RnExecutorchError); + EXPECT_THROW( + (void)model.generateFromString("nonexistent_image.jpg", 0.5, {}), + RnExecutorchError); } TEST(ObjectDetectionGenerateTests, EmptyImagePathThrows) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - EXPECT_THROW((void)model.generateFromString("", 0.5), RnExecutorchError); + EXPECT_THROW((void)model.generateFromString("", 0.5, {}), RnExecutorchError); } TEST(ObjectDetectionGenerateTests, MalformedURIThrows) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - EXPECT_THROW((void)model.generateFromString("not_a_valid_uri://bad", 0.5), - RnExecutorchError); + EXPECT_THROW( + (void)model.generateFromString("not_a_valid_uri://bad", 0.5, {}), + RnExecutorchError); } TEST(ObjectDetectionGenerateTests, NegativeThresholdThrows) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - EXPECT_THROW((void)model.generateFromString(kValidTestImagePath, -0.1), - RnExecutorchError); + EXPECT_THROW( + (void)model.generateFromString(kValidTestImagePath, -0.1, {}), + RnExecutorchError); } TEST(ObjectDetectionGenerateTests, ThresholdAboveOneThrows) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - EXPECT_THROW((void)model.generateFromString(kValidTestImagePath, 1.1), - RnExecutorchError); + EXPECT_THROW( + (void)model.generateFromString(kValidTestImagePath, 1.1, {}), + RnExecutorchError); } TEST(ObjectDetectionGenerateTests, ValidImageReturnsResults) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - auto results = model.generateFromString(kValidTestImagePath, 0.3); + auto results = model.generateFromString(kValidTestImagePath, 0.3, {}); EXPECT_GE(results.size(), 0u); } TEST(ObjectDetectionGenerateTests, HighThresholdReturnsFewerResults) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - auto lowThresholdResults = model.generateFromString(kValidTestImagePath, 0.1); + auto lowThresholdResults = + model.generateFromString(kValidTestImagePath, 0.1, {}); auto highThresholdResults = - model.generateFromString(kValidTestImagePath, 0.9); + model.generateFromString(kValidTestImagePath, 0.9, {}); EXPECT_GE(lowThresholdResults.size(), highThresholdResults.size()); } TEST(ObjectDetectionGenerateTests, DetectionsHaveValidBoundingBoxes) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - auto results = model.generateFromString(kValidTestImagePath, 0.3); + auto results = model.generateFromString(kValidTestImagePath, 0.3, {}); for (const auto &detection : results) { EXPECT_LE(detection.x1, detection.x2); @@ -100,7 +105,7 @@ TEST(ObjectDetectionGenerateTests, DetectionsHaveValidBoundingBoxes) { TEST(ObjectDetectionGenerateTests, DetectionsHaveValidScores) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - auto results = model.generateFromString(kValidTestImagePath, 0.3); + auto results = model.generateFromString(kValidTestImagePath, 0.3, {}); for (const auto &detection : results) { EXPECT_GE(detection.score, 0.0f); @@ -110,11 +115,11 @@ TEST(ObjectDetectionGenerateTests, DetectionsHaveValidScores) { TEST(ObjectDetectionGenerateTests, DetectionsHaveValidLabels) { ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - auto results = model.generateFromString(kValidTestImagePath, 0.3); - - for (const auto &detection : results) { - EXPECT_GE(detection.label, 0); - } + // Label names are resolved by the caller (e.g. ObjectDetectionModule). + // Passing {} produces empty-string labels; this test just verifies the + // returned detections have the expected structure without throwing. + EXPECT_NO_THROW( + (void)model.generateFromString(kValidTestImagePath, 0.3, {})); } // ============================================================================ @@ -127,7 +132,7 @@ TEST(ObjectDetectionPixelTests, ValidPixelDataReturnsResults) { JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Byte}; - auto results = model.generateFromPixels(tensorView, 0.3); + auto results = model.generateFromPixels(tensorView, 0.3, {}); EXPECT_GE(results.size(), 0u); } @@ -136,7 +141,7 @@ TEST(ObjectDetectionPixelTests, WrongSizesLengthThrows) { std::vector pixelData(16, 0); JSTensorViewIn tensorView{ pixelData.data(), {4, 4}, executorch::aten::ScalarType::Byte}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5), + EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5, {}), RnExecutorchError); } @@ -147,7 +152,7 @@ TEST(ObjectDetectionPixelTests, WrongChannelCountThrows) { JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Byte}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5), + EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5, {}), RnExecutorchError); } @@ -158,7 +163,7 @@ TEST(ObjectDetectionPixelTests, WrongScalarTypeThrows) { JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Float}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5), + EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5, {}), RnExecutorchError); } @@ -169,7 +174,7 @@ TEST(ObjectDetectionPixelTests, NegativeThresholdThrows) { JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Byte}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, -0.1), + EXPECT_THROW((void)model.generateFromPixels(tensorView, -0.1, {}), RnExecutorchError); } @@ -180,7 +185,7 @@ TEST(ObjectDetectionPixelTests, ThresholdAboveOneThrows) { JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Byte}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, 1.1), + EXPECT_THROW((void)model.generateFromPixels(tensorView, 1.1, {}), RnExecutorchError); } diff --git a/packages/react-native-executorch/src/constants/modelUrls.ts b/packages/react-native-executorch/src/constants/modelUrls.ts index 499abf63a..c648775e0 100644 --- a/packages/react-native-executorch/src/constants/modelUrls.ts +++ b/packages/react-native-executorch/src/constants/modelUrls.ts @@ -386,13 +386,24 @@ export const EFFICIENTNET_V2_S = { // Object detection const SSDLITE_320_MOBILENET_V3_LARGE_MODEL = `${URL_PREFIX}-ssdlite320-mobilenet-v3-large/${VERSION_TAG}/ssdlite320-mobilenetv3-large.pte`; +// const RF_DETR_NANO_MODEL = `${URL_PREFIX}-rf-detr-nano/${VERSION_TAG}/rf-detr-nano.pte`; +const RF_DETR_NANO_MODEL = `https://ai.swmansion.com/storage/jc_tests/rfdetr_det.pte`; /** * @category Models - Object Detection */ export const SSDLITE_320_MOBILENET_V3_LARGE = { + modelName: 'ssdlite-320-mobilenet-v3-large', modelSource: SSDLITE_320_MOBILENET_V3_LARGE_MODEL, -}; +} as const; + +/** + * @category Models - Object Detection + */ +export const RF_DETR_NANO = { + modelName: 'rf-detr-nano', + modelSource: RF_DETR_NANO_MODEL, +} as const; // Style transfer const STYLE_TRANSFER_CANDY_MODEL = diff --git a/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts b/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts index 27be7478c..92ed9bfe9 100644 --- a/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts +++ b/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts @@ -1,24 +1,115 @@ -import { useModule } from '../useModule'; -import { ObjectDetectionModule } from '../../modules/computer_vision/ObjectDetectionModule'; +import { useState, useEffect } from 'react'; import { + ObjectDetectionModule, + ObjectDetectionLabels, +} from '../../modules/computer_vision/ObjectDetectionModule'; +import { + Detection, + ObjectDetectionModelSources, ObjectDetectionProps, ObjectDetectionType, } from '../../types/objectDetection'; +import { Frame, PixelData } from '../../types/common'; +import { RnExecutorchErrorCode } from '../../errors/ErrorCodes'; +import { RnExecutorchError, parseUnknownError } from '../../errors/errorUtils'; /** * React hook for managing an Object Detection model instance. * + * @typeParam C - A {@link ObjectDetectionModelSources} config specifying which built-in model to load. * @category Hooks - * @param ObjectDetectionProps - Configuration object containing `model` source and optional `preventLoad` flag. - * @returns Ready to use Object Detection model. + * @param props - Configuration object containing `model` config and optional `preventLoad` flag. + * @returns An object with model state (`error`, `isReady`, `isGenerating`, `downloadProgress`) and typed `forward` and `runOnFrame` functions. */ -export const useObjectDetection = ({ +export const useObjectDetection = ({ model, preventLoad = false, -}: ObjectDetectionProps): ObjectDetectionType => { - return useModule({ - module: ObjectDetectionModule, - model, - preventLoad: preventLoad, - }); +}: ObjectDetectionProps): ObjectDetectionType< + ObjectDetectionLabels +> => { + const [error, setError] = useState(null); + const [isReady, setIsReady] = useState(false); + const [isGenerating, setIsGenerating] = useState(false); + const [downloadProgress, setDownloadProgress] = useState(0); + const [instance, setInstance] = useState | null>(null); + const [runOnFrame, setRunOnFrame] = useState< + | (( + frame: Frame, + detectionThreshold: number + ) => Detection>[]) + | null + >(null); + + useEffect(() => { + if (preventLoad) return; + + let currentInstance: ObjectDetectionModule | null = null; + + (async () => { + setDownloadProgress(0); + setError(null); + setIsReady(false); + setRunOnFrame(null); + try { + currentInstance = await ObjectDetectionModule.fromModelName( + model, + setDownloadProgress + ); + setInstance(currentInstance); + // Functional setState form is required when storing a function in state, + // to prevent React from calling it as an updater. + setRunOnFrame(() => currentInstance!.runOnFrame); + setIsReady(true); + } catch (err) { + setError(parseUnknownError(err)); + } + })(); + + return () => { + currentInstance?.delete(); + }; + + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [model.modelName, model.modelSource, preventLoad]); + + const forward = async ( + input: string | PixelData, + detectionThreshold?: number + ) => { + if (!isReady || !instance) { + throw new RnExecutorchError( + RnExecutorchErrorCode.ModuleNotLoaded, + 'The model is currently not loaded. Please load the model before calling forward().' + ); + } + if (isGenerating) { + throw new RnExecutorchError( + RnExecutorchErrorCode.ModelGenerating, + 'The model is currently generating. Please wait until previous model run is complete.' + ); + } + try { + setIsGenerating(true); + return (await instance.forward(input, detectionThreshold)) as Awaited< + ReturnType< + ObjectDetectionType< + ObjectDetectionLabels + >['forward'] + > + >; + } finally { + setIsGenerating(false); + } + }; + + return { + error, + isReady, + isGenerating, + downloadProgress, + forward, + runOnFrame, + }; }; diff --git a/packages/react-native-executorch/src/index.ts b/packages/react-native-executorch/src/index.ts index dd7557ca2..21c19e10e 100644 --- a/packages/react-native-executorch/src/index.ts +++ b/packages/react-native-executorch/src/index.ts @@ -42,7 +42,11 @@ declare global { normStd: Triple | [] ) => any; var loadClassification: (source: string) => any; - var loadObjectDetection: (source: string) => any; + var loadObjectDetection: ( + source: string, + normMean: Triple | [], + normStd: Triple | [] + ) => any; var loadExecutorchModule: (source: string) => any; var loadTokenizerModule: (source: string) => any; var loadImageEmbeddings: (source: string) => any; diff --git a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts index f056cff62..105ef155b 100644 --- a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts +++ b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts @@ -1,52 +1,220 @@ import { ResourceFetcher } from '../../utils/ResourceFetcher'; -import { ResourceSource, PixelData } from '../../types/common'; -import { Detection } from '../../types/objectDetection'; +import { LabelEnum, ResourceSource, PixelData, Frame } from '../../types/common'; +import { + CocoLabel, + Detection, + ObjectDetectionConfig, + ObjectDetectionModelName, + ObjectDetectionModelSources, +} from '../../types/objectDetection'; import { RnExecutorchErrorCode } from '../../errors/ErrorCodes'; -import { parseUnknownError, RnExecutorchError } from '../../errors/errorUtils'; -import { Logger } from '../../common/Logger'; -import { VisionModule } from './VisionModule'; +import { RnExecutorchError } from '../../errors/errorUtils'; +import { BaseModule } from '../BaseModule'; +import { IMAGENET1K_MEAN, IMAGENET1K_STD } from '../../constants/commonVision'; + +const ModelConfigs = { + 'ssdlite-320-mobilenet-v3-large': { + labelMap: CocoLabel, + preprocessorConfig: undefined, + }, + 'rf-detr-nano': { + labelMap: CocoLabel, + preprocessorConfig: { normMean: IMAGENET1K_MEAN, normStd: IMAGENET1K_STD }, + }, +} as const satisfies Record< + ObjectDetectionModelName, + ObjectDetectionConfig +>; + +type ModelConfigsType = typeof ModelConfigs; + +/** + * Resolves the {@link LabelEnum} for a given built-in object detection model name. + * + * @typeParam M - A built-in model name from {@link ObjectDetectionModelName}. + * + * @category Types + */ +export type ObjectDetectionLabels = + ModelConfigsType[M]['labelMap']; + +type ModelNameOf = C['modelName']; + +/** + * @internal + * Resolves the label type: if `T` is a {@link ObjectDetectionModelName}, looks up its labels + * from the built-in config; otherwise uses `T` directly as a {@link LabelEnum}. + */ +type ResolveLabels = + T extends ObjectDetectionModelName ? ObjectDetectionLabels : T; /** - * Module for object detection tasks. + * Generic object detection module with type-safe label maps. + * + * @typeParam T - Either a built-in model name (e.g. `'ssdlite-320-mobilenet-v3-large'`) + * or a custom {@link LabelEnum} label map. * * @category Typescript API */ -export class ObjectDetectionModule extends VisionModule { +export class ObjectDetectionModule< + T extends ObjectDetectionModelName | LabelEnum, +> extends BaseModule { + private labelMap: ResolveLabels; + private allLabelNames: string[]; + + private constructor(labelMap: ResolveLabels, nativeModule: unknown) { + super(); + this.labelMap = labelMap; + this.allLabelNames = []; + for (const [name, value] of Object.entries(this.labelMap)) { + if (typeof value === 'number') { + this.allLabelNames[value] = name; + } + } + for (let i = 0; i < this.allLabelNames.length; i++) { + if (this.allLabelNames[i] == null) { + this.allLabelNames[i] = ''; + } + } + this.nativeModule = nativeModule; + } + + // TODO: figure it out so we can delete this (we need this because of basemodule inheritance) + override async load() {} + /** - * Loads the model, where `modelSource` is a string that specifies the location of the model binary. - * To track the download progress, supply a callback function `onDownloadProgressCallback`. + * Creates an object detection instance for a built-in model. * - * @param model - Object containing `modelSource`. - * @param onDownloadProgressCallback - Optional callback to monitor download progress. + * @param config - A {@link ObjectDetectionModelSources} object specifying which model to load and where to fetch it from. + * @param onDownloadProgress - Optional callback to monitor download progress, receiving a value between 0 and 1. + * @returns A Promise resolving to an `ObjectDetectionModule` instance typed to the chosen model's label map. */ - async load( - model: { modelSource: ResourceSource }, - onDownloadProgressCallback: (progress: number) => void = () => {} - ): Promise { - try { - const paths = await ResourceFetcher.fetch( - onDownloadProgressCallback, - model.modelSource + static async fromModelName( + config: C, + onDownloadProgress: (progress: number) => void = () => {} + ): Promise>> { + const { modelSource } = config; + const { labelMap, preprocessorConfig } = ModelConfigs[ + config.modelName + ] as ObjectDetectionConfig; + const normMean = preprocessorConfig?.normMean ?? []; + const normStd = preprocessorConfig?.normStd ?? []; + const paths = await ResourceFetcher.fetch(onDownloadProgress, modelSource); + if (!paths?.[0]) { + throw new RnExecutorchError( + RnExecutorchErrorCode.DownloadInterrupted, + 'The download has been interrupted. Please retry.' ); + } + const nativeModule = global.loadObjectDetection( + paths[0], + normMean, + normStd + ); + return new ObjectDetectionModule>( + labelMap as ResolveLabels>, + nativeModule + ); + } - if (!paths?.[0]) { - throw new RnExecutorchError( - RnExecutorchErrorCode.DownloadInterrupted, - 'The download has been interrupted. As a result, not every file was downloaded. Please retry the download.' - ); - } - - this.nativeModule = global.loadObjectDetection(paths[0]); - } catch (error) { - Logger.error('Load failed:', error); - throw parseUnknownError(error); + /** + * Creates an object detection instance with a user-provided label map and custom config. + * + * @param modelSource - A fetchable resource pointing to the model binary. + * @param config - A {@link ObjectDetectionConfig} object with the label map. + * @param onDownloadProgress - Optional callback to monitor download progress, receiving a value between 0 and 1. + * @returns A Promise resolving to an `ObjectDetectionModule` instance typed to the provided label map. + */ + static async fromCustomConfig( + modelSource: ResourceSource, + config: ObjectDetectionConfig, + onDownloadProgress: (progress: number) => void = () => {} + ): Promise> { + const normMean = config.preprocessorConfig?.normMean ?? []; + const normStd = config.preprocessorConfig?.normStd ?? []; + const paths = await ResourceFetcher.fetch(onDownloadProgress, modelSource); + if (!paths?.[0]) { + throw new RnExecutorchError( + RnExecutorchErrorCode.DownloadInterrupted, + 'The download has been interrupted. Please retry.' + ); } + const nativeModule = global.loadObjectDetection( + paths[0], + normMean, + normStd + ); + return new ObjectDetectionModule( + config.labelMap as ResolveLabels, + nativeModule + ); } + /** + * Executes the model's forward pass to detect objects within the provided image. + * + * @param input - A string image source (file path, URI, or Base64) or a {@link PixelData} object. + * @param detectionThreshold - Minimum confidence score for a detection to be included. Default is 0.7. + * @returns A Promise resolving to an array of {@link Detection} objects. + * @throws {RnExecutorchError} If the model is not loaded. + */ async forward( input: string | PixelData, - detectionThreshold: number = 0.5 - ): Promise { - return super.forward(input, detectionThreshold); + detectionThreshold: number = 0.7 + ): Promise>[]> { + if (this.nativeModule == null) { + throw new RnExecutorchError( + RnExecutorchErrorCode.ModuleNotLoaded, + 'The model is currently not loaded.' + ); + } + if (typeof input === 'string') { + return await this.nativeModule.generateFromString( + input, + detectionThreshold, + this.allLabelNames + ); + } + return await this.nativeModule.generateFromPixels( + input, + detectionThreshold, + this.allLabelNames + ); + } + + /** + * Synchronous worklet function for real-time VisionCamera frame processing. + * The label names are captured from the module instance — no need to pass them per frame. + * + * **Use this for VisionCamera frame processing in worklets.** + * For async processing, use `forward()` instead. + */ + get runOnFrame(): + | ((frame: Frame, detectionThreshold: number) => Detection>[]) + | null { + if (!this.nativeModule?.generateFromFrame) { + return null; + } + + const nativeGenerateFromFrame = this.nativeModule.generateFromFrame; + const allLabelNames = this.allLabelNames; + + return ( + frame: Frame, + detectionThreshold: number + ): Detection>[] => { + 'worklet'; + + let nativeBuffer: ReturnType | null = null; + try { + nativeBuffer = frame.getNativeBuffer(); + const frameData = { nativeBuffer: nativeBuffer.pointer }; + return nativeGenerateFromFrame(frameData, detectionThreshold, allLabelNames); + } finally { + if (nativeBuffer?.release) { + nativeBuffer.release(); + } + } + }; } } diff --git a/packages/react-native-executorch/src/types/objectDetection.ts b/packages/react-native-executorch/src/types/objectDetection.ts index 5aaf81833..58b90589a 100644 --- a/packages/react-native-executorch/src/types/objectDetection.ts +++ b/packages/react-native-executorch/src/types/objectDetection.ts @@ -1,5 +1,5 @@ import { RnExecutorchError } from '../errors/errorUtils'; -import { ResourceSource, PixelData, Frame } from './common'; +import { LabelEnum, Triple, ResourceSource, PixelData, Frame } from './common'; /** * Represents a bounding box for a detected object in an image. @@ -21,13 +21,14 @@ export interface Bbox { * Represents a detected object within an image, including its bounding box, label, and confidence score. * * @category Types + * @typeParam L - The label enum type for the detected object. Defaults to {@link CocoLabel}. * @property {Bbox} bbox - The bounding box of the detected object, defined by its top-left (x1, y1) and bottom-right (x2, y2) coordinates. - * @property {keyof typeof CocoLabel} label - The class label of the detected object, represented as a key from the `CocoLabel` enum. + * @property {keyof L} label - The class label of the detected object. * @property {number} score - The confidence score of the detection, typically ranging from 0 to 1. */ -export interface Detection { +export interface Detection { bbox: Bbox; - label: keyof typeof CocoLabel; + label: keyof L; score: number; } @@ -129,16 +130,43 @@ export enum CocoLabel { HAIR_BRUSH = 91, } +/** + * Per-model config for {@link ObjectDetectionModule.fromModelName}. + * Each model name maps to its required fields. + * + * @category Types + */ +export type ObjectDetectionModelSources = + | { modelName: 'ssdlite-320-mobilenet-v3-large'; modelSource: ResourceSource } + | { modelName: 'rf-detr-nano'; modelSource: ResourceSource }; + +/** + * Union of all built-in object detection model names. + * + * @category Types + */ +export type ObjectDetectionModelName = ObjectDetectionModelSources['modelName']; + +/** + * Configuration for a custom object detection model. + * + * @category Types + */ +export type ObjectDetectionConfig = { + labelMap: T; + preprocessorConfig?: { normMean?: Triple; normStd?: Triple }; +}; + /** * Props for the `useObjectDetection` hook. * + * @typeParam C - A {@link ObjectDetectionModelSources} config specifying which built-in model to load. * @category Types - * @property {Object} model - An object containing the model source. - * @property {ResourceSource} model.modelSource - The source of the object detection model binary. + * @property model - The model config containing `modelName` and `modelSource`. * @property {boolean} [preventLoad] - Boolean that can prevent automatic model loading (and downloading the data if you load it for the first time) after running the hook. */ -export interface ObjectDetectionProps { - model: { modelSource: ResourceSource }; +export interface ObjectDetectionProps { + model: C; preventLoad?: boolean; } @@ -146,9 +174,11 @@ export interface ObjectDetectionProps { * Return type for the `useObjectDetection` hook. * Manages the state and operations for Computer Vision object detection tasks. * + * @typeParam L - The {@link LabelEnum} representing the model's class labels. + * * @category Types */ -export interface ObjectDetectionType { +export interface ObjectDetectionType { /** * Contains the error object if the model failed to load, download, or encountered a runtime error during detection. */ @@ -172,34 +202,15 @@ export interface ObjectDetectionType { /** * Executes the model's forward pass with automatic input type detection. * - * Supports two input types: - * 1. **String path/URI**: File path, URL, or Base64-encoded string - * 2. **PixelData**: Raw pixel data from image libraries (e.g., NitroImage) - * - * **Note**: For VisionCamera frame processing, use `runOnFrame` instead. - * - * @param input - Image source (string or PixelData object) - * @param detectionThreshold - An optional number between 0 and 1 representing the minimum confidence score. Default is 0.5. + * @param input - Image source (string path/URI or PixelData object) + * @param detectionThreshold - An optional number between 0 and 1 representing the minimum confidence score. Default is 0.7. * @returns A Promise that resolves to an array of `Detection` objects. * @throws {RnExecutorchError} If the model is not loaded or is currently processing another image. - * - * @example - * ```typescript - * // String path - * const detections1 = await model.forward('file:///path/to/image.jpg'); - * - * // Pixel data - * const detections2 = await model.forward({ - * dataPtr: new Uint8Array(rgbPixels), - * sizes: [480, 640, 3], - * scalarType: ScalarType.BYTE - * }); - * ``` */ forward: ( input: string | PixelData, detectionThreshold?: number - ) => Promise; + ) => Promise[]>; /** * Synchronous worklet function for real-time VisionCamera frame processing. @@ -210,25 +221,11 @@ export interface ObjectDetectionType { * * Available after model is loaded (`isReady: true`). * - * @example - * ```typescript - * const { runOnFrame, isReady } = useObjectDetection({ model: MODEL }); - * - * const frameOutput = useFrameOutput({ - * onFrame(frame) { - * 'worklet'; - * if (!runOnFrame) return; - * const detections = runOnFrame(frame, 0.5); - * frame.dispose(); - * } - * }); - * ``` - * * @param frame - VisionCamera Frame object * @param detectionThreshold - The threshold for detection sensitivity. * @returns Array of Detection objects representing detected items in the frame. */ runOnFrame: - | ((frame: Frame, detectionThreshold: number) => Detection[]) + | ((frame: Frame, detectionThreshold: number) => Detection[]) | null; } From a044cc90426872139bda310ad796d69251eccd47 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Fri, 20 Feb 2026 13:48:21 +0100 Subject: [PATCH 02/24] wip --- .../src/constants/commonVision.ts | 98 +++++++++++++++++ .../computer_vision/useObjectDetection.ts | 101 +++--------------- .../useSemanticSegmentation.ts | 93 +++------------- .../src/hooks/useModuleFactory.ts | 85 +++++++++++++++ .../src/modules/BaseLabeledModule.ts | 62 +++++++++++ .../computer_vision/ObjectDetectionModule.ts | 56 ++++------ .../SemanticSegmentationModule.ts | 45 +++----- .../src/types/objectDetection.ts | 100 +---------------- 8 files changed, 309 insertions(+), 331 deletions(-) create mode 100644 packages/react-native-executorch/src/hooks/useModuleFactory.ts create mode 100644 packages/react-native-executorch/src/modules/BaseLabeledModule.ts diff --git a/packages/react-native-executorch/src/constants/commonVision.ts b/packages/react-native-executorch/src/constants/commonVision.ts index 05eeba759..ba89c9f84 100644 --- a/packages/react-native-executorch/src/constants/commonVision.ts +++ b/packages/react-native-executorch/src/constants/commonVision.ts @@ -2,3 +2,101 @@ import { Triple } from '../types/common'; export const IMAGENET1K_MEAN: Triple = [0.485, 0.456, 0.406]; export const IMAGENET1K_STD: Triple = [0.229, 0.224, 0.225]; + +/** + * COCO dataset class labels used for object detection. + * + * @category Types + */ +export enum CocoLabel { + PERSON = 1, + BICYCLE = 2, + CAR = 3, + MOTORCYCLE = 4, + AIRPLANE = 5, + BUS = 6, + TRAIN = 7, + TRUCK = 8, + BOAT = 9, + TRAFFIC_LIGHT = 10, + FIRE_HYDRANT = 11, + STREET_SIGN = 12, + STOP_SIGN = 13, + PARKING = 14, + BENCH = 15, + BIRD = 16, + CAT = 17, + DOG = 18, + HORSE = 19, + SHEEP = 20, + COW = 21, + ELEPHANT = 22, + BEAR = 23, + ZEBRA = 24, + GIRAFFE = 25, + HAT = 26, + BACKPACK = 27, + UMBRELLA = 28, + SHOE = 29, + EYE = 30, + HANDBAG = 31, + TIE = 32, + SUITCASE = 33, + FRISBEE = 34, + SKIS = 35, + SNOWBOARD = 36, + SPORTS = 37, + KITE = 38, + BASEBALL = 39, + SKATEBOARD = 41, + SURFBOARD = 42, + TENNIS_RACKET = 43, + BOTTLE = 44, + PLATE = 45, + WINE_GLASS = 46, + CUP = 47, + FORK = 48, + KNIFE = 49, + SPOON = 50, + BOWL = 51, + BANANA = 52, + APPLE = 53, + SANDWICH = 54, + ORANGE = 55, + BROCCOLI = 56, + CARROT = 57, + HOT_DOG = 58, + PIZZA = 59, + DONUT = 60, + CAKE = 61, + CHAIR = 62, + COUCH = 63, + POTTED_PLANT = 64, + BED = 65, + MIRROR = 66, + DINING_TABLE = 67, + WINDOW = 68, + DESK = 69, + TOILET = 70, + DOOR = 71, + TV = 72, + LAPTOP = 73, + MOUSE = 74, + REMOTE = 75, + KEYBOARD = 76, + CELL_PHONE = 77, + MICROWAVE = 78, + OVEN = 79, + TOASTER = 80, + SINK = 81, + REFRIGERATOR = 82, + BLENDER = 83, + BOOK = 84, + CLOCK = 85, + VASE = 86, + SCISSORS = 87, + TEDDY_BEAR = 88, + HAIR_DRIER = 89, + TOOTHBRUSH = 90, + HAIR_BRUSH = 91, +} diff --git a/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts b/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts index 92ed9bfe9..daaeca476 100644 --- a/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts +++ b/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts @@ -1,17 +1,15 @@ -import { useState, useEffect } from 'react'; import { ObjectDetectionModule, ObjectDetectionLabels, } from '../../modules/computer_vision/ObjectDetectionModule'; import { - Detection, ObjectDetectionModelSources, ObjectDetectionProps, ObjectDetectionType, } from '../../types/objectDetection'; -import { Frame, PixelData } from '../../types/common'; -import { RnExecutorchErrorCode } from '../../errors/ErrorCodes'; -import { RnExecutorchError, parseUnknownError } from '../../errors/errorUtils'; +import { useMemo } from 'react'; +import { PixelData } from '../../types/common'; +import { useModuleFactory } from '../useModuleFactory'; /** * React hook for managing an Object Detection model instance. @@ -27,89 +25,18 @@ export const useObjectDetection = ({ }: ObjectDetectionProps): ObjectDetectionType< ObjectDetectionLabels > => { - const [error, setError] = useState(null); - const [isReady, setIsReady] = useState(false); - const [isGenerating, setIsGenerating] = useState(false); - const [downloadProgress, setDownloadProgress] = useState(0); - const [instance, setInstance] = useState | null>(null); - const [runOnFrame, setRunOnFrame] = useState< - | (( - frame: Frame, - detectionThreshold: number - ) => Detection>[]) - | null - >(null); + const { error, isReady, isGenerating, downloadProgress, runForward, instance } = + useModuleFactory({ + factory: (config, onProgress) => + ObjectDetectionModule.fromModelName(config, onProgress), + config: model, + preventLoad, + }); - useEffect(() => { - if (preventLoad) return; + const forward = (input: string | PixelData, detectionThreshold?: number) => + runForward((inst) => inst.forward(input, detectionThreshold)); - let currentInstance: ObjectDetectionModule | null = null; + const runOnFrame = useMemo(() => instance?.runOnFrame ?? null, [instance]); - (async () => { - setDownloadProgress(0); - setError(null); - setIsReady(false); - setRunOnFrame(null); - try { - currentInstance = await ObjectDetectionModule.fromModelName( - model, - setDownloadProgress - ); - setInstance(currentInstance); - // Functional setState form is required when storing a function in state, - // to prevent React from calling it as an updater. - setRunOnFrame(() => currentInstance!.runOnFrame); - setIsReady(true); - } catch (err) { - setError(parseUnknownError(err)); - } - })(); - - return () => { - currentInstance?.delete(); - }; - - // eslint-disable-next-line react-hooks/exhaustive-deps - }, [model.modelName, model.modelSource, preventLoad]); - - const forward = async ( - input: string | PixelData, - detectionThreshold?: number - ) => { - if (!isReady || !instance) { - throw new RnExecutorchError( - RnExecutorchErrorCode.ModuleNotLoaded, - 'The model is currently not loaded. Please load the model before calling forward().' - ); - } - if (isGenerating) { - throw new RnExecutorchError( - RnExecutorchErrorCode.ModelGenerating, - 'The model is currently generating. Please wait until previous model run is complete.' - ); - } - try { - setIsGenerating(true); - return (await instance.forward(input, detectionThreshold)) as Awaited< - ReturnType< - ObjectDetectionType< - ObjectDetectionLabels - >['forward'] - > - >; - } finally { - setIsGenerating(false); - } - }; - - return { - error, - isReady, - isGenerating, - downloadProgress, - forward, - runOnFrame, - }; + return { error, isReady, isGenerating, downloadProgress, forward, runOnFrame }; }; diff --git a/packages/react-native-executorch/src/hooks/computer_vision/useSemanticSegmentation.ts b/packages/react-native-executorch/src/hooks/computer_vision/useSemanticSegmentation.ts index 5cfd1af18..cad249110 100644 --- a/packages/react-native-executorch/src/hooks/computer_vision/useSemanticSegmentation.ts +++ b/packages/react-native-executorch/src/hooks/computer_vision/useSemanticSegmentation.ts @@ -1,4 +1,3 @@ -import { useState, useEffect } from 'react'; import { SemanticSegmentationModule, SegmentationLabels, @@ -9,8 +8,7 @@ import { ModelNameOf, SemanticSegmentationModelSources, } from '../../types/semanticSegmentation'; -import { RnExecutorchErrorCode } from '../../errors/ErrorCodes'; -import { RnExecutorchError, parseUnknownError } from '../../errors/errorUtils'; +import { useModuleFactory } from '../useModuleFactory'; /** * React hook for managing a Semantic Segmentation model instance. @@ -36,83 +34,22 @@ export const useSemanticSegmentation = < }: SemanticSegmentationProps): SemanticSegmentationType< SegmentationLabels> > => { - const [error, setError] = useState(null); - const [isReady, setIsReady] = useState(false); - const [isGenerating, setIsGenerating] = useState(false); - const [downloadProgress, setDownloadProgress] = useState(0); - const [instance, setInstance] = useState - > | null>(null); - - useEffect(() => { - if (preventLoad) return; - - let isMounted = true; - let currentInstance: SemanticSegmentationModule> | null = - null; - - (async () => { - setDownloadProgress(0); - setError(null); - setIsReady(false); - try { - currentInstance = await SemanticSegmentationModule.fromModelName( - model, - (progress) => { - if (isMounted) setDownloadProgress(progress); - } - ); - if (isMounted) { - setInstance(currentInstance); - setIsReady(true); - } - } catch (err) { - if (isMounted) setError(parseUnknownError(err)); - } - })(); - - return () => { - isMounted = false; - currentInstance?.delete(); - }; - - // eslint-disable-next-line react-hooks/exhaustive-deps - }, [model.modelName, model.modelSource, preventLoad]); - - const forward = async >>( + const { error, isReady, isGenerating, downloadProgress, runForward } = + useModuleFactory({ + factory: (config, onProgress) => + SemanticSegmentationModule.fromModelName(config, onProgress), + config: model, + preventLoad, + }); + + const forward = >>( imageSource: string, classesOfInterest: K[] = [], resizeToInput: boolean = true - ) => { - if (!isReady || !instance) { - throw new RnExecutorchError( - RnExecutorchErrorCode.ModuleNotLoaded, - 'The model is currently not loaded. Please load the model before calling forward().' - ); - } - if (isGenerating) { - throw new RnExecutorchError( - RnExecutorchErrorCode.ModelGenerating, - 'The model is currently generating. Please wait until previous model run is complete.' - ); - } - try { - setIsGenerating(true); - return await instance.forward( - imageSource, - classesOfInterest, - resizeToInput - ); - } finally { - setIsGenerating(false); - } - }; + ) => + runForward((inst) => + inst.forward(imageSource, classesOfInterest, resizeToInput) + ); - return { - error, - isReady, - isGenerating, - downloadProgress, - forward, - }; + return { error, isReady, isGenerating, downloadProgress, forward }; }; diff --git a/packages/react-native-executorch/src/hooks/useModuleFactory.ts b/packages/react-native-executorch/src/hooks/useModuleFactory.ts new file mode 100644 index 000000000..ef4dc841c --- /dev/null +++ b/packages/react-native-executorch/src/hooks/useModuleFactory.ts @@ -0,0 +1,85 @@ +import { useState, useEffect } from 'react'; +import { RnExecutorchErrorCode } from '../errors/ErrorCodes'; +import { RnExecutorchError, parseUnknownError } from '../errors/errorUtils'; + +type Deletable = { delete: () => void }; + +/** + * Shared hook for modules that are instantiated via an async static factory + * (i.e. `SomeModule.fromModelName(config, onProgress)`). + * + * Handles model loading, download progress, error state, and enforces the + * not-loaded / already-generating guards so individual hooks only need to + * define their typed `forward` wrapper. + * + * @internal + */ +export function useModuleFactory< + M extends Deletable, + Config extends { modelName: string; modelSource: unknown }, +>({ + factory, + config, + preventLoad = false, +}: { + factory: ( + config: Config, + onProgress: (progress: number) => void + ) => Promise; + config: Config; + preventLoad?: boolean; +}) { + const [error, setError] = useState(null); + const [isReady, setIsReady] = useState(false); + const [isGenerating, setIsGenerating] = useState(false); + const [downloadProgress, setDownloadProgress] = useState(0); + const [instance, setInstance] = useState(null); + + useEffect(() => { + if (preventLoad) return; + + let currentInstance: M | null = null; + + (async () => { + setDownloadProgress(0); + setError(null); + setIsReady(false); + try { + currentInstance = await factory(config, setDownloadProgress); + setInstance(currentInstance); + setIsReady(true); + } catch (err) { + setError(parseUnknownError(err)); + } + })(); + + return () => { + currentInstance?.delete(); + }; + + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [config.modelName, config.modelSource, preventLoad]); + + const runForward = async (fn: (instance: M) => Promise): Promise => { + if (!isReady || !instance) { + throw new RnExecutorchError( + RnExecutorchErrorCode.ModuleNotLoaded, + 'The model is currently not loaded. Please load the model before calling forward().' + ); + } + if (isGenerating) { + throw new RnExecutorchError( + RnExecutorchErrorCode.ModelGenerating, + 'The model is currently generating. Please wait until previous model run is complete.' + ); + } + try { + setIsGenerating(true); + return await fn(instance); + } finally { + setIsGenerating(false); + } + }; + + return { error, isReady, isGenerating, downloadProgress, runForward, instance }; +} diff --git a/packages/react-native-executorch/src/modules/BaseLabeledModule.ts b/packages/react-native-executorch/src/modules/BaseLabeledModule.ts new file mode 100644 index 000000000..6d8719b65 --- /dev/null +++ b/packages/react-native-executorch/src/modules/BaseLabeledModule.ts @@ -0,0 +1,62 @@ +import { ResourceFetcher } from '../utils/ResourceFetcher'; +import { LabelEnum, ResourceSource } from '../types/common'; +import { RnExecutorchErrorCode } from '../errors/ErrorCodes'; +import { RnExecutorchError } from '../errors/errorUtils'; +import { BaseModule } from './BaseModule'; + +/** + * Fetches a model binary and returns its local path, throwing if the download + * was interrupted (paused or cancelled). + * + * @internal + */ +export async function fetchModelPath( + source: ResourceSource, + onDownloadProgress: (progress: number) => void +): Promise { + const paths = await ResourceFetcher.fetch(onDownloadProgress, source); + if (!paths?.[0]) { + throw new RnExecutorchError( + RnExecutorchErrorCode.DownloadInterrupted, + 'The download has been interrupted. Please retry.' + ); + } + return paths[0]; +} + +/** + * Given a model configs record (mapping model names to `{ labelMap }`) and a + * type `T` (either a model name key or a raw {@link LabelEnum}), resolves to + * the label map for that model or `T` itself. + * + * @internal + */ +export type ResolveLabels< + T, + Configs extends Record, +> = T extends keyof Configs + ? Configs[T]['labelMap'] + : T extends LabelEnum + ? T + : never; + +/** + * Base class for vision modules that carry a type-safe label map. + * + * @typeParam LabelMap - The resolved {@link LabelEnum} for the model's output classes. + * @internal + */ +export abstract class BaseLabeledModule< + LabelMap extends LabelEnum, +> extends BaseModule { + protected readonly labelMap: LabelMap; + + protected constructor(labelMap: LabelMap, nativeModule: unknown) { + super(); + this.labelMap = labelMap; + this.nativeModule = nativeModule; + } + + // TODO: figure it out so we can delete this (we need this because of basemodule inheritance) + override async load() {} +} diff --git a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts index 105ef155b..2fdb3b1af 100644 --- a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts +++ b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts @@ -1,16 +1,22 @@ -import { ResourceFetcher } from '../../utils/ResourceFetcher'; import { LabelEnum, ResourceSource, PixelData, Frame } from '../../types/common'; import { - CocoLabel, Detection, ObjectDetectionConfig, ObjectDetectionModelName, ObjectDetectionModelSources, } from '../../types/objectDetection'; +import { + CocoLabel, + IMAGENET1K_MEAN, + IMAGENET1K_STD, +} from '../../constants/commonVision'; import { RnExecutorchErrorCode } from '../../errors/ErrorCodes'; import { RnExecutorchError } from '../../errors/errorUtils'; -import { BaseModule } from '../BaseModule'; -import { IMAGENET1K_MEAN, IMAGENET1K_STD } from '../../constants/commonVision'; +import { + BaseLabeledModule, + fetchModelPath, + ResolveLabels as ResolveLabelsFor, +} from '../BaseLabeledModule'; const ModelConfigs = { 'ssdlite-320-mobilenet-v3-large': { @@ -35,18 +41,14 @@ type ModelConfigsType = typeof ModelConfigs; * * @category Types */ -export type ObjectDetectionLabels = - ModelConfigsType[M]['labelMap']; +export type ObjectDetectionLabels = + ResolveLabelsFor; type ModelNameOf = C['modelName']; -/** - * @internal - * Resolves the label type: if `T` is a {@link ObjectDetectionModelName}, looks up its labels - * from the built-in config; otherwise uses `T` directly as a {@link LabelEnum}. - */ +/** @internal */ type ResolveLabels = - T extends ObjectDetectionModelName ? ObjectDetectionLabels : T; + ResolveLabelsFor; /** * Generic object detection module with type-safe label maps. @@ -58,13 +60,11 @@ type ResolveLabels = */ export class ObjectDetectionModule< T extends ObjectDetectionModelName | LabelEnum, -> extends BaseModule { - private labelMap: ResolveLabels; +> extends BaseLabeledModule> { private allLabelNames: string[]; private constructor(labelMap: ResolveLabels, nativeModule: unknown) { - super(); - this.labelMap = labelMap; + super(labelMap, nativeModule); this.allLabelNames = []; for (const [name, value] of Object.entries(this.labelMap)) { if (typeof value === 'number') { @@ -76,12 +76,8 @@ export class ObjectDetectionModule< this.allLabelNames[i] = ''; } } - this.nativeModule = nativeModule; } - // TODO: figure it out so we can delete this (we need this because of basemodule inheritance) - override async load() {} - /** * Creates an object detection instance for a built-in model. * @@ -99,15 +95,9 @@ export class ObjectDetectionModule< ] as ObjectDetectionConfig; const normMean = preprocessorConfig?.normMean ?? []; const normStd = preprocessorConfig?.normStd ?? []; - const paths = await ResourceFetcher.fetch(onDownloadProgress, modelSource); - if (!paths?.[0]) { - throw new RnExecutorchError( - RnExecutorchErrorCode.DownloadInterrupted, - 'The download has been interrupted. Please retry.' - ); - } + const modelPath = await fetchModelPath(modelSource, onDownloadProgress); const nativeModule = global.loadObjectDetection( - paths[0], + modelPath, normMean, normStd ); @@ -132,15 +122,9 @@ export class ObjectDetectionModule< ): Promise> { const normMean = config.preprocessorConfig?.normMean ?? []; const normStd = config.preprocessorConfig?.normStd ?? []; - const paths = await ResourceFetcher.fetch(onDownloadProgress, modelSource); - if (!paths?.[0]) { - throw new RnExecutorchError( - RnExecutorchErrorCode.DownloadInterrupted, - 'The download has been interrupted. Please retry.' - ); - } + const modelPath = await fetchModelPath(modelSource, onDownloadProgress); const nativeModule = global.loadObjectDetection( - paths[0], + modelPath, normMean, normStd ); diff --git a/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts b/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts index b3b7030b3..db28b50be 100644 --- a/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts +++ b/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts @@ -1,4 +1,3 @@ -import { ResourceFetcher } from '../../utils/ResourceFetcher'; import { ResourceSource, LabelEnum } from '../../types/common'; import { DeeplabLabel, @@ -10,8 +9,12 @@ import { } from '../../types/semanticSegmentation'; import { RnExecutorchErrorCode } from '../../errors/ErrorCodes'; import { RnExecutorchError } from '../../errors/errorUtils'; -import { BaseModule } from '../BaseModule'; import { IMAGENET1K_MEAN, IMAGENET1K_STD } from '../../constants/commonVision'; +import { + BaseLabeledModule, + fetchModelPath, + ResolveLabels as ResolveLabelsFor, +} from '../BaseLabeledModule'; const PascalVocSegmentationConfig = { labelMap: DeeplabLabel, @@ -55,13 +58,9 @@ type ModelConfigsType = typeof ModelConfigs; export type SegmentationLabels = ModelConfigsType[M]['labelMap']; -/** - * @internal - * Resolves the label type: if `T` is a {@link SemanticSegmentationModelName}, looks up its labels - * from the built-in config; otherwise uses `T` directly as a {@link LabelEnum}. - */ +/** @internal */ type ResolveLabels = - T extends SemanticSegmentationModelName ? SegmentationLabels : T; + ResolveLabelsFor; /** * Generic semantic segmentation module with type-safe label maps. @@ -80,22 +79,16 @@ type ResolveLabels = */ export class SemanticSegmentationModule< T extends SemanticSegmentationModelName | LabelEnum, -> extends BaseModule { - private labelMap: ResolveLabels; +> extends BaseLabeledModule> { private allClassNames: string[]; private constructor(labelMap: ResolveLabels, nativeModule: unknown) { - super(); - this.labelMap = labelMap; + super(labelMap, nativeModule); this.allClassNames = Object.keys(this.labelMap).filter((k) => isNaN(Number(k)) ); - this.nativeModule = nativeModule; } - // TODO: figure it out so we can delete this (we need this because of basemodule inheritance) - override async load() {} - /** * Creates a segmentation instance for a built-in model. * The config object is discriminated by `modelName` — each model can require different fields. @@ -124,15 +117,9 @@ export class SemanticSegmentationModule< ] as SemanticSegmentationConfig; const normMean = preprocessorConfig?.normMean ?? []; const normStd = preprocessorConfig?.normStd ?? []; - const paths = await ResourceFetcher.fetch(onDownloadProgress, modelSource); - if (!paths?.[0]) { - throw new RnExecutorchError( - RnExecutorchErrorCode.DownloadInterrupted, - 'The download has been interrupted. As a result, not every file was downloaded. Please retry the download.' - ); - } + const modelPath = await fetchModelPath(modelSource, onDownloadProgress); const nativeModule = global.loadSemanticSegmentation( - paths[0], + modelPath, normMean, normStd ); @@ -165,17 +152,11 @@ export class SemanticSegmentationModule< config: SemanticSegmentationConfig, onDownloadProgress: (progress: number) => void = () => {} ): Promise> { - const paths = await ResourceFetcher.fetch(onDownloadProgress, modelSource); - if (!paths?.[0]) { - throw new RnExecutorchError( - RnExecutorchErrorCode.DownloadInterrupted, - 'The download has been interrupted. Please retry.' - ); - } const normMean = config.preprocessorConfig?.normMean ?? []; const normStd = config.preprocessorConfig?.normStd ?? []; + const modelPath = await fetchModelPath(modelSource, onDownloadProgress); const nativeModule = global.loadSemanticSegmentation( - paths[0], + modelPath, normMean, normStd ); diff --git a/packages/react-native-executorch/src/types/objectDetection.ts b/packages/react-native-executorch/src/types/objectDetection.ts index 58b90589a..aa25e9c41 100644 --- a/packages/react-native-executorch/src/types/objectDetection.ts +++ b/packages/react-native-executorch/src/types/objectDetection.ts @@ -1,5 +1,7 @@ import { RnExecutorchError } from '../errors/errorUtils'; import { LabelEnum, Triple, ResourceSource, PixelData, Frame } from './common'; +import { CocoLabel } from '../constants/commonVision'; +export { CocoLabel }; /** * Represents a bounding box for a detected object in an image. @@ -32,104 +34,6 @@ export interface Detection { score: number; } -/** - * COCO dataset class labels used for object detection. - * - * @category Types - */ -export enum CocoLabel { - PERSON = 1, - BICYCLE = 2, - CAR = 3, - MOTORCYCLE = 4, - AIRPLANE = 5, - BUS = 6, - TRAIN = 7, - TRUCK = 8, - BOAT = 9, - TRAFFIC_LIGHT = 10, - FIRE_HYDRANT = 11, - STREET_SIGN = 12, - STOP_SIGN = 13, - PARKING = 14, - BENCH = 15, - BIRD = 16, - CAT = 17, - DOG = 18, - HORSE = 19, - SHEEP = 20, - COW = 21, - ELEPHANT = 22, - BEAR = 23, - ZEBRA = 24, - GIRAFFE = 25, - HAT = 26, - BACKPACK = 27, - UMBRELLA = 28, - SHOE = 29, - EYE = 30, - HANDBAG = 31, - TIE = 32, - SUITCASE = 33, - FRISBEE = 34, - SKIS = 35, - SNOWBOARD = 36, - SPORTS = 37, - KITE = 38, - BASEBALL = 39, - SKATEBOARD = 41, - SURFBOARD = 42, - TENNIS_RACKET = 43, - BOTTLE = 44, - PLATE = 45, - WINE_GLASS = 46, - CUP = 47, - FORK = 48, - KNIFE = 49, - SPOON = 50, - BOWL = 51, - BANANA = 52, - APPLE = 53, - SANDWICH = 54, - ORANGE = 55, - BROCCOLI = 56, - CARROT = 57, - HOT_DOG = 58, - PIZZA = 59, - DONUT = 60, - CAKE = 61, - CHAIR = 62, - COUCH = 63, - POTTED_PLANT = 64, - BED = 65, - MIRROR = 66, - DINING_TABLE = 67, - WINDOW = 68, - DESK = 69, - TOILET = 70, - DOOR = 71, - TV = 72, - LAPTOP = 73, - MOUSE = 74, - REMOTE = 75, - KEYBOARD = 76, - CELL_PHONE = 77, - MICROWAVE = 78, - OVEN = 79, - TOASTER = 80, - SINK = 81, - REFRIGERATOR = 82, - BLENDER = 83, - BOOK = 84, - CLOCK = 85, - VASE = 86, - SCISSORS = 87, - TEDDY_BEAR = 88, - HAIR_DRIER = 89, - TOOTHBRUSH = 90, - HAIR_BRUSH = 91, -} - /** * Per-model config for {@link ObjectDetectionModule.fromModelName}. * Each model name maps to its required fields. From 2a76e3caf11f6c07327af40ef9b7ec6ea67351aa Mon Sep 17 00:00:00 2001 From: chmjkb Date: Fri, 20 Feb 2026 14:27:37 +0100 Subject: [PATCH 03/24] delete claude.md --- CLAUDE.md | 100 ------------------------------------------------------ 1 file changed, 100 deletions(-) delete mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 5e1d4d9c2..000000000 --- a/CLAUDE.md +++ /dev/null @@ -1,100 +0,0 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## Project Overview - -React Native ExecuTorch — a React Native library for running AI models on-device using Meta's ExecuTorch framework. Bridges React Native (TypeScript) with native C++/iOS/Android to run local inference for LLMs, computer vision, speech, and more. - -**Requirements**: iOS 17.0+, Android 13+, React Native 0.81+, New Architecture only. - -## Monorepo Structure - -Yarn 4.1.1 workspaces: - -- `packages/react-native-executorch/` — the library (published to npm) -- `apps/` — demo apps (computer-vision, llm, speech, text-embeddings, etc.) -- `docs/` — documentation site -- `scripts/` — build utilities - -## Commands - -```bash -# Root level -yarn lint # ESLint across all packages -yarn typecheck # TypeScript checking across all packages - -# Library (packages/react-native-executorch/) -yarn typecheck -yarn lint -yarn clean -yarn prepare # Build with react-native-builder-bob -``` - -No unit test suite — testing is done by building and running the demo apps on device. - -## Architecture - -### TypeScript → C++ Bridge - -The bridge uses JSI (JavaScript Interface) with global loader functions: - -1. **App startup**: `ETInstallerNativeModule.install()` (TurboModule, synchronous) calls C++ `RnExecutorchInstaller::injectJSIBindings()` -2. **Binding injection**: C++ template `loadModel()` registers global functions like `global.loadLLM()`, `global.loadImageSegmentation()`, etc. -3. **Model loading**: JS calls global function → C++ constructs model → returns `ModelHostObject` (JSI HostObject wrapping C++ model) -4. **Inference**: JS calls methods on the host object (e.g., `.generate()`, `.forward()`) → C++ executes ExecuTorch model → returns results as JSI TypedArrays - -Global loader declarations are in `src/index.ts`. - -### C++ Model Registration - -Models use the `REGISTER_CONSTRUCTOR` macro (in their header) to declare their constructor signature for JSI binding: - -```cpp -REGISTER_CONSTRUCTOR(models::image_segmentation::ImageSegmentation, - std::string, std::vector, std::vector, - std::shared_ptr); -``` - -The `CallInvoker` argument is always last and injected automatically — not passed from JS. - -### TypeScript Layers - -- **Modules** (`src/modules/`): Classes wrapping native host objects. Each has static factory methods or a `load()` method. Extend `BaseModule`. -- **Hooks** (`src/hooks/`): React hooks that manage module lifecycle (load, forward, delete, error/progress state). Two patterns: - - `useModule` — for constructor-based modules (`new Module()` + `load()`) - - `useModuleFactory` — for factory-based modules (async static factory) -- **Types** (`src/types/`): Shared type definitions per domain. -- **Controllers** (`src/controllers/`): Business logic layer (used by LLM). - -### C++ Side (`common/`) - -- `rnexecutorch/` — project C++ code (~16k lines) - - `models/` — model implementations, each with a `BaseModel` subclass - - `data_processing/` — image processing (OpenCV), numerical utils - - `host_objects/` — JSI interop (`ModelHostObject`, `JsiConversions`) - - `RnExecutorchInstaller.h` — JSI binding registration (template metaprogramming) - - `jsi/OwningArrayBuffer.h` — owns data returned to JS as ArrayBuffer -- `ada/` — vendored URL parser (~28k lines, don't modify) -- `runner/` — ExecuTorch runner utilities - -### Native Platform Code - -- **iOS** (`ios/`): `ETInstaller.mm` — ObjC++ TurboModule, loads frameworks, calls C++ installer -- **Android** (`android/`): `ETInstaller.kt` — Kotlin TurboModule + JNI bridge to C++ installer - -## Key Patterns - -- Models return TypedArrays (Int32Array, Float32Array) to JS, not plain arrays -- `ResourceFetcher.ts` handles model binary downloads with progress callbacks -- `OwningArrayBuffer` in C++ owns data that gets exposed as JS ArrayBuffer -- Image processing uses OpenCV (`cv::Mat` ↔ tensor conversions) -- Error handling: C++ `RnExecutorchError` → JS `RnExecutorchError` with error codes -- Cross-platform logging: `#include `, use `rnexecutorch::log(LOG_LEVEL::Info, ...)` - -## Formatting - -- **TypeScript**: Prettier (single quotes, 2-space indent, trailing commas) + ESLint with `@react-native` config -- **C++**: clang-format, C++20 standard -- **Kotlin**: ktlint -- Pre-commit hooks via lefthook enforce all of the above From 4e83924252b95819fe1eb2ea15399555d47f3441 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Wed, 25 Feb 2026 12:58:32 +0100 Subject: [PATCH 04/24] docs/fix: minor cpp change, add docs --- .../02-computer-vision/useObjectDetection.md | 99 ++++++++++--------- .../object_detection/ObjectDetection.cpp | 22 +++-- .../models/object_detection/ObjectDetection.h | 6 +- 3 files changed, 70 insertions(+), 57 deletions(-) diff --git a/docs/docs/03-hooks/02-computer-vision/useObjectDetection.md b/docs/docs/03-hooks/02-computer-vision/useObjectDetection.md index 4dc96eafb..69ac3c79a 100644 --- a/docs/docs/03-hooks/02-computer-vision/useObjectDetection.md +++ b/docs/docs/03-hooks/02-computer-vision/useObjectDetection.md @@ -2,8 +2,7 @@ title: useObjectDetection --- -Object detection is a computer vision technique that identifies and locates objects within images or video. It’s commonly used in applications like image recognition, video surveillance or autonomous driving. -`useObjectDetection` is a hook that allows you to seamlessly integrate object detection into your React Native applications. +Object detection is a computer vision technique that identifies and locates objects within images. Unlike image classification, which assigns a single label to the whole image, object detection returns a list of detected objects — each with a bounding box, a class label, and a confidence score. React Native ExecuTorch offers a dedicated hook `useObjectDetection` for this task. :::warning It is recommended to use models provided by us, which are available at our [Hugging Face repository](https://huggingface.co/collections/software-mansion/object-detection-68d0ea936cd0906843cbba7d). You can also use [constants](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts) shipped with our library. @@ -16,22 +15,23 @@ It is recommended to use models provided by us, which are available at our [Hugg ## High Level Overview -```tsx +```typescript import { useObjectDetection, SSDLITE_320_MOBILENET_V3_LARGE, } from 'react-native-executorch'; -function App() { - const ssdlite = useObjectDetection({ model: SSDLITE_320_MOBILENET_V3_LARGE }); +const model = useObjectDetection({ + model: SSDLITE_320_MOBILENET_V3_LARGE, +}); - // ... - for (const detection of await ssdlite.forward('https://url-to-image.jpg')) { - console.log('Bounding box: ', detection.bbox); - console.log('Bounding label: ', detection.label); - console.log('Bounding score: ', detection.score); - } - // ... +const imageUri = 'file:///Users/.../photo.jpg'; + +try { + const detections = await model.forward(imageUri); + // detections is an array of Detection objects +} catch (error) { + console.error(error); } ``` @@ -39,9 +39,13 @@ function App() { `useObjectDetection` takes [`ObjectDetectionProps`](../../06-api-reference/interfaces/ObjectDetectionProps.md) that consists of: -- `model` containing [`modelSource`](../../06-api-reference/interfaces/ObjectDetectionProps.md#modelsource). +- `model` - An object containing: + - `modelName` - The name of a built-in model. See [`ObjectDetectionModelSources`](../../06-api-reference/interfaces/ObjectDetectionProps.md) for the list of supported models. + - `modelSource` - The location of the model binary (a URL or a bundled resource). - An optional flag [`preventLoad`](../../06-api-reference/interfaces/ObjectDetectionProps.md#preventload) which prevents auto-loading of the model. +The hook is generic over the model config — TypeScript automatically infers the correct label type based on the `modelName` you provide. No explicit generic parameter is needed. + You need more details? Check the following resources: - For detailed information about `useObjectDetection` arguments check this section: [`useObjectDetection` arguments](../../06-api-reference/functions/useObjectDetection.md#parameters). @@ -50,54 +54,56 @@ You need more details? Check the following resources: ### Returns -`useObjectDetection` returns an object called `ObjectDetectionType` containing bunch of functions to interact with object detection models. To get more details please read: [`ObjectDetectionType` API Reference](../../06-api-reference/interfaces/ObjectDetectionType.md). +`useObjectDetection` returns an [`ObjectDetectionType`](../../06-api-reference/interfaces/ObjectDetectionType.md) object containing: -## Running the model - -To run the model, you can use the [`forward`](../../06-api-reference/interfaces/ObjectDetectionType.md#forward) method. It accepts one argument, which is the image. The image can be a remote URL, a local file URI, or a base64-encoded image (whole URI or only raw base64). The function returns an array of [`Detection`](../../06-api-reference/interfaces/Detection.md) objects. Each object contains coordinates of the bounding box, the label of the detected object, and the confidence score. For more information, please refer to the reference or type definitions. +- `isReady` - Whether the model is loaded and ready to process images. +- `isGenerating` - Whether the model is currently processing an image. +- `error` - An error object if the model failed to load or encountered a runtime error. +- `downloadProgress` - A value between 0 and 1 representing the download progress of the model binary. +- `forward` - A function to run inference on an image. -## Detection object +## Running the model -The detection object is specified as follows: +To run the model, use the [`forward`](../../06-api-reference/interfaces/ObjectDetectionType.md#forward) method. It accepts two arguments: -```typescript -interface Bbox { - x1: number; - y1: number; - x2: number; - y2: number; -} +- `imageSource` (required) - The image to process. Can be a remote URL, a local file URI, or a base64-encoded image (whole URI or only raw base64). +- `detectionThreshold` (optional) - A number between 0 and 1 representing the minimum confidence score for a detection to be included in the results. Defaults to `0.7`. -interface Detection { - bbox: Bbox; - label: keyof typeof CocoLabels; - score: number; -} -``` +`forward` returns a promise resolving to an array of [`Detection`](../../06-api-reference/interfaces/Detection.md) objects, each containing: -The `bbox` property contains information about the bounding box of detected objects. It is represented as two points: one at the bottom-left corner of the bounding box (`x1`, `y1`) and the other at the top-right corner (`x2`, `y2`). -The `label` property contains the name of the detected object, which corresponds to one of the [`CocoLabels`](../../06-api-reference/enumerations/CocoLabel.md). The `score` represents the confidence score of the detected object. +- `bbox` - A [`Bbox`](../../06-api-reference/interfaces/Bbox.md) object with `x1`, `y1` (top-left corner) and `x2`, `y2` (bottom-right corner) coordinates in the original image's pixel space. +- `label` - The class name of the detected object, typed to the label map of the chosen model. +- `score` - The confidence score of the detection, between 0 and 1. ## Example -```tsx -import { - useObjectDetection, - SSDLITE_320_MOBILENET_V3_LARGE, -} from 'react-native-executorch'; +```typescript +import { useObjectDetection, RF_DETR_NANO } from 'react-native-executorch'; function App() { - const ssdlite = useObjectDetection({ model: SSDLITE_320_MOBILENET_V3_LARGE }); + const model = useObjectDetection({ + model: RF_DETR_NANO, + }); + + const handleDetect = async () => { + if (!model.isReady) return; + + const imageUri = 'file:///Users/.../photo.jpg'; - const runModel = async () => { - const detections = await ssdlite.forward('https://url-to-image.jpg'); + try { + const detections = await model.forward(imageUri, 0.5); - for (const detection of detections) { - console.log('Bounding box: ', detection.bbox); - console.log('Bounding label: ', detection.label); - console.log('Bounding score: ', detection.score); + for (const detection of detections) { + console.log('Label:', detection.label); + console.log('Score:', detection.score); + console.log('Bounding box:', detection.bbox); + } + } catch (error) { + console.error(error); } }; + + // ... } ``` @@ -106,3 +112,4 @@ function App() { | Model | Number of classes | Class list | | ----------------------------------------------------------------------------------------------------------------------------- | ----------------- | -------------------------------------------------------- | | [SSDLite320 MobileNetV3 Large](https://huggingface.co/software-mansion/react-native-executorch-ssdlite320-mobilenet-v3-large) | 91 | [COCO](../../06-api-reference/enumerations/CocoLabel.md) | +| [RF-DETR Nano](https://huggingface.co/software-mansion/react-native-executorch-rf-detr-nano) | 80 | [COCO](../../06-api-reference/enumerations/CocoLabel.md) | diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp index 8eccb6ac5..37b4114b2 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp @@ -36,11 +36,17 @@ ObjectDetection::ObjectDetection( const std::string &modelSource, std::vector normMean, std::vector normStd, std::shared_ptr callInvoker) : ObjectDetection(modelSource, callInvoker) { - if (normMean.size() >= 3) { - normMean_ = cv::Scalar(normMean[0], normMean[1], normMean[2]); + if (normMean.size() == 3) { + normMean_ = std::move(normMean); + } else if (!normMean.empty()) { + log(LOG_LEVEL::Warn, + "normMean must have 3 elements — ignoring provided value."); } - if (normStd.size() >= 3) { - normStd_ = cv::Scalar(normStd[0], normStd[1], normStd[2]); + if (normStd.size() == 3) { + normStd_ = std::move(normStd); + } else if (!normStd.empty()) { + log(LOG_LEVEL::Warn, + "normStd must have 3 elements — ignoring provided value."); } } @@ -133,9 +139,11 @@ ObjectDetection::runInference(cv::Mat image, double detectionThreshold, const std::vector tensorDims = getAllInputShapes()[0]; auto inputTensor = - (normMean_ && normStd_) - ? image_processing::getTensorFromMatrix(tensorDims, preprocessed, - *normMean_, *normStd_) + (!normMean_.empty() && !normStd_.empty()) + ? image_processing::getTensorFromMatrix( + tensorDims, preprocessed, + cv::Scalar(normMean_[0], normMean_[1], normMean_[2]), + cv::Scalar(normStd_[0], normStd_[1], normStd_[2])) : image_processing::getTensorFromMatrix(tensorDims, preprocessed); auto forwardResult = BaseModel::forward(inputTensor); diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h index 800764664..9453cd94e 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h @@ -1,7 +1,5 @@ #pragma once -#include - #include #include #include @@ -47,8 +45,8 @@ class ObjectDetection : public VisionModel { const std::vector &labelNames); cv::Size modelImageSize{0, 0}; - std::optional normMean_; - std::optional normStd_; + std::vector normMean_; + std::vector normStd_; }; } // namespace models::object_detection From 87f4be7b27376b6703213707186c0baab58c5148 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Wed, 25 Feb 2026 16:44:21 +0100 Subject: [PATCH 05/24] chore: export cocolabel --- packages/react-native-executorch/src/index.ts | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/react-native-executorch/src/index.ts b/packages/react-native-executorch/src/index.ts index 21c19e10e..7f390e670 100644 --- a/packages/react-native-executorch/src/index.ts +++ b/packages/react-native-executorch/src/index.ts @@ -177,6 +177,7 @@ export * from './types/styleTransfer'; export * from './types/tti'; // constants +export * from './constants/commonVision'; export * from './constants/modelUrls'; export * from './constants/ocr/models'; export * from './constants/tts/models'; From 3a3c598c8dfd48d1e3e225eeea034a38910fb642 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Wed, 25 Feb 2026 17:21:59 +0100 Subject: [PATCH 06/24] chore: add api reference --- .../classes/ObjectDetectionModule.md | 305 +++++++----------- .../enumerations/CocoLabel.md | 182 +++++------ .../functions/useObjectDetection.md | 22 +- docs/docs/06-api-reference/index.md | 44 ++- docs/docs/06-api-reference/interfaces/Bbox.md | 10 +- .../06-api-reference/interfaces/Detection.md | 22 +- .../interfaces/ObjectDetectionProps.md | 24 +- .../interfaces/ObjectDetectionType.md | 102 ++---- .../type-aliases/ObjectDetectionConfig.md | 37 +++ .../type-aliases/ObjectDetectionLabels.md | 15 + .../type-aliases/ObjectDetectionModelName.md | 7 + .../ObjectDetectionModelSources.md | 8 + .../type-aliases/SegmentationLabels.md | 6 +- .../docs/06-api-reference/typedoc-sidebar.cjs | 2 +- .../variables/ALL_MINILM_L6_V2.md | 2 +- .../variables/ALL_MPNET_BASE_V2.md | 2 +- .../variables/BK_SDM_TINY_VPRED_256.md | 2 +- .../variables/BK_SDM_TINY_VPRED_512.md | 2 +- .../variables/CLIP_VIT_BASE_PATCH32_IMAGE.md | 2 +- .../variables/CLIP_VIT_BASE_PATCH32_TEXT.md | 2 +- .../06-api-reference/variables/FSMN_VAD.md | 2 +- .../variables/IMAGENET1K_MEAN.md | 5 + .../variables/IMAGENET1K_STD.md | 5 + .../variables/MULTI_QA_MINILM_L6_COS_V1.md | 2 +- .../variables/MULTI_QA_MPNET_BASE_DOT_V1.md | 2 +- .../variables/RF_DETR_NANO.md | 15 + .../variables/SELFIE_SEGMENTATION.md | 2 +- .../SSDLITE_320_MOBILENET_V3_LARGE.md | 8 +- .../variables/STYLE_TRANSFER_CANDY.md | 2 +- .../variables/STYLE_TRANSFER_MOSAIC.md | 2 +- .../variables/STYLE_TRANSFER_RAIN_PRINCESS.md | 2 +- .../variables/STYLE_TRANSFER_UDNIE.md | 2 +- .../variables/WHISPER_BASE.md | 2 +- .../variables/WHISPER_BASE_EN.md | 2 +- .../variables/WHISPER_SMALL.md | 2 +- .../variables/WHISPER_SMALL_EN.md | 2 +- .../variables/WHISPER_TINY.md | 2 +- .../variables/WHISPER_TINY_EN.md | 2 +- .../variables/WHISPER_TINY_EN_QUANTIZED.md | 2 +- 39 files changed, 414 insertions(+), 447 deletions(-) create mode 100644 docs/docs/06-api-reference/type-aliases/ObjectDetectionConfig.md create mode 100644 docs/docs/06-api-reference/type-aliases/ObjectDetectionLabels.md create mode 100644 docs/docs/06-api-reference/type-aliases/ObjectDetectionModelName.md create mode 100644 docs/docs/06-api-reference/type-aliases/ObjectDetectionModelSources.md create mode 100644 docs/docs/06-api-reference/variables/IMAGENET1K_MEAN.md create mode 100644 docs/docs/06-api-reference/variables/IMAGENET1K_STD.md create mode 100644 docs/docs/06-api-reference/variables/RF_DETR_NANO.md diff --git a/docs/docs/06-api-reference/classes/ObjectDetectionModule.md b/docs/docs/06-api-reference/classes/ObjectDetectionModule.md index fadb2536c..67536491e 100644 --- a/docs/docs/06-api-reference/classes/ObjectDetectionModule.md +++ b/docs/docs/06-api-reference/classes/ObjectDetectionModule.md @@ -1,98 +1,33 @@ -# Class: ObjectDetectionModule +# Class: ObjectDetectionModule\ -Defined in: [modules/computer_vision/ObjectDetectionModule.ts:14](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L14) +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L61) -Module for object detection tasks. +Generic object detection module with type-safe label maps. ## Extends -- `VisionModule`\<[`Detection`](../interfaces/Detection.md)[]\> +- `BaseLabeledModule`\<`ResolveLabels`\<`T`\>\> -## Constructors +## Type Parameters -### Constructor +### T -> **new ObjectDetectionModule**(): `ObjectDetectionModule` +`T` _extends_ [`ObjectDetectionModelName`](../type-aliases/ObjectDetectionModelName.md) \| [`LabelEnum`](../type-aliases/LabelEnum.md) -#### Returns - -`ObjectDetectionModule` - -#### Inherited from - -`VisionModule.constructor` +Either a built-in model name (e.g. `'ssdlite-320-mobilenet-v3-large'`) +or a custom [LabelEnum](../type-aliases/LabelEnum.md) label map. ## Properties -### generateFromFrame() - -> **generateFromFrame**: (`frameData`, ...`args`) => `any` - -Defined in: [modules/BaseModule.ts:56](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L56) - -Process a camera frame directly for real-time inference. - -This method is bound to a native JSI function after calling `load()`, -making it worklet-compatible and safe to call from VisionCamera's -frame processor thread. - -**Performance characteristics:** - -- **Zero-copy path**: When using `frame.getNativeBuffer()` from VisionCamera v5, - frame data is accessed directly without copying (fastest, recommended). -- **Copy path**: When using `frame.toArrayBuffer()`, pixel data is copied - from native to JS, then accessed from native code (slower, fallback). - -**Usage with VisionCamera:** - -```typescript -const frameOutput = useFrameOutput({ - pixelFormat: 'rgb', - onFrame(frame) { - 'worklet'; - // Zero-copy approach (recommended) - const nativeBuffer = frame.getNativeBuffer(); - const result = model.generateFromFrame( - { - nativeBuffer: nativeBuffer.pointer, - width: frame.width, - height: frame.height, - }, - ...args - ); - nativeBuffer.release(); - frame.dispose(); - }, -}); -``` - -#### Parameters - -##### frameData - -[`Frame`](../interfaces/Frame.md) - -Frame data object with either nativeBuffer (zero-copy) or data (ArrayBuffer) - -##### args - -...`any`[] - -Additional model-specific arguments (e.g., threshold, options) - -#### Returns - -`any` +### labelMap -Model-specific output (e.g., detections, classifications, embeddings) +> `protected` `readonly` **labelMap**: `ResolveLabels` -#### See - -[Frame](../interfaces/Frame.md) for frame data format details +Defined in: [modules/BaseLabeledModule.ts:52](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseLabeledModule.ts#L52) #### Inherited from -`VisionModule.generateFromFrame` +`BaseLabeledModule.labelMap` --- @@ -100,61 +35,13 @@ Model-specific output (e.g., detections, classifications, embeddings) > **nativeModule**: `any` = `null` -Defined in: [modules/BaseModule.ts:17](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L17) - -**`Internal`** - -Native module instance (JSI Host Object) - -#### Inherited from - -`VisionModule.nativeModule` - -## Accessors - -### runOnFrame - -#### Get Signature - -> **get** **runOnFrame**(): (`frame`, ...`args`) => `TOutput` \| `null` - -Defined in: [modules/computer_vision/VisionModule.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/VisionModule.ts#L61) - -Synchronous worklet function for real-time VisionCamera frame processing. +Defined in: [modules/BaseModule.ts:8](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L8) -Only available after the model is loaded. Returns null if not loaded. - -**Use this for VisionCamera frame processing in worklets.** -For async processing, use `forward()` instead. - -##### Example - -```typescript -const model = new ClassificationModule(); -await model.load({ modelSource: MODEL }); - -// Use the functional form of setState to store the worklet — passing it -// directly would cause React to invoke it immediately as an updater fn. -const [runOnFrame, setRunOnFrame] = useState(null); -setRunOnFrame(() => model.runOnFrame); - -const frameOutput = useFrameOutput({ - onFrame(frame) { - 'worklet'; - if (!runOnFrame) return; - const result = runOnFrame(frame); - frame.dispose(); - }, -}); -``` - -##### Returns - -(`frame`, ...`args`) => `TOutput` \| `null` +Native module instance #### Inherited from -`VisionModule.runOnFrame` +`BaseLabeledModule.nativeModule` ## Methods @@ -162,11 +49,9 @@ const frameOutput = useFrameOutput({ > **delete**(): `void` -Defined in: [modules/BaseModule.ts:100](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L100) - -Unloads the model from memory and releases native resources. +Defined in: [modules/BaseModule.ts:41](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L41) -Always call this method when you're done with a model to prevent memory leaks. +Unloads the model from memory. #### Returns @@ -174,70 +59,41 @@ Always call this method when you're done with a model to prevent memory leaks. #### Inherited from -`VisionModule.delete` +`BaseLabeledModule.delete` --- ### forward() -> **forward**(`input`, `detectionThreshold?`): `Promise`\<[`Detection`](../interfaces/Detection.md)[]\> - -Defined in: [modules/computer_vision/ObjectDetectionModule.ts:46](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L46) +> **forward**(`imageSource`, `detectionThreshold`): `Promise`\<[`Detection`](../interfaces/Detection.md)\<`ResolveLabels`\<`T`, \{ `rf-detr-nano`: \{ `labelMap`: _typeof_ [`CocoLabel`](../enumerations/CocoLabel.md); `preprocessorConfig`: \{ `normMean`: [`Triple`](../type-aliases/Triple.md)\<`number`\>; `normStd`: [`Triple`](../type-aliases/Triple.md)\<`number`\>; \}; \}; `ssdlite-320-mobilenet-v3-large`: \{ `labelMap`: _typeof_ [`CocoLabel`](../enumerations/CocoLabel.md); `preprocessorConfig`: `undefined`; \}; \}\>\>[]\> -Executes the model's forward pass with automatic input type detection. +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:145](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L145) -Supports two input types: - -1. **String path/URI**: File path, URL, or Base64-encoded string -2. **PixelData**: Raw pixel data from image libraries (e.g., NitroImage) - -**Note**: For VisionCamera frame processing, use `runOnFrame` instead. -This method is async and cannot be called in worklet context. +Executes the model's forward pass to detect objects within the provided image. #### Parameters -##### input - -Image source (string path or PixelData object) +##### imageSource -`string` | [`PixelData`](../interfaces/PixelData.md) - -##### detectionThreshold? - -`number` = `0.5` +`string` -#### Returns +A string representing the image source (e.g., a file path, URI, or base64 string). -`Promise`\<[`Detection`](../interfaces/Detection.md)[]\> +##### detectionThreshold -A Promise that resolves to the model output. +`number` = `0.7` -#### Example +Minimum confidence score for a detection to be included. Default is 0.7. -```typescript -// String path (async) -const result1 = await model.forward('file:///path/to/image.jpg'); +#### Returns -// Pixel data (async) -const result2 = await model.forward({ - dataPtr: new Uint8Array(pixelBuffer), - sizes: [480, 640, 3], - scalarType: ScalarType.BYTE, -}); +`Promise`\<[`Detection`](../interfaces/Detection.md)\<`ResolveLabels`\<`T`, \{ `rf-detr-nano`: \{ `labelMap`: _typeof_ [`CocoLabel`](../enumerations/CocoLabel.md); `preprocessorConfig`: \{ `normMean`: [`Triple`](../type-aliases/Triple.md)\<`number`\>; `normStd`: [`Triple`](../type-aliases/Triple.md)\<`number`\>; \}; \}; `ssdlite-320-mobilenet-v3-large`: \{ `labelMap`: _typeof_ [`CocoLabel`](../enumerations/CocoLabel.md); `preprocessorConfig`: `undefined`; \}; \}\>\>[]\> -// For VisionCamera frames, use runOnFrame in worklet: -const frameOutput = useFrameOutput({ - onFrame(frame) { - 'worklet'; - if (!model.runOnFrame) return; - const result = model.runOnFrame(frame); - }, -}); -``` +A Promise resolving to an array of [Detection](../interfaces/Detection.md) objects. -#### Overrides +#### Throws -`VisionModule.forward` +If the model is not loaded. --- @@ -245,9 +101,7 @@ const frameOutput = useFrameOutput({ > `protected` **forwardET**(`inputTensor`): `Promise`\<[`TensorPtr`](../interfaces/TensorPtr.md)[]\> -Defined in: [modules/BaseModule.ts:80](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L80) - -**`Internal`** +Defined in: [modules/BaseModule.ts:23](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L23) Runs the model's forward method with the given input tensors. It returns the output tensors that mimic the structure of output from ExecuTorch. @@ -268,7 +122,7 @@ Array of output tensors. #### Inherited from -`VisionModule.forwardET` +`BaseLabeledModule.forwardET` --- @@ -276,7 +130,7 @@ Array of output tensors. > **getInputShape**(`methodName`, `index`): `Promise`\<`number`[]\> -Defined in: [modules/BaseModule.ts:91](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L91) +Defined in: [modules/BaseModule.ts:34](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L34) Gets the input shape for a given method and index. @@ -302,39 +156,98 @@ The input shape as an array of numbers. #### Inherited from -`VisionModule.getInputShape` +`BaseLabeledModule.getInputShape` --- ### load() -> **load**(`model`, `onDownloadProgressCallback?`): `Promise`\<`void`\> +> **load**(): `Promise`\<`void`\> -Defined in: [modules/computer_vision/ObjectDetectionModule.ts:22](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L22) +Defined in: [modules/BaseLabeledModule.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseLabeledModule.ts#L61) -Loads the model, where `modelSource` is a string that specifies the location of the model binary. -To track the download progress, supply a callback function `onDownloadProgressCallback`. +#### Returns -#### Parameters +`Promise`\<`void`\> + +#### Inherited from + +`BaseLabeledModule.load` -##### model +--- + +### fromCustomConfig() + +> `static` **fromCustomConfig**\<`L`\>(`modelSource`, `config`, `onDownloadProgress`): `Promise`\<`ObjectDetectionModule`\<`L`\>\> + +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:118](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L118) + +Creates an object detection instance with a user-provided label map and custom config. + +#### Type Parameters -Object containing `modelSource`. +##### L -###### modelSource +`L` _extends_ `Readonly`\<`Record`\<`string`, `string` \| `number`\>\> + +#### Parameters + +##### modelSource [`ResourceSource`](../type-aliases/ResourceSource.md) -##### onDownloadProgressCallback? +A fetchable resource pointing to the model binary. + +##### config + +[`ObjectDetectionConfig`](../type-aliases/ObjectDetectionConfig.md)\<`L`\> + +A [ObjectDetectionConfig](../type-aliases/ObjectDetectionConfig.md) object with the label map. + +##### onDownloadProgress (`progress`) => `void` -Optional callback to monitor download progress. +Optional callback to monitor download progress, receiving a value between 0 and 1. #### Returns -`Promise`\<`void`\> +`Promise`\<`ObjectDetectionModule`\<`L`\>\> + +A Promise resolving to an `ObjectDetectionModule` instance typed to the provided label map. + +--- + +### fromModelName() + +> `static` **fromModelName**\<`C`\>(`config`, `onDownloadProgress`): `Promise`\<`ObjectDetectionModule`\<`ModelNameOf`\<`C`\>\>\> + +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:88](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L88) + +Creates an object detection instance for a built-in model. + +#### Type Parameters + +##### C + +`C` _extends_ [`ObjectDetectionModelSources`](../type-aliases/ObjectDetectionModelSources.md) + +#### Parameters + +##### config + +`C` + +A [ObjectDetectionModelSources](../type-aliases/ObjectDetectionModelSources.md) object specifying which model to load and where to fetch it from. + +##### onDownloadProgress + +(`progress`) => `void` + +Optional callback to monitor download progress, receiving a value between 0 and 1. + +#### Returns -#### Overrides +`Promise`\<`ObjectDetectionModule`\<`ModelNameOf`\<`C`\>\>\> -`VisionModule.load` +A Promise resolving to an `ObjectDetectionModule` instance typed to the chosen model's label map. diff --git a/docs/docs/06-api-reference/enumerations/CocoLabel.md b/docs/docs/06-api-reference/enumerations/CocoLabel.md index b94e3e9c6..8a15fbe25 100644 --- a/docs/docs/06-api-reference/enumerations/CocoLabel.md +++ b/docs/docs/06-api-reference/enumerations/CocoLabel.md @@ -1,6 +1,6 @@ # Enumeration: CocoLabel -Defined in: [types/objectDetection.ts:39](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L39) +Defined in: [constants/commonVision.ts:11](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L11) COCO dataset class labels used for object detection. @@ -10,7 +10,7 @@ COCO dataset class labels used for object detection. > **AIRPLANE**: `5` -Defined in: [types/objectDetection.ts:44](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L44) +Defined in: [constants/commonVision.ts:16](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L16) --- @@ -18,7 +18,7 @@ Defined in: [types/objectDetection.ts:44](https://github.com/software-mansion/re > **APPLE**: `53` -Defined in: [types/objectDetection.ts:91](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L91) +Defined in: [constants/commonVision.ts:63](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L63) --- @@ -26,7 +26,7 @@ Defined in: [types/objectDetection.ts:91](https://github.com/software-mansion/re > **BACKPACK**: `27` -Defined in: [types/objectDetection.ts:66](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L66) +Defined in: [constants/commonVision.ts:38](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L38) --- @@ -34,7 +34,7 @@ Defined in: [types/objectDetection.ts:66](https://github.com/software-mansion/re > **BANANA**: `52` -Defined in: [types/objectDetection.ts:90](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L90) +Defined in: [constants/commonVision.ts:62](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L62) --- @@ -42,7 +42,7 @@ Defined in: [types/objectDetection.ts:90](https://github.com/software-mansion/re > **BASEBALL**: `39` -Defined in: [types/objectDetection.ts:78](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L78) +Defined in: [constants/commonVision.ts:50](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L50) --- @@ -50,7 +50,7 @@ Defined in: [types/objectDetection.ts:78](https://github.com/software-mansion/re > **BEAR**: `23` -Defined in: [types/objectDetection.ts:62](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L62) +Defined in: [constants/commonVision.ts:34](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L34) --- @@ -58,7 +58,7 @@ Defined in: [types/objectDetection.ts:62](https://github.com/software-mansion/re > **BED**: `65` -Defined in: [types/objectDetection.ts:103](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L103) +Defined in: [constants/commonVision.ts:75](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L75) --- @@ -66,7 +66,7 @@ Defined in: [types/objectDetection.ts:103](https://github.com/software-mansion/r > **BENCH**: `15` -Defined in: [types/objectDetection.ts:54](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L54) +Defined in: [constants/commonVision.ts:26](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L26) --- @@ -74,7 +74,7 @@ Defined in: [types/objectDetection.ts:54](https://github.com/software-mansion/re > **BICYCLE**: `2` -Defined in: [types/objectDetection.ts:41](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L41) +Defined in: [constants/commonVision.ts:13](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L13) --- @@ -82,7 +82,7 @@ Defined in: [types/objectDetection.ts:41](https://github.com/software-mansion/re > **BIRD**: `16` -Defined in: [types/objectDetection.ts:55](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L55) +Defined in: [constants/commonVision.ts:27](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L27) --- @@ -90,7 +90,7 @@ Defined in: [types/objectDetection.ts:55](https://github.com/software-mansion/re > **BLENDER**: `83` -Defined in: [types/objectDetection.ts:121](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L121) +Defined in: [constants/commonVision.ts:93](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L93) --- @@ -98,7 +98,7 @@ Defined in: [types/objectDetection.ts:121](https://github.com/software-mansion/r > **BOAT**: `9` -Defined in: [types/objectDetection.ts:48](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L48) +Defined in: [constants/commonVision.ts:20](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L20) --- @@ -106,7 +106,7 @@ Defined in: [types/objectDetection.ts:48](https://github.com/software-mansion/re > **BOOK**: `84` -Defined in: [types/objectDetection.ts:122](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L122) +Defined in: [constants/commonVision.ts:94](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L94) --- @@ -114,7 +114,7 @@ Defined in: [types/objectDetection.ts:122](https://github.com/software-mansion/r > **BOTTLE**: `44` -Defined in: [types/objectDetection.ts:82](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L82) +Defined in: [constants/commonVision.ts:54](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L54) --- @@ -122,7 +122,7 @@ Defined in: [types/objectDetection.ts:82](https://github.com/software-mansion/re > **BOWL**: `51` -Defined in: [types/objectDetection.ts:89](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L89) +Defined in: [constants/commonVision.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L61) --- @@ -130,7 +130,7 @@ Defined in: [types/objectDetection.ts:89](https://github.com/software-mansion/re > **BROCCOLI**: `56` -Defined in: [types/objectDetection.ts:94](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L94) +Defined in: [constants/commonVision.ts:66](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L66) --- @@ -138,7 +138,7 @@ Defined in: [types/objectDetection.ts:94](https://github.com/software-mansion/re > **BUS**: `6` -Defined in: [types/objectDetection.ts:45](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L45) +Defined in: [constants/commonVision.ts:17](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L17) --- @@ -146,7 +146,7 @@ Defined in: [types/objectDetection.ts:45](https://github.com/software-mansion/re > **CAKE**: `61` -Defined in: [types/objectDetection.ts:99](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L99) +Defined in: [constants/commonVision.ts:71](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L71) --- @@ -154,7 +154,7 @@ Defined in: [types/objectDetection.ts:99](https://github.com/software-mansion/re > **CAR**: `3` -Defined in: [types/objectDetection.ts:42](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L42) +Defined in: [constants/commonVision.ts:14](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L14) --- @@ -162,7 +162,7 @@ Defined in: [types/objectDetection.ts:42](https://github.com/software-mansion/re > **CARROT**: `57` -Defined in: [types/objectDetection.ts:95](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L95) +Defined in: [constants/commonVision.ts:67](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L67) --- @@ -170,7 +170,7 @@ Defined in: [types/objectDetection.ts:95](https://github.com/software-mansion/re > **CAT**: `17` -Defined in: [types/objectDetection.ts:56](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L56) +Defined in: [constants/commonVision.ts:28](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L28) --- @@ -178,7 +178,7 @@ Defined in: [types/objectDetection.ts:56](https://github.com/software-mansion/re > **CELL_PHONE**: `77` -Defined in: [types/objectDetection.ts:115](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L115) +Defined in: [constants/commonVision.ts:87](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L87) --- @@ -186,7 +186,7 @@ Defined in: [types/objectDetection.ts:115](https://github.com/software-mansion/r > **CHAIR**: `62` -Defined in: [types/objectDetection.ts:100](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L100) +Defined in: [constants/commonVision.ts:72](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L72) --- @@ -194,7 +194,7 @@ Defined in: [types/objectDetection.ts:100](https://github.com/software-mansion/r > **CLOCK**: `85` -Defined in: [types/objectDetection.ts:123](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L123) +Defined in: [constants/commonVision.ts:95](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L95) --- @@ -202,7 +202,7 @@ Defined in: [types/objectDetection.ts:123](https://github.com/software-mansion/r > **COUCH**: `63` -Defined in: [types/objectDetection.ts:101](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L101) +Defined in: [constants/commonVision.ts:73](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L73) --- @@ -210,7 +210,7 @@ Defined in: [types/objectDetection.ts:101](https://github.com/software-mansion/r > **COW**: `21` -Defined in: [types/objectDetection.ts:60](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L60) +Defined in: [constants/commonVision.ts:32](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L32) --- @@ -218,7 +218,7 @@ Defined in: [types/objectDetection.ts:60](https://github.com/software-mansion/re > **CUP**: `47` -Defined in: [types/objectDetection.ts:85](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L85) +Defined in: [constants/commonVision.ts:57](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L57) --- @@ -226,7 +226,7 @@ Defined in: [types/objectDetection.ts:85](https://github.com/software-mansion/re > **DESK**: `69` -Defined in: [types/objectDetection.ts:107](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L107) +Defined in: [constants/commonVision.ts:79](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L79) --- @@ -234,7 +234,7 @@ Defined in: [types/objectDetection.ts:107](https://github.com/software-mansion/r > **DINING_TABLE**: `67` -Defined in: [types/objectDetection.ts:105](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L105) +Defined in: [constants/commonVision.ts:77](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L77) --- @@ -242,7 +242,7 @@ Defined in: [types/objectDetection.ts:105](https://github.com/software-mansion/r > **DOG**: `18` -Defined in: [types/objectDetection.ts:57](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L57) +Defined in: [constants/commonVision.ts:29](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L29) --- @@ -250,7 +250,7 @@ Defined in: [types/objectDetection.ts:57](https://github.com/software-mansion/re > **DONUT**: `60` -Defined in: [types/objectDetection.ts:98](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L98) +Defined in: [constants/commonVision.ts:70](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L70) --- @@ -258,7 +258,7 @@ Defined in: [types/objectDetection.ts:98](https://github.com/software-mansion/re > **DOOR**: `71` -Defined in: [types/objectDetection.ts:109](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L109) +Defined in: [constants/commonVision.ts:81](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L81) --- @@ -266,7 +266,7 @@ Defined in: [types/objectDetection.ts:109](https://github.com/software-mansion/r > **ELEPHANT**: `22` -Defined in: [types/objectDetection.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L61) +Defined in: [constants/commonVision.ts:33](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L33) --- @@ -274,7 +274,7 @@ Defined in: [types/objectDetection.ts:61](https://github.com/software-mansion/re > **EYE**: `30` -Defined in: [types/objectDetection.ts:69](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L69) +Defined in: [constants/commonVision.ts:41](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L41) --- @@ -282,7 +282,7 @@ Defined in: [types/objectDetection.ts:69](https://github.com/software-mansion/re > **FIRE_HYDRANT**: `11` -Defined in: [types/objectDetection.ts:50](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L50) +Defined in: [constants/commonVision.ts:22](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L22) --- @@ -290,7 +290,7 @@ Defined in: [types/objectDetection.ts:50](https://github.com/software-mansion/re > **FORK**: `48` -Defined in: [types/objectDetection.ts:86](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L86) +Defined in: [constants/commonVision.ts:58](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L58) --- @@ -298,7 +298,7 @@ Defined in: [types/objectDetection.ts:86](https://github.com/software-mansion/re > **FRISBEE**: `34` -Defined in: [types/objectDetection.ts:73](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L73) +Defined in: [constants/commonVision.ts:45](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L45) --- @@ -306,7 +306,7 @@ Defined in: [types/objectDetection.ts:73](https://github.com/software-mansion/re > **GIRAFFE**: `25` -Defined in: [types/objectDetection.ts:64](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L64) +Defined in: [constants/commonVision.ts:36](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L36) --- @@ -314,7 +314,7 @@ Defined in: [types/objectDetection.ts:64](https://github.com/software-mansion/re > **HAIR_BRUSH**: `91` -Defined in: [types/objectDetection.ts:129](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L129) +Defined in: [constants/commonVision.ts:101](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L101) --- @@ -322,7 +322,7 @@ Defined in: [types/objectDetection.ts:129](https://github.com/software-mansion/r > **HAIR_DRIER**: `89` -Defined in: [types/objectDetection.ts:127](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L127) +Defined in: [constants/commonVision.ts:99](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L99) --- @@ -330,7 +330,7 @@ Defined in: [types/objectDetection.ts:127](https://github.com/software-mansion/r > **HANDBAG**: `31` -Defined in: [types/objectDetection.ts:70](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L70) +Defined in: [constants/commonVision.ts:42](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L42) --- @@ -338,7 +338,7 @@ Defined in: [types/objectDetection.ts:70](https://github.com/software-mansion/re > **HAT**: `26` -Defined in: [types/objectDetection.ts:65](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L65) +Defined in: [constants/commonVision.ts:37](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L37) --- @@ -346,7 +346,7 @@ Defined in: [types/objectDetection.ts:65](https://github.com/software-mansion/re > **HORSE**: `19` -Defined in: [types/objectDetection.ts:58](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L58) +Defined in: [constants/commonVision.ts:30](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L30) --- @@ -354,7 +354,7 @@ Defined in: [types/objectDetection.ts:58](https://github.com/software-mansion/re > **HOT_DOG**: `58` -Defined in: [types/objectDetection.ts:96](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L96) +Defined in: [constants/commonVision.ts:68](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L68) --- @@ -362,7 +362,7 @@ Defined in: [types/objectDetection.ts:96](https://github.com/software-mansion/re > **KEYBOARD**: `76` -Defined in: [types/objectDetection.ts:114](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L114) +Defined in: [constants/commonVision.ts:86](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L86) --- @@ -370,7 +370,7 @@ Defined in: [types/objectDetection.ts:114](https://github.com/software-mansion/r > **KITE**: `38` -Defined in: [types/objectDetection.ts:77](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L77) +Defined in: [constants/commonVision.ts:49](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L49) --- @@ -378,7 +378,7 @@ Defined in: [types/objectDetection.ts:77](https://github.com/software-mansion/re > **KNIFE**: `49` -Defined in: [types/objectDetection.ts:87](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L87) +Defined in: [constants/commonVision.ts:59](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L59) --- @@ -386,7 +386,7 @@ Defined in: [types/objectDetection.ts:87](https://github.com/software-mansion/re > **LAPTOP**: `73` -Defined in: [types/objectDetection.ts:111](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L111) +Defined in: [constants/commonVision.ts:83](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L83) --- @@ -394,7 +394,7 @@ Defined in: [types/objectDetection.ts:111](https://github.com/software-mansion/r > **MICROWAVE**: `78` -Defined in: [types/objectDetection.ts:116](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L116) +Defined in: [constants/commonVision.ts:88](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L88) --- @@ -402,7 +402,7 @@ Defined in: [types/objectDetection.ts:116](https://github.com/software-mansion/r > **MIRROR**: `66` -Defined in: [types/objectDetection.ts:104](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L104) +Defined in: [constants/commonVision.ts:76](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L76) --- @@ -410,7 +410,7 @@ Defined in: [types/objectDetection.ts:104](https://github.com/software-mansion/r > **MOTORCYCLE**: `4` -Defined in: [types/objectDetection.ts:43](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L43) +Defined in: [constants/commonVision.ts:15](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L15) --- @@ -418,7 +418,7 @@ Defined in: [types/objectDetection.ts:43](https://github.com/software-mansion/re > **MOUSE**: `74` -Defined in: [types/objectDetection.ts:112](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L112) +Defined in: [constants/commonVision.ts:84](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L84) --- @@ -426,7 +426,7 @@ Defined in: [types/objectDetection.ts:112](https://github.com/software-mansion/r > **ORANGE**: `55` -Defined in: [types/objectDetection.ts:93](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L93) +Defined in: [constants/commonVision.ts:65](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L65) --- @@ -434,7 +434,7 @@ Defined in: [types/objectDetection.ts:93](https://github.com/software-mansion/re > **OVEN**: `79` -Defined in: [types/objectDetection.ts:117](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L117) +Defined in: [constants/commonVision.ts:89](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L89) --- @@ -442,7 +442,7 @@ Defined in: [types/objectDetection.ts:117](https://github.com/software-mansion/r > **PARKING**: `14` -Defined in: [types/objectDetection.ts:53](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L53) +Defined in: [constants/commonVision.ts:25](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L25) --- @@ -450,7 +450,7 @@ Defined in: [types/objectDetection.ts:53](https://github.com/software-mansion/re > **PERSON**: `1` -Defined in: [types/objectDetection.ts:40](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L40) +Defined in: [constants/commonVision.ts:12](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L12) --- @@ -458,7 +458,7 @@ Defined in: [types/objectDetection.ts:40](https://github.com/software-mansion/re > **PIZZA**: `59` -Defined in: [types/objectDetection.ts:97](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L97) +Defined in: [constants/commonVision.ts:69](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L69) --- @@ -466,7 +466,7 @@ Defined in: [types/objectDetection.ts:97](https://github.com/software-mansion/re > **PLATE**: `45` -Defined in: [types/objectDetection.ts:83](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L83) +Defined in: [constants/commonVision.ts:55](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L55) --- @@ -474,7 +474,7 @@ Defined in: [types/objectDetection.ts:83](https://github.com/software-mansion/re > **POTTED_PLANT**: `64` -Defined in: [types/objectDetection.ts:102](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L102) +Defined in: [constants/commonVision.ts:74](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L74) --- @@ -482,7 +482,7 @@ Defined in: [types/objectDetection.ts:102](https://github.com/software-mansion/r > **REFRIGERATOR**: `82` -Defined in: [types/objectDetection.ts:120](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L120) +Defined in: [constants/commonVision.ts:92](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L92) --- @@ -490,7 +490,7 @@ Defined in: [types/objectDetection.ts:120](https://github.com/software-mansion/r > **REMOTE**: `75` -Defined in: [types/objectDetection.ts:113](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L113) +Defined in: [constants/commonVision.ts:85](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L85) --- @@ -498,7 +498,7 @@ Defined in: [types/objectDetection.ts:113](https://github.com/software-mansion/r > **SANDWICH**: `54` -Defined in: [types/objectDetection.ts:92](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L92) +Defined in: [constants/commonVision.ts:64](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L64) --- @@ -506,7 +506,7 @@ Defined in: [types/objectDetection.ts:92](https://github.com/software-mansion/re > **SCISSORS**: `87` -Defined in: [types/objectDetection.ts:125](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L125) +Defined in: [constants/commonVision.ts:97](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L97) --- @@ -514,7 +514,7 @@ Defined in: [types/objectDetection.ts:125](https://github.com/software-mansion/r > **SHEEP**: `20` -Defined in: [types/objectDetection.ts:59](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L59) +Defined in: [constants/commonVision.ts:31](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L31) --- @@ -522,7 +522,7 @@ Defined in: [types/objectDetection.ts:59](https://github.com/software-mansion/re > **SHOE**: `29` -Defined in: [types/objectDetection.ts:68](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L68) +Defined in: [constants/commonVision.ts:40](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L40) --- @@ -530,7 +530,7 @@ Defined in: [types/objectDetection.ts:68](https://github.com/software-mansion/re > **SINK**: `81` -Defined in: [types/objectDetection.ts:119](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L119) +Defined in: [constants/commonVision.ts:91](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L91) --- @@ -538,7 +538,7 @@ Defined in: [types/objectDetection.ts:119](https://github.com/software-mansion/r > **SKATEBOARD**: `41` -Defined in: [types/objectDetection.ts:79](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L79) +Defined in: [constants/commonVision.ts:51](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L51) --- @@ -546,7 +546,7 @@ Defined in: [types/objectDetection.ts:79](https://github.com/software-mansion/re > **SKIS**: `35` -Defined in: [types/objectDetection.ts:74](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L74) +Defined in: [constants/commonVision.ts:46](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L46) --- @@ -554,7 +554,7 @@ Defined in: [types/objectDetection.ts:74](https://github.com/software-mansion/re > **SNOWBOARD**: `36` -Defined in: [types/objectDetection.ts:75](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L75) +Defined in: [constants/commonVision.ts:47](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L47) --- @@ -562,7 +562,7 @@ Defined in: [types/objectDetection.ts:75](https://github.com/software-mansion/re > **SPOON**: `50` -Defined in: [types/objectDetection.ts:88](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L88) +Defined in: [constants/commonVision.ts:60](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L60) --- @@ -570,7 +570,7 @@ Defined in: [types/objectDetection.ts:88](https://github.com/software-mansion/re > **SPORTS**: `37` -Defined in: [types/objectDetection.ts:76](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L76) +Defined in: [constants/commonVision.ts:48](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L48) --- @@ -578,7 +578,7 @@ Defined in: [types/objectDetection.ts:76](https://github.com/software-mansion/re > **STOP_SIGN**: `13` -Defined in: [types/objectDetection.ts:52](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L52) +Defined in: [constants/commonVision.ts:24](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L24) --- @@ -586,7 +586,7 @@ Defined in: [types/objectDetection.ts:52](https://github.com/software-mansion/re > **STREET_SIGN**: `12` -Defined in: [types/objectDetection.ts:51](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L51) +Defined in: [constants/commonVision.ts:23](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L23) --- @@ -594,7 +594,7 @@ Defined in: [types/objectDetection.ts:51](https://github.com/software-mansion/re > **SUITCASE**: `33` -Defined in: [types/objectDetection.ts:72](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L72) +Defined in: [constants/commonVision.ts:44](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L44) --- @@ -602,7 +602,7 @@ Defined in: [types/objectDetection.ts:72](https://github.com/software-mansion/re > **SURFBOARD**: `42` -Defined in: [types/objectDetection.ts:80](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L80) +Defined in: [constants/commonVision.ts:52](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L52) --- @@ -610,7 +610,7 @@ Defined in: [types/objectDetection.ts:80](https://github.com/software-mansion/re > **TEDDY_BEAR**: `88` -Defined in: [types/objectDetection.ts:126](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L126) +Defined in: [constants/commonVision.ts:98](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L98) --- @@ -618,7 +618,7 @@ Defined in: [types/objectDetection.ts:126](https://github.com/software-mansion/r > **TENNIS_RACKET**: `43` -Defined in: [types/objectDetection.ts:81](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L81) +Defined in: [constants/commonVision.ts:53](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L53) --- @@ -626,7 +626,7 @@ Defined in: [types/objectDetection.ts:81](https://github.com/software-mansion/re > **TIE**: `32` -Defined in: [types/objectDetection.ts:71](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L71) +Defined in: [constants/commonVision.ts:43](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L43) --- @@ -634,7 +634,7 @@ Defined in: [types/objectDetection.ts:71](https://github.com/software-mansion/re > **TOASTER**: `80` -Defined in: [types/objectDetection.ts:118](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L118) +Defined in: [constants/commonVision.ts:90](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L90) --- @@ -642,7 +642,7 @@ Defined in: [types/objectDetection.ts:118](https://github.com/software-mansion/r > **TOILET**: `70` -Defined in: [types/objectDetection.ts:108](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L108) +Defined in: [constants/commonVision.ts:80](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L80) --- @@ -650,7 +650,7 @@ Defined in: [types/objectDetection.ts:108](https://github.com/software-mansion/r > **TOOTHBRUSH**: `90` -Defined in: [types/objectDetection.ts:128](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L128) +Defined in: [constants/commonVision.ts:100](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L100) --- @@ -658,7 +658,7 @@ Defined in: [types/objectDetection.ts:128](https://github.com/software-mansion/r > **TRAFFIC_LIGHT**: `10` -Defined in: [types/objectDetection.ts:49](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L49) +Defined in: [constants/commonVision.ts:21](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L21) --- @@ -666,7 +666,7 @@ Defined in: [types/objectDetection.ts:49](https://github.com/software-mansion/re > **TRAIN**: `7` -Defined in: [types/objectDetection.ts:46](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L46) +Defined in: [constants/commonVision.ts:18](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L18) --- @@ -674,7 +674,7 @@ Defined in: [types/objectDetection.ts:46](https://github.com/software-mansion/re > **TRUCK**: `8` -Defined in: [types/objectDetection.ts:47](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L47) +Defined in: [constants/commonVision.ts:19](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L19) --- @@ -682,7 +682,7 @@ Defined in: [types/objectDetection.ts:47](https://github.com/software-mansion/re > **TV**: `72` -Defined in: [types/objectDetection.ts:110](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L110) +Defined in: [constants/commonVision.ts:82](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L82) --- @@ -690,7 +690,7 @@ Defined in: [types/objectDetection.ts:110](https://github.com/software-mansion/r > **UMBRELLA**: `28` -Defined in: [types/objectDetection.ts:67](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L67) +Defined in: [constants/commonVision.ts:39](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L39) --- @@ -698,7 +698,7 @@ Defined in: [types/objectDetection.ts:67](https://github.com/software-mansion/re > **VASE**: `86` -Defined in: [types/objectDetection.ts:124](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L124) +Defined in: [constants/commonVision.ts:96](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L96) --- @@ -706,7 +706,7 @@ Defined in: [types/objectDetection.ts:124](https://github.com/software-mansion/r > **WINDOW**: `68` -Defined in: [types/objectDetection.ts:106](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L106) +Defined in: [constants/commonVision.ts:78](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L78) --- @@ -714,7 +714,7 @@ Defined in: [types/objectDetection.ts:106](https://github.com/software-mansion/r > **WINE_GLASS**: `46` -Defined in: [types/objectDetection.ts:84](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L84) +Defined in: [constants/commonVision.ts:56](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L56) --- @@ -722,4 +722,4 @@ Defined in: [types/objectDetection.ts:84](https://github.com/software-mansion/re > **ZEBRA**: `24` -Defined in: [types/objectDetection.ts:63](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L63) +Defined in: [constants/commonVision.ts:35](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L35) diff --git a/docs/docs/06-api-reference/functions/useObjectDetection.md b/docs/docs/06-api-reference/functions/useObjectDetection.md index 8cc0ba419..5c2d7abed 100644 --- a/docs/docs/06-api-reference/functions/useObjectDetection.md +++ b/docs/docs/06-api-reference/functions/useObjectDetection.md @@ -1,21 +1,29 @@ # Function: useObjectDetection() -> **useObjectDetection**(`ObjectDetectionProps`): [`ObjectDetectionType`](../interfaces/ObjectDetectionType.md) +> **useObjectDetection**\<`C`\>(`props`): [`ObjectDetectionType`](../interfaces/ObjectDetectionType.md)\<[`ObjectDetectionLabels`](../type-aliases/ObjectDetectionLabels.md)\<`C`\[`"modelName"`\]\>\> -Defined in: [hooks/computer_vision/useObjectDetection.ts:15](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts#L15) +Defined in: [hooks/computer_vision/useObjectDetection.ts:20](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts#L20) React hook for managing an Object Detection model instance. +## Type Parameters + +### C + +`C` _extends_ [`ObjectDetectionModelSources`](../type-aliases/ObjectDetectionModelSources.md) + +A [ObjectDetectionModelSources](../type-aliases/ObjectDetectionModelSources.md) config specifying which built-in model to load. + ## Parameters -### ObjectDetectionProps +### props -[`ObjectDetectionProps`](../interfaces/ObjectDetectionProps.md) +[`ObjectDetectionProps`](../interfaces/ObjectDetectionProps.md)\<`C`\> -Configuration object containing `model` source and optional `preventLoad` flag. +Configuration object containing `model` config and optional `preventLoad` flag. ## Returns -[`ObjectDetectionType`](../interfaces/ObjectDetectionType.md) +[`ObjectDetectionType`](../interfaces/ObjectDetectionType.md)\<[`ObjectDetectionLabels`](../type-aliases/ObjectDetectionLabels.md)\<`C`\[`"modelName"`\]\>\> -Ready to use Object Detection model. +An object with model state (`error`, `isReady`, `isGenerating`, `downloadProgress`) and a typed `forward` function. diff --git a/docs/docs/06-api-reference/index.md b/docs/docs/06-api-reference/index.md index 7bffecd63..07e63aa17 100644 --- a/docs/docs/06-api-reference/index.md +++ b/docs/docs/06-api-reference/index.md @@ -5,10 +5,10 @@ - [useClassification](functions/useClassification.md) - [useExecutorchModule](functions/useExecutorchModule.md) - [useImageEmbeddings](functions/useImageEmbeddings.md) +- [useImageSegmentation](functions/useImageSegmentation.md) - [useLLM](functions/useLLM.md) - [useObjectDetection](functions/useObjectDetection.md) - [useOCR](functions/useOCR.md) -- [useSemanticSegmentation](functions/useSemanticSegmentation.md) - [useSpeechToText](functions/useSpeechToText.md) - [useStyleTransfer](functions/useStyleTransfer.md) - [useTextEmbeddings](functions/useTextEmbeddings.md) @@ -35,6 +35,11 @@ - [BK_SDM_TINY_VPRED_256](variables/BK_SDM_TINY_VPRED_256.md) - [BK_SDM_TINY_VPRED_512](variables/BK_SDM_TINY_VPRED_512.md) +## Models - Image Segmentation + +- [DEEPLAB_V3_RESNET50](variables/DEEPLAB_V3_RESNET50.md) +- [SELFIE_SEGMENTATION](variables/SELFIE_SEGMENTATION.md) + ## Models - LMM - [HAMMER2_1_0_5B](variables/HAMMER2_1_0_5B.md) @@ -74,24 +79,9 @@ ## Models - Object Detection +- [RF_DETR_NANO](variables/RF_DETR_NANO.md) - [SSDLITE_320_MOBILENET_V3_LARGE](variables/SSDLITE_320_MOBILENET_V3_LARGE.md) -## Models - Semantic Segmentation - -- [DEEPLAB_V3_MOBILENET_V3_LARGE](variables/DEEPLAB_V3_MOBILENET_V3_LARGE.md) -- [DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED](variables/DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED.md) -- [DEEPLAB_V3_RESNET101](variables/DEEPLAB_V3_RESNET101.md) -- [DEEPLAB_V3_RESNET101_QUANTIZED](variables/DEEPLAB_V3_RESNET101_QUANTIZED.md) -- [DEEPLAB_V3_RESNET50](variables/DEEPLAB_V3_RESNET50.md) -- [DEEPLAB_V3_RESNET50_QUANTIZED](variables/DEEPLAB_V3_RESNET50_QUANTIZED.md) -- [FCN_RESNET101](variables/FCN_RESNET101.md) -- [FCN_RESNET101_QUANTIZED](variables/FCN_RESNET101_QUANTIZED.md) -- [FCN_RESNET50](variables/FCN_RESNET50.md) -- [FCN_RESNET50_QUANTIZED](variables/FCN_RESNET50_QUANTIZED.md) -- [LRASPP_MOBILENET_V3_LARGE](variables/LRASPP_MOBILENET_V3_LARGE.md) -- [LRASPP_MOBILENET_V3_LARGE_QUANTIZED](variables/LRASPP_MOBILENET_V3_LARGE_QUANTIZED.md) -- [SELFIE_SEGMENTATION](variables/SELFIE_SEGMENTATION.md) - ## Models - Speech To Text - [WHISPER_BASE](variables/WHISPER_BASE.md) @@ -197,7 +187,8 @@ - [RnExecutorchErrorCode](enumerations/RnExecutorchErrorCode.md) - [Logger](classes/Logger.md) - [RnExecutorchError](classes/RnExecutorchError.md) -- [Frame](interfaces/Frame.md) +- [IMAGENET1K_MEAN](variables/IMAGENET1K_MEAN.md) +- [IMAGENET1K_STD](variables/IMAGENET1K_STD.md) ## TTS Supported Voices @@ -231,6 +222,8 @@ - [GenerationConfig](interfaces/GenerationConfig.md) - [ImageEmbeddingsProps](interfaces/ImageEmbeddingsProps.md) - [ImageEmbeddingsType](interfaces/ImageEmbeddingsType.md) +- [ImageSegmentationProps](interfaces/ImageSegmentationProps.md) +- [ImageSegmentationType](interfaces/ImageSegmentationType.md) - [KokoroConfig](interfaces/KokoroConfig.md) - [KokoroVoiceExtras](interfaces/KokoroVoiceExtras.md) - [LLMConfig](interfaces/LLMConfig.md) @@ -242,11 +235,8 @@ - [OCRDetection](interfaces/OCRDetection.md) - [OCRProps](interfaces/OCRProps.md) - [OCRType](interfaces/OCRType.md) -- [PixelData](interfaces/PixelData.md) - [Point](interfaces/Point.md) - [Segment](interfaces/Segment.md) -- [SemanticSegmentationProps](interfaces/SemanticSegmentationProps.md) -- [SemanticSegmentationType](interfaces/SemanticSegmentationType.md) - [SpeechToTextModelConfig](interfaces/SpeechToTextModelConfig.md) - [SpeechToTextProps](interfaces/SpeechToTextProps.md) - [SpeechToTextType](interfaces/SpeechToTextType.md) @@ -277,12 +267,16 @@ - [LLMTool](type-aliases/LLMTool.md) - [MessageRole](type-aliases/MessageRole.md) - [ModelNameOf](type-aliases/ModelNameOf.md) +- [ModelSources](type-aliases/ModelSources.md) +- [ObjectDetectionConfig](type-aliases/ObjectDetectionConfig.md) +- [ObjectDetectionLabels](type-aliases/ObjectDetectionLabels.md) +- [ObjectDetectionModelName](type-aliases/ObjectDetectionModelName.md) +- [ObjectDetectionModelSources](type-aliases/ObjectDetectionModelSources.md) - [OCRLanguage](type-aliases/OCRLanguage.md) - [ResourceSource](type-aliases/ResourceSource.md) +- [SegmentationConfig](type-aliases/SegmentationConfig.md) - [SegmentationLabels](type-aliases/SegmentationLabels.md) -- [SemanticSegmentationConfig](type-aliases/SemanticSegmentationConfig.md) -- [SemanticSegmentationModelName](type-aliases/SemanticSegmentationModelName.md) -- [SemanticSegmentationModelSources](type-aliases/SemanticSegmentationModelSources.md) +- [SegmentationModelName](type-aliases/SegmentationModelName.md) - [SpeechToTextLanguage](type-aliases/SpeechToTextLanguage.md) - [TensorBuffer](type-aliases/TensorBuffer.md) - [TextToSpeechLanguage](type-aliases/TextToSpeechLanguage.md) @@ -294,10 +288,10 @@ - [ClassificationModule](classes/ClassificationModule.md) - [ExecutorchModule](classes/ExecutorchModule.md) - [ImageEmbeddingsModule](classes/ImageEmbeddingsModule.md) +- [ImageSegmentationModule](classes/ImageSegmentationModule.md) - [LLMModule](classes/LLMModule.md) - [ObjectDetectionModule](classes/ObjectDetectionModule.md) - [OCRModule](classes/OCRModule.md) -- [SemanticSegmentationModule](classes/SemanticSegmentationModule.md) - [SpeechToTextModule](classes/SpeechToTextModule.md) - [StyleTransferModule](classes/StyleTransferModule.md) - [TextEmbeddingsModule](classes/TextEmbeddingsModule.md) diff --git a/docs/docs/06-api-reference/interfaces/Bbox.md b/docs/docs/06-api-reference/interfaces/Bbox.md index 98fec21a8..864152f14 100644 --- a/docs/docs/06-api-reference/interfaces/Bbox.md +++ b/docs/docs/06-api-reference/interfaces/Bbox.md @@ -1,6 +1,6 @@ # Interface: Bbox -Defined in: [types/objectDetection.ts:13](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L13) +Defined in: [types/objectDetection.ts:14](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L14) Represents a bounding box for a detected object in an image. @@ -10,7 +10,7 @@ Represents a bounding box for a detected object in an image. > **x1**: `number` -Defined in: [types/objectDetection.ts:14](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L14) +Defined in: [types/objectDetection.ts:15](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L15) The x-coordinate of the bottom-left corner of the bounding box. @@ -20,7 +20,7 @@ The x-coordinate of the bottom-left corner of the bounding box. > **x2**: `number` -Defined in: [types/objectDetection.ts:15](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L15) +Defined in: [types/objectDetection.ts:16](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L16) The x-coordinate of the top-right corner of the bounding box. @@ -30,7 +30,7 @@ The x-coordinate of the top-right corner of the bounding box. > **y1**: `number` -Defined in: [types/objectDetection.ts:16](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L16) +Defined in: [types/objectDetection.ts:17](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L17) The y-coordinate of the bottom-left corner of the bounding box. @@ -40,6 +40,6 @@ The y-coordinate of the bottom-left corner of the bounding box. > **y2**: `number` -Defined in: [types/objectDetection.ts:17](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L17) +Defined in: [types/objectDetection.ts:18](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L18) The y-coordinate of the top-right corner of the bounding box. diff --git a/docs/docs/06-api-reference/interfaces/Detection.md b/docs/docs/06-api-reference/interfaces/Detection.md index 55bfa2a2c..695460596 100644 --- a/docs/docs/06-api-reference/interfaces/Detection.md +++ b/docs/docs/06-api-reference/interfaces/Detection.md @@ -1,16 +1,24 @@ -# Interface: Detection +# Interface: Detection\ -Defined in: [types/objectDetection.ts:28](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L28) +Defined in: [types/objectDetection.ts:30](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L30) Represents a detected object within an image, including its bounding box, label, and confidence score. +## Type Parameters + +### L + +`L` _extends_ [`LabelEnum`](../type-aliases/LabelEnum.md) = _typeof_ [`CocoLabel`](../enumerations/CocoLabel.md) + +The label enum type for the detected object. Defaults to [CocoLabel](../enumerations/CocoLabel.md). + ## Properties ### bbox > **bbox**: [`Bbox`](Bbox.md) -Defined in: [types/objectDetection.ts:29](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L29) +Defined in: [types/objectDetection.ts:31](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L31) The bounding box of the detected object, defined by its top-left (x1, y1) and bottom-right (x2, y2) coordinates. @@ -18,11 +26,11 @@ The bounding box of the detected object, defined by its top-left (x1, y1) and bo ### label -> **label**: `"BICYCLE"` \| `"BIRD"` \| `"BOAT"` \| `"BOTTLE"` \| `"BUS"` \| `"CAR"` \| `"CAT"` \| `"CHAIR"` \| `"COW"` \| `"DOG"` \| `"HORSE"` \| `"PERSON"` \| `"SHEEP"` \| `"TRAIN"` \| `"MOTORCYCLE"` \| `"AIRPLANE"` \| `"TRUCK"` \| `"TRAFFIC_LIGHT"` \| `"FIRE_HYDRANT"` \| `"STREET_SIGN"` \| `"STOP_SIGN"` \| `"PARKING"` \| `"BENCH"` \| `"ELEPHANT"` \| `"BEAR"` \| `"ZEBRA"` \| `"GIRAFFE"` \| `"HAT"` \| `"BACKPACK"` \| `"UMBRELLA"` \| `"SHOE"` \| `"EYE"` \| `"HANDBAG"` \| `"TIE"` \| `"SUITCASE"` \| `"FRISBEE"` \| `"SKIS"` \| `"SNOWBOARD"` \| `"SPORTS"` \| `"KITE"` \| `"BASEBALL"` \| `"SKATEBOARD"` \| `"SURFBOARD"` \| `"TENNIS_RACKET"` \| `"PLATE"` \| `"WINE_GLASS"` \| `"CUP"` \| `"FORK"` \| `"KNIFE"` \| `"SPOON"` \| `"BOWL"` \| `"BANANA"` \| `"APPLE"` \| `"SANDWICH"` \| `"ORANGE"` \| `"BROCCOLI"` \| `"CARROT"` \| `"HOT_DOG"` \| `"PIZZA"` \| `"DONUT"` \| `"CAKE"` \| `"COUCH"` \| `"POTTED_PLANT"` \| `"BED"` \| `"MIRROR"` \| `"DINING_TABLE"` \| `"WINDOW"` \| `"DESK"` \| `"TOILET"` \| `"DOOR"` \| `"TV"` \| `"LAPTOP"` \| `"MOUSE"` \| `"REMOTE"` \| `"KEYBOARD"` \| `"CELL_PHONE"` \| `"MICROWAVE"` \| `"OVEN"` \| `"TOASTER"` \| `"SINK"` \| `"REFRIGERATOR"` \| `"BLENDER"` \| `"BOOK"` \| `"CLOCK"` \| `"VASE"` \| `"SCISSORS"` \| `"TEDDY_BEAR"` \| `"HAIR_DRIER"` \| `"TOOTHBRUSH"` \| `"HAIR_BRUSH"` +> **label**: keyof `L` -Defined in: [types/objectDetection.ts:30](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L30) +Defined in: [types/objectDetection.ts:32](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L32) -The class label of the detected object, represented as a key from the `CocoLabel` enum. +The class label of the detected object. --- @@ -30,6 +38,6 @@ The class label of the detected object, represented as a key from the `CocoLabel > **score**: `number` -Defined in: [types/objectDetection.ts:31](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L31) +Defined in: [types/objectDetection.ts:33](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L33) The confidence score of the detection, typically ranging from 0 to 1. diff --git a/docs/docs/06-api-reference/interfaces/ObjectDetectionProps.md b/docs/docs/06-api-reference/interfaces/ObjectDetectionProps.md index 069c4d3e1..06f3b0e8e 100644 --- a/docs/docs/06-api-reference/interfaces/ObjectDetectionProps.md +++ b/docs/docs/06-api-reference/interfaces/ObjectDetectionProps.md @@ -1,22 +1,26 @@ -# Interface: ObjectDetectionProps +# Interface: ObjectDetectionProps\ -Defined in: [types/objectDetection.ts:140](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L140) +Defined in: [types/objectDetection.ts:71](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L71) Props for the `useObjectDetection` hook. -## Properties +## Type Parameters -### model +### C + +`C` _extends_ [`ObjectDetectionModelSources`](../type-aliases/ObjectDetectionModelSources.md) -> **model**: `object` +A [ObjectDetectionModelSources](../type-aliases/ObjectDetectionModelSources.md) config specifying which built-in model to load. -Defined in: [types/objectDetection.ts:141](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L141) +## Properties + +### model -An object containing the model source. +> **model**: `C` -#### modelSource +Defined in: [types/objectDetection.ts:72](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L72) -> **modelSource**: [`ResourceSource`](../type-aliases/ResourceSource.md) +The model config containing `modelName` and `modelSource`. --- @@ -24,6 +28,6 @@ An object containing the model source. > `optional` **preventLoad**: `boolean` -Defined in: [types/objectDetection.ts:142](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L142) +Defined in: [types/objectDetection.ts:73](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L73) Boolean that can prevent automatic model loading (and downloading the data if you load it for the first time) after running the hook. diff --git a/docs/docs/06-api-reference/interfaces/ObjectDetectionType.md b/docs/docs/06-api-reference/interfaces/ObjectDetectionType.md index 4bd5dba98..a9e8a97b7 100644 --- a/docs/docs/06-api-reference/interfaces/ObjectDetectionType.md +++ b/docs/docs/06-api-reference/interfaces/ObjectDetectionType.md @@ -1,17 +1,25 @@ -# Interface: ObjectDetectionType +# Interface: ObjectDetectionType\ -Defined in: [types/objectDetection.ts:151](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L151) +Defined in: [types/objectDetection.ts:84](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L84) Return type for the `useObjectDetection` hook. Manages the state and operations for Computer Vision object detection tasks. +## Type Parameters + +### L + +`L` _extends_ [`LabelEnum`](../type-aliases/LabelEnum.md) + +The [LabelEnum](../type-aliases/LabelEnum.md) representing the model's class labels. + ## Properties ### downloadProgress > **downloadProgress**: `number` -Defined in: [types/objectDetection.ts:170](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L170) +Defined in: [types/objectDetection.ts:103](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L103) Represents the download progress of the model binary as a value between 0 and 1. @@ -21,7 +29,7 @@ Represents the download progress of the model binary as a value between 0 and 1. > **error**: [`RnExecutorchError`](../classes/RnExecutorchError.md) \| `null` -Defined in: [types/objectDetection.ts:155](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L155) +Defined in: [types/objectDetection.ts:88](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L88) Contains the error object if the model failed to load, download, or encountered a runtime error during detection. @@ -29,64 +37,43 @@ Contains the error object if the model failed to load, download, or encountered ### forward() -> **forward**: (`input`, `detectionThreshold?`) => `Promise`\<[`Detection`](Detection.md)[]\> - -Defined in: [types/objectDetection.ts:199](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L199) - -Executes the model's forward pass with automatic input type detection. - -Supports two input types: +> **forward**: (`imageSource`, `detectionThreshold?`) => `Promise`\<[`Detection`](Detection.md)\<`L`\>[]\> -1. **String path/URI**: File path, URL, or Base64-encoded string -2. **PixelData**: Raw pixel data from image libraries (e.g., NitroImage) +Defined in: [types/objectDetection.ts:112](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L112) -**Note**: For VisionCamera frame processing, use `runOnFrame` instead. +Executes the model's forward pass to detect objects within the provided image. #### Parameters -##### input +##### imageSource -Image source (string or PixelData object) +`string` -`string` | [`PixelData`](PixelData.md) +A string representing the image source (e.g., a file path, URI, or base64 string) to be processed. ##### detectionThreshold? `number` -An optional number between 0 and 1 representing the minimum confidence score. Default is 0.5. +An optional number between 0 and 1 representing the minimum confidence score required for an object to be included in the results. Default is 0.7. #### Returns -`Promise`\<[`Detection`](Detection.md)[]\> +`Promise`\<[`Detection`](Detection.md)\<`L`\>[]\> -A Promise that resolves to an array of `Detection` objects. +A Promise that resolves to an array of `Detection` objects, where each object typically contains bounding box coordinates, a class label, and a confidence score. #### Throws If the model is not loaded or is currently processing another image. -#### Example - -```typescript -// String path -const detections1 = await model.forward('file:///path/to/image.jpg'); - -// Pixel data -const detections2 = await model.forward({ - dataPtr: new Uint8Array(rgbPixels), - sizes: [480, 640, 3], - scalarType: ScalarType.BYTE, -}); -``` - --- ### isGenerating > **isGenerating**: `boolean` -Defined in: [types/objectDetection.ts:165](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L165) +Defined in: [types/objectDetection.ts:98](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L98) Indicates whether the model is currently processing an image. @@ -96,49 +83,6 @@ Indicates whether the model is currently processing an image. > **isReady**: `boolean` -Defined in: [types/objectDetection.ts:160](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L160) +Defined in: [types/objectDetection.ts:93](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L93) Indicates whether the object detection model is loaded and ready to process images. - ---- - -### runOnFrame - -> **runOnFrame**: (`frame`, `detectionThreshold`) => [`Detection`](Detection.md)[] \| `null` - -Defined in: [types/objectDetection.ts:231](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L231) - -Synchronous worklet function for real-time VisionCamera frame processing. -Automatically handles native buffer extraction and cleanup. - -**Use this for VisionCamera frame processing in worklets.** -For async processing, use `forward()` instead. - -Available after model is loaded (`isReady: true`). - -#### Example - -```typescript -const { runOnFrame, isReady } = useObjectDetection({ model: MODEL }); - -const frameOutput = useFrameOutput({ - onFrame(frame) { - 'worklet'; - if (!runOnFrame) return; - const detections = runOnFrame(frame, 0.5); - frame.dispose(); - }, -}); -``` - -#### Param - -VisionCamera Frame object - -#### Param - -The threshold for detection sensitivity. - -#### Returns - -Array of Detection objects representing detected items in the frame. diff --git a/docs/docs/06-api-reference/type-aliases/ObjectDetectionConfig.md b/docs/docs/06-api-reference/type-aliases/ObjectDetectionConfig.md new file mode 100644 index 000000000..aefbc33c5 --- /dev/null +++ b/docs/docs/06-api-reference/type-aliases/ObjectDetectionConfig.md @@ -0,0 +1,37 @@ +# Type Alias: ObjectDetectionConfig\ + +> **ObjectDetectionConfig**\<`T`\> = `object` + +Defined in: [types/objectDetection.ts:58](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L58) + +Configuration for a custom object detection model. + +## Type Parameters + +### T + +`T` _extends_ [`LabelEnum`](LabelEnum.md) + +## Properties + +### labelMap + +> **labelMap**: `T` + +Defined in: [types/objectDetection.ts:59](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L59) + +--- + +### preprocessorConfig? + +> `optional` **preprocessorConfig**: `object` + +Defined in: [types/objectDetection.ts:60](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L60) + +#### normMean? + +> `optional` **normMean**: [`Triple`](Triple.md)\<`number`\> + +#### normStd? + +> `optional` **normStd**: [`Triple`](Triple.md)\<`number`\> diff --git a/docs/docs/06-api-reference/type-aliases/ObjectDetectionLabels.md b/docs/docs/06-api-reference/type-aliases/ObjectDetectionLabels.md new file mode 100644 index 000000000..2a412796f --- /dev/null +++ b/docs/docs/06-api-reference/type-aliases/ObjectDetectionLabels.md @@ -0,0 +1,15 @@ +# Type Alias: ObjectDetectionLabels\ + +> **ObjectDetectionLabels**\<`M`\> = `ResolveLabelsFor`\<`M`, `ModelConfigsType`\> + +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:44](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L44) + +Resolves the [LabelEnum](LabelEnum.md) for a given built-in object detection model name. + +## Type Parameters + +### M + +`M` _extends_ [`ObjectDetectionModelName`](ObjectDetectionModelName.md) + +A built-in model name from [ObjectDetectionModelName](ObjectDetectionModelName.md). diff --git a/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelName.md b/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelName.md new file mode 100644 index 000000000..4f9872946 --- /dev/null +++ b/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelName.md @@ -0,0 +1,7 @@ +# Type Alias: ObjectDetectionModelName + +> **ObjectDetectionModelName** = [`ObjectDetectionModelSources`](ObjectDetectionModelSources.md)\[`"modelName"`\] + +Defined in: [types/objectDetection.ts:51](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L51) + +Union of all built-in object detection model names. diff --git a/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelSources.md b/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelSources.md new file mode 100644 index 000000000..1191ca8f0 --- /dev/null +++ b/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelSources.md @@ -0,0 +1,8 @@ +# Type Alias: ObjectDetectionModelSources + +> **ObjectDetectionModelSources** = \{ `modelName`: `"ssdlite-320-mobilenet-v3-large"`; `modelSource`: [`ResourceSource`](ResourceSource.md); \} \| \{ `modelName`: `"rf-detr-nano"`; `modelSource`: [`ResourceSource`](ResourceSource.md); \} + +Defined in: [types/objectDetection.ts:42](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L42) + +Per-model config for [ObjectDetectionModule.fromModelName](../classes/ObjectDetectionModule.md#frommodelname). +Each model name maps to its required fields. diff --git a/docs/docs/06-api-reference/type-aliases/SegmentationLabels.md b/docs/docs/06-api-reference/type-aliases/SegmentationLabels.md index eaa40fe6a..5e5968dd3 100644 --- a/docs/docs/06-api-reference/type-aliases/SegmentationLabels.md +++ b/docs/docs/06-api-reference/type-aliases/SegmentationLabels.md @@ -2,7 +2,7 @@ > **SegmentationLabels**\<`M`\> = `ModelConfigsType`\[`M`\]\[`"labelMap"`\] -Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:55](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L55) +Defined in: [modules/computer_vision/ImageSegmentationModule.ts:42](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ImageSegmentationModule.ts#L42) Resolves the [LabelEnum](LabelEnum.md) for a given built-in model name. @@ -10,6 +10,6 @@ Resolves the [LabelEnum](LabelEnum.md) for a given built-in model name. ### M -`M` _extends_ [`SemanticSegmentationModelName`](SemanticSegmentationModelName.md) +`M` _extends_ [`SegmentationModelName`](SegmentationModelName.md) -A built-in model name from [SemanticSegmentationModelName](SemanticSegmentationModelName.md). +A built-in model name from [SegmentationModelName](SegmentationModelName.md). diff --git a/docs/docs/06-api-reference/typedoc-sidebar.cjs b/docs/docs/06-api-reference/typedoc-sidebar.cjs index 00abf2925..9a646b7fe 100644 --- a/docs/docs/06-api-reference/typedoc-sidebar.cjs +++ b/docs/docs/06-api-reference/typedoc-sidebar.cjs @@ -1,4 +1,4 @@ // @ts-check /** @type {import("@docusaurus/plugin-content-docs").SidebarsConfig} */ -const typedocSidebar = {items:[{type:"category",label:"Hooks",items:[{type:"doc",id:"06-api-reference/functions/useClassification",label:"useClassification"},{type:"doc",id:"06-api-reference/functions/useExecutorchModule",label:"useExecutorchModule"},{type:"doc",id:"06-api-reference/functions/useImageEmbeddings",label:"useImageEmbeddings"},{type:"doc",id:"06-api-reference/functions/useLLM",label:"useLLM"},{type:"doc",id:"06-api-reference/functions/useObjectDetection",label:"useObjectDetection"},{type:"doc",id:"06-api-reference/functions/useOCR",label:"useOCR"},{type:"doc",id:"06-api-reference/functions/useSemanticSegmentation",label:"useSemanticSegmentation"},{type:"doc",id:"06-api-reference/functions/useSpeechToText",label:"useSpeechToText"},{type:"doc",id:"06-api-reference/functions/useStyleTransfer",label:"useStyleTransfer"},{type:"doc",id:"06-api-reference/functions/useTextEmbeddings",label:"useTextEmbeddings"},{type:"doc",id:"06-api-reference/functions/useTextToImage",label:"useTextToImage"},{type:"doc",id:"06-api-reference/functions/useTextToSpeech",label:"useTextToSpeech"},{type:"doc",id:"06-api-reference/functions/useTokenizer",label:"useTokenizer"},{type:"doc",id:"06-api-reference/functions/useVAD",label:"useVAD"},{type:"doc",id:"06-api-reference/functions/useVerticalOCR",label:"useVerticalOCR"}]},{type:"category",label:"Interfaces",items:[{type:"doc",id:"06-api-reference/interfaces/ResourceSourceExtended",label:"ResourceSourceExtended"}]},{type:"category",label:"Models - Classification",items:[{type:"doc",id:"06-api-reference/variables/EFFICIENTNET_V2_S",label:"EFFICIENTNET_V2_S"}]},{type:"category",label:"Models - Image Embeddings",items:[{type:"doc",id:"06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE",label:"CLIP_VIT_BASE_PATCH32_IMAGE"}]},{type:"category",label:"Models - Image Generation",items:[{type:"doc",id:"06-api-reference/variables/BK_SDM_TINY_VPRED_256",label:"BK_SDM_TINY_VPRED_256"},{type:"doc",id:"06-api-reference/variables/BK_SDM_TINY_VPRED_512",label:"BK_SDM_TINY_VPRED_512"}]},{type:"category",label:"Models - LMM",items:[{type:"doc",id:"06-api-reference/variables/HAMMER2_1_0_5B",label:"HAMMER2_1_0_5B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_0_5B_QUANTIZED",label:"HAMMER2_1_0_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_1_5B",label:"HAMMER2_1_1_5B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_1_5B_QUANTIZED",label:"HAMMER2_1_1_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_3B",label:"HAMMER2_1_3B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_3B_QUANTIZED",label:"HAMMER2_1_3B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LFM2_5_1_2B_INSTRUCT",label:"LFM2_5_1_2B_INSTRUCT"},{type:"doc",id:"06-api-reference/variables/LFM2_5_1_2B_INSTRUCT_QUANTIZED",label:"LFM2_5_1_2B_INSTRUCT_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B",label:"LLAMA3_2_1B"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B_QLORA",label:"LLAMA3_2_1B_QLORA"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B_SPINQUANT",label:"LLAMA3_2_1B_SPINQUANT"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B",label:"LLAMA3_2_3B"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B_QLORA",label:"LLAMA3_2_3B_QLORA"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B_SPINQUANT",label:"LLAMA3_2_3B_SPINQUANT"},{type:"doc",id:"06-api-reference/variables/PHI_4_MINI_4B",label:"PHI_4_MINI_4B"},{type:"doc",id:"06-api-reference/variables/PHI_4_MINI_4B_QUANTIZED",label:"PHI_4_MINI_4B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_0_5B",label:"QWEN2_5_0_5B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_0_5B_QUANTIZED",label:"QWEN2_5_0_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_1_5B",label:"QWEN2_5_1_5B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_1_5B_QUANTIZED",label:"QWEN2_5_1_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_3B",label:"QWEN2_5_3B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_3B_QUANTIZED",label:"QWEN2_5_3B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_0_6B",label:"QWEN3_0_6B"},{type:"doc",id:"06-api-reference/variables/QWEN3_0_6B_QUANTIZED",label:"QWEN3_0_6B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_1_7B",label:"QWEN3_1_7B"},{type:"doc",id:"06-api-reference/variables/QWEN3_1_7B_QUANTIZED",label:"QWEN3_1_7B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_4B",label:"QWEN3_4B"},{type:"doc",id:"06-api-reference/variables/QWEN3_4B_QUANTIZED",label:"QWEN3_4B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_1_7B",label:"SMOLLM2_1_1_7B"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_1_7B_QUANTIZED",label:"SMOLLM2_1_1_7B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_135M",label:"SMOLLM2_1_135M"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_135M_QUANTIZED",label:"SMOLLM2_1_135M_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_360M",label:"SMOLLM2_1_360M"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_360M_QUANTIZED",label:"SMOLLM2_1_360M_QUANTIZED"}]},{type:"category",label:"Models - Object Detection",items:[{type:"doc",id:"06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE",label:"SSDLITE_320_MOBILENET_V3_LARGE"}]},{type:"category",label:"Models - Semantic Segmentation",items:[{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE",label:"DEEPLAB_V3_MOBILENET_V3_LARGE"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED",label:"DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET101",label:"DEEPLAB_V3_RESNET101"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET101_QUANTIZED",label:"DEEPLAB_V3_RESNET101_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET50",label:"DEEPLAB_V3_RESNET50"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET50_QUANTIZED",label:"DEEPLAB_V3_RESNET50_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/FCN_RESNET101",label:"FCN_RESNET101"},{type:"doc",id:"06-api-reference/variables/FCN_RESNET101_QUANTIZED",label:"FCN_RESNET101_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/FCN_RESNET50",label:"FCN_RESNET50"},{type:"doc",id:"06-api-reference/variables/FCN_RESNET50_QUANTIZED",label:"FCN_RESNET50_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE",label:"LRASPP_MOBILENET_V3_LARGE"},{type:"doc",id:"06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE_QUANTIZED",label:"LRASPP_MOBILENET_V3_LARGE_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SELFIE_SEGMENTATION",label:"SELFIE_SEGMENTATION"}]},{type:"category",label:"Models - Speech To Text",items:[{type:"doc",id:"06-api-reference/variables/WHISPER_BASE",label:"WHISPER_BASE"},{type:"doc",id:"06-api-reference/variables/WHISPER_BASE_EN",label:"WHISPER_BASE_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_SMALL",label:"WHISPER_SMALL"},{type:"doc",id:"06-api-reference/variables/WHISPER_SMALL_EN",label:"WHISPER_SMALL_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY",label:"WHISPER_TINY"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY_EN",label:"WHISPER_TINY_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED",label:"WHISPER_TINY_EN_QUANTIZED"}]},{type:"category",label:"Models - Style Transfer",items:[{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_CANDY",label:"STYLE_TRANSFER_CANDY"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_MOSAIC",label:"STYLE_TRANSFER_MOSAIC"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS",label:"STYLE_TRANSFER_RAIN_PRINCESS"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_UDNIE",label:"STYLE_TRANSFER_UDNIE"}]},{type:"category",label:"Models - Text Embeddings",items:[{type:"doc",id:"06-api-reference/variables/ALL_MINILM_L6_V2",label:"ALL_MINILM_L6_V2"},{type:"doc",id:"06-api-reference/variables/ALL_MPNET_BASE_V2",label:"ALL_MPNET_BASE_V2"},{type:"doc",id:"06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT",label:"CLIP_VIT_BASE_PATCH32_TEXT"},{type:"doc",id:"06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1",label:"MULTI_QA_MINILM_L6_COS_V1"},{type:"doc",id:"06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1",label:"MULTI_QA_MPNET_BASE_DOT_V1"}]},{type:"category",label:"Models - Text to Speech",items:[{type:"doc",id:"06-api-reference/variables/KOKORO_MEDIUM",label:"KOKORO_MEDIUM"},{type:"doc",id:"06-api-reference/variables/KOKORO_SMALL",label:"KOKORO_SMALL"}]},{type:"category",label:"Models - Voice Activity Detection",items:[{type:"doc",id:"06-api-reference/variables/FSMN_VAD",label:"FSMN_VAD"}]},{type:"category",label:"OCR Supported Alphabets",items:[{type:"doc",id:"06-api-reference/variables/OCR_ABAZA",label:"OCR_ABAZA"},{type:"doc",id:"06-api-reference/variables/OCR_ADYGHE",label:"OCR_ADYGHE"},{type:"doc",id:"06-api-reference/variables/OCR_AFRIKAANS",label:"OCR_AFRIKAANS"},{type:"doc",id:"06-api-reference/variables/OCR_ALBANIAN",label:"OCR_ALBANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_AVAR",label:"OCR_AVAR"},{type:"doc",id:"06-api-reference/variables/OCR_AZERBAIJANI",label:"OCR_AZERBAIJANI"},{type:"doc",id:"06-api-reference/variables/OCR_BELARUSIAN",label:"OCR_BELARUSIAN"},{type:"doc",id:"06-api-reference/variables/OCR_BOSNIAN",label:"OCR_BOSNIAN"},{type:"doc",id:"06-api-reference/variables/OCR_BULGARIAN",label:"OCR_BULGARIAN"},{type:"doc",id:"06-api-reference/variables/OCR_CHECHEN",label:"OCR_CHECHEN"},{type:"doc",id:"06-api-reference/variables/OCR_CROATIAN",label:"OCR_CROATIAN"},{type:"doc",id:"06-api-reference/variables/OCR_CZECH",label:"OCR_CZECH"},{type:"doc",id:"06-api-reference/variables/OCR_DANISH",label:"OCR_DANISH"},{type:"doc",id:"06-api-reference/variables/OCR_DARGWA",label:"OCR_DARGWA"},{type:"doc",id:"06-api-reference/variables/OCR_DUTCH",label:"OCR_DUTCH"},{type:"doc",id:"06-api-reference/variables/OCR_ENGLISH",label:"OCR_ENGLISH"},{type:"doc",id:"06-api-reference/variables/OCR_ESTONIAN",label:"OCR_ESTONIAN"},{type:"doc",id:"06-api-reference/variables/OCR_FRENCH",label:"OCR_FRENCH"},{type:"doc",id:"06-api-reference/variables/OCR_GERMAN",label:"OCR_GERMAN"},{type:"doc",id:"06-api-reference/variables/OCR_HUNGARIAN",label:"OCR_HUNGARIAN"},{type:"doc",id:"06-api-reference/variables/OCR_ICELANDIC",label:"OCR_ICELANDIC"},{type:"doc",id:"06-api-reference/variables/OCR_INDONESIAN",label:"OCR_INDONESIAN"},{type:"doc",id:"06-api-reference/variables/OCR_INGUSH",label:"OCR_INGUSH"},{type:"doc",id:"06-api-reference/variables/OCR_IRISH",label:"OCR_IRISH"},{type:"doc",id:"06-api-reference/variables/OCR_ITALIAN",label:"OCR_ITALIAN"},{type:"doc",id:"06-api-reference/variables/OCR_JAPANESE",label:"OCR_JAPANESE"},{type:"doc",id:"06-api-reference/variables/OCR_KANNADA",label:"OCR_KANNADA"},{type:"doc",id:"06-api-reference/variables/OCR_KARBADIAN",label:"OCR_KARBADIAN"},{type:"doc",id:"06-api-reference/variables/OCR_KOREAN",label:"OCR_KOREAN"},{type:"doc",id:"06-api-reference/variables/OCR_KURDISH",label:"OCR_KURDISH"},{type:"doc",id:"06-api-reference/variables/OCR_LAK",label:"OCR_LAK"},{type:"doc",id:"06-api-reference/variables/OCR_LATIN",label:"OCR_LATIN"},{type:"doc",id:"06-api-reference/variables/OCR_LATVIAN",label:"OCR_LATVIAN"},{type:"doc",id:"06-api-reference/variables/OCR_LEZGHIAN",label:"OCR_LEZGHIAN"},{type:"doc",id:"06-api-reference/variables/OCR_LITHUANIAN",label:"OCR_LITHUANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_MALAY",label:"OCR_MALAY"},{type:"doc",id:"06-api-reference/variables/OCR_MALTESE",label:"OCR_MALTESE"},{type:"doc",id:"06-api-reference/variables/OCR_MAORI",label:"OCR_MAORI"},{type:"doc",id:"06-api-reference/variables/OCR_MONGOLIAN",label:"OCR_MONGOLIAN"},{type:"doc",id:"06-api-reference/variables/OCR_NORWEGIAN",label:"OCR_NORWEGIAN"},{type:"doc",id:"06-api-reference/variables/OCR_OCCITAN",label:"OCR_OCCITAN"},{type:"doc",id:"06-api-reference/variables/OCR_PALI",label:"OCR_PALI"},{type:"doc",id:"06-api-reference/variables/OCR_POLISH",label:"OCR_POLISH"},{type:"doc",id:"06-api-reference/variables/OCR_PORTUGUESE",label:"OCR_PORTUGUESE"},{type:"doc",id:"06-api-reference/variables/OCR_ROMANIAN",label:"OCR_ROMANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_RUSSIAN",label:"OCR_RUSSIAN"},{type:"doc",id:"06-api-reference/variables/OCR_SERBIAN_CYRILLIC",label:"OCR_SERBIAN_CYRILLIC"},{type:"doc",id:"06-api-reference/variables/OCR_SERBIAN_LATIN",label:"OCR_SERBIAN_LATIN"},{type:"doc",id:"06-api-reference/variables/OCR_SIMPLIFIED_CHINESE",label:"OCR_SIMPLIFIED_CHINESE"},{type:"doc",id:"06-api-reference/variables/OCR_SLOVAK",label:"OCR_SLOVAK"},{type:"doc",id:"06-api-reference/variables/OCR_SLOVENIAN",label:"OCR_SLOVENIAN"},{type:"doc",id:"06-api-reference/variables/OCR_SPANISH",label:"OCR_SPANISH"},{type:"doc",id:"06-api-reference/variables/OCR_SWAHILI",label:"OCR_SWAHILI"},{type:"doc",id:"06-api-reference/variables/OCR_SWEDISH",label:"OCR_SWEDISH"},{type:"doc",id:"06-api-reference/variables/OCR_TABASSARAN",label:"OCR_TABASSARAN"},{type:"doc",id:"06-api-reference/variables/OCR_TAGALOG",label:"OCR_TAGALOG"},{type:"doc",id:"06-api-reference/variables/OCR_TAJIK",label:"OCR_TAJIK"},{type:"doc",id:"06-api-reference/variables/OCR_TELUGU",label:"OCR_TELUGU"},{type:"doc",id:"06-api-reference/variables/OCR_TURKISH",label:"OCR_TURKISH"},{type:"doc",id:"06-api-reference/variables/OCR_UKRAINIAN",label:"OCR_UKRAINIAN"},{type:"doc",id:"06-api-reference/variables/OCR_UZBEK",label:"OCR_UZBEK"},{type:"doc",id:"06-api-reference/variables/OCR_VIETNAMESE",label:"OCR_VIETNAMESE"},{type:"doc",id:"06-api-reference/variables/OCR_WELSH",label:"OCR_WELSH"}]},{type:"category",label:"Other",items:[{type:"doc",id:"06-api-reference/enumerations/RnExecutorchErrorCode",label:"RnExecutorchErrorCode"},{type:"doc",id:"06-api-reference/classes/Logger",label:"Logger"},{type:"doc",id:"06-api-reference/classes/RnExecutorchError",label:"RnExecutorchError"},{type:"doc",id:"06-api-reference/interfaces/Frame",label:"Frame"}]},{type:"category",label:"TTS Supported Voices",items:[{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_HEART",label:"KOKORO_VOICE_AF_HEART"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_RIVER",label:"KOKORO_VOICE_AF_RIVER"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_SARAH",label:"KOKORO_VOICE_AF_SARAH"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_ADAM",label:"KOKORO_VOICE_AM_ADAM"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_MICHAEL",label:"KOKORO_VOICE_AM_MICHAEL"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_SANTA",label:"KOKORO_VOICE_AM_SANTA"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_BF_EMMA",label:"KOKORO_VOICE_BF_EMMA"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_BM_DANIEL",label:"KOKORO_VOICE_BM_DANIEL"}]},{type:"category",label:"Types",items:[{type:"doc",id:"06-api-reference/enumerations/CocoLabel",label:"CocoLabel"},{type:"doc",id:"06-api-reference/enumerations/DeeplabLabel",label:"DeeplabLabel"},{type:"doc",id:"06-api-reference/enumerations/DownloadStatus",label:"DownloadStatus"},{type:"doc",id:"06-api-reference/enumerations/HTTP_CODE",label:"HTTP_CODE"},{type:"doc",id:"06-api-reference/enumerations/ScalarType",label:"ScalarType"},{type:"doc",id:"06-api-reference/enumerations/SelfieSegmentationLabel",label:"SelfieSegmentationLabel"},{type:"doc",id:"06-api-reference/enumerations/SourceType",label:"SourceType"},{type:"doc",id:"06-api-reference/interfaces/Bbox",label:"Bbox"},{type:"doc",id:"06-api-reference/interfaces/ChatConfig",label:"ChatConfig"},{type:"doc",id:"06-api-reference/interfaces/ClassificationProps",label:"ClassificationProps"},{type:"doc",id:"06-api-reference/interfaces/ClassificationType",label:"ClassificationType"},{type:"doc",id:"06-api-reference/interfaces/ContextStrategy",label:"ContextStrategy"},{type:"doc",id:"06-api-reference/interfaces/DecodingOptions",label:"DecodingOptions"},{type:"doc",id:"06-api-reference/interfaces/Detection",label:"Detection"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchModuleProps",label:"ExecutorchModuleProps"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchModuleType",label:"ExecutorchModuleType"},{type:"doc",id:"06-api-reference/interfaces/GenerationConfig",label:"GenerationConfig"},{type:"doc",id:"06-api-reference/interfaces/ImageEmbeddingsProps",label:"ImageEmbeddingsProps"},{type:"doc",id:"06-api-reference/interfaces/ImageEmbeddingsType",label:"ImageEmbeddingsType"},{type:"doc",id:"06-api-reference/interfaces/KokoroConfig",label:"KokoroConfig"},{type:"doc",id:"06-api-reference/interfaces/KokoroVoiceExtras",label:"KokoroVoiceExtras"},{type:"doc",id:"06-api-reference/interfaces/LLMConfig",label:"LLMConfig"},{type:"doc",id:"06-api-reference/interfaces/LLMProps",label:"LLMProps"},{type:"doc",id:"06-api-reference/interfaces/LLMType",label:"LLMType"},{type:"doc",id:"06-api-reference/interfaces/Message",label:"Message"},{type:"doc",id:"06-api-reference/interfaces/ObjectDetectionProps",label:"ObjectDetectionProps"},{type:"doc",id:"06-api-reference/interfaces/ObjectDetectionType",label:"ObjectDetectionType"},{type:"doc",id:"06-api-reference/interfaces/OCRDetection",label:"OCRDetection"},{type:"doc",id:"06-api-reference/interfaces/OCRProps",label:"OCRProps"},{type:"doc",id:"06-api-reference/interfaces/OCRType",label:"OCRType"},{type:"doc",id:"06-api-reference/interfaces/PixelData",label:"PixelData"},{type:"doc",id:"06-api-reference/interfaces/Point",label:"Point"},{type:"doc",id:"06-api-reference/interfaces/Segment",label:"Segment"},{type:"doc",id:"06-api-reference/interfaces/SemanticSegmentationProps",label:"SemanticSegmentationProps"},{type:"doc",id:"06-api-reference/interfaces/SemanticSegmentationType",label:"SemanticSegmentationType"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextModelConfig",label:"SpeechToTextModelConfig"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextProps",label:"SpeechToTextProps"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextType",label:"SpeechToTextType"},{type:"doc",id:"06-api-reference/interfaces/StyleTransferProps",label:"StyleTransferProps"},{type:"doc",id:"06-api-reference/interfaces/StyleTransferType",label:"StyleTransferType"},{type:"doc",id:"06-api-reference/interfaces/TensorPtr",label:"TensorPtr"},{type:"doc",id:"06-api-reference/interfaces/TextEmbeddingsProps",label:"TextEmbeddingsProps"},{type:"doc",id:"06-api-reference/interfaces/TextEmbeddingsType",label:"TextEmbeddingsType"},{type:"doc",id:"06-api-reference/interfaces/TextToImageProps",label:"TextToImageProps"},{type:"doc",id:"06-api-reference/interfaces/TextToImageType",label:"TextToImageType"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechConfig",label:"TextToSpeechConfig"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechInput",label:"TextToSpeechInput"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechProps",label:"TextToSpeechProps"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechStreamingInput",label:"TextToSpeechStreamingInput"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechType",label:"TextToSpeechType"},{type:"doc",id:"06-api-reference/interfaces/TokenizerProps",label:"TokenizerProps"},{type:"doc",id:"06-api-reference/interfaces/TokenizerType",label:"TokenizerType"},{type:"doc",id:"06-api-reference/interfaces/ToolCall",label:"ToolCall"},{type:"doc",id:"06-api-reference/interfaces/ToolsConfig",label:"ToolsConfig"},{type:"doc",id:"06-api-reference/interfaces/TranscriptionResult",label:"TranscriptionResult"},{type:"doc",id:"06-api-reference/interfaces/TranscriptionSegment",label:"TranscriptionSegment"},{type:"doc",id:"06-api-reference/interfaces/VADProps",label:"VADProps"},{type:"doc",id:"06-api-reference/interfaces/VADType",label:"VADType"},{type:"doc",id:"06-api-reference/interfaces/VerticalOCRProps",label:"VerticalOCRProps"},{type:"doc",id:"06-api-reference/interfaces/VoiceConfig",label:"VoiceConfig"},{type:"doc",id:"06-api-reference/interfaces/Word",label:"Word"},{type:"doc",id:"06-api-reference/type-aliases/LabelEnum",label:"LabelEnum"},{type:"doc",id:"06-api-reference/type-aliases/LLMTool",label:"LLMTool"},{type:"doc",id:"06-api-reference/type-aliases/MessageRole",label:"MessageRole"},{type:"doc",id:"06-api-reference/type-aliases/ModelNameOf",label:"ModelNameOf"},{type:"doc",id:"06-api-reference/type-aliases/OCRLanguage",label:"OCRLanguage"},{type:"doc",id:"06-api-reference/type-aliases/ResourceSource",label:"ResourceSource"},{type:"doc",id:"06-api-reference/type-aliases/SegmentationLabels",label:"SegmentationLabels"},{type:"doc",id:"06-api-reference/type-aliases/SemanticSegmentationConfig",label:"SemanticSegmentationConfig"},{type:"doc",id:"06-api-reference/type-aliases/SemanticSegmentationModelName",label:"SemanticSegmentationModelName"},{type:"doc",id:"06-api-reference/type-aliases/SemanticSegmentationModelSources",label:"SemanticSegmentationModelSources"},{type:"doc",id:"06-api-reference/type-aliases/SpeechToTextLanguage",label:"SpeechToTextLanguage"},{type:"doc",id:"06-api-reference/type-aliases/TensorBuffer",label:"TensorBuffer"},{type:"doc",id:"06-api-reference/type-aliases/TextToSpeechLanguage",label:"TextToSpeechLanguage"},{type:"doc",id:"06-api-reference/type-aliases/Triple",label:"Triple"},{type:"doc",id:"06-api-reference/variables/SPECIAL_TOKENS",label:"SPECIAL_TOKENS"}]},{type:"category",label:"Typescript API",items:[{type:"doc",id:"06-api-reference/classes/ClassificationModule",label:"ClassificationModule"},{type:"doc",id:"06-api-reference/classes/ExecutorchModule",label:"ExecutorchModule"},{type:"doc",id:"06-api-reference/classes/ImageEmbeddingsModule",label:"ImageEmbeddingsModule"},{type:"doc",id:"06-api-reference/classes/LLMModule",label:"LLMModule"},{type:"doc",id:"06-api-reference/classes/ObjectDetectionModule",label:"ObjectDetectionModule"},{type:"doc",id:"06-api-reference/classes/OCRModule",label:"OCRModule"},{type:"doc",id:"06-api-reference/classes/SemanticSegmentationModule",label:"SemanticSegmentationModule"},{type:"doc",id:"06-api-reference/classes/SpeechToTextModule",label:"SpeechToTextModule"},{type:"doc",id:"06-api-reference/classes/StyleTransferModule",label:"StyleTransferModule"},{type:"doc",id:"06-api-reference/classes/TextEmbeddingsModule",label:"TextEmbeddingsModule"},{type:"doc",id:"06-api-reference/classes/TextToImageModule",label:"TextToImageModule"},{type:"doc",id:"06-api-reference/classes/TextToSpeechModule",label:"TextToSpeechModule"},{type:"doc",id:"06-api-reference/classes/TokenizerModule",label:"TokenizerModule"},{type:"doc",id:"06-api-reference/classes/VADModule",label:"VADModule"},{type:"doc",id:"06-api-reference/classes/VerticalOCRModule",label:"VerticalOCRModule"}]},{type:"category",label:"Utilities - General",items:[{type:"category",label:"ResourceFetcherUtils",items:[{type:"category",label:"Functions",items:[{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/calculateDownloadProgress",label:"calculateDownloadProgress"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/getFilenameFromUri",label:"getFilenameFromUri"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/hashObject",label:"hashObject"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/removeFilePrefix",label:"removeFilePrefix"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/triggerHuggingFaceDownloadCounter",label:"triggerHuggingFaceDownloadCounter"}]}],link:{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/index"}},{type:"doc",id:"06-api-reference/classes/ResourceFetcher",label:"ResourceFetcher"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchConfig",label:"ExecutorchConfig"},{type:"doc",id:"06-api-reference/interfaces/ResourceFetcherAdapter",label:"ResourceFetcherAdapter"},{type:"doc",id:"06-api-reference/functions/cleanupExecutorch",label:"cleanupExecutorch"},{type:"doc",id:"06-api-reference/functions/initExecutorch",label:"initExecutorch"}]},{type:"category",label:"Utilities - LLM",items:[{type:"doc",id:"06-api-reference/variables/DEFAULT_CHAT_CONFIG",label:"DEFAULT_CHAT_CONFIG"},{type:"doc",id:"06-api-reference/variables/DEFAULT_CONTEXT_BUFFER_TOKENS",label:"DEFAULT_CONTEXT_BUFFER_TOKENS"},{type:"doc",id:"06-api-reference/variables/DEFAULT_MESSAGE_HISTORY",label:"DEFAULT_MESSAGE_HISTORY"},{type:"doc",id:"06-api-reference/variables/DEFAULT_SYSTEM_PROMPT",label:"DEFAULT_SYSTEM_PROMPT"},{type:"doc",id:"06-api-reference/variables/parseToolCall",label:"parseToolCall"},{type:"doc",id:"06-api-reference/functions/DEFAULT_STRUCTURED_OUTPUT_PROMPT",label:"DEFAULT_STRUCTURED_OUTPUT_PROMPT"},{type:"doc",id:"06-api-reference/functions/fixAndValidateStructuredOutput",label:"fixAndValidateStructuredOutput"},{type:"doc",id:"06-api-reference/functions/getStructuredOutputPrompt",label:"getStructuredOutputPrompt"}]},{type:"category",label:"Utils",items:[{type:"doc",id:"06-api-reference/classes/MessageCountContextStrategy",label:"MessageCountContextStrategy"},{type:"doc",id:"06-api-reference/classes/NoopContextStrategy",label:"NoopContextStrategy"},{type:"doc",id:"06-api-reference/classes/SlidingWindowContextStrategy",label:"SlidingWindowContextStrategy"}]}]}; +const typedocSidebar = {items:[{type:"category",label:"Hooks",items:[{type:"doc",id:"06-api-reference/functions/useClassification",label:"useClassification"},{type:"doc",id:"06-api-reference/functions/useExecutorchModule",label:"useExecutorchModule"},{type:"doc",id:"06-api-reference/functions/useImageEmbeddings",label:"useImageEmbeddings"},{type:"doc",id:"06-api-reference/functions/useImageSegmentation",label:"useImageSegmentation"},{type:"doc",id:"06-api-reference/functions/useLLM",label:"useLLM"},{type:"doc",id:"06-api-reference/functions/useObjectDetection",label:"useObjectDetection"},{type:"doc",id:"06-api-reference/functions/useOCR",label:"useOCR"},{type:"doc",id:"06-api-reference/functions/useSpeechToText",label:"useSpeechToText"},{type:"doc",id:"06-api-reference/functions/useStyleTransfer",label:"useStyleTransfer"},{type:"doc",id:"06-api-reference/functions/useTextEmbeddings",label:"useTextEmbeddings"},{type:"doc",id:"06-api-reference/functions/useTextToImage",label:"useTextToImage"},{type:"doc",id:"06-api-reference/functions/useTextToSpeech",label:"useTextToSpeech"},{type:"doc",id:"06-api-reference/functions/useTokenizer",label:"useTokenizer"},{type:"doc",id:"06-api-reference/functions/useVAD",label:"useVAD"},{type:"doc",id:"06-api-reference/functions/useVerticalOCR",label:"useVerticalOCR"}]},{type:"category",label:"Interfaces",items:[{type:"doc",id:"06-api-reference/interfaces/ResourceSourceExtended",label:"ResourceSourceExtended"}]},{type:"category",label:"Models - Classification",items:[{type:"doc",id:"06-api-reference/variables/EFFICIENTNET_V2_S",label:"EFFICIENTNET_V2_S"}]},{type:"category",label:"Models - Image Embeddings",items:[{type:"doc",id:"06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE",label:"CLIP_VIT_BASE_PATCH32_IMAGE"}]},{type:"category",label:"Models - Image Generation",items:[{type:"doc",id:"06-api-reference/variables/BK_SDM_TINY_VPRED_256",label:"BK_SDM_TINY_VPRED_256"},{type:"doc",id:"06-api-reference/variables/BK_SDM_TINY_VPRED_512",label:"BK_SDM_TINY_VPRED_512"}]},{type:"category",label:"Models - Image Segmentation",items:[{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET50",label:"DEEPLAB_V3_RESNET50"},{type:"doc",id:"06-api-reference/variables/SELFIE_SEGMENTATION",label:"SELFIE_SEGMENTATION"}]},{type:"category",label:"Models - LMM",items:[{type:"doc",id:"06-api-reference/variables/HAMMER2_1_0_5B",label:"HAMMER2_1_0_5B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_0_5B_QUANTIZED",label:"HAMMER2_1_0_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_1_5B",label:"HAMMER2_1_1_5B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_1_5B_QUANTIZED",label:"HAMMER2_1_1_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_3B",label:"HAMMER2_1_3B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_3B_QUANTIZED",label:"HAMMER2_1_3B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LFM2_5_1_2B_INSTRUCT",label:"LFM2_5_1_2B_INSTRUCT"},{type:"doc",id:"06-api-reference/variables/LFM2_5_1_2B_INSTRUCT_QUANTIZED",label:"LFM2_5_1_2B_INSTRUCT_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B",label:"LLAMA3_2_1B"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B_QLORA",label:"LLAMA3_2_1B_QLORA"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B_SPINQUANT",label:"LLAMA3_2_1B_SPINQUANT"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B",label:"LLAMA3_2_3B"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B_QLORA",label:"LLAMA3_2_3B_QLORA"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B_SPINQUANT",label:"LLAMA3_2_3B_SPINQUANT"},{type:"doc",id:"06-api-reference/variables/PHI_4_MINI_4B",label:"PHI_4_MINI_4B"},{type:"doc",id:"06-api-reference/variables/PHI_4_MINI_4B_QUANTIZED",label:"PHI_4_MINI_4B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_0_5B",label:"QWEN2_5_0_5B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_0_5B_QUANTIZED",label:"QWEN2_5_0_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_1_5B",label:"QWEN2_5_1_5B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_1_5B_QUANTIZED",label:"QWEN2_5_1_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_3B",label:"QWEN2_5_3B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_3B_QUANTIZED",label:"QWEN2_5_3B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_0_6B",label:"QWEN3_0_6B"},{type:"doc",id:"06-api-reference/variables/QWEN3_0_6B_QUANTIZED",label:"QWEN3_0_6B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_1_7B",label:"QWEN3_1_7B"},{type:"doc",id:"06-api-reference/variables/QWEN3_1_7B_QUANTIZED",label:"QWEN3_1_7B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_4B",label:"QWEN3_4B"},{type:"doc",id:"06-api-reference/variables/QWEN3_4B_QUANTIZED",label:"QWEN3_4B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_1_7B",label:"SMOLLM2_1_1_7B"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_1_7B_QUANTIZED",label:"SMOLLM2_1_1_7B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_135M",label:"SMOLLM2_1_135M"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_135M_QUANTIZED",label:"SMOLLM2_1_135M_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_360M",label:"SMOLLM2_1_360M"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_360M_QUANTIZED",label:"SMOLLM2_1_360M_QUANTIZED"}]},{type:"category",label:"Models - Object Detection",items:[{type:"doc",id:"06-api-reference/variables/RF_DETR_NANO",label:"RF_DETR_NANO"},{type:"doc",id:"06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE",label:"SSDLITE_320_MOBILENET_V3_LARGE"}]},{type:"category",label:"Models - Speech To Text",items:[{type:"doc",id:"06-api-reference/variables/WHISPER_BASE",label:"WHISPER_BASE"},{type:"doc",id:"06-api-reference/variables/WHISPER_BASE_EN",label:"WHISPER_BASE_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_SMALL",label:"WHISPER_SMALL"},{type:"doc",id:"06-api-reference/variables/WHISPER_SMALL_EN",label:"WHISPER_SMALL_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY",label:"WHISPER_TINY"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY_EN",label:"WHISPER_TINY_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED",label:"WHISPER_TINY_EN_QUANTIZED"}]},{type:"category",label:"Models - Style Transfer",items:[{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_CANDY",label:"STYLE_TRANSFER_CANDY"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_MOSAIC",label:"STYLE_TRANSFER_MOSAIC"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS",label:"STYLE_TRANSFER_RAIN_PRINCESS"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_UDNIE",label:"STYLE_TRANSFER_UDNIE"}]},{type:"category",label:"Models - Text Embeddings",items:[{type:"doc",id:"06-api-reference/variables/ALL_MINILM_L6_V2",label:"ALL_MINILM_L6_V2"},{type:"doc",id:"06-api-reference/variables/ALL_MPNET_BASE_V2",label:"ALL_MPNET_BASE_V2"},{type:"doc",id:"06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT",label:"CLIP_VIT_BASE_PATCH32_TEXT"},{type:"doc",id:"06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1",label:"MULTI_QA_MINILM_L6_COS_V1"},{type:"doc",id:"06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1",label:"MULTI_QA_MPNET_BASE_DOT_V1"}]},{type:"category",label:"Models - Text to Speech",items:[{type:"doc",id:"06-api-reference/variables/KOKORO_MEDIUM",label:"KOKORO_MEDIUM"},{type:"doc",id:"06-api-reference/variables/KOKORO_SMALL",label:"KOKORO_SMALL"}]},{type:"category",label:"Models - Voice Activity Detection",items:[{type:"doc",id:"06-api-reference/variables/FSMN_VAD",label:"FSMN_VAD"}]},{type:"category",label:"OCR Supported Alphabets",items:[{type:"doc",id:"06-api-reference/variables/OCR_ABAZA",label:"OCR_ABAZA"},{type:"doc",id:"06-api-reference/variables/OCR_ADYGHE",label:"OCR_ADYGHE"},{type:"doc",id:"06-api-reference/variables/OCR_AFRIKAANS",label:"OCR_AFRIKAANS"},{type:"doc",id:"06-api-reference/variables/OCR_ALBANIAN",label:"OCR_ALBANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_AVAR",label:"OCR_AVAR"},{type:"doc",id:"06-api-reference/variables/OCR_AZERBAIJANI",label:"OCR_AZERBAIJANI"},{type:"doc",id:"06-api-reference/variables/OCR_BELARUSIAN",label:"OCR_BELARUSIAN"},{type:"doc",id:"06-api-reference/variables/OCR_BOSNIAN",label:"OCR_BOSNIAN"},{type:"doc",id:"06-api-reference/variables/OCR_BULGARIAN",label:"OCR_BULGARIAN"},{type:"doc",id:"06-api-reference/variables/OCR_CHECHEN",label:"OCR_CHECHEN"},{type:"doc",id:"06-api-reference/variables/OCR_CROATIAN",label:"OCR_CROATIAN"},{type:"doc",id:"06-api-reference/variables/OCR_CZECH",label:"OCR_CZECH"},{type:"doc",id:"06-api-reference/variables/OCR_DANISH",label:"OCR_DANISH"},{type:"doc",id:"06-api-reference/variables/OCR_DARGWA",label:"OCR_DARGWA"},{type:"doc",id:"06-api-reference/variables/OCR_DUTCH",label:"OCR_DUTCH"},{type:"doc",id:"06-api-reference/variables/OCR_ENGLISH",label:"OCR_ENGLISH"},{type:"doc",id:"06-api-reference/variables/OCR_ESTONIAN",label:"OCR_ESTONIAN"},{type:"doc",id:"06-api-reference/variables/OCR_FRENCH",label:"OCR_FRENCH"},{type:"doc",id:"06-api-reference/variables/OCR_GERMAN",label:"OCR_GERMAN"},{type:"doc",id:"06-api-reference/variables/OCR_HUNGARIAN",label:"OCR_HUNGARIAN"},{type:"doc",id:"06-api-reference/variables/OCR_ICELANDIC",label:"OCR_ICELANDIC"},{type:"doc",id:"06-api-reference/variables/OCR_INDONESIAN",label:"OCR_INDONESIAN"},{type:"doc",id:"06-api-reference/variables/OCR_INGUSH",label:"OCR_INGUSH"},{type:"doc",id:"06-api-reference/variables/OCR_IRISH",label:"OCR_IRISH"},{type:"doc",id:"06-api-reference/variables/OCR_ITALIAN",label:"OCR_ITALIAN"},{type:"doc",id:"06-api-reference/variables/OCR_JAPANESE",label:"OCR_JAPANESE"},{type:"doc",id:"06-api-reference/variables/OCR_KANNADA",label:"OCR_KANNADA"},{type:"doc",id:"06-api-reference/variables/OCR_KARBADIAN",label:"OCR_KARBADIAN"},{type:"doc",id:"06-api-reference/variables/OCR_KOREAN",label:"OCR_KOREAN"},{type:"doc",id:"06-api-reference/variables/OCR_KURDISH",label:"OCR_KURDISH"},{type:"doc",id:"06-api-reference/variables/OCR_LAK",label:"OCR_LAK"},{type:"doc",id:"06-api-reference/variables/OCR_LATIN",label:"OCR_LATIN"},{type:"doc",id:"06-api-reference/variables/OCR_LATVIAN",label:"OCR_LATVIAN"},{type:"doc",id:"06-api-reference/variables/OCR_LEZGHIAN",label:"OCR_LEZGHIAN"},{type:"doc",id:"06-api-reference/variables/OCR_LITHUANIAN",label:"OCR_LITHUANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_MALAY",label:"OCR_MALAY"},{type:"doc",id:"06-api-reference/variables/OCR_MALTESE",label:"OCR_MALTESE"},{type:"doc",id:"06-api-reference/variables/OCR_MAORI",label:"OCR_MAORI"},{type:"doc",id:"06-api-reference/variables/OCR_MONGOLIAN",label:"OCR_MONGOLIAN"},{type:"doc",id:"06-api-reference/variables/OCR_NORWEGIAN",label:"OCR_NORWEGIAN"},{type:"doc",id:"06-api-reference/variables/OCR_OCCITAN",label:"OCR_OCCITAN"},{type:"doc",id:"06-api-reference/variables/OCR_PALI",label:"OCR_PALI"},{type:"doc",id:"06-api-reference/variables/OCR_POLISH",label:"OCR_POLISH"},{type:"doc",id:"06-api-reference/variables/OCR_PORTUGUESE",label:"OCR_PORTUGUESE"},{type:"doc",id:"06-api-reference/variables/OCR_ROMANIAN",label:"OCR_ROMANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_RUSSIAN",label:"OCR_RUSSIAN"},{type:"doc",id:"06-api-reference/variables/OCR_SERBIAN_CYRILLIC",label:"OCR_SERBIAN_CYRILLIC"},{type:"doc",id:"06-api-reference/variables/OCR_SERBIAN_LATIN",label:"OCR_SERBIAN_LATIN"},{type:"doc",id:"06-api-reference/variables/OCR_SIMPLIFIED_CHINESE",label:"OCR_SIMPLIFIED_CHINESE"},{type:"doc",id:"06-api-reference/variables/OCR_SLOVAK",label:"OCR_SLOVAK"},{type:"doc",id:"06-api-reference/variables/OCR_SLOVENIAN",label:"OCR_SLOVENIAN"},{type:"doc",id:"06-api-reference/variables/OCR_SPANISH",label:"OCR_SPANISH"},{type:"doc",id:"06-api-reference/variables/OCR_SWAHILI",label:"OCR_SWAHILI"},{type:"doc",id:"06-api-reference/variables/OCR_SWEDISH",label:"OCR_SWEDISH"},{type:"doc",id:"06-api-reference/variables/OCR_TABASSARAN",label:"OCR_TABASSARAN"},{type:"doc",id:"06-api-reference/variables/OCR_TAGALOG",label:"OCR_TAGALOG"},{type:"doc",id:"06-api-reference/variables/OCR_TAJIK",label:"OCR_TAJIK"},{type:"doc",id:"06-api-reference/variables/OCR_TELUGU",label:"OCR_TELUGU"},{type:"doc",id:"06-api-reference/variables/OCR_TURKISH",label:"OCR_TURKISH"},{type:"doc",id:"06-api-reference/variables/OCR_UKRAINIAN",label:"OCR_UKRAINIAN"},{type:"doc",id:"06-api-reference/variables/OCR_UZBEK",label:"OCR_UZBEK"},{type:"doc",id:"06-api-reference/variables/OCR_VIETNAMESE",label:"OCR_VIETNAMESE"},{type:"doc",id:"06-api-reference/variables/OCR_WELSH",label:"OCR_WELSH"}]},{type:"category",label:"Other",items:[{type:"doc",id:"06-api-reference/enumerations/RnExecutorchErrorCode",label:"RnExecutorchErrorCode"},{type:"doc",id:"06-api-reference/classes/Logger",label:"Logger"},{type:"doc",id:"06-api-reference/classes/RnExecutorchError",label:"RnExecutorchError"},{type:"doc",id:"06-api-reference/variables/IMAGENET1K_MEAN",label:"IMAGENET1K_MEAN"},{type:"doc",id:"06-api-reference/variables/IMAGENET1K_STD",label:"IMAGENET1K_STD"}]},{type:"category",label:"TTS Supported Voices",items:[{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_HEART",label:"KOKORO_VOICE_AF_HEART"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_RIVER",label:"KOKORO_VOICE_AF_RIVER"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_SARAH",label:"KOKORO_VOICE_AF_SARAH"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_ADAM",label:"KOKORO_VOICE_AM_ADAM"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_MICHAEL",label:"KOKORO_VOICE_AM_MICHAEL"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_SANTA",label:"KOKORO_VOICE_AM_SANTA"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_BF_EMMA",label:"KOKORO_VOICE_BF_EMMA"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_BM_DANIEL",label:"KOKORO_VOICE_BM_DANIEL"}]},{type:"category",label:"Types",items:[{type:"doc",id:"06-api-reference/enumerations/CocoLabel",label:"CocoLabel"},{type:"doc",id:"06-api-reference/enumerations/DeeplabLabel",label:"DeeplabLabel"},{type:"doc",id:"06-api-reference/enumerations/DownloadStatus",label:"DownloadStatus"},{type:"doc",id:"06-api-reference/enumerations/HTTP_CODE",label:"HTTP_CODE"},{type:"doc",id:"06-api-reference/enumerations/ScalarType",label:"ScalarType"},{type:"doc",id:"06-api-reference/enumerations/SelfieSegmentationLabel",label:"SelfieSegmentationLabel"},{type:"doc",id:"06-api-reference/enumerations/SourceType",label:"SourceType"},{type:"doc",id:"06-api-reference/interfaces/Bbox",label:"Bbox"},{type:"doc",id:"06-api-reference/interfaces/ChatConfig",label:"ChatConfig"},{type:"doc",id:"06-api-reference/interfaces/ClassificationProps",label:"ClassificationProps"},{type:"doc",id:"06-api-reference/interfaces/ClassificationType",label:"ClassificationType"},{type:"doc",id:"06-api-reference/interfaces/ContextStrategy",label:"ContextStrategy"},{type:"doc",id:"06-api-reference/interfaces/DecodingOptions",label:"DecodingOptions"},{type:"doc",id:"06-api-reference/interfaces/Detection",label:"Detection"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchModuleProps",label:"ExecutorchModuleProps"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchModuleType",label:"ExecutorchModuleType"},{type:"doc",id:"06-api-reference/interfaces/GenerationConfig",label:"GenerationConfig"},{type:"doc",id:"06-api-reference/interfaces/ImageEmbeddingsProps",label:"ImageEmbeddingsProps"},{type:"doc",id:"06-api-reference/interfaces/ImageEmbeddingsType",label:"ImageEmbeddingsType"},{type:"doc",id:"06-api-reference/interfaces/ImageSegmentationProps",label:"ImageSegmentationProps"},{type:"doc",id:"06-api-reference/interfaces/ImageSegmentationType",label:"ImageSegmentationType"},{type:"doc",id:"06-api-reference/interfaces/KokoroConfig",label:"KokoroConfig"},{type:"doc",id:"06-api-reference/interfaces/KokoroVoiceExtras",label:"KokoroVoiceExtras"},{type:"doc",id:"06-api-reference/interfaces/LLMConfig",label:"LLMConfig"},{type:"doc",id:"06-api-reference/interfaces/LLMProps",label:"LLMProps"},{type:"doc",id:"06-api-reference/interfaces/LLMType",label:"LLMType"},{type:"doc",id:"06-api-reference/interfaces/Message",label:"Message"},{type:"doc",id:"06-api-reference/interfaces/ObjectDetectionProps",label:"ObjectDetectionProps"},{type:"doc",id:"06-api-reference/interfaces/ObjectDetectionType",label:"ObjectDetectionType"},{type:"doc",id:"06-api-reference/interfaces/OCRDetection",label:"OCRDetection"},{type:"doc",id:"06-api-reference/interfaces/OCRProps",label:"OCRProps"},{type:"doc",id:"06-api-reference/interfaces/OCRType",label:"OCRType"},{type:"doc",id:"06-api-reference/interfaces/Point",label:"Point"},{type:"doc",id:"06-api-reference/interfaces/Segment",label:"Segment"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextModelConfig",label:"SpeechToTextModelConfig"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextProps",label:"SpeechToTextProps"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextType",label:"SpeechToTextType"},{type:"doc",id:"06-api-reference/interfaces/StyleTransferProps",label:"StyleTransferProps"},{type:"doc",id:"06-api-reference/interfaces/StyleTransferType",label:"StyleTransferType"},{type:"doc",id:"06-api-reference/interfaces/TensorPtr",label:"TensorPtr"},{type:"doc",id:"06-api-reference/interfaces/TextEmbeddingsProps",label:"TextEmbeddingsProps"},{type:"doc",id:"06-api-reference/interfaces/TextEmbeddingsType",label:"TextEmbeddingsType"},{type:"doc",id:"06-api-reference/interfaces/TextToImageProps",label:"TextToImageProps"},{type:"doc",id:"06-api-reference/interfaces/TextToImageType",label:"TextToImageType"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechConfig",label:"TextToSpeechConfig"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechInput",label:"TextToSpeechInput"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechProps",label:"TextToSpeechProps"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechStreamingInput",label:"TextToSpeechStreamingInput"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechType",label:"TextToSpeechType"},{type:"doc",id:"06-api-reference/interfaces/TokenizerProps",label:"TokenizerProps"},{type:"doc",id:"06-api-reference/interfaces/TokenizerType",label:"TokenizerType"},{type:"doc",id:"06-api-reference/interfaces/ToolCall",label:"ToolCall"},{type:"doc",id:"06-api-reference/interfaces/ToolsConfig",label:"ToolsConfig"},{type:"doc",id:"06-api-reference/interfaces/TranscriptionResult",label:"TranscriptionResult"},{type:"doc",id:"06-api-reference/interfaces/TranscriptionSegment",label:"TranscriptionSegment"},{type:"doc",id:"06-api-reference/interfaces/VADProps",label:"VADProps"},{type:"doc",id:"06-api-reference/interfaces/VADType",label:"VADType"},{type:"doc",id:"06-api-reference/interfaces/VerticalOCRProps",label:"VerticalOCRProps"},{type:"doc",id:"06-api-reference/interfaces/VoiceConfig",label:"VoiceConfig"},{type:"doc",id:"06-api-reference/interfaces/Word",label:"Word"},{type:"doc",id:"06-api-reference/type-aliases/LabelEnum",label:"LabelEnum"},{type:"doc",id:"06-api-reference/type-aliases/LLMTool",label:"LLMTool"},{type:"doc",id:"06-api-reference/type-aliases/MessageRole",label:"MessageRole"},{type:"doc",id:"06-api-reference/type-aliases/ModelNameOf",label:"ModelNameOf"},{type:"doc",id:"06-api-reference/type-aliases/ModelSources",label:"ModelSources"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionConfig",label:"ObjectDetectionConfig"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionLabels",label:"ObjectDetectionLabels"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionModelName",label:"ObjectDetectionModelName"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionModelSources",label:"ObjectDetectionModelSources"},{type:"doc",id:"06-api-reference/type-aliases/OCRLanguage",label:"OCRLanguage"},{type:"doc",id:"06-api-reference/type-aliases/ResourceSource",label:"ResourceSource"},{type:"doc",id:"06-api-reference/type-aliases/SegmentationConfig",label:"SegmentationConfig"},{type:"doc",id:"06-api-reference/type-aliases/SegmentationLabels",label:"SegmentationLabels"},{type:"doc",id:"06-api-reference/type-aliases/SegmentationModelName",label:"SegmentationModelName"},{type:"doc",id:"06-api-reference/type-aliases/SpeechToTextLanguage",label:"SpeechToTextLanguage"},{type:"doc",id:"06-api-reference/type-aliases/TensorBuffer",label:"TensorBuffer"},{type:"doc",id:"06-api-reference/type-aliases/TextToSpeechLanguage",label:"TextToSpeechLanguage"},{type:"doc",id:"06-api-reference/type-aliases/Triple",label:"Triple"},{type:"doc",id:"06-api-reference/variables/SPECIAL_TOKENS",label:"SPECIAL_TOKENS"}]},{type:"category",label:"Typescript API",items:[{type:"doc",id:"06-api-reference/classes/ClassificationModule",label:"ClassificationModule"},{type:"doc",id:"06-api-reference/classes/ExecutorchModule",label:"ExecutorchModule"},{type:"doc",id:"06-api-reference/classes/ImageEmbeddingsModule",label:"ImageEmbeddingsModule"},{type:"doc",id:"06-api-reference/classes/ImageSegmentationModule",label:"ImageSegmentationModule"},{type:"doc",id:"06-api-reference/classes/LLMModule",label:"LLMModule"},{type:"doc",id:"06-api-reference/classes/ObjectDetectionModule",label:"ObjectDetectionModule"},{type:"doc",id:"06-api-reference/classes/OCRModule",label:"OCRModule"},{type:"doc",id:"06-api-reference/classes/SpeechToTextModule",label:"SpeechToTextModule"},{type:"doc",id:"06-api-reference/classes/StyleTransferModule",label:"StyleTransferModule"},{type:"doc",id:"06-api-reference/classes/TextEmbeddingsModule",label:"TextEmbeddingsModule"},{type:"doc",id:"06-api-reference/classes/TextToImageModule",label:"TextToImageModule"},{type:"doc",id:"06-api-reference/classes/TextToSpeechModule",label:"TextToSpeechModule"},{type:"doc",id:"06-api-reference/classes/TokenizerModule",label:"TokenizerModule"},{type:"doc",id:"06-api-reference/classes/VADModule",label:"VADModule"},{type:"doc",id:"06-api-reference/classes/VerticalOCRModule",label:"VerticalOCRModule"}]},{type:"category",label:"Utilities - General",items:[{type:"category",label:"ResourceFetcherUtils",items:[{type:"category",label:"Functions",items:[{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/calculateDownloadProgress",label:"calculateDownloadProgress"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/getFilenameFromUri",label:"getFilenameFromUri"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/hashObject",label:"hashObject"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/removeFilePrefix",label:"removeFilePrefix"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/triggerHuggingFaceDownloadCounter",label:"triggerHuggingFaceDownloadCounter"}]}],link:{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/index"}},{type:"doc",id:"06-api-reference/classes/ResourceFetcher",label:"ResourceFetcher"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchConfig",label:"ExecutorchConfig"},{type:"doc",id:"06-api-reference/interfaces/ResourceFetcherAdapter",label:"ResourceFetcherAdapter"},{type:"doc",id:"06-api-reference/functions/cleanupExecutorch",label:"cleanupExecutorch"},{type:"doc",id:"06-api-reference/functions/initExecutorch",label:"initExecutorch"}]},{type:"category",label:"Utilities - LLM",items:[{type:"doc",id:"06-api-reference/variables/DEFAULT_CHAT_CONFIG",label:"DEFAULT_CHAT_CONFIG"},{type:"doc",id:"06-api-reference/variables/DEFAULT_CONTEXT_BUFFER_TOKENS",label:"DEFAULT_CONTEXT_BUFFER_TOKENS"},{type:"doc",id:"06-api-reference/variables/DEFAULT_MESSAGE_HISTORY",label:"DEFAULT_MESSAGE_HISTORY"},{type:"doc",id:"06-api-reference/variables/DEFAULT_SYSTEM_PROMPT",label:"DEFAULT_SYSTEM_PROMPT"},{type:"doc",id:"06-api-reference/variables/parseToolCall",label:"parseToolCall"},{type:"doc",id:"06-api-reference/functions/DEFAULT_STRUCTURED_OUTPUT_PROMPT",label:"DEFAULT_STRUCTURED_OUTPUT_PROMPT"},{type:"doc",id:"06-api-reference/functions/fixAndValidateStructuredOutput",label:"fixAndValidateStructuredOutput"},{type:"doc",id:"06-api-reference/functions/getStructuredOutputPrompt",label:"getStructuredOutputPrompt"}]},{type:"category",label:"Utils",items:[{type:"doc",id:"06-api-reference/classes/MessageCountContextStrategy",label:"MessageCountContextStrategy"},{type:"doc",id:"06-api-reference/classes/NoopContextStrategy",label:"NoopContextStrategy"},{type:"doc",id:"06-api-reference/classes/SlidingWindowContextStrategy",label:"SlidingWindowContextStrategy"}]}]}; module.exports = typedocSidebar.items; \ No newline at end of file diff --git a/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md b/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md index 7bd9749d6..511160773 100644 --- a/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md +++ b/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md @@ -2,7 +2,7 @@ > `const` **ALL_MINILM_L6_V2**: `object` -Defined in: [constants/modelUrls.ts:685](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L685) +Defined in: [constants/modelUrls.ts:597](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L597) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md b/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md index 64af2c816..4d142ab81 100644 --- a/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md +++ b/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md @@ -2,7 +2,7 @@ > `const` **ALL_MPNET_BASE_V2**: `object` -Defined in: [constants/modelUrls.ts:693](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L693) +Defined in: [constants/modelUrls.ts:605](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L605) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md index b52fa6de3..d58a00f63 100644 --- a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md +++ b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md @@ -2,7 +2,7 @@ > `const` **BK_SDM_TINY_VPRED_256**: `object` -Defined in: [constants/modelUrls.ts:738](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L738) +Defined in: [constants/modelUrls.ts:650](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L650) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md index 68be8ecdf..6b07fc463 100644 --- a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md +++ b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md @@ -2,7 +2,7 @@ > `const` **BK_SDM_TINY_VPRED_512**: `object` -Defined in: [constants/modelUrls.ts:727](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L727) +Defined in: [constants/modelUrls.ts:639](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L639) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md index c186e0d7b..4094e20d3 100644 --- a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md +++ b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md @@ -2,7 +2,7 @@ > `const` **CLIP_VIT_BASE_PATCH32_IMAGE**: `object` -Defined in: [constants/modelUrls.ts:666](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L666) +Defined in: [constants/modelUrls.ts:578](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L578) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md index 4f769e47b..62ce4f0aa 100644 --- a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md +++ b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md @@ -2,7 +2,7 @@ > `const` **CLIP_VIT_BASE_PATCH32_TEXT**: `object` -Defined in: [constants/modelUrls.ts:717](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L717) +Defined in: [constants/modelUrls.ts:629](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L629) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/FSMN_VAD.md b/docs/docs/06-api-reference/variables/FSMN_VAD.md index f8a961a7b..67164479d 100644 --- a/docs/docs/06-api-reference/variables/FSMN_VAD.md +++ b/docs/docs/06-api-reference/variables/FSMN_VAD.md @@ -2,7 +2,7 @@ > `const` **FSMN_VAD**: `object` -Defined in: [constants/modelUrls.ts:752](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L752) +Defined in: [constants/modelUrls.ts:664](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L664) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/IMAGENET1K_MEAN.md b/docs/docs/06-api-reference/variables/IMAGENET1K_MEAN.md new file mode 100644 index 000000000..6bd352c29 --- /dev/null +++ b/docs/docs/06-api-reference/variables/IMAGENET1K_MEAN.md @@ -0,0 +1,5 @@ +# Variable: IMAGENET1K_MEAN + +> `const` **IMAGENET1K_MEAN**: [`Triple`](../type-aliases/Triple.md)\<`number`\> + +Defined in: [constants/commonVision.ts:3](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L3) diff --git a/docs/docs/06-api-reference/variables/IMAGENET1K_STD.md b/docs/docs/06-api-reference/variables/IMAGENET1K_STD.md new file mode 100644 index 000000000..4d7e72b62 --- /dev/null +++ b/docs/docs/06-api-reference/variables/IMAGENET1K_STD.md @@ -0,0 +1,5 @@ +# Variable: IMAGENET1K_STD + +> `const` **IMAGENET1K_STD**: [`Triple`](../type-aliases/Triple.md)\<`number`\> + +Defined in: [constants/commonVision.ts:4](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/commonVision.ts#L4) diff --git a/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md b/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md index 5bde1d509..cb84c3f8a 100644 --- a/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md +++ b/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md @@ -2,7 +2,7 @@ > `const` **MULTI_QA_MINILM_L6_COS_V1**: `object` -Defined in: [constants/modelUrls.ts:701](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L701) +Defined in: [constants/modelUrls.ts:613](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L613) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md b/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md index 7ab08cb1d..084cd8537 100644 --- a/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md +++ b/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md @@ -2,7 +2,7 @@ > `const` **MULTI_QA_MPNET_BASE_DOT_V1**: `object` -Defined in: [constants/modelUrls.ts:709](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L709) +Defined in: [constants/modelUrls.ts:621](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L621) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/RF_DETR_NANO.md b/docs/docs/06-api-reference/variables/RF_DETR_NANO.md new file mode 100644 index 000000000..5f7c5df5d --- /dev/null +++ b/docs/docs/06-api-reference/variables/RF_DETR_NANO.md @@ -0,0 +1,15 @@ +# Variable: RF_DETR_NANO + +> `const` **RF_DETR_NANO**: `object` + +Defined in: [constants/modelUrls.ts:403](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L403) + +## Type Declaration + +### modelName + +> `readonly` **modelName**: `"rf-detr-nano"` = `'rf-detr-nano'` + +### modelSource + +> `readonly` **modelSource**: `"https://ai.swmansion.com/storage/jc_tests/rfdetr_det.pte"` = `RF_DETR_NANO_MODEL` diff --git a/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md b/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md index 5ecb1b70d..48a80df41 100644 --- a/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md +++ b/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md @@ -2,7 +2,7 @@ > `const` **SELFIE_SEGMENTATION**: `object` -Defined in: [constants/modelUrls.ts:655](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L655) +Defined in: [constants/modelUrls.ts:567](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L567) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE.md b/docs/docs/06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE.md index 3c7e92612..97c7caa67 100644 --- a/docs/docs/06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE.md +++ b/docs/docs/06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE.md @@ -2,10 +2,14 @@ > `const` **SSDLITE_320_MOBILENET_V3_LARGE**: `object` -Defined in: [constants/modelUrls.ts:393](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L393) +Defined in: [constants/modelUrls.ts:395](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L395) ## Type Declaration +### modelName + +> `readonly` **modelName**: `"ssdlite-320-mobilenet-v3-large"` = `'ssdlite-320-mobilenet-v3-large'` + ### modelSource -> **modelSource**: `string` = `SSDLITE_320_MOBILENET_V3_LARGE_MODEL` +> `readonly` **modelSource**: `"https://huggingface.co/software-mansion/react-native-executorch-ssdlite320-mobilenet-v3-large/resolve/v0.7.0/ssdlite320-mobilenetv3-large.pte"` = `SSDLITE_320_MOBILENET_V3_LARGE_MODEL` diff --git a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_CANDY.md b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_CANDY.md index 1816ba019..f05899c97 100644 --- a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_CANDY.md +++ b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_CANDY.md @@ -2,7 +2,7 @@ > `const` **STYLE_TRANSFER_CANDY**: `object` -Defined in: [constants/modelUrls.ts:418](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L418) +Defined in: [constants/modelUrls.ts:429](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L429) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_MOSAIC.md b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_MOSAIC.md index a0e662ebf..ad5815694 100644 --- a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_MOSAIC.md +++ b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_MOSAIC.md @@ -2,7 +2,7 @@ > `const` **STYLE_TRANSFER_MOSAIC**: `object` -Defined in: [constants/modelUrls.ts:425](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L425) +Defined in: [constants/modelUrls.ts:436](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L436) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS.md b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS.md index 495456d46..c3dc51d4c 100644 --- a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS.md +++ b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS.md @@ -2,7 +2,7 @@ > `const` **STYLE_TRANSFER_RAIN_PRINCESS**: `object` -Defined in: [constants/modelUrls.ts:432](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L432) +Defined in: [constants/modelUrls.ts:443](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L443) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_UDNIE.md b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_UDNIE.md index 9a75aa0b4..61e62868e 100644 --- a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_UDNIE.md +++ b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_UDNIE.md @@ -2,7 +2,7 @@ > `const` **STYLE_TRANSFER_UDNIE**: `object` -Defined in: [constants/modelUrls.ts:439](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L439) +Defined in: [constants/modelUrls.ts:450](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L450) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_BASE.md b/docs/docs/06-api-reference/variables/WHISPER_BASE.md index 9c3ddcf7c..9a2639f90 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_BASE.md +++ b/docs/docs/06-api-reference/variables/WHISPER_BASE.md @@ -2,7 +2,7 @@ > `const` **WHISPER_BASE**: `object` -Defined in: [constants/modelUrls.ts:524](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L524) +Defined in: [constants/modelUrls.ts:535](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L535) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_BASE_EN.md b/docs/docs/06-api-reference/variables/WHISPER_BASE_EN.md index 6e3d13151..8458f5e4d 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_BASE_EN.md +++ b/docs/docs/06-api-reference/variables/WHISPER_BASE_EN.md @@ -2,7 +2,7 @@ > `const` **WHISPER_BASE_EN**: `object` -Defined in: [constants/modelUrls.ts:494](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L494) +Defined in: [constants/modelUrls.ts:505](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L505) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_SMALL.md b/docs/docs/06-api-reference/variables/WHISPER_SMALL.md index 73af6e193..556bfd9f1 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_SMALL.md +++ b/docs/docs/06-api-reference/variables/WHISPER_SMALL.md @@ -2,7 +2,7 @@ > `const` **WHISPER_SMALL**: `object` -Defined in: [constants/modelUrls.ts:534](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L534) +Defined in: [constants/modelUrls.ts:545](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L545) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_SMALL_EN.md b/docs/docs/06-api-reference/variables/WHISPER_SMALL_EN.md index d3efa2af5..4298dd126 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_SMALL_EN.md +++ b/docs/docs/06-api-reference/variables/WHISPER_SMALL_EN.md @@ -2,7 +2,7 @@ > `const` **WHISPER_SMALL_EN**: `object` -Defined in: [constants/modelUrls.ts:504](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L504) +Defined in: [constants/modelUrls.ts:515](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L515) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_TINY.md b/docs/docs/06-api-reference/variables/WHISPER_TINY.md index 1922e8578..8f2488d08 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_TINY.md +++ b/docs/docs/06-api-reference/variables/WHISPER_TINY.md @@ -2,7 +2,7 @@ > `const` **WHISPER_TINY**: `object` -Defined in: [constants/modelUrls.ts:514](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L514) +Defined in: [constants/modelUrls.ts:525](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L525) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_TINY_EN.md b/docs/docs/06-api-reference/variables/WHISPER_TINY_EN.md index 6d37a6f8c..02b02eea3 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_TINY_EN.md +++ b/docs/docs/06-api-reference/variables/WHISPER_TINY_EN.md @@ -2,7 +2,7 @@ > `const` **WHISPER_TINY_EN**: `object` -Defined in: [constants/modelUrls.ts:474](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L474) +Defined in: [constants/modelUrls.ts:485](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L485) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED.md b/docs/docs/06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED.md index f0e86324e..9700c48c4 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED.md +++ b/docs/docs/06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED.md @@ -2,7 +2,7 @@ > `const` **WHISPER_TINY_EN_QUANTIZED**: `object` -Defined in: [constants/modelUrls.ts:484](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L484) +Defined in: [constants/modelUrls.ts:495](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L495) ## Type Declaration From 071639ca4c645c92175485a31f0b74dba0199feb Mon Sep 17 00:00:00 2001 From: chmjkb Date: Wed, 25 Feb 2026 18:51:49 +0100 Subject: [PATCH 07/24] fix: add proper model url --- packages/react-native-executorch/src/constants/modelUrls.ts | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/packages/react-native-executorch/src/constants/modelUrls.ts b/packages/react-native-executorch/src/constants/modelUrls.ts index c648775e0..f3d382375 100644 --- a/packages/react-native-executorch/src/constants/modelUrls.ts +++ b/packages/react-native-executorch/src/constants/modelUrls.ts @@ -386,8 +386,7 @@ export const EFFICIENTNET_V2_S = { // Object detection const SSDLITE_320_MOBILENET_V3_LARGE_MODEL = `${URL_PREFIX}-ssdlite320-mobilenet-v3-large/${VERSION_TAG}/ssdlite320-mobilenetv3-large.pte`; -// const RF_DETR_NANO_MODEL = `${URL_PREFIX}-rf-detr-nano/${VERSION_TAG}/rf-detr-nano.pte`; -const RF_DETR_NANO_MODEL = `https://ai.swmansion.com/storage/jc_tests/rfdetr_det.pte`; +const RF_DETR_NANO_MODEL = `${URL_PREFIX}-rf-detr-nano-detector/${VERSION_TAG}/rf-detr-nano.pte`; /** * @category Models - Object Detection From 62f87fcb80855c19db49f82e35765d9e000438b1 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Wed, 25 Feb 2026 19:02:30 +0100 Subject: [PATCH 08/24] fix: use proper version tag for rfdetr --- packages/react-native-executorch/src/constants/modelUrls.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/react-native-executorch/src/constants/modelUrls.ts b/packages/react-native-executorch/src/constants/modelUrls.ts index f3d382375..f2cadc2dd 100644 --- a/packages/react-native-executorch/src/constants/modelUrls.ts +++ b/packages/react-native-executorch/src/constants/modelUrls.ts @@ -386,7 +386,7 @@ export const EFFICIENTNET_V2_S = { // Object detection const SSDLITE_320_MOBILENET_V3_LARGE_MODEL = `${URL_PREFIX}-ssdlite320-mobilenet-v3-large/${VERSION_TAG}/ssdlite320-mobilenetv3-large.pte`; -const RF_DETR_NANO_MODEL = `${URL_PREFIX}-rf-detr-nano-detector/${VERSION_TAG}/rf-detr-nano.pte`; +const RF_DETR_NANO_MODEL = `${URL_PREFIX}-rf-detr-nano-detector/${NEXT_VERSION_TAG}/rf-detr-nano.pte`; /** * @category Models - Object Detection From 2e57377456092472a0903fdabd9f0225487c7d3a Mon Sep 17 00:00:00 2001 From: chmjkb Date: Thu, 26 Feb 2026 10:39:39 +0100 Subject: [PATCH 09/24] docs: update api reference --- docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md | 2 +- docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md | 2 +- docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md | 2 +- docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md | 2 +- .../06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md | 2 +- .../06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md | 2 +- docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50.md | 2 +- docs/docs/06-api-reference/variables/FSMN_VAD.md | 2 +- .../06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md | 2 +- .../06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md | 2 +- docs/docs/06-api-reference/variables/RF_DETR_NANO.md | 4 ++-- docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md | 2 +- .../variables/SSDLITE_320_MOBILENET_V3_LARGE.md | 2 +- docs/docs/06-api-reference/variables/STYLE_TRANSFER_CANDY.md | 2 +- docs/docs/06-api-reference/variables/STYLE_TRANSFER_MOSAIC.md | 2 +- .../variables/STYLE_TRANSFER_RAIN_PRINCESS.md | 2 +- docs/docs/06-api-reference/variables/STYLE_TRANSFER_UDNIE.md | 2 +- docs/docs/06-api-reference/variables/WHISPER_BASE.md | 2 +- docs/docs/06-api-reference/variables/WHISPER_BASE_EN.md | 2 +- docs/docs/06-api-reference/variables/WHISPER_SMALL.md | 2 +- docs/docs/06-api-reference/variables/WHISPER_SMALL_EN.md | 2 +- docs/docs/06-api-reference/variables/WHISPER_TINY.md | 2 +- docs/docs/06-api-reference/variables/WHISPER_TINY_EN.md | 2 +- .../06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED.md | 2 +- 24 files changed, 25 insertions(+), 25 deletions(-) diff --git a/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md b/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md index 511160773..9df9d5e64 100644 --- a/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md +++ b/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md @@ -2,7 +2,7 @@ > `const` **ALL_MINILM_L6_V2**: `object` -Defined in: [constants/modelUrls.ts:597](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L597) +Defined in: [constants/modelUrls.ts:596](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L596) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md b/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md index 4d142ab81..b996eeda7 100644 --- a/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md +++ b/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md @@ -2,7 +2,7 @@ > `const` **ALL_MPNET_BASE_V2**: `object` -Defined in: [constants/modelUrls.ts:605](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L605) +Defined in: [constants/modelUrls.ts:604](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L604) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md index d58a00f63..01dd61729 100644 --- a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md +++ b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md @@ -2,7 +2,7 @@ > `const` **BK_SDM_TINY_VPRED_256**: `object` -Defined in: [constants/modelUrls.ts:650](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L650) +Defined in: [constants/modelUrls.ts:649](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L649) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md index 6b07fc463..6e4f4c0c4 100644 --- a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md +++ b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md @@ -2,7 +2,7 @@ > `const` **BK_SDM_TINY_VPRED_512**: `object` -Defined in: [constants/modelUrls.ts:639](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L639) +Defined in: [constants/modelUrls.ts:638](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L638) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md index 4094e20d3..1ae2347d6 100644 --- a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md +++ b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md @@ -2,7 +2,7 @@ > `const` **CLIP_VIT_BASE_PATCH32_IMAGE**: `object` -Defined in: [constants/modelUrls.ts:578](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L578) +Defined in: [constants/modelUrls.ts:577](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L577) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md index 62ce4f0aa..99df027e5 100644 --- a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md +++ b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md @@ -2,7 +2,7 @@ > `const` **CLIP_VIT_BASE_PATCH32_TEXT**: `object` -Defined in: [constants/modelUrls.ts:629](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L629) +Defined in: [constants/modelUrls.ts:628](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L628) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50.md b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50.md index 55ed0b222..09f2f1641 100644 --- a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50.md +++ b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50.md @@ -2,7 +2,7 @@ > `const` **DEEPLAB_V3_RESNET50**: `object` -Defined in: [constants/modelUrls.ts:558](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L558) +Defined in: [constants/modelUrls.ts:557](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L557) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/FSMN_VAD.md b/docs/docs/06-api-reference/variables/FSMN_VAD.md index 67164479d..f91c2cb97 100644 --- a/docs/docs/06-api-reference/variables/FSMN_VAD.md +++ b/docs/docs/06-api-reference/variables/FSMN_VAD.md @@ -2,7 +2,7 @@ > `const` **FSMN_VAD**: `object` -Defined in: [constants/modelUrls.ts:664](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L664) +Defined in: [constants/modelUrls.ts:663](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L663) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md b/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md index cb84c3f8a..ebbed0cf4 100644 --- a/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md +++ b/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md @@ -2,7 +2,7 @@ > `const` **MULTI_QA_MINILM_L6_COS_V1**: `object` -Defined in: [constants/modelUrls.ts:613](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L613) +Defined in: [constants/modelUrls.ts:612](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L612) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md b/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md index 084cd8537..e689be26d 100644 --- a/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md +++ b/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md @@ -2,7 +2,7 @@ > `const` **MULTI_QA_MPNET_BASE_DOT_V1**: `object` -Defined in: [constants/modelUrls.ts:621](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L621) +Defined in: [constants/modelUrls.ts:620](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L620) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/RF_DETR_NANO.md b/docs/docs/06-api-reference/variables/RF_DETR_NANO.md index 5f7c5df5d..857746ce0 100644 --- a/docs/docs/06-api-reference/variables/RF_DETR_NANO.md +++ b/docs/docs/06-api-reference/variables/RF_DETR_NANO.md @@ -2,7 +2,7 @@ > `const` **RF_DETR_NANO**: `object` -Defined in: [constants/modelUrls.ts:403](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L403) +Defined in: [constants/modelUrls.ts:402](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L402) ## Type Declaration @@ -12,4 +12,4 @@ Defined in: [constants/modelUrls.ts:403](https://github.com/software-mansion/rea ### modelSource -> `readonly` **modelSource**: `"https://ai.swmansion.com/storage/jc_tests/rfdetr_det.pte"` = `RF_DETR_NANO_MODEL` +> `readonly` **modelSource**: `"https://huggingface.co/software-mansion/react-native-executorch-rf-detr-nano-detector/resolve/v0.8.0/rf-detr-nano.pte"` = `RF_DETR_NANO_MODEL` diff --git a/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md b/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md index 48a80df41..90b55917a 100644 --- a/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md +++ b/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md @@ -2,7 +2,7 @@ > `const` **SELFIE_SEGMENTATION**: `object` -Defined in: [constants/modelUrls.ts:567](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L567) +Defined in: [constants/modelUrls.ts:566](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L566) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE.md b/docs/docs/06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE.md index 97c7caa67..4c28e4ff4 100644 --- a/docs/docs/06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE.md +++ b/docs/docs/06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE.md @@ -2,7 +2,7 @@ > `const` **SSDLITE_320_MOBILENET_V3_LARGE**: `object` -Defined in: [constants/modelUrls.ts:395](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L395) +Defined in: [constants/modelUrls.ts:394](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L394) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_CANDY.md b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_CANDY.md index f05899c97..c82979642 100644 --- a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_CANDY.md +++ b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_CANDY.md @@ -2,7 +2,7 @@ > `const` **STYLE_TRANSFER_CANDY**: `object` -Defined in: [constants/modelUrls.ts:429](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L429) +Defined in: [constants/modelUrls.ts:428](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L428) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_MOSAIC.md b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_MOSAIC.md index ad5815694..f57bb8859 100644 --- a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_MOSAIC.md +++ b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_MOSAIC.md @@ -2,7 +2,7 @@ > `const` **STYLE_TRANSFER_MOSAIC**: `object` -Defined in: [constants/modelUrls.ts:436](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L436) +Defined in: [constants/modelUrls.ts:435](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L435) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS.md b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS.md index c3dc51d4c..8cf07fd19 100644 --- a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS.md +++ b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS.md @@ -2,7 +2,7 @@ > `const` **STYLE_TRANSFER_RAIN_PRINCESS**: `object` -Defined in: [constants/modelUrls.ts:443](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L443) +Defined in: [constants/modelUrls.ts:442](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L442) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_UDNIE.md b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_UDNIE.md index 61e62868e..20cd7cb3b 100644 --- a/docs/docs/06-api-reference/variables/STYLE_TRANSFER_UDNIE.md +++ b/docs/docs/06-api-reference/variables/STYLE_TRANSFER_UDNIE.md @@ -2,7 +2,7 @@ > `const` **STYLE_TRANSFER_UDNIE**: `object` -Defined in: [constants/modelUrls.ts:450](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L450) +Defined in: [constants/modelUrls.ts:449](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L449) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_BASE.md b/docs/docs/06-api-reference/variables/WHISPER_BASE.md index 9a2639f90..c48856a93 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_BASE.md +++ b/docs/docs/06-api-reference/variables/WHISPER_BASE.md @@ -2,7 +2,7 @@ > `const` **WHISPER_BASE**: `object` -Defined in: [constants/modelUrls.ts:535](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L535) +Defined in: [constants/modelUrls.ts:534](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L534) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_BASE_EN.md b/docs/docs/06-api-reference/variables/WHISPER_BASE_EN.md index 8458f5e4d..ab44fad16 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_BASE_EN.md +++ b/docs/docs/06-api-reference/variables/WHISPER_BASE_EN.md @@ -2,7 +2,7 @@ > `const` **WHISPER_BASE_EN**: `object` -Defined in: [constants/modelUrls.ts:505](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L505) +Defined in: [constants/modelUrls.ts:504](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L504) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_SMALL.md b/docs/docs/06-api-reference/variables/WHISPER_SMALL.md index 556bfd9f1..dde353ccc 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_SMALL.md +++ b/docs/docs/06-api-reference/variables/WHISPER_SMALL.md @@ -2,7 +2,7 @@ > `const` **WHISPER_SMALL**: `object` -Defined in: [constants/modelUrls.ts:545](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L545) +Defined in: [constants/modelUrls.ts:544](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L544) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_SMALL_EN.md b/docs/docs/06-api-reference/variables/WHISPER_SMALL_EN.md index 4298dd126..b832832c3 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_SMALL_EN.md +++ b/docs/docs/06-api-reference/variables/WHISPER_SMALL_EN.md @@ -2,7 +2,7 @@ > `const` **WHISPER_SMALL_EN**: `object` -Defined in: [constants/modelUrls.ts:515](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L515) +Defined in: [constants/modelUrls.ts:514](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L514) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_TINY.md b/docs/docs/06-api-reference/variables/WHISPER_TINY.md index 8f2488d08..27f0abd75 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_TINY.md +++ b/docs/docs/06-api-reference/variables/WHISPER_TINY.md @@ -2,7 +2,7 @@ > `const` **WHISPER_TINY**: `object` -Defined in: [constants/modelUrls.ts:525](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L525) +Defined in: [constants/modelUrls.ts:524](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L524) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_TINY_EN.md b/docs/docs/06-api-reference/variables/WHISPER_TINY_EN.md index 02b02eea3..b54b13ba8 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_TINY_EN.md +++ b/docs/docs/06-api-reference/variables/WHISPER_TINY_EN.md @@ -2,7 +2,7 @@ > `const` **WHISPER_TINY_EN**: `object` -Defined in: [constants/modelUrls.ts:485](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L485) +Defined in: [constants/modelUrls.ts:484](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L484) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED.md b/docs/docs/06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED.md index 9700c48c4..29266d1b7 100644 --- a/docs/docs/06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED.md +++ b/docs/docs/06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED.md @@ -2,7 +2,7 @@ > `const` **WHISPER_TINY_EN_QUANTIZED**: `object` -Defined in: [constants/modelUrls.ts:495](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L495) +Defined in: [constants/modelUrls.ts:494](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L494) ## Type Declaration From d52a05f6ba83c68f30a88f13cf23ef0ff7c82eb9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mateusz=20S=C5=82uszniak?= Date: Thu, 26 Feb 2026 11:48:22 +0100 Subject: [PATCH 10/24] chore: fix link name --- packages/react-native-executorch/src/constants/modelUrls.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/react-native-executorch/src/constants/modelUrls.ts b/packages/react-native-executorch/src/constants/modelUrls.ts index f2cadc2dd..e19801cfd 100644 --- a/packages/react-native-executorch/src/constants/modelUrls.ts +++ b/packages/react-native-executorch/src/constants/modelUrls.ts @@ -386,7 +386,7 @@ export const EFFICIENTNET_V2_S = { // Object detection const SSDLITE_320_MOBILENET_V3_LARGE_MODEL = `${URL_PREFIX}-ssdlite320-mobilenet-v3-large/${VERSION_TAG}/ssdlite320-mobilenetv3-large.pte`; -const RF_DETR_NANO_MODEL = `${URL_PREFIX}-rf-detr-nano-detector/${NEXT_VERSION_TAG}/rf-detr-nano.pte`; +const RF_DETR_NANO_MODEL = `${URL_PREFIX}-rfdetr-nano-detector/${NEXT_VERSION_TAG}/rfdetr_detector.pte`; /** * @category Models - Object Detection From f2aafbd33e359576266e8e0e413dceb68167ee52 Mon Sep 17 00:00:00 2001 From: Jakub Chmura <92989966+chmjkb@users.noreply.github.com> Date: Fri, 27 Feb 2026 18:20:10 +0100 Subject: [PATCH 11/24] Update packages/react-native-executorch/common/rnexecutorch/models/object_detection/Constants.h Co-authored-by: Mateusz Sluszniak <56299341+msluszniak@users.noreply.github.com> --- .../common/rnexecutorch/models/object_detection/Constants.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Constants.h b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Constants.h index 9b3fd147a..95d58d596 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Constants.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/Constants.h @@ -1,5 +1,5 @@ #pragma once namespace rnexecutorch::models::object_detection::constants { -inline constexpr float IOU_THRESHOLD = 0.55; +inline constexpr float IOU_THRESHOLD = 0.55f; } // namespace rnexecutorch::models::object_detection::constants \ No newline at end of file From 6351ad6e79549c12d70821d90fafd88c1d9d7173 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Fri, 27 Feb 2026 18:27:06 +0100 Subject: [PATCH 12/24] chore: remove unused constructor for image segmenattion --- .../semantic_segmentation/BaseSemanticSegmentation.cpp | 7 ------- .../semantic_segmentation/BaseSemanticSegmentation.h | 3 --- 2 files changed, 10 deletions(-) diff --git a/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.cpp b/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.cpp index 740e78d43..4b1e8b313 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.cpp @@ -11,13 +11,6 @@ namespace rnexecutorch::models::semantic_segmentation { -BaseSemanticSegmentation::BaseSemanticSegmentation( - const std::string &modelSource, - std::shared_ptr callInvoker) - : BaseModel(modelSource, callInvoker) { - initModelImageSize(); -} - BaseSemanticSegmentation::BaseSemanticSegmentation( const std::string &modelSource, std::vector normMean, std::vector normStd, std::shared_ptr callInvoker) diff --git a/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.h b/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.h index 16a9ecfe7..2f015b64a 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.h @@ -19,9 +19,6 @@ using executorch::extension::TensorPtr; class BaseSemanticSegmentation : public BaseModel { public: - BaseSemanticSegmentation(const std::string &modelSource, - std::shared_ptr callInvoker); - BaseSemanticSegmentation(const std::string &modelSource, std::vector normMean, std::vector normStd, From 6701a48e60671c701630029ca7b7ba2233a4eeef Mon Sep 17 00:00:00 2001 From: chmjkb Date: Mon, 2 Mar 2026 11:51:39 +0100 Subject: [PATCH 13/24] review suggestions --- .../object_detection/ObjectDetection.cpp | 43 +++--- .../models/object_detection/ObjectDetection.h | 22 ++- .../BaseSemanticSegmentation.cpp | 13 +- .../BaseSemanticSegmentation.h | 5 +- .../tests/integration/ObjectDetectionTest.cpp | 126 ++++++++++++------ packages/react-native-executorch/src/index.ts | 6 +- .../computer_vision/ObjectDetectionModule.ts | 42 +++--- .../SemanticSegmentationModule.ts | 16 +-- 8 files changed, 153 insertions(+), 120 deletions(-) diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp index 37b4114b2..9abea1377 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp @@ -10,9 +10,10 @@ namespace rnexecutorch::models::object_detection { ObjectDetection::ObjectDetection( - const std::string &modelSource, + const std::string &modelSource, std::vector normMean, + std::vector normStd, std::vector labelNames, std::shared_ptr callInvoker) - : VisionModel(modelSource, callInvoker) { + : VisionModel(modelSource, callInvoker), labelNames_(std::move(labelNames)) { auto inputTensors = getAllInputShapes(); if (inputTensors.empty()) { throw RnExecutorchError(RnExecutorchErrorCode::UnexpectedNumInputs, @@ -30,20 +31,14 @@ ObjectDetection::ObjectDetection( } modelImageSize = cv::Size(modelInputShape[modelInputShape.size() - 1], modelInputShape[modelInputShape.size() - 2]); -} - -ObjectDetection::ObjectDetection( - const std::string &modelSource, std::vector normMean, - std::vector normStd, std::shared_ptr callInvoker) - : ObjectDetection(modelSource, callInvoker) { if (normMean.size() == 3) { - normMean_ = std::move(normMean); + normMean_ = cv::Scalar(normMean[0], normMean[1], normMean[2]); } else if (!normMean.empty()) { log(LOG_LEVEL::Warn, "normMean must have 3 elements — ignoring provided value."); } if (normStd.size() == 3) { - normStd_ = std::move(normStd); + normStd_ = cv::Scalar(normStd[0], normStd[1], normStd[2]); } else if (!normStd.empty()) { log(LOG_LEVEL::Warn, "normStd must have 3 elements — ignoring provided value."); @@ -126,8 +121,7 @@ ObjectDetection::postprocess(const std::vector &tensors, } std::vector -ObjectDetection::runInference(cv::Mat image, double detectionThreshold, - const std::vector &labelNames) { +ObjectDetection::runInference(cv::Mat image, double detectionThreshold) { if (detectionThreshold < 0.0 || detectionThreshold > 1.0) { throw RnExecutorchError(RnExecutorchErrorCode::InvalidUserInput, "detectionThreshold must be in range [0, 1]"); @@ -139,11 +133,9 @@ ObjectDetection::runInference(cv::Mat image, double detectionThreshold, const std::vector tensorDims = getAllInputShapes()[0]; auto inputTensor = - (!normMean_.empty() && !normStd_.empty()) - ? image_processing::getTensorFromMatrix( - tensorDims, preprocessed, - cv::Scalar(normMean_[0], normMean_[1], normMean_[2]), - cv::Scalar(normStd_[0], normStd_[1], normStd_[2])) + (normMean_ && normStd_) + ? image_processing::getTensorFromMatrix(tensorDims, preprocessed, + *normMean_, *normStd_) : image_processing::getTensorFromMatrix(tensorDims, preprocessed); auto forwardResult = BaseModel::forward(inputTensor); @@ -154,38 +146,35 @@ ObjectDetection::runInference(cv::Mat image, double detectionThreshold, } return postprocess(forwardResult.get(), originalSize, detectionThreshold, - labelNames); + labelNames_); } std::vector ObjectDetection::generateFromString(std::string imageSource, - double detectionThreshold, - std::vector labelNames) { + double detectionThreshold) { cv::Mat imageBGR = image_processing::readImage(imageSource); cv::Mat imageRGB; cv::cvtColor(imageBGR, imageRGB, cv::COLOR_BGR2RGB); - return runInference(imageRGB, detectionThreshold, labelNames); + return runInference(imageRGB, detectionThreshold); } std::vector ObjectDetection::generateFromFrame(jsi::Runtime &runtime, const jsi::Value &frameData, - double detectionThreshold, - std::vector labelNames) { + double detectionThreshold) { auto frameObj = frameData.asObject(runtime); cv::Mat frame = rnexecutorch::utils::extractFrame(runtime, frameObj); - return runInference(frame, detectionThreshold, labelNames); + return runInference(frame, detectionThreshold); } std::vector ObjectDetection::generateFromPixels(JSTensorViewIn pixelData, - double detectionThreshold, - std::vector labelNames) { + double detectionThreshold) { cv::Mat image = extractFromPixels(pixelData); - return runInference(image, detectionThreshold, labelNames); + return runInference(image, detectionThreshold); } } // namespace rnexecutorch::models::object_detection diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h index 9453cd94e..2fc23c710 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h @@ -3,6 +3,7 @@ #include #include #include +#include #include "Types.h" #include "rnexecutorch/metaprogramming/ConstructorHelpers.h" @@ -16,26 +17,21 @@ using executorch::runtime::EValue; class ObjectDetection : public VisionModel { public: - ObjectDetection(const std::string &modelSource, - std::shared_ptr callInvoker); ObjectDetection(const std::string &modelSource, std::vector normMean, std::vector normStd, + std::vector labelNames, std::shared_ptr callInvoker); [[nodiscard("Registered non-void function")]] std::vector - generateFromString(std::string imageSource, double detectionThreshold, - std::vector labelNames); + generateFromString(std::string imageSource, double detectionThreshold); [[nodiscard("Registered non-void function")]] std::vector generateFromFrame(jsi::Runtime &runtime, const jsi::Value &frameData, - double detectionThreshold, - std::vector labelNames); + double detectionThreshold); [[nodiscard("Registered non-void function")]] std::vector - generateFromPixels(JSTensorViewIn pixelData, double detectionThreshold, - std::vector labelNames); + generateFromPixels(JSTensorViewIn pixelData, double detectionThreshold); protected: std::vector - runInference(cv::Mat image, double detectionThreshold, - const std::vector &labelNames); + runInference(cv::Mat image, double detectionThreshold); cv::Mat preprocessFrame(const cv::Mat &frame) const override; private: @@ -45,12 +41,14 @@ class ObjectDetection : public VisionModel { const std::vector &labelNames); cv::Size modelImageSize{0, 0}; - std::vector normMean_; - std::vector normStd_; + std::optional normMean_; + std::optional normStd_; + std::vector labelNames_; }; } // namespace models::object_detection REGISTER_CONSTRUCTOR(models::object_detection::ObjectDetection, std::string, std::vector, std::vector, + std::vector, std::shared_ptr); } // namespace rnexecutorch diff --git a/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.cpp b/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.cpp index 4b1e8b313..fc6a04ebc 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.cpp @@ -13,18 +13,19 @@ namespace rnexecutorch::models::semantic_segmentation { BaseSemanticSegmentation::BaseSemanticSegmentation( const std::string &modelSource, std::vector normMean, - std::vector normStd, std::shared_ptr callInvoker) - : BaseModel(modelSource, callInvoker) { + std::vector normStd, std::vector allClasses, + std::shared_ptr callInvoker) + : BaseModel(modelSource, callInvoker), allClasses_(std::move(allClasses)) { initModelImageSize(); if (normMean.size() == 3) { normMean_ = cv::Scalar(normMean[0], normMean[1], normMean[2]); - } else { + } else if (!normMean.empty()) { log(LOG_LEVEL::Warn, "normMean must have 3 elements — ignoring provided value."); } if (normStd.size() == 3) { normStd_ = cv::Scalar(normStd[0], normStd[1], normStd[2]); - } else { + } else if (!normStd.empty()) { log(LOG_LEVEL::Warn, "normStd must have 3 elements — ignoring provided value."); } @@ -57,7 +58,7 @@ TensorPtr BaseSemanticSegmentation::preprocess(const std::string &imageSource, } std::shared_ptr BaseSemanticSegmentation::generate( - std::string imageSource, std::vector allClasses, + std::string imageSource, std::set> classesOfInterest, bool resize) { cv::Size originalSize; @@ -71,7 +72,7 @@ std::shared_ptr BaseSemanticSegmentation::generate( "Ensure the model input is correct."); } - return postprocess(forwardResult->at(0).toTensor(), originalSize, allClasses, + return postprocess(forwardResult->at(0).toTensor(), originalSize, allClasses_, classesOfInterest, resize); } diff --git a/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.h b/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.h index 2f015b64a..d39a7e5d4 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/semantic_segmentation/BaseSemanticSegmentation.h @@ -22,10 +22,11 @@ class BaseSemanticSegmentation : public BaseModel { BaseSemanticSegmentation(const std::string &modelSource, std::vector normMean, std::vector normStd, + std::vector allClasses, std::shared_ptr callInvoker); [[nodiscard("Registered non-void function")]] std::shared_ptr - generate(std::string imageSource, std::vector allClasses, + generate(std::string imageSource, std::set> classesOfInterest, bool resize); protected: @@ -41,6 +42,7 @@ class BaseSemanticSegmentation : public BaseModel { std::size_t numModelPixels; std::optional normMean_; std::optional normStd_; + std::vector allClasses_; std::shared_ptr populateDictionary( std::shared_ptr argmax, @@ -55,5 +57,6 @@ class BaseSemanticSegmentation : public BaseModel { REGISTER_CONSTRUCTOR(models::semantic_segmentation::BaseSemanticSegmentation, std::string, std::vector, std::vector, + std::vector, std::shared_ptr); } // namespace rnexecutorch diff --git a/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp b/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp index 73540e07d..43f7104de 100644 --- a/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp @@ -15,6 +15,23 @@ constexpr auto kValidObjectDetectionModelPath = constexpr auto kValidTestImagePath = "file:///data/local/tmp/rnexecutorch_tests/test_image.jpg"; +// clang-format off +const std::vector kCocoLabels = { + "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", + "truck", "boat", "traffic light", "fire hydrant", "stop sign", + "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", + "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", + "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", + "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", + "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", + "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", + "donut", "cake", "chair", "couch", "potted plant", "bed", "dining table", + "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone", + "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", + "vase", "scissors", "teddy bear", "hair drier", "toothbrush" +}; +// clang-format on + // ============================================================================ // Common tests via typed test suite // ============================================================================ @@ -23,15 +40,16 @@ template <> struct ModelTraits { using ModelType = ObjectDetection; static ModelType createValid() { - return ModelType(kValidObjectDetectionModelPath, nullptr); + return ModelType(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); } static ModelType createInvalid() { - return ModelType("nonexistent.pte", nullptr); + return ModelType("nonexistent.pte", {}, {}, {}, nullptr); } static void callGenerate(ModelType &model) { - (void)model.generateFromString(kValidTestImagePath, 0.5, {}); + (void)model.generateFromString(kValidTestImagePath, 0.5); } }; } // namespace model_tests @@ -44,56 +62,64 @@ INSTANTIATE_TYPED_TEST_SUITE_P(ObjectDetection, CommonModelTest, // Model-specific tests // ============================================================================ TEST(ObjectDetectionGenerateTests, InvalidImagePathThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); EXPECT_THROW( - (void)model.generateFromString("nonexistent_image.jpg", 0.5, {}), + (void)model.generateFromString("nonexistent_image.jpg", 0.5), RnExecutorchError); } TEST(ObjectDetectionGenerateTests, EmptyImagePathThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - EXPECT_THROW((void)model.generateFromString("", 0.5, {}), RnExecutorchError); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); + EXPECT_THROW((void)model.generateFromString("", 0.5), RnExecutorchError); } TEST(ObjectDetectionGenerateTests, MalformedURIThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); EXPECT_THROW( - (void)model.generateFromString("not_a_valid_uri://bad", 0.5, {}), + (void)model.generateFromString("not_a_valid_uri://bad", 0.5), RnExecutorchError); } TEST(ObjectDetectionGenerateTests, NegativeThresholdThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); EXPECT_THROW( - (void)model.generateFromString(kValidTestImagePath, -0.1, {}), + (void)model.generateFromString(kValidTestImagePath, -0.1), RnExecutorchError); } TEST(ObjectDetectionGenerateTests, ThresholdAboveOneThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); EXPECT_THROW( - (void)model.generateFromString(kValidTestImagePath, 1.1, {}), + (void)model.generateFromString(kValidTestImagePath, 1.1), RnExecutorchError); } TEST(ObjectDetectionGenerateTests, ValidImageReturnsResults) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - auto results = model.generateFromString(kValidTestImagePath, 0.3, {}); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); + auto results = model.generateFromString(kValidTestImagePath, 0.3); EXPECT_GE(results.size(), 0u); } TEST(ObjectDetectionGenerateTests, HighThresholdReturnsFewerResults) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); auto lowThresholdResults = - model.generateFromString(kValidTestImagePath, 0.1, {}); + model.generateFromString(kValidTestImagePath, 0.1); auto highThresholdResults = - model.generateFromString(kValidTestImagePath, 0.9, {}); + model.generateFromString(kValidTestImagePath, 0.9); EXPECT_GE(lowThresholdResults.size(), highThresholdResults.size()); } TEST(ObjectDetectionGenerateTests, DetectionsHaveValidBoundingBoxes) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - auto results = model.generateFromString(kValidTestImagePath, 0.3, {}); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); + auto results = model.generateFromString(kValidTestImagePath, 0.3); for (const auto &detection : results) { EXPECT_LE(detection.x1, detection.x2); @@ -104,8 +130,9 @@ TEST(ObjectDetectionGenerateTests, DetectionsHaveValidBoundingBoxes) { } TEST(ObjectDetectionGenerateTests, DetectionsHaveValidScores) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - auto results = model.generateFromString(kValidTestImagePath, 0.3, {}); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); + auto results = model.generateFromString(kValidTestImagePath, 0.3); for (const auto &detection : results) { EXPECT_GE(detection.score, 0.0f); @@ -114,83 +141,94 @@ TEST(ObjectDetectionGenerateTests, DetectionsHaveValidScores) { } TEST(ObjectDetectionGenerateTests, DetectionsHaveValidLabels) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); - // Label names are resolved by the caller (e.g. ObjectDetectionModule). - // Passing {} produces empty-string labels; this test just verifies the - // returned detections have the expected structure without throwing. - EXPECT_NO_THROW( - (void)model.generateFromString(kValidTestImagePath, 0.3, {})); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); + auto results = model.generateFromString(kValidTestImagePath, 0.3); + + for (const auto &detection : results) { + const auto &label = detection.label; + EXPECT_FALSE(label.empty()); + EXPECT_NE(std::find(kCocoLabels.begin(), kCocoLabels.end(), label), + kCocoLabels.end()); + } } // ============================================================================ // generateFromPixels tests // ============================================================================ TEST(ObjectDetectionPixelTests, ValidPixelDataReturnsResults) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); constexpr int32_t width = 4, height = 4, channels = 3; std::vector pixelData(width * height * channels, 128); JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Byte}; - auto results = model.generateFromPixels(tensorView, 0.3, {}); + auto results = model.generateFromPixels(tensorView, 0.3); EXPECT_GE(results.size(), 0u); } TEST(ObjectDetectionPixelTests, WrongSizesLengthThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); std::vector pixelData(16, 0); JSTensorViewIn tensorView{ pixelData.data(), {4, 4}, executorch::aten::ScalarType::Byte}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5, {}), + EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5), RnExecutorchError); } TEST(ObjectDetectionPixelTests, WrongChannelCountThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); constexpr int32_t width = 4, height = 4, channels = 4; std::vector pixelData(width * height * channels, 0); JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Byte}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5, {}), + EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5), RnExecutorchError); } TEST(ObjectDetectionPixelTests, WrongScalarTypeThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); constexpr int32_t width = 4, height = 4, channels = 3; std::vector pixelData(width * height * channels, 0); JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Float}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5, {}), + EXPECT_THROW((void)model.generateFromPixels(tensorView, 0.5), RnExecutorchError); } TEST(ObjectDetectionPixelTests, NegativeThresholdThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); constexpr int32_t width = 4, height = 4, channels = 3; std::vector pixelData(width * height * channels, 128); JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Byte}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, -0.1, {}), + EXPECT_THROW((void)model.generateFromPixels(tensorView, -0.1), RnExecutorchError); } TEST(ObjectDetectionPixelTests, ThresholdAboveOneThrows) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); constexpr int32_t width = 4, height = 4, channels = 3; std::vector pixelData(width * height * channels, 128); JSTensorViewIn tensorView{pixelData.data(), {height, width, channels}, executorch::aten::ScalarType::Byte}; - EXPECT_THROW((void)model.generateFromPixels(tensorView, 1.1, {}), + EXPECT_THROW((void)model.generateFromPixels(tensorView, 1.1), RnExecutorchError); } TEST(ObjectDetectionInheritedTests, GetInputShapeWorks) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); auto shape = model.getInputShape("forward", 0); EXPECT_EQ(shape.size(), 4); EXPECT_EQ(shape[0], 1); @@ -198,13 +236,15 @@ TEST(ObjectDetectionInheritedTests, GetInputShapeWorks) { } TEST(ObjectDetectionInheritedTests, GetAllInputShapesWorks) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); auto shapes = model.getAllInputShapes("forward"); EXPECT_FALSE(shapes.empty()); } TEST(ObjectDetectionInheritedTests, GetMethodMetaWorks) { - ObjectDetection model(kValidObjectDetectionModelPath, nullptr); + ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, + nullptr); auto result = model.getMethodMeta("forward"); EXPECT_TRUE(result.ok()); } diff --git a/packages/react-native-executorch/src/index.ts b/packages/react-native-executorch/src/index.ts index 7f390e670..2febddda0 100644 --- a/packages/react-native-executorch/src/index.ts +++ b/packages/react-native-executorch/src/index.ts @@ -39,13 +39,15 @@ declare global { var loadSemanticSegmentation: ( source: string, normMean: Triple | [], - normStd: Triple | [] + normStd: Triple | [], + allClasses: string[] ) => any; var loadClassification: (source: string) => any; var loadObjectDetection: ( source: string, normMean: Triple | [], - normStd: Triple | [] + normStd: Triple | [], + labelNames: string[] ) => any; var loadExecutorchModule: (source: string) => any; var loadTokenizerModule: (source: string) => any; diff --git a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts index 2fdb3b1af..58d016201 100644 --- a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts +++ b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts @@ -61,21 +61,8 @@ type ResolveLabels = export class ObjectDetectionModule< T extends ObjectDetectionModelName | LabelEnum, > extends BaseLabeledModule> { - private allLabelNames: string[]; - private constructor(labelMap: ResolveLabels, nativeModule: unknown) { super(labelMap, nativeModule); - this.allLabelNames = []; - for (const [name, value] of Object.entries(this.labelMap)) { - if (typeof value === 'number') { - this.allLabelNames[value] = name; - } - } - for (let i = 0; i < this.allLabelNames.length; i++) { - if (this.allLabelNames[i] == null) { - this.allLabelNames[i] = ''; - } - } } /** @@ -95,11 +82,19 @@ export class ObjectDetectionModule< ] as ObjectDetectionConfig; const normMean = preprocessorConfig?.normMean ?? []; const normStd = preprocessorConfig?.normStd ?? []; + const allLabelNames: string[] = []; + for (const [name, value] of Object.entries(labelMap)) { + if (typeof value === 'number') allLabelNames[value] = name; + } + for (let i = 0; i < allLabelNames.length; i++) { + if (allLabelNames[i] == null) allLabelNames[i] = ''; + } const modelPath = await fetchModelPath(modelSource, onDownloadProgress); const nativeModule = global.loadObjectDetection( modelPath, normMean, - normStd + normStd, + allLabelNames ); return new ObjectDetectionModule>( labelMap as ResolveLabels>, @@ -122,11 +117,19 @@ export class ObjectDetectionModule< ): Promise> { const normMean = config.preprocessorConfig?.normMean ?? []; const normStd = config.preprocessorConfig?.normStd ?? []; + const allLabelNames: string[] = []; + for (const [name, value] of Object.entries(config.labelMap)) { + if (typeof value === 'number') allLabelNames[value] = name; + } + for (let i = 0; i < allLabelNames.length; i++) { + if (allLabelNames[i] == null) allLabelNames[i] = ''; + } const modelPath = await fetchModelPath(modelSource, onDownloadProgress); const nativeModule = global.loadObjectDetection( modelPath, normMean, - normStd + normStd, + allLabelNames ); return new ObjectDetectionModule( config.labelMap as ResolveLabels, @@ -155,14 +158,12 @@ export class ObjectDetectionModule< if (typeof input === 'string') { return await this.nativeModule.generateFromString( input, - detectionThreshold, - this.allLabelNames + detectionThreshold ); } return await this.nativeModule.generateFromPixels( input, - detectionThreshold, - this.allLabelNames + detectionThreshold ); } @@ -181,7 +182,6 @@ export class ObjectDetectionModule< } const nativeGenerateFromFrame = this.nativeModule.generateFromFrame; - const allLabelNames = this.allLabelNames; return ( frame: Frame, @@ -193,7 +193,7 @@ export class ObjectDetectionModule< try { nativeBuffer = frame.getNativeBuffer(); const frameData = { nativeBuffer: nativeBuffer.pointer }; - return nativeGenerateFromFrame(frameData, detectionThreshold, allLabelNames); + return nativeGenerateFromFrame(frameData, detectionThreshold); } finally { if (nativeBuffer?.release) { nativeBuffer.release(); diff --git a/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts b/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts index db28b50be..b2d08e6f5 100644 --- a/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts +++ b/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts @@ -80,13 +80,8 @@ type ResolveLabels = export class SemanticSegmentationModule< T extends SemanticSegmentationModelName | LabelEnum, > extends BaseLabeledModule> { - private allClassNames: string[]; - private constructor(labelMap: ResolveLabels, nativeModule: unknown) { super(labelMap, nativeModule); - this.allClassNames = Object.keys(this.labelMap).filter((k) => - isNaN(Number(k)) - ); } /** @@ -117,11 +112,13 @@ export class SemanticSegmentationModule< ] as SemanticSegmentationConfig; const normMean = preprocessorConfig?.normMean ?? []; const normStd = preprocessorConfig?.normStd ?? []; + const allClassNames = Object.keys(labelMap).filter((k) => isNaN(Number(k))); const modelPath = await fetchModelPath(modelSource, onDownloadProgress); const nativeModule = global.loadSemanticSegmentation( modelPath, normMean, - normStd + normStd, + allClassNames ); return new SemanticSegmentationModule>( labelMap as ResolveLabels>, @@ -154,11 +151,15 @@ export class SemanticSegmentationModule< ): Promise> { const normMean = config.preprocessorConfig?.normMean ?? []; const normStd = config.preprocessorConfig?.normStd ?? []; + const allClassNames = Object.keys(config.labelMap).filter((k) => + isNaN(Number(k)) + ); const modelPath = await fetchModelPath(modelSource, onDownloadProgress); const nativeModule = global.loadSemanticSegmentation( modelPath, normMean, - normStd + normStd, + allClassNames ); return new SemanticSegmentationModule( config.labelMap as ResolveLabels, @@ -193,7 +194,6 @@ export class SemanticSegmentationModule< const nativeResult = await this.nativeModule.generate( imageSource, - this.allClassNames, classesOfInterestNames, resizeToInput ); From 22b29fee4a55f8633fe1e0369822b49e2457b7a0 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Mon, 2 Mar 2026 11:56:08 +0100 Subject: [PATCH 14/24] add doxygen comments --- .../models/object_detection/ObjectDetection.h | 63 +++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h index 2fc23c710..7740994fe 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h @@ -15,12 +15,54 @@ namespace models::object_detection { using executorch::extension::TensorPtr; using executorch::runtime::EValue; +/** + * @brief Object detection model that detects and localises objects in images. + * + * Wraps an ExecuTorch model and exposes a single @ref generate call that + * preprocesses an input image, runs the forward pass, and returns a filtered, + * non-max-suppressed list of @ref types::Detection results. + */ class ObjectDetection : public VisionModel { public: + /** + * @brief Constructs an ObjectDetection model and loads it from disk. + * + * @param modelSource Path to the `.pte` model file. + * @param normMean Per-channel mean values used for input normalisation + * (must be exactly 3 elements, or empty to skip). + * @param normStd Per-channel standard-deviation values used for input + * normalisation (must be exactly 3 elements, or empty to + * skip). + * @param labelNames Ordered list of class label strings. Index @c i must + * correspond to class index @c i produced by the model. + * @param callInvoker JSI call invoker used to marshal results back to JS. + * + * @throws RnExecutorchError if the model cannot be loaded or its input shape + * is incompatible. + */ ObjectDetection(const std::string &modelSource, std::vector normMean, std::vector normStd, std::vector labelNames, std::shared_ptr callInvoker); + + /** + * @brief Runs object detection on a single image. + * + * Preprocesses the image, executes the model's forward pass, and returns + * all detections whose confidence score meets @p detectionThreshold after + * non-maximum suppression. + * + * @param imageSource URI or file path of the input image. + * @param detectionThreshold Minimum confidence score in (0, 1] for a + * detection to be included in the output. + * + * @return A vector of @ref types::Detection objects with bounding boxes, + * label strings (resolved via the label names passed to the + * constructor), and confidence scores. + * + * @throws RnExecutorchError if the image cannot be read or the forward pass + * fails. + */ [[nodiscard("Registered non-void function")]] std::vector generateFromString(std::string imageSource, double detectionThreshold); [[nodiscard("Registered non-void function")]] std::vector @@ -35,14 +77,35 @@ class ObjectDetection : public VisionModel { cv::Mat preprocessFrame(const cv::Mat &frame) const override; private: + /** + * @brief Decodes raw model output tensors into a list of detections. + * + * @param tensors Raw EValue outputs from the forward pass + * (bboxes at index 0, scores at index 1, + * labels at index 2). + * @param originalSize Original image dimensions used to scale + * bounding boxes back to input coordinates. + * @param detectionThreshold Confidence threshold below which detections + * are discarded. + * @param labelNames Label strings indexed by class id. + * + * @return Non-max-suppressed detections above the threshold. + */ std::vector postprocess(const std::vector &tensors, cv::Size originalSize, double detectionThreshold, const std::vector &labelNames); + /// Expected input image dimensions derived from the model's input shape. cv::Size modelImageSize{0, 0}; + + /// Optional per-channel mean for input normalisation (set in constructor). std::optional normMean_; + + /// Optional per-channel standard deviation for input normalisation. std::optional normStd_; + + /// Ordered label strings mapping class indices to human-readable names. std::vector labelNames_; }; } // namespace models::object_detection From 349a7b5498de6e9373eecc7e510d76d975d4e80f Mon Sep 17 00:00:00 2001 From: chmjkb Date: Mon, 2 Mar 2026 12:01:49 +0100 Subject: [PATCH 15/24] add norm and std tests --- .../tests/integration/ObjectDetectionTest.cpp | 52 ++++++++++++++----- 1 file changed, 38 insertions(+), 14 deletions(-) diff --git a/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp b/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp index 43f7104de..8964f2013 100644 --- a/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/tests/integration/ObjectDetectionTest.cpp @@ -64,9 +64,8 @@ INSTANTIATE_TYPED_TEST_SUITE_P(ObjectDetection, CommonModelTest, TEST(ObjectDetectionGenerateTests, InvalidImagePathThrows) { ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, nullptr); - EXPECT_THROW( - (void)model.generateFromString("nonexistent_image.jpg", 0.5), - RnExecutorchError); + EXPECT_THROW((void)model.generateFromString("nonexistent_image.jpg", 0.5), + RnExecutorchError); } TEST(ObjectDetectionGenerateTests, EmptyImagePathThrows) { @@ -78,25 +77,22 @@ TEST(ObjectDetectionGenerateTests, EmptyImagePathThrows) { TEST(ObjectDetectionGenerateTests, MalformedURIThrows) { ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, nullptr); - EXPECT_THROW( - (void)model.generateFromString("not_a_valid_uri://bad", 0.5), - RnExecutorchError); + EXPECT_THROW((void)model.generateFromString("not_a_valid_uri://bad", 0.5), + RnExecutorchError); } TEST(ObjectDetectionGenerateTests, NegativeThresholdThrows) { ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, nullptr); - EXPECT_THROW( - (void)model.generateFromString(kValidTestImagePath, -0.1), - RnExecutorchError); + EXPECT_THROW((void)model.generateFromString(kValidTestImagePath, -0.1), + RnExecutorchError); } TEST(ObjectDetectionGenerateTests, ThresholdAboveOneThrows) { ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, nullptr); - EXPECT_THROW( - (void)model.generateFromString(kValidTestImagePath, 1.1), - RnExecutorchError); + EXPECT_THROW((void)model.generateFromString(kValidTestImagePath, 1.1), + RnExecutorchError); } TEST(ObjectDetectionGenerateTests, ValidImageReturnsResults) { @@ -109,8 +105,7 @@ TEST(ObjectDetectionGenerateTests, ValidImageReturnsResults) { TEST(ObjectDetectionGenerateTests, HighThresholdReturnsFewerResults) { ObjectDetection model(kValidObjectDetectionModelPath, {}, {}, kCocoLabels, nullptr); - auto lowThresholdResults = - model.generateFromString(kValidTestImagePath, 0.1); + auto lowThresholdResults = model.generateFromString(kValidTestImagePath, 0.1); auto highThresholdResults = model.generateFromString(kValidTestImagePath, 0.9); EXPECT_GE(lowThresholdResults.size(), highThresholdResults.size()); @@ -248,3 +243,32 @@ TEST(ObjectDetectionInheritedTests, GetMethodMetaWorks) { auto result = model.getMethodMeta("forward"); EXPECT_TRUE(result.ok()); } + +// ============================================================================ +// Normalisation tests +// ============================================================================ +TEST(ObjectDetectionNormTests, ValidNormParamsDoesntThrow) { + const std::vector mean = {0.485f, 0.456f, 0.406f}; + const std::vector std = {0.229f, 0.224f, 0.225f}; + EXPECT_NO_THROW( + ObjectDetection(kValidObjectDetectionModelPath, mean, std, {}, nullptr)); +} + +TEST(ObjectDetectionNormTests, InvalidNormMeanSizeDoesntThrow) { + EXPECT_NO_THROW(ObjectDetection(kValidObjectDetectionModelPath, {0.5f}, + {0.229f, 0.224f, 0.225f}, {}, nullptr)); +} + +TEST(ObjectDetectionNormTests, InvalidNormStdSizeDoesntThrow) { + EXPECT_NO_THROW(ObjectDetection(kValidObjectDetectionModelPath, + {0.485f, 0.456f, 0.406f}, {0.5f}, {}, + nullptr)); +} + +TEST(ObjectDetectionNormTests, ValidNormParamsGenerateSucceeds) { + const std::vector mean = {0.485f, 0.456f, 0.406f}; + const std::vector std = {0.229f, 0.224f, 0.225f}; + ObjectDetection model(kValidObjectDetectionModelPath, mean, std, kCocoLabels, + nullptr); + EXPECT_NO_THROW((void)model.generateFromString(kValidTestImagePath, 0.5)); +} From e4af32c154c2fc5c094871d6aed61f4b67a9341a Mon Sep 17 00:00:00 2001 From: chmjkb Date: Tue, 3 Mar 2026 08:05:28 +0100 Subject: [PATCH 16/24] chore: add vision labeled module --- .../computer_vision/ObjectDetectionModule.ts | 73 +------------------ .../computer_vision/VisionLabeledModule.ts | 26 +++++++ 2 files changed, 29 insertions(+), 70 deletions(-) create mode 100644 packages/react-native-executorch/src/modules/computer_vision/VisionLabeledModule.ts diff --git a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts index 58d016201..0c570dbbc 100644 --- a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts +++ b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts @@ -1,4 +1,4 @@ -import { LabelEnum, ResourceSource, PixelData, Frame } from '../../types/common'; +import { LabelEnum, ResourceSource } from '../../types/common'; import { Detection, ObjectDetectionConfig, @@ -10,13 +10,11 @@ import { IMAGENET1K_MEAN, IMAGENET1K_STD, } from '../../constants/commonVision'; -import { RnExecutorchErrorCode } from '../../errors/ErrorCodes'; -import { RnExecutorchError } from '../../errors/errorUtils'; import { - BaseLabeledModule, fetchModelPath, ResolveLabels as ResolveLabelsFor, } from '../BaseLabeledModule'; +import { VisionLabeledModule } from './VisionLabeledModule'; const ModelConfigs = { 'ssdlite-320-mobilenet-v3-large': { @@ -60,7 +58,7 @@ type ResolveLabels = */ export class ObjectDetectionModule< T extends ObjectDetectionModelName | LabelEnum, -> extends BaseLabeledModule> { +> extends VisionLabeledModule>[], ResolveLabels> { private constructor(labelMap: ResolveLabels, nativeModule: unknown) { super(labelMap, nativeModule); } @@ -136,69 +134,4 @@ export class ObjectDetectionModule< nativeModule ); } - - /** - * Executes the model's forward pass to detect objects within the provided image. - * - * @param input - A string image source (file path, URI, or Base64) or a {@link PixelData} object. - * @param detectionThreshold - Minimum confidence score for a detection to be included. Default is 0.7. - * @returns A Promise resolving to an array of {@link Detection} objects. - * @throws {RnExecutorchError} If the model is not loaded. - */ - async forward( - input: string | PixelData, - detectionThreshold: number = 0.7 - ): Promise>[]> { - if (this.nativeModule == null) { - throw new RnExecutorchError( - RnExecutorchErrorCode.ModuleNotLoaded, - 'The model is currently not loaded.' - ); - } - if (typeof input === 'string') { - return await this.nativeModule.generateFromString( - input, - detectionThreshold - ); - } - return await this.nativeModule.generateFromPixels( - input, - detectionThreshold - ); - } - - /** - * Synchronous worklet function for real-time VisionCamera frame processing. - * The label names are captured from the module instance — no need to pass them per frame. - * - * **Use this for VisionCamera frame processing in worklets.** - * For async processing, use `forward()` instead. - */ - get runOnFrame(): - | ((frame: Frame, detectionThreshold: number) => Detection>[]) - | null { - if (!this.nativeModule?.generateFromFrame) { - return null; - } - - const nativeGenerateFromFrame = this.nativeModule.generateFromFrame; - - return ( - frame: Frame, - detectionThreshold: number - ): Detection>[] => { - 'worklet'; - - let nativeBuffer: ReturnType | null = null; - try { - nativeBuffer = frame.getNativeBuffer(); - const frameData = { nativeBuffer: nativeBuffer.pointer }; - return nativeGenerateFromFrame(frameData, detectionThreshold); - } finally { - if (nativeBuffer?.release) { - nativeBuffer.release(); - } - } - }; - } } diff --git a/packages/react-native-executorch/src/modules/computer_vision/VisionLabeledModule.ts b/packages/react-native-executorch/src/modules/computer_vision/VisionLabeledModule.ts new file mode 100644 index 000000000..914ea6195 --- /dev/null +++ b/packages/react-native-executorch/src/modules/computer_vision/VisionLabeledModule.ts @@ -0,0 +1,26 @@ +import { LabelEnum } from '../../types/common'; +import { VisionModule } from './VisionModule'; + +/** + * Base class for computer vision modules that carry a type-safe label map + * and support the full VisionModule API (string/PixelData forward + runOnFrame). + * + * @typeParam TOutput - The model's output type. + * @typeParam LabelMap - The resolved {@link LabelEnum} for the model's output classes. + * @internal + */ +export abstract class VisionLabeledModule< + TOutput, + LabelMap extends LabelEnum, +> extends VisionModule { + protected readonly labelMap: LabelMap; + + protected constructor(labelMap: LabelMap, nativeModule: unknown) { + super(); + this.labelMap = labelMap; + this.nativeModule = nativeModule; + } + + // TODO: figure it out so we can delete this (we need this because of basemodule inheritance) + override async load() {} +} From 15efb7cc8fe69fa7034a7fab2cb4d4a14a23ba57 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Tue, 3 Mar 2026 08:54:29 +0100 Subject: [PATCH 17/24] fix: object detection --- .../computer_vision/ObjectDetectionModule.ts | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts index 0c570dbbc..e15f6981f 100644 --- a/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts +++ b/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts @@ -1,4 +1,4 @@ -import { LabelEnum, ResourceSource } from '../../types/common'; +import { LabelEnum, PixelData, ResourceSource } from '../../types/common'; import { Detection, ObjectDetectionConfig, @@ -108,6 +108,20 @@ export class ObjectDetectionModule< * @param onDownloadProgress - Optional callback to monitor download progress, receiving a value between 0 and 1. * @returns A Promise resolving to an `ObjectDetectionModule` instance typed to the provided label map. */ + /** + * Executes the model's forward pass to detect objects within the provided image. + * + * @param input - A string image source (file path, URI, or Base64) or a {@link PixelData} object. + * @param detectionThreshold - Minimum confidence score for a detection to be included. Default is 0.7. + * @returns A Promise resolving to an array of {@link Detection} objects. + */ + override async forward( + input: string | PixelData, + detectionThreshold = 0.7 + ): Promise>[]> { + return super.forward(input, detectionThreshold); + } + static async fromCustomConfig( modelSource: ResourceSource, config: ObjectDetectionConfig, From dff5a8f0fafa704d6cc47875d5ca866505dced6e Mon Sep 17 00:00:00 2001 From: Norbert Klockiewicz Date: Tue, 3 Mar 2026 09:08:21 +0100 Subject: [PATCH 18/24] chore: switch object detection example to RF_DETR_NANO model --- .../app/object_detection/index.tsx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/apps/computer-vision/app/object_detection/index.tsx b/apps/computer-vision/app/object_detection/index.tsx index 6a43dd920..2f8fa6d58 100644 --- a/apps/computer-vision/app/object_detection/index.tsx +++ b/apps/computer-vision/app/object_detection/index.tsx @@ -4,7 +4,7 @@ import { getImage } from '../../utils'; import { Detection, useObjectDetection, - SSDLITE_320_MOBILENET_V3_LARGE, + RF_DETR_NANO, } from 'react-native-executorch'; import { View, StyleSheet, Image } from 'react-native'; import ImageWithBboxes from '../../components/ImageWithBboxes'; @@ -20,11 +20,11 @@ export default function ObjectDetectionScreen() { height: number; }>(); - const ssdLite = useObjectDetection({ model: SSDLITE_320_MOBILENET_V3_LARGE }); + const rfDetr = useObjectDetection({ model: RF_DETR_NANO }); const { setGlobalGenerating } = useContext(GeneratingContext); useEffect(() => { - setGlobalGenerating(ssdLite.isGenerating); - }, [ssdLite.isGenerating, setGlobalGenerating]); + setGlobalGenerating(rfDetr.isGenerating); + }, [rfDetr.isGenerating, setGlobalGenerating]); const handleCameraPress = async (isCamera: boolean) => { const image = await getImage(isCamera); @@ -42,7 +42,7 @@ export default function ObjectDetectionScreen() { const runForward = async () => { if (imageUri) { try { - const output = await ssdLite.forward(imageUri); + const output = await rfDetr.forward(imageUri); setResults(output); } catch (e) { console.error(e); @@ -50,11 +50,11 @@ export default function ObjectDetectionScreen() { } }; - if (!ssdLite.isReady) { + if (!rfDetr.isReady) { return ( ); } From 630e3924cf58c5f4050298881ae2e15a47577936 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Tue, 3 Mar 2026 12:51:41 +0100 Subject: [PATCH 19/24] chore: review suggestions --- .../object_detection/ObjectDetection.cpp | 20 +++++++++++-------- .../models/object_detection/ObjectDetection.h | 18 ++++++++++------- 2 files changed, 23 insertions(+), 15 deletions(-) diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp index 9abea1377..1dad1a61a 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.cpp @@ -13,7 +13,8 @@ ObjectDetection::ObjectDetection( const std::string &modelSource, std::vector normMean, std::vector normStd, std::vector labelNames, std::shared_ptr callInvoker) - : VisionModel(modelSource, callInvoker), labelNames_(std::move(labelNames)) { + : VisionModel(modelSource, callInvoker), + labelNames_(std::move(labelNames)) { auto inputTensors = getAllInputShapes(); if (inputTensors.empty()) { throw RnExecutorchError(RnExecutorchErrorCode::UnexpectedNumInputs, @@ -80,8 +81,7 @@ cv::Mat ObjectDetection::preprocessFrame(const cv::Mat &frame) const { std::vector ObjectDetection::postprocess(const std::vector &tensors, - cv::Size originalSize, double detectionThreshold, - const std::vector &labelNames) { + cv::Size originalSize, double detectionThreshold) { float widthRatio = static_cast(originalSize.width) / modelImageSize.width; float heightRatio = @@ -112,9 +112,14 @@ ObjectDetection::postprocess(const std::vector &tensors, float x2 = bboxes[i * 4 + 2] * widthRatio; float y2 = bboxes[i * 4 + 3] * heightRatio; auto labelIdx = static_cast(labels[i]); - std::string labelName = - labelIdx < labelNames.size() ? labelNames[labelIdx] : ""; - detections.emplace_back(x1, y1, x2, y2, labelName, scores[i]); + if (labelIdx >= labelNames_.size()) { + throw RnExecutorchError( + RnExecutorchErrorCode::InvalidConfig, + "Model output class index " + std::to_string(labelIdx) + + " exceeds labelNames size " + std::to_string(labelNames_.size()) + + ". Ensure the labelMap covers all model output classes."); + } + detections.emplace_back(x1, y1, x2, y2, labelNames_[labelIdx], scores[i]); } return utils::nonMaxSuppression(detections); @@ -145,8 +150,7 @@ ObjectDetection::runInference(cv::Mat image, double detectionThreshold) { "Ensure the model input is correct."); } - return postprocess(forwardResult.get(), originalSize, detectionThreshold, - labelNames_); + return postprocess(forwardResult.get(), originalSize, detectionThreshold); } std::vector diff --git a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h index 7740994fe..f1159d88e 100644 --- a/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h +++ b/packages/react-native-executorch/common/rnexecutorch/models/object_detection/ObjectDetection.h @@ -35,6 +35,9 @@ class ObjectDetection : public VisionModel { * skip). * @param labelNames Ordered list of class label strings. Index @c i must * correspond to class index @c i produced by the model. + * This is a runtime value passed from JS side, + * dependant the model. The user can pass his own, custom + * labels. * @param callInvoker JSI call invoker used to marshal results back to JS. * * @throws RnExecutorchError if the model cannot be loaded or its input shape @@ -72,8 +75,8 @@ class ObjectDetection : public VisionModel { generateFromPixels(JSTensorViewIn pixelData, double detectionThreshold); protected: - std::vector - runInference(cv::Mat image, double detectionThreshold); + std::vector runInference(cv::Mat image, + double detectionThreshold); cv::Mat preprocessFrame(const cv::Mat &frame) const override; private: @@ -87,14 +90,15 @@ class ObjectDetection : public VisionModel { * bounding boxes back to input coordinates. * @param detectionThreshold Confidence threshold below which detections * are discarded. - * @param labelNames Label strings indexed by class id. * * @return Non-max-suppressed detections above the threshold. + * + * @throws RnExecutorchError if the model outputs a class index that exceeds + * the size of @ref labelNames_. */ - std::vector - postprocess(const std::vector &tensors, cv::Size originalSize, - double detectionThreshold, - const std::vector &labelNames); + std::vector postprocess(const std::vector &tensors, + cv::Size originalSize, + double detectionThreshold); /// Expected input image dimensions derived from the model's input shape. cv::Size modelImageSize{0, 0}; From b9fcad8b1d59345e7a5f1da574acfc1fe2315244 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Tue, 3 Mar 2026 12:55:38 +0100 Subject: [PATCH 20/24] chore: lint --- .../computer_vision/useObjectDetection.ts | 29 ++++++++++++++----- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts b/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts index daaeca476..5333b8a32 100644 --- a/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts +++ b/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts @@ -25,18 +25,31 @@ export const useObjectDetection = ({ }: ObjectDetectionProps): ObjectDetectionType< ObjectDetectionLabels > => { - const { error, isReady, isGenerating, downloadProgress, runForward, instance } = - useModuleFactory({ - factory: (config, onProgress) => - ObjectDetectionModule.fromModelName(config, onProgress), - config: model, - preventLoad, - }); + const { + error, + isReady, + isGenerating, + downloadProgress, + runForward, + instance, + } = useModuleFactory({ + factory: (config, onProgress) => + ObjectDetectionModule.fromModelName(config, onProgress), + config: model, + preventLoad, + }); const forward = (input: string | PixelData, detectionThreshold?: number) => runForward((inst) => inst.forward(input, detectionThreshold)); const runOnFrame = useMemo(() => instance?.runOnFrame ?? null, [instance]); - return { error, isReady, isGenerating, downloadProgress, forward, runOnFrame }; + return { + error, + isReady, + isGenerating, + downloadProgress, + forward, + runOnFrame, + }; }; From b4143029dc92a3722d6de104dd41e057d6401762 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Tue, 3 Mar 2026 12:55:51 +0100 Subject: [PATCH 21/24] chore: lint --- .../src/hooks/useModuleFactory.ts | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/packages/react-native-executorch/src/hooks/useModuleFactory.ts b/packages/react-native-executorch/src/hooks/useModuleFactory.ts index ef4dc841c..2be882126 100644 --- a/packages/react-native-executorch/src/hooks/useModuleFactory.ts +++ b/packages/react-native-executorch/src/hooks/useModuleFactory.ts @@ -81,5 +81,12 @@ export function useModuleFactory< } }; - return { error, isReady, isGenerating, downloadProgress, runForward, instance }; + return { + error, + isReady, + isGenerating, + downloadProgress, + runForward, + instance, + }; } From 61d8f7a65309b74ab1df05c7edb1af2e4ed6512b Mon Sep 17 00:00:00 2001 From: chmjkb Date: Tue, 3 Mar 2026 13:49:31 +0100 Subject: [PATCH 22/24] tests: fix style transfer and basemodule tests --- .../common/rnexecutorch/tests/integration/BaseModelTest.cpp | 2 +- .../common/rnexecutorch/tests/integration/StyleTransferTest.cpp | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/react-native-executorch/common/rnexecutorch/tests/integration/BaseModelTest.cpp b/packages/react-native-executorch/common/rnexecutorch/tests/integration/BaseModelTest.cpp index faf21f5b5..efee033be 100644 --- a/packages/react-native-executorch/common/rnexecutorch/tests/integration/BaseModelTest.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/tests/integration/BaseModelTest.cpp @@ -13,7 +13,7 @@ using namespace model_tests; using executorch::runtime::EValue; constexpr auto kValidStyleTransferModelPath = - "style_transfer_candy_xnnpack.pte"; + "style_transfer_candy_xnnpack_fp32.pte"; // ============================================================================ // Common tests via typed test suite diff --git a/packages/react-native-executorch/common/rnexecutorch/tests/integration/StyleTransferTest.cpp b/packages/react-native-executorch/common/rnexecutorch/tests/integration/StyleTransferTest.cpp index 3e6951617..d5427ce61 100644 --- a/packages/react-native-executorch/common/rnexecutorch/tests/integration/StyleTransferTest.cpp +++ b/packages/react-native-executorch/common/rnexecutorch/tests/integration/StyleTransferTest.cpp @@ -10,7 +10,7 @@ using namespace rnexecutorch::models::style_transfer; using namespace model_tests; constexpr auto kValidStyleTransferModelPath = - "style_transfer_candy_xnnpack.pte"; + "style_transfer_candy_xnnpack_fp32.pte"; constexpr auto kValidTestImagePath = "file:///data/local/tmp/rnexecutorch_tests/test_image.jpg"; From a16074eeb9bb628f90c4dcf31751e83fbabb92c8 Mon Sep 17 00:00:00 2001 From: chmjkb Date: Tue, 3 Mar 2026 13:56:29 +0100 Subject: [PATCH 23/24] docs: update API reference --- .../classes/ObjectDetectionModule.md | 196 ++++++++++++++---- .../classes/SemanticSegmentationModule.md | 38 ++-- .../functions/useObjectDetection.md | 4 +- .../functions/useSemanticSegmentation.md | 2 +- docs/docs/06-api-reference/index.md | 37 ++-- docs/docs/06-api-reference/interfaces/Bbox.md | 10 +- .../06-api-reference/interfaces/Detection.md | 8 +- .../interfaces/ObjectDetectionProps.md | 6 +- .../interfaces/ObjectDetectionType.md | 54 +++-- .../type-aliases/ObjectDetectionConfig.md | 6 +- .../type-aliases/ObjectDetectionLabels.md | 2 +- .../type-aliases/ObjectDetectionModelName.md | 2 +- .../ObjectDetectionModelSources.md | 2 +- .../type-aliases/SegmentationLabels.md | 6 +- .../docs/06-api-reference/typedoc-sidebar.cjs | 2 +- .../variables/ALL_MINILM_L6_V2.md | 2 +- .../variables/ALL_MPNET_BASE_V2.md | 2 +- .../variables/BK_SDM_TINY_VPRED_256.md | 2 +- .../variables/BK_SDM_TINY_VPRED_512.md | 2 +- .../variables/CLIP_VIT_BASE_PATCH32_IMAGE.md | 2 +- .../variables/CLIP_VIT_BASE_PATCH32_TEXT.md | 2 +- .../DEEPLAB_V3_MOBILENET_V3_LARGE.md | 2 +- ...DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED.md | 2 +- .../variables/DEEPLAB_V3_RESNET101.md | 2 +- .../DEEPLAB_V3_RESNET101_QUANTIZED.md | 2 +- .../variables/DEEPLAB_V3_RESNET50.md | 2 +- .../DEEPLAB_V3_RESNET50_QUANTIZED.md | 2 +- .../variables/FCN_RESNET101.md | 2 +- .../variables/FCN_RESNET101_QUANTIZED.md | 2 +- .../variables/FCN_RESNET50.md | 2 +- .../variables/FCN_RESNET50_QUANTIZED.md | 2 +- .../06-api-reference/variables/FSMN_VAD.md | 2 +- .../variables/LRASPP_MOBILENET_V3_LARGE.md | 2 +- .../LRASPP_MOBILENET_V3_LARGE_QUANTIZED.md | 2 +- .../variables/MULTI_QA_MINILM_L6_COS_V1.md | 2 +- .../variables/MULTI_QA_MPNET_BASE_DOT_V1.md | 2 +- .../variables/RF_DETR_NANO.md | 2 +- .../variables/SELFIE_SEGMENTATION.md | 2 +- 38 files changed, 295 insertions(+), 126 deletions(-) diff --git a/docs/docs/06-api-reference/classes/ObjectDetectionModule.md b/docs/docs/06-api-reference/classes/ObjectDetectionModule.md index 67536491e..e36777857 100644 --- a/docs/docs/06-api-reference/classes/ObjectDetectionModule.md +++ b/docs/docs/06-api-reference/classes/ObjectDetectionModule.md @@ -1,12 +1,12 @@ # Class: ObjectDetectionModule\ -Defined in: [modules/computer_vision/ObjectDetectionModule.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L61) +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:59](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L59) Generic object detection module with type-safe label maps. ## Extends -- `BaseLabeledModule`\<`ResolveLabels`\<`T`\>\> +- `VisionLabeledModule`\<[`Detection`](../interfaces/Detection.md)\<`ResolveLabels`\<`T`\>\>[], `ResolveLabels`\<`T`\>\> ## Type Parameters @@ -19,15 +19,87 @@ or a custom [LabelEnum](../type-aliases/LabelEnum.md) label map. ## Properties +### generateFromFrame() + +> **generateFromFrame**: (`frameData`, ...`args`) => `any` + +Defined in: [modules/BaseModule.ts:56](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L56) + +Process a camera frame directly for real-time inference. + +This method is bound to a native JSI function after calling `load()`, +making it worklet-compatible and safe to call from VisionCamera's +frame processor thread. + +**Performance characteristics:** + +- **Zero-copy path**: When using `frame.getNativeBuffer()` from VisionCamera v5, + frame data is accessed directly without copying (fastest, recommended). +- **Copy path**: When using `frame.toArrayBuffer()`, pixel data is copied + from native to JS, then accessed from native code (slower, fallback). + +**Usage with VisionCamera:** + +```typescript +const frameOutput = useFrameOutput({ + pixelFormat: 'rgb', + onFrame(frame) { + 'worklet'; + // Zero-copy approach (recommended) + const nativeBuffer = frame.getNativeBuffer(); + const result = model.generateFromFrame( + { + nativeBuffer: nativeBuffer.pointer, + width: frame.width, + height: frame.height, + }, + ...args + ); + nativeBuffer.release(); + frame.dispose(); + }, +}); +``` + +#### Parameters + +##### frameData + +[`Frame`](../interfaces/Frame.md) + +Frame data object with either nativeBuffer (zero-copy) or data (ArrayBuffer) + +##### args + +...`any`[] + +Additional model-specific arguments (e.g., threshold, options) + +#### Returns + +`any` + +Model-specific output (e.g., detections, classifications, embeddings) + +#### See + +[Frame](../interfaces/Frame.md) for frame data format details + +#### Inherited from + +`VisionLabeledModule.generateFromFrame` + +--- + ### labelMap > `protected` `readonly` **labelMap**: `ResolveLabels` -Defined in: [modules/BaseLabeledModule.ts:52](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseLabeledModule.ts#L52) +Defined in: [modules/computer_vision/VisionLabeledModule.ts:16](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/VisionLabeledModule.ts#L16) #### Inherited from -`BaseLabeledModule.labelMap` +`VisionLabeledModule.labelMap` --- @@ -35,13 +107,61 @@ Defined in: [modules/BaseLabeledModule.ts:52](https://github.com/software-mansio > **nativeModule**: `any` = `null` -Defined in: [modules/BaseModule.ts:8](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L8) +Defined in: [modules/BaseModule.ts:17](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L17) + +**`Internal`** -Native module instance +Native module instance (JSI Host Object) #### Inherited from -`BaseLabeledModule.nativeModule` +`VisionLabeledModule.nativeModule` + +## Accessors + +### runOnFrame + +#### Get Signature + +> **get** **runOnFrame**(): (`frame`, ...`args`) => `TOutput` \| `null` + +Defined in: [modules/computer_vision/VisionModule.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/VisionModule.ts#L61) + +Synchronous worklet function for real-time VisionCamera frame processing. + +Only available after the model is loaded. Returns null if not loaded. + +**Use this for VisionCamera frame processing in worklets.** +For async processing, use `forward()` instead. + +##### Example + +```typescript +const model = new ClassificationModule(); +await model.load({ modelSource: MODEL }); + +// Use the functional form of setState to store the worklet — passing it +// directly would cause React to invoke it immediately as an updater fn. +const [runOnFrame, setRunOnFrame] = useState(null); +setRunOnFrame(() => model.runOnFrame); + +const frameOutput = useFrameOutput({ + onFrame(frame) { + 'worklet'; + if (!runOnFrame) return; + const result = runOnFrame(frame); + frame.dispose(); + }, +}); +``` + +##### Returns + +(`frame`, ...`args`) => `TOutput` \| `null` + +#### Inherited from + +`VisionLabeledModule.runOnFrame` ## Methods @@ -49,9 +169,11 @@ Native module instance > **delete**(): `void` -Defined in: [modules/BaseModule.ts:41](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L41) +Defined in: [modules/BaseModule.ts:100](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L100) -Unloads the model from memory. +Unloads the model from memory and releases native resources. + +Always call this method when you're done with a model to prevent memory leaks. #### Returns @@ -59,27 +181,27 @@ Unloads the model from memory. #### Inherited from -`BaseLabeledModule.delete` +`VisionLabeledModule.delete` --- ### forward() -> **forward**(`imageSource`, `detectionThreshold`): `Promise`\<[`Detection`](../interfaces/Detection.md)\<`ResolveLabels`\<`T`, \{ `rf-detr-nano`: \{ `labelMap`: _typeof_ [`CocoLabel`](../enumerations/CocoLabel.md); `preprocessorConfig`: \{ `normMean`: [`Triple`](../type-aliases/Triple.md)\<`number`\>; `normStd`: [`Triple`](../type-aliases/Triple.md)\<`number`\>; \}; \}; `ssdlite-320-mobilenet-v3-large`: \{ `labelMap`: _typeof_ [`CocoLabel`](../enumerations/CocoLabel.md); `preprocessorConfig`: `undefined`; \}; \}\>\>[]\> +> **forward**(`input`, `detectionThreshold?`): `Promise`\<[`Detection`](../interfaces/Detection.md)\<`ResolveLabels`\<`T`, \{ `rf-detr-nano`: \{ `labelMap`: _typeof_ [`CocoLabel`](../enumerations/CocoLabel.md); `preprocessorConfig`: \{ `normMean`: [`Triple`](../type-aliases/Triple.md)\<`number`\>; `normStd`: [`Triple`](../type-aliases/Triple.md)\<`number`\>; \}; \}; `ssdlite-320-mobilenet-v3-large`: \{ `labelMap`: _typeof_ [`CocoLabel`](../enumerations/CocoLabel.md); `preprocessorConfig`: `undefined`; \}; \}\>\>[]\> -Defined in: [modules/computer_vision/ObjectDetectionModule.ts:145](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L145) +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:118](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L118) Executes the model's forward pass to detect objects within the provided image. #### Parameters -##### imageSource +##### input -`string` +A string image source (file path, URI, or Base64) or a [PixelData](../interfaces/PixelData.md) object. -A string representing the image source (e.g., a file path, URI, or base64 string). +`string` | [`PixelData`](../interfaces/PixelData.md) -##### detectionThreshold +##### detectionThreshold? `number` = `0.7` @@ -91,9 +213,9 @@ Minimum confidence score for a detection to be included. Default is 0.7. A Promise resolving to an array of [Detection](../interfaces/Detection.md) objects. -#### Throws +#### Overrides -If the model is not loaded. +`VisionLabeledModule.forward` --- @@ -101,7 +223,9 @@ If the model is not loaded. > `protected` **forwardET**(`inputTensor`): `Promise`\<[`TensorPtr`](../interfaces/TensorPtr.md)[]\> -Defined in: [modules/BaseModule.ts:23](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L23) +Defined in: [modules/BaseModule.ts:80](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L80) + +**`Internal`** Runs the model's forward method with the given input tensors. It returns the output tensors that mimic the structure of output from ExecuTorch. @@ -122,7 +246,7 @@ Array of output tensors. #### Inherited from -`BaseLabeledModule.forwardET` +`VisionLabeledModule.forwardET` --- @@ -130,7 +254,7 @@ Array of output tensors. > **getInputShape**(`methodName`, `index`): `Promise`\<`number`[]\> -Defined in: [modules/BaseModule.ts:34](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L34) +Defined in: [modules/BaseModule.ts:91](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseModule.ts#L91) Gets the input shape for a given method and index. @@ -156,7 +280,7 @@ The input shape as an array of numbers. #### Inherited from -`BaseLabeledModule.getInputShape` +`VisionLabeledModule.getInputShape` --- @@ -164,7 +288,9 @@ The input shape as an array of numbers. > **load**(): `Promise`\<`void`\> -Defined in: [modules/BaseLabeledModule.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseLabeledModule.ts#L61) +Defined in: [modules/computer_vision/VisionLabeledModule.ts:25](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/VisionLabeledModule.ts#L25) + +Load the model and prepare it for inference. #### Returns @@ -172,17 +298,15 @@ Defined in: [modules/BaseLabeledModule.ts:61](https://github.com/software-mansio #### Inherited from -`BaseLabeledModule.load` +`VisionLabeledModule.load` --- ### fromCustomConfig() -> `static` **fromCustomConfig**\<`L`\>(`modelSource`, `config`, `onDownloadProgress`): `Promise`\<`ObjectDetectionModule`\<`L`\>\> +> `static` **fromCustomConfig**\<`L`\>(`modelSource`, `config`, `onDownloadProgress?`): `Promise`\<`ObjectDetectionModule`\<`L`\>\> -Defined in: [modules/computer_vision/ObjectDetectionModule.ts:118](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L118) - -Creates an object detection instance with a user-provided label map and custom config. +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:125](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L125) #### Type Parameters @@ -196,33 +320,25 @@ Creates an object detection instance with a user-provided label map and custom c [`ResourceSource`](../type-aliases/ResourceSource.md) -A fetchable resource pointing to the model binary. - ##### config [`ObjectDetectionConfig`](../type-aliases/ObjectDetectionConfig.md)\<`L`\> -A [ObjectDetectionConfig](../type-aliases/ObjectDetectionConfig.md) object with the label map. - -##### onDownloadProgress +##### onDownloadProgress? (`progress`) => `void` -Optional callback to monitor download progress, receiving a value between 0 and 1. - #### Returns `Promise`\<`ObjectDetectionModule`\<`L`\>\> -A Promise resolving to an `ObjectDetectionModule` instance typed to the provided label map. - --- ### fromModelName() -> `static` **fromModelName**\<`C`\>(`config`, `onDownloadProgress`): `Promise`\<`ObjectDetectionModule`\<`ModelNameOf`\<`C`\>\>\> +> `static` **fromModelName**\<`C`\>(`config`, `onDownloadProgress?`): `Promise`\<`ObjectDetectionModule`\<`ModelNameOf`\<`C`\>\>\> -Defined in: [modules/computer_vision/ObjectDetectionModule.ts:88](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L88) +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:73](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L73) Creates an object detection instance for a built-in model. @@ -240,7 +356,7 @@ Creates an object detection instance for a built-in model. A [ObjectDetectionModelSources](../type-aliases/ObjectDetectionModelSources.md) object specifying which model to load and where to fetch it from. -##### onDownloadProgress +##### onDownloadProgress? (`progress`) => `void` diff --git a/docs/docs/06-api-reference/classes/SemanticSegmentationModule.md b/docs/docs/06-api-reference/classes/SemanticSegmentationModule.md index b6f23cd74..97f33ea24 100644 --- a/docs/docs/06-api-reference/classes/SemanticSegmentationModule.md +++ b/docs/docs/06-api-reference/classes/SemanticSegmentationModule.md @@ -1,6 +1,6 @@ # Class: SemanticSegmentationModule\ -Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:81](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L81) +Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:80](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L80) Generic semantic segmentation module with type-safe label maps. Use a model name (e.g. `'deeplab-v3-resnet50'`) as the generic parameter for built-in models, @@ -8,7 +8,7 @@ or a custom label enum for custom configs. ## Extends -- `BaseModule` +- `BaseLabeledModule`\<`ResolveLabels`\<`T`\>\> ## Type Parameters @@ -94,7 +94,19 @@ Model-specific output (e.g., detections, classifications, embeddings) #### Inherited from -`BaseModule.generateFromFrame` +`BaseLabeledModule.generateFromFrame` + +--- + +### labelMap + +> `protected` `readonly` **labelMap**: `ResolveLabels` + +Defined in: [modules/BaseLabeledModule.ts:52](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseLabeledModule.ts#L52) + +#### Inherited from + +`BaseLabeledModule.labelMap` --- @@ -110,7 +122,7 @@ Native module instance (JSI Host Object) #### Inherited from -`BaseModule.nativeModule` +`BaseLabeledModule.nativeModule` ## Methods @@ -130,7 +142,7 @@ Always call this method when you're done with a model to prevent memory leaks. #### Inherited from -`BaseModule.delete` +`BaseLabeledModule.delete` --- @@ -138,7 +150,7 @@ Always call this method when you're done with a model to prevent memory leaks. > **forward**\<`K`\>(`imageSource`, `classesOfInterest?`, `resizeToInput?`): `Promise`\<`Record`\<`"ARGMAX"`, `Int32Array`\<`ArrayBufferLike`\>\> & `Record`\<`K`, `Float32Array`\<`ArrayBufferLike`\>\>\> -Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:197](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L197) +Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:179](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L179) Executes the model's forward pass to perform semantic segmentation on the provided image. @@ -207,7 +219,7 @@ Array of output tensors. #### Inherited from -`BaseModule.forwardET` +`BaseLabeledModule.forwardET` --- @@ -241,7 +253,7 @@ The input shape as an array of numbers. #### Inherited from -`BaseModule.getInputShape` +`BaseLabeledModule.getInputShape` --- @@ -249,7 +261,7 @@ The input shape as an array of numbers. > **load**(): `Promise`\<`void`\> -Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:97](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L97) +Defined in: [modules/BaseLabeledModule.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/BaseLabeledModule.ts#L61) Load the model and prepare it for inference. @@ -257,9 +269,9 @@ Load the model and prepare it for inference. `Promise`\<`void`\> -#### Overrides +#### Inherited from -`BaseModule.load` +`BaseLabeledModule.load` --- @@ -267,7 +279,7 @@ Load the model and prepare it for inference. > `static` **fromCustomConfig**\<`L`\>(`modelSource`, `config`, `onDownloadProgress?`): `Promise`\<`SemanticSegmentationModule`\<`L`\>\> -Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:163](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L163) +Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:147](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L147) Creates a segmentation instance with a user-provided label map and custom config. Use this when working with a custom-exported segmentation model that is not one of the built-in models. @@ -320,7 +332,7 @@ const segmentation = await SemanticSegmentationModule.fromCustomConfig( > `static` **fromModelName**\<`C`\>(`config`, `onDownloadProgress?`): `Promise`\<`SemanticSegmentationModule`\<[`ModelNameOf`](../type-aliases/ModelNameOf.md)\<`C`\>\>\> -Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:116](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L116) +Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:104](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L104) Creates a segmentation instance for a built-in model. The config object is discriminated by `modelName` — each model can require different fields. diff --git a/docs/docs/06-api-reference/functions/useObjectDetection.md b/docs/docs/06-api-reference/functions/useObjectDetection.md index 5c2d7abed..7ed581760 100644 --- a/docs/docs/06-api-reference/functions/useObjectDetection.md +++ b/docs/docs/06-api-reference/functions/useObjectDetection.md @@ -2,7 +2,7 @@ > **useObjectDetection**\<`C`\>(`props`): [`ObjectDetectionType`](../interfaces/ObjectDetectionType.md)\<[`ObjectDetectionLabels`](../type-aliases/ObjectDetectionLabels.md)\<`C`\[`"modelName"`\]\>\> -Defined in: [hooks/computer_vision/useObjectDetection.ts:20](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts#L20) +Defined in: [hooks/computer_vision/useObjectDetection.ts:22](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/hooks/computer_vision/useObjectDetection.ts#L22) React hook for managing an Object Detection model instance. @@ -26,4 +26,4 @@ Configuration object containing `model` config and optional `preventLoad` flag. [`ObjectDetectionType`](../interfaces/ObjectDetectionType.md)\<[`ObjectDetectionLabels`](../type-aliases/ObjectDetectionLabels.md)\<`C`\[`"modelName"`\]\>\> -An object with model state (`error`, `isReady`, `isGenerating`, `downloadProgress`) and a typed `forward` function. +An object with model state (`error`, `isReady`, `isGenerating`, `downloadProgress`) and typed `forward` and `runOnFrame` functions. diff --git a/docs/docs/06-api-reference/functions/useSemanticSegmentation.md b/docs/docs/06-api-reference/functions/useSemanticSegmentation.md index ebd154086..ee260c7f1 100644 --- a/docs/docs/06-api-reference/functions/useSemanticSegmentation.md +++ b/docs/docs/06-api-reference/functions/useSemanticSegmentation.md @@ -2,7 +2,7 @@ > **useSemanticSegmentation**\<`C`\>(`props`): [`SemanticSegmentationType`](../interfaces/SemanticSegmentationType.md)\<[`SegmentationLabels`](../type-aliases/SegmentationLabels.md)\<[`ModelNameOf`](../type-aliases/ModelNameOf.md)\<`C`\>\>\> -Defined in: [hooks/computer_vision/useSemanticSegmentation.ts:31](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/hooks/computer_vision/useSemanticSegmentation.ts#L31) +Defined in: [hooks/computer_vision/useSemanticSegmentation.ts:29](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/hooks/computer_vision/useSemanticSegmentation.ts#L29) React hook for managing a Semantic Segmentation model instance. diff --git a/docs/docs/06-api-reference/index.md b/docs/docs/06-api-reference/index.md index 07e63aa17..c059a9cc8 100644 --- a/docs/docs/06-api-reference/index.md +++ b/docs/docs/06-api-reference/index.md @@ -5,10 +5,10 @@ - [useClassification](functions/useClassification.md) - [useExecutorchModule](functions/useExecutorchModule.md) - [useImageEmbeddings](functions/useImageEmbeddings.md) -- [useImageSegmentation](functions/useImageSegmentation.md) - [useLLM](functions/useLLM.md) - [useObjectDetection](functions/useObjectDetection.md) - [useOCR](functions/useOCR.md) +- [useSemanticSegmentation](functions/useSemanticSegmentation.md) - [useSpeechToText](functions/useSpeechToText.md) - [useStyleTransfer](functions/useStyleTransfer.md) - [useTextEmbeddings](functions/useTextEmbeddings.md) @@ -35,11 +35,6 @@ - [BK_SDM_TINY_VPRED_256](variables/BK_SDM_TINY_VPRED_256.md) - [BK_SDM_TINY_VPRED_512](variables/BK_SDM_TINY_VPRED_512.md) -## Models - Image Segmentation - -- [DEEPLAB_V3_RESNET50](variables/DEEPLAB_V3_RESNET50.md) -- [SELFIE_SEGMENTATION](variables/SELFIE_SEGMENTATION.md) - ## Models - LMM - [HAMMER2_1_0_5B](variables/HAMMER2_1_0_5B.md) @@ -82,6 +77,22 @@ - [RF_DETR_NANO](variables/RF_DETR_NANO.md) - [SSDLITE_320_MOBILENET_V3_LARGE](variables/SSDLITE_320_MOBILENET_V3_LARGE.md) +## Models - Semantic Segmentation + +- [DEEPLAB_V3_MOBILENET_V3_LARGE](variables/DEEPLAB_V3_MOBILENET_V3_LARGE.md) +- [DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED](variables/DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED.md) +- [DEEPLAB_V3_RESNET101](variables/DEEPLAB_V3_RESNET101.md) +- [DEEPLAB_V3_RESNET101_QUANTIZED](variables/DEEPLAB_V3_RESNET101_QUANTIZED.md) +- [DEEPLAB_V3_RESNET50](variables/DEEPLAB_V3_RESNET50.md) +- [DEEPLAB_V3_RESNET50_QUANTIZED](variables/DEEPLAB_V3_RESNET50_QUANTIZED.md) +- [FCN_RESNET101](variables/FCN_RESNET101.md) +- [FCN_RESNET101_QUANTIZED](variables/FCN_RESNET101_QUANTIZED.md) +- [FCN_RESNET50](variables/FCN_RESNET50.md) +- [FCN_RESNET50_QUANTIZED](variables/FCN_RESNET50_QUANTIZED.md) +- [LRASPP_MOBILENET_V3_LARGE](variables/LRASPP_MOBILENET_V3_LARGE.md) +- [LRASPP_MOBILENET_V3_LARGE_QUANTIZED](variables/LRASPP_MOBILENET_V3_LARGE_QUANTIZED.md) +- [SELFIE_SEGMENTATION](variables/SELFIE_SEGMENTATION.md) + ## Models - Speech To Text - [WHISPER_BASE](variables/WHISPER_BASE.md) @@ -187,6 +198,7 @@ - [RnExecutorchErrorCode](enumerations/RnExecutorchErrorCode.md) - [Logger](classes/Logger.md) - [RnExecutorchError](classes/RnExecutorchError.md) +- [Frame](interfaces/Frame.md) - [IMAGENET1K_MEAN](variables/IMAGENET1K_MEAN.md) - [IMAGENET1K_STD](variables/IMAGENET1K_STD.md) @@ -222,8 +234,6 @@ - [GenerationConfig](interfaces/GenerationConfig.md) - [ImageEmbeddingsProps](interfaces/ImageEmbeddingsProps.md) - [ImageEmbeddingsType](interfaces/ImageEmbeddingsType.md) -- [ImageSegmentationProps](interfaces/ImageSegmentationProps.md) -- [ImageSegmentationType](interfaces/ImageSegmentationType.md) - [KokoroConfig](interfaces/KokoroConfig.md) - [KokoroVoiceExtras](interfaces/KokoroVoiceExtras.md) - [LLMConfig](interfaces/LLMConfig.md) @@ -235,8 +245,11 @@ - [OCRDetection](interfaces/OCRDetection.md) - [OCRProps](interfaces/OCRProps.md) - [OCRType](interfaces/OCRType.md) +- [PixelData](interfaces/PixelData.md) - [Point](interfaces/Point.md) - [Segment](interfaces/Segment.md) +- [SemanticSegmentationProps](interfaces/SemanticSegmentationProps.md) +- [SemanticSegmentationType](interfaces/SemanticSegmentationType.md) - [SpeechToTextModelConfig](interfaces/SpeechToTextModelConfig.md) - [SpeechToTextProps](interfaces/SpeechToTextProps.md) - [SpeechToTextType](interfaces/SpeechToTextType.md) @@ -267,16 +280,16 @@ - [LLMTool](type-aliases/LLMTool.md) - [MessageRole](type-aliases/MessageRole.md) - [ModelNameOf](type-aliases/ModelNameOf.md) -- [ModelSources](type-aliases/ModelSources.md) - [ObjectDetectionConfig](type-aliases/ObjectDetectionConfig.md) - [ObjectDetectionLabels](type-aliases/ObjectDetectionLabels.md) - [ObjectDetectionModelName](type-aliases/ObjectDetectionModelName.md) - [ObjectDetectionModelSources](type-aliases/ObjectDetectionModelSources.md) - [OCRLanguage](type-aliases/OCRLanguage.md) - [ResourceSource](type-aliases/ResourceSource.md) -- [SegmentationConfig](type-aliases/SegmentationConfig.md) - [SegmentationLabels](type-aliases/SegmentationLabels.md) -- [SegmentationModelName](type-aliases/SegmentationModelName.md) +- [SemanticSegmentationConfig](type-aliases/SemanticSegmentationConfig.md) +- [SemanticSegmentationModelName](type-aliases/SemanticSegmentationModelName.md) +- [SemanticSegmentationModelSources](type-aliases/SemanticSegmentationModelSources.md) - [SpeechToTextLanguage](type-aliases/SpeechToTextLanguage.md) - [TensorBuffer](type-aliases/TensorBuffer.md) - [TextToSpeechLanguage](type-aliases/TextToSpeechLanguage.md) @@ -288,10 +301,10 @@ - [ClassificationModule](classes/ClassificationModule.md) - [ExecutorchModule](classes/ExecutorchModule.md) - [ImageEmbeddingsModule](classes/ImageEmbeddingsModule.md) -- [ImageSegmentationModule](classes/ImageSegmentationModule.md) - [LLMModule](classes/LLMModule.md) - [ObjectDetectionModule](classes/ObjectDetectionModule.md) - [OCRModule](classes/OCRModule.md) +- [SemanticSegmentationModule](classes/SemanticSegmentationModule.md) - [SpeechToTextModule](classes/SpeechToTextModule.md) - [StyleTransferModule](classes/StyleTransferModule.md) - [TextEmbeddingsModule](classes/TextEmbeddingsModule.md) diff --git a/docs/docs/06-api-reference/interfaces/Bbox.md b/docs/docs/06-api-reference/interfaces/Bbox.md index 864152f14..6eadedfb1 100644 --- a/docs/docs/06-api-reference/interfaces/Bbox.md +++ b/docs/docs/06-api-reference/interfaces/Bbox.md @@ -1,6 +1,6 @@ # Interface: Bbox -Defined in: [types/objectDetection.ts:14](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L14) +Defined in: [types/objectDetection.ts:15](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L15) Represents a bounding box for a detected object in an image. @@ -10,7 +10,7 @@ Represents a bounding box for a detected object in an image. > **x1**: `number` -Defined in: [types/objectDetection.ts:15](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L15) +Defined in: [types/objectDetection.ts:16](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L16) The x-coordinate of the bottom-left corner of the bounding box. @@ -20,7 +20,7 @@ The x-coordinate of the bottom-left corner of the bounding box. > **x2**: `number` -Defined in: [types/objectDetection.ts:16](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L16) +Defined in: [types/objectDetection.ts:17](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L17) The x-coordinate of the top-right corner of the bounding box. @@ -30,7 +30,7 @@ The x-coordinate of the top-right corner of the bounding box. > **y1**: `number` -Defined in: [types/objectDetection.ts:17](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L17) +Defined in: [types/objectDetection.ts:18](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L18) The y-coordinate of the bottom-left corner of the bounding box. @@ -40,6 +40,6 @@ The y-coordinate of the bottom-left corner of the bounding box. > **y2**: `number` -Defined in: [types/objectDetection.ts:18](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L18) +Defined in: [types/objectDetection.ts:19](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L19) The y-coordinate of the top-right corner of the bounding box. diff --git a/docs/docs/06-api-reference/interfaces/Detection.md b/docs/docs/06-api-reference/interfaces/Detection.md index 695460596..44a7324aa 100644 --- a/docs/docs/06-api-reference/interfaces/Detection.md +++ b/docs/docs/06-api-reference/interfaces/Detection.md @@ -1,6 +1,6 @@ # Interface: Detection\ -Defined in: [types/objectDetection.ts:30](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L30) +Defined in: [types/objectDetection.ts:31](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L31) Represents a detected object within an image, including its bounding box, label, and confidence score. @@ -18,7 +18,7 @@ The label enum type for the detected object. Defaults to [CocoLabel](../enumerat > **bbox**: [`Bbox`](Bbox.md) -Defined in: [types/objectDetection.ts:31](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L31) +Defined in: [types/objectDetection.ts:32](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L32) The bounding box of the detected object, defined by its top-left (x1, y1) and bottom-right (x2, y2) coordinates. @@ -28,7 +28,7 @@ The bounding box of the detected object, defined by its top-left (x1, y1) and bo > **label**: keyof `L` -Defined in: [types/objectDetection.ts:32](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L32) +Defined in: [types/objectDetection.ts:33](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L33) The class label of the detected object. @@ -38,6 +38,6 @@ The class label of the detected object. > **score**: `number` -Defined in: [types/objectDetection.ts:33](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L33) +Defined in: [types/objectDetection.ts:34](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L34) The confidence score of the detection, typically ranging from 0 to 1. diff --git a/docs/docs/06-api-reference/interfaces/ObjectDetectionProps.md b/docs/docs/06-api-reference/interfaces/ObjectDetectionProps.md index 06f3b0e8e..7199fe504 100644 --- a/docs/docs/06-api-reference/interfaces/ObjectDetectionProps.md +++ b/docs/docs/06-api-reference/interfaces/ObjectDetectionProps.md @@ -1,6 +1,6 @@ # Interface: ObjectDetectionProps\ -Defined in: [types/objectDetection.ts:71](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L71) +Defined in: [types/objectDetection.ts:72](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L72) Props for the `useObjectDetection` hook. @@ -18,7 +18,7 @@ A [ObjectDetectionModelSources](../type-aliases/ObjectDetectionModelSources.md) > **model**: `C` -Defined in: [types/objectDetection.ts:72](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L72) +Defined in: [types/objectDetection.ts:73](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L73) The model config containing `modelName` and `modelSource`. @@ -28,6 +28,6 @@ The model config containing `modelName` and `modelSource`. > `optional` **preventLoad**: `boolean` -Defined in: [types/objectDetection.ts:73](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L73) +Defined in: [types/objectDetection.ts:74](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L74) Boolean that can prevent automatic model loading (and downloading the data if you load it for the first time) after running the hook. diff --git a/docs/docs/06-api-reference/interfaces/ObjectDetectionType.md b/docs/docs/06-api-reference/interfaces/ObjectDetectionType.md index a9e8a97b7..95a2bd407 100644 --- a/docs/docs/06-api-reference/interfaces/ObjectDetectionType.md +++ b/docs/docs/06-api-reference/interfaces/ObjectDetectionType.md @@ -1,6 +1,6 @@ # Interface: ObjectDetectionType\ -Defined in: [types/objectDetection.ts:84](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L84) +Defined in: [types/objectDetection.ts:85](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L85) Return type for the `useObjectDetection` hook. Manages the state and operations for Computer Vision object detection tasks. @@ -19,7 +19,7 @@ The [LabelEnum](../type-aliases/LabelEnum.md) representing the model's class lab > **downloadProgress**: `number` -Defined in: [types/objectDetection.ts:103](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L103) +Defined in: [types/objectDetection.ts:104](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L104) Represents the download progress of the model binary as a value between 0 and 1. @@ -29,7 +29,7 @@ Represents the download progress of the model binary as a value between 0 and 1. > **error**: [`RnExecutorchError`](../classes/RnExecutorchError.md) \| `null` -Defined in: [types/objectDetection.ts:88](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L88) +Defined in: [types/objectDetection.ts:89](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L89) Contains the error object if the model failed to load, download, or encountered a runtime error during detection. @@ -37,31 +37,31 @@ Contains the error object if the model failed to load, download, or encountered ### forward() -> **forward**: (`imageSource`, `detectionThreshold?`) => `Promise`\<[`Detection`](Detection.md)\<`L`\>[]\> +> **forward**: (`input`, `detectionThreshold?`) => `Promise`\<[`Detection`](Detection.md)\<`L`\>[]\> -Defined in: [types/objectDetection.ts:112](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L112) +Defined in: [types/objectDetection.ts:114](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L114) -Executes the model's forward pass to detect objects within the provided image. +Executes the model's forward pass with automatic input type detection. #### Parameters -##### imageSource +##### input -`string` +Image source (string path/URI or PixelData object) -A string representing the image source (e.g., a file path, URI, or base64 string) to be processed. +`string` | [`PixelData`](PixelData.md) ##### detectionThreshold? `number` -An optional number between 0 and 1 representing the minimum confidence score required for an object to be included in the results. Default is 0.7. +An optional number between 0 and 1 representing the minimum confidence score. Default is 0.7. #### Returns `Promise`\<[`Detection`](Detection.md)\<`L`\>[]\> -A Promise that resolves to an array of `Detection` objects, where each object typically contains bounding box coordinates, a class label, and a confidence score. +A Promise that resolves to an array of `Detection` objects. #### Throws @@ -73,7 +73,7 @@ If the model is not loaded or is currently processing another image. > **isGenerating**: `boolean` -Defined in: [types/objectDetection.ts:98](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L98) +Defined in: [types/objectDetection.ts:99](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L99) Indicates whether the model is currently processing an image. @@ -83,6 +83,34 @@ Indicates whether the model is currently processing an image. > **isReady**: `boolean` -Defined in: [types/objectDetection.ts:93](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L93) +Defined in: [types/objectDetection.ts:94](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L94) Indicates whether the object detection model is loaded and ready to process images. + +--- + +### runOnFrame + +> **runOnFrame**: (`frame`, `detectionThreshold`) => [`Detection`](Detection.md)\<`L`\>[] \| `null` + +Defined in: [types/objectDetection.ts:132](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L132) + +Synchronous worklet function for real-time VisionCamera frame processing. +Automatically handles native buffer extraction and cleanup. + +**Use this for VisionCamera frame processing in worklets.** +For async processing, use `forward()` instead. + +Available after model is loaded (`isReady: true`). + +#### Param + +VisionCamera Frame object + +#### Param + +The threshold for detection sensitivity. + +#### Returns + +Array of Detection objects representing detected items in the frame. diff --git a/docs/docs/06-api-reference/type-aliases/ObjectDetectionConfig.md b/docs/docs/06-api-reference/type-aliases/ObjectDetectionConfig.md index aefbc33c5..5705f34d2 100644 --- a/docs/docs/06-api-reference/type-aliases/ObjectDetectionConfig.md +++ b/docs/docs/06-api-reference/type-aliases/ObjectDetectionConfig.md @@ -2,7 +2,7 @@ > **ObjectDetectionConfig**\<`T`\> = `object` -Defined in: [types/objectDetection.ts:58](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L58) +Defined in: [types/objectDetection.ts:59](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L59) Configuration for a custom object detection model. @@ -18,7 +18,7 @@ Configuration for a custom object detection model. > **labelMap**: `T` -Defined in: [types/objectDetection.ts:59](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L59) +Defined in: [types/objectDetection.ts:60](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L60) --- @@ -26,7 +26,7 @@ Defined in: [types/objectDetection.ts:59](https://github.com/software-mansion/re > `optional` **preprocessorConfig**: `object` -Defined in: [types/objectDetection.ts:60](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L60) +Defined in: [types/objectDetection.ts:61](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L61) #### normMean? diff --git a/docs/docs/06-api-reference/type-aliases/ObjectDetectionLabels.md b/docs/docs/06-api-reference/type-aliases/ObjectDetectionLabels.md index 2a412796f..21453f6aa 100644 --- a/docs/docs/06-api-reference/type-aliases/ObjectDetectionLabels.md +++ b/docs/docs/06-api-reference/type-aliases/ObjectDetectionLabels.md @@ -2,7 +2,7 @@ > **ObjectDetectionLabels**\<`M`\> = `ResolveLabelsFor`\<`M`, `ModelConfigsType`\> -Defined in: [modules/computer_vision/ObjectDetectionModule.ts:44](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L44) +Defined in: [modules/computer_vision/ObjectDetectionModule.ts:42](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ObjectDetectionModule.ts#L42) Resolves the [LabelEnum](LabelEnum.md) for a given built-in object detection model name. diff --git a/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelName.md b/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelName.md index 4f9872946..8bfff7d4a 100644 --- a/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelName.md +++ b/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelName.md @@ -2,6 +2,6 @@ > **ObjectDetectionModelName** = [`ObjectDetectionModelSources`](ObjectDetectionModelSources.md)\[`"modelName"`\] -Defined in: [types/objectDetection.ts:51](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L51) +Defined in: [types/objectDetection.ts:52](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L52) Union of all built-in object detection model names. diff --git a/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelSources.md b/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelSources.md index 1191ca8f0..91be33999 100644 --- a/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelSources.md +++ b/docs/docs/06-api-reference/type-aliases/ObjectDetectionModelSources.md @@ -2,7 +2,7 @@ > **ObjectDetectionModelSources** = \{ `modelName`: `"ssdlite-320-mobilenet-v3-large"`; `modelSource`: [`ResourceSource`](ResourceSource.md); \} \| \{ `modelName`: `"rf-detr-nano"`; `modelSource`: [`ResourceSource`](ResourceSource.md); \} -Defined in: [types/objectDetection.ts:42](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L42) +Defined in: [types/objectDetection.ts:43](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/types/objectDetection.ts#L43) Per-model config for [ObjectDetectionModule.fromModelName](../classes/ObjectDetectionModule.md#frommodelname). Each model name maps to its required fields. diff --git a/docs/docs/06-api-reference/type-aliases/SegmentationLabels.md b/docs/docs/06-api-reference/type-aliases/SegmentationLabels.md index 5e5968dd3..5f62875c0 100644 --- a/docs/docs/06-api-reference/type-aliases/SegmentationLabels.md +++ b/docs/docs/06-api-reference/type-aliases/SegmentationLabels.md @@ -2,7 +2,7 @@ > **SegmentationLabels**\<`M`\> = `ModelConfigsType`\[`M`\]\[`"labelMap"`\] -Defined in: [modules/computer_vision/ImageSegmentationModule.ts:42](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/ImageSegmentationModule.ts#L42) +Defined in: [modules/computer_vision/SemanticSegmentationModule.ts:58](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/modules/computer_vision/SemanticSegmentationModule.ts#L58) Resolves the [LabelEnum](LabelEnum.md) for a given built-in model name. @@ -10,6 +10,6 @@ Resolves the [LabelEnum](LabelEnum.md) for a given built-in model name. ### M -`M` _extends_ [`SegmentationModelName`](SegmentationModelName.md) +`M` _extends_ [`SemanticSegmentationModelName`](SemanticSegmentationModelName.md) -A built-in model name from [SegmentationModelName](SegmentationModelName.md). +A built-in model name from [SemanticSegmentationModelName](SemanticSegmentationModelName.md). diff --git a/docs/docs/06-api-reference/typedoc-sidebar.cjs b/docs/docs/06-api-reference/typedoc-sidebar.cjs index 9a646b7fe..faccf365c 100644 --- a/docs/docs/06-api-reference/typedoc-sidebar.cjs +++ b/docs/docs/06-api-reference/typedoc-sidebar.cjs @@ -1,4 +1,4 @@ // @ts-check /** @type {import("@docusaurus/plugin-content-docs").SidebarsConfig} */ -const typedocSidebar = {items:[{type:"category",label:"Hooks",items:[{type:"doc",id:"06-api-reference/functions/useClassification",label:"useClassification"},{type:"doc",id:"06-api-reference/functions/useExecutorchModule",label:"useExecutorchModule"},{type:"doc",id:"06-api-reference/functions/useImageEmbeddings",label:"useImageEmbeddings"},{type:"doc",id:"06-api-reference/functions/useImageSegmentation",label:"useImageSegmentation"},{type:"doc",id:"06-api-reference/functions/useLLM",label:"useLLM"},{type:"doc",id:"06-api-reference/functions/useObjectDetection",label:"useObjectDetection"},{type:"doc",id:"06-api-reference/functions/useOCR",label:"useOCR"},{type:"doc",id:"06-api-reference/functions/useSpeechToText",label:"useSpeechToText"},{type:"doc",id:"06-api-reference/functions/useStyleTransfer",label:"useStyleTransfer"},{type:"doc",id:"06-api-reference/functions/useTextEmbeddings",label:"useTextEmbeddings"},{type:"doc",id:"06-api-reference/functions/useTextToImage",label:"useTextToImage"},{type:"doc",id:"06-api-reference/functions/useTextToSpeech",label:"useTextToSpeech"},{type:"doc",id:"06-api-reference/functions/useTokenizer",label:"useTokenizer"},{type:"doc",id:"06-api-reference/functions/useVAD",label:"useVAD"},{type:"doc",id:"06-api-reference/functions/useVerticalOCR",label:"useVerticalOCR"}]},{type:"category",label:"Interfaces",items:[{type:"doc",id:"06-api-reference/interfaces/ResourceSourceExtended",label:"ResourceSourceExtended"}]},{type:"category",label:"Models - Classification",items:[{type:"doc",id:"06-api-reference/variables/EFFICIENTNET_V2_S",label:"EFFICIENTNET_V2_S"}]},{type:"category",label:"Models - Image Embeddings",items:[{type:"doc",id:"06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE",label:"CLIP_VIT_BASE_PATCH32_IMAGE"}]},{type:"category",label:"Models - Image Generation",items:[{type:"doc",id:"06-api-reference/variables/BK_SDM_TINY_VPRED_256",label:"BK_SDM_TINY_VPRED_256"},{type:"doc",id:"06-api-reference/variables/BK_SDM_TINY_VPRED_512",label:"BK_SDM_TINY_VPRED_512"}]},{type:"category",label:"Models - Image Segmentation",items:[{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET50",label:"DEEPLAB_V3_RESNET50"},{type:"doc",id:"06-api-reference/variables/SELFIE_SEGMENTATION",label:"SELFIE_SEGMENTATION"}]},{type:"category",label:"Models - LMM",items:[{type:"doc",id:"06-api-reference/variables/HAMMER2_1_0_5B",label:"HAMMER2_1_0_5B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_0_5B_QUANTIZED",label:"HAMMER2_1_0_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_1_5B",label:"HAMMER2_1_1_5B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_1_5B_QUANTIZED",label:"HAMMER2_1_1_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_3B",label:"HAMMER2_1_3B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_3B_QUANTIZED",label:"HAMMER2_1_3B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LFM2_5_1_2B_INSTRUCT",label:"LFM2_5_1_2B_INSTRUCT"},{type:"doc",id:"06-api-reference/variables/LFM2_5_1_2B_INSTRUCT_QUANTIZED",label:"LFM2_5_1_2B_INSTRUCT_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B",label:"LLAMA3_2_1B"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B_QLORA",label:"LLAMA3_2_1B_QLORA"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B_SPINQUANT",label:"LLAMA3_2_1B_SPINQUANT"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B",label:"LLAMA3_2_3B"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B_QLORA",label:"LLAMA3_2_3B_QLORA"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B_SPINQUANT",label:"LLAMA3_2_3B_SPINQUANT"},{type:"doc",id:"06-api-reference/variables/PHI_4_MINI_4B",label:"PHI_4_MINI_4B"},{type:"doc",id:"06-api-reference/variables/PHI_4_MINI_4B_QUANTIZED",label:"PHI_4_MINI_4B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_0_5B",label:"QWEN2_5_0_5B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_0_5B_QUANTIZED",label:"QWEN2_5_0_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_1_5B",label:"QWEN2_5_1_5B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_1_5B_QUANTIZED",label:"QWEN2_5_1_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_3B",label:"QWEN2_5_3B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_3B_QUANTIZED",label:"QWEN2_5_3B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_0_6B",label:"QWEN3_0_6B"},{type:"doc",id:"06-api-reference/variables/QWEN3_0_6B_QUANTIZED",label:"QWEN3_0_6B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_1_7B",label:"QWEN3_1_7B"},{type:"doc",id:"06-api-reference/variables/QWEN3_1_7B_QUANTIZED",label:"QWEN3_1_7B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_4B",label:"QWEN3_4B"},{type:"doc",id:"06-api-reference/variables/QWEN3_4B_QUANTIZED",label:"QWEN3_4B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_1_7B",label:"SMOLLM2_1_1_7B"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_1_7B_QUANTIZED",label:"SMOLLM2_1_1_7B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_135M",label:"SMOLLM2_1_135M"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_135M_QUANTIZED",label:"SMOLLM2_1_135M_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_360M",label:"SMOLLM2_1_360M"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_360M_QUANTIZED",label:"SMOLLM2_1_360M_QUANTIZED"}]},{type:"category",label:"Models - Object Detection",items:[{type:"doc",id:"06-api-reference/variables/RF_DETR_NANO",label:"RF_DETR_NANO"},{type:"doc",id:"06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE",label:"SSDLITE_320_MOBILENET_V3_LARGE"}]},{type:"category",label:"Models - Speech To Text",items:[{type:"doc",id:"06-api-reference/variables/WHISPER_BASE",label:"WHISPER_BASE"},{type:"doc",id:"06-api-reference/variables/WHISPER_BASE_EN",label:"WHISPER_BASE_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_SMALL",label:"WHISPER_SMALL"},{type:"doc",id:"06-api-reference/variables/WHISPER_SMALL_EN",label:"WHISPER_SMALL_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY",label:"WHISPER_TINY"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY_EN",label:"WHISPER_TINY_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED",label:"WHISPER_TINY_EN_QUANTIZED"}]},{type:"category",label:"Models - Style Transfer",items:[{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_CANDY",label:"STYLE_TRANSFER_CANDY"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_MOSAIC",label:"STYLE_TRANSFER_MOSAIC"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS",label:"STYLE_TRANSFER_RAIN_PRINCESS"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_UDNIE",label:"STYLE_TRANSFER_UDNIE"}]},{type:"category",label:"Models - Text Embeddings",items:[{type:"doc",id:"06-api-reference/variables/ALL_MINILM_L6_V2",label:"ALL_MINILM_L6_V2"},{type:"doc",id:"06-api-reference/variables/ALL_MPNET_BASE_V2",label:"ALL_MPNET_BASE_V2"},{type:"doc",id:"06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT",label:"CLIP_VIT_BASE_PATCH32_TEXT"},{type:"doc",id:"06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1",label:"MULTI_QA_MINILM_L6_COS_V1"},{type:"doc",id:"06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1",label:"MULTI_QA_MPNET_BASE_DOT_V1"}]},{type:"category",label:"Models - Text to Speech",items:[{type:"doc",id:"06-api-reference/variables/KOKORO_MEDIUM",label:"KOKORO_MEDIUM"},{type:"doc",id:"06-api-reference/variables/KOKORO_SMALL",label:"KOKORO_SMALL"}]},{type:"category",label:"Models - Voice Activity Detection",items:[{type:"doc",id:"06-api-reference/variables/FSMN_VAD",label:"FSMN_VAD"}]},{type:"category",label:"OCR Supported Alphabets",items:[{type:"doc",id:"06-api-reference/variables/OCR_ABAZA",label:"OCR_ABAZA"},{type:"doc",id:"06-api-reference/variables/OCR_ADYGHE",label:"OCR_ADYGHE"},{type:"doc",id:"06-api-reference/variables/OCR_AFRIKAANS",label:"OCR_AFRIKAANS"},{type:"doc",id:"06-api-reference/variables/OCR_ALBANIAN",label:"OCR_ALBANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_AVAR",label:"OCR_AVAR"},{type:"doc",id:"06-api-reference/variables/OCR_AZERBAIJANI",label:"OCR_AZERBAIJANI"},{type:"doc",id:"06-api-reference/variables/OCR_BELARUSIAN",label:"OCR_BELARUSIAN"},{type:"doc",id:"06-api-reference/variables/OCR_BOSNIAN",label:"OCR_BOSNIAN"},{type:"doc",id:"06-api-reference/variables/OCR_BULGARIAN",label:"OCR_BULGARIAN"},{type:"doc",id:"06-api-reference/variables/OCR_CHECHEN",label:"OCR_CHECHEN"},{type:"doc",id:"06-api-reference/variables/OCR_CROATIAN",label:"OCR_CROATIAN"},{type:"doc",id:"06-api-reference/variables/OCR_CZECH",label:"OCR_CZECH"},{type:"doc",id:"06-api-reference/variables/OCR_DANISH",label:"OCR_DANISH"},{type:"doc",id:"06-api-reference/variables/OCR_DARGWA",label:"OCR_DARGWA"},{type:"doc",id:"06-api-reference/variables/OCR_DUTCH",label:"OCR_DUTCH"},{type:"doc",id:"06-api-reference/variables/OCR_ENGLISH",label:"OCR_ENGLISH"},{type:"doc",id:"06-api-reference/variables/OCR_ESTONIAN",label:"OCR_ESTONIAN"},{type:"doc",id:"06-api-reference/variables/OCR_FRENCH",label:"OCR_FRENCH"},{type:"doc",id:"06-api-reference/variables/OCR_GERMAN",label:"OCR_GERMAN"},{type:"doc",id:"06-api-reference/variables/OCR_HUNGARIAN",label:"OCR_HUNGARIAN"},{type:"doc",id:"06-api-reference/variables/OCR_ICELANDIC",label:"OCR_ICELANDIC"},{type:"doc",id:"06-api-reference/variables/OCR_INDONESIAN",label:"OCR_INDONESIAN"},{type:"doc",id:"06-api-reference/variables/OCR_INGUSH",label:"OCR_INGUSH"},{type:"doc",id:"06-api-reference/variables/OCR_IRISH",label:"OCR_IRISH"},{type:"doc",id:"06-api-reference/variables/OCR_ITALIAN",label:"OCR_ITALIAN"},{type:"doc",id:"06-api-reference/variables/OCR_JAPANESE",label:"OCR_JAPANESE"},{type:"doc",id:"06-api-reference/variables/OCR_KANNADA",label:"OCR_KANNADA"},{type:"doc",id:"06-api-reference/variables/OCR_KARBADIAN",label:"OCR_KARBADIAN"},{type:"doc",id:"06-api-reference/variables/OCR_KOREAN",label:"OCR_KOREAN"},{type:"doc",id:"06-api-reference/variables/OCR_KURDISH",label:"OCR_KURDISH"},{type:"doc",id:"06-api-reference/variables/OCR_LAK",label:"OCR_LAK"},{type:"doc",id:"06-api-reference/variables/OCR_LATIN",label:"OCR_LATIN"},{type:"doc",id:"06-api-reference/variables/OCR_LATVIAN",label:"OCR_LATVIAN"},{type:"doc",id:"06-api-reference/variables/OCR_LEZGHIAN",label:"OCR_LEZGHIAN"},{type:"doc",id:"06-api-reference/variables/OCR_LITHUANIAN",label:"OCR_LITHUANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_MALAY",label:"OCR_MALAY"},{type:"doc",id:"06-api-reference/variables/OCR_MALTESE",label:"OCR_MALTESE"},{type:"doc",id:"06-api-reference/variables/OCR_MAORI",label:"OCR_MAORI"},{type:"doc",id:"06-api-reference/variables/OCR_MONGOLIAN",label:"OCR_MONGOLIAN"},{type:"doc",id:"06-api-reference/variables/OCR_NORWEGIAN",label:"OCR_NORWEGIAN"},{type:"doc",id:"06-api-reference/variables/OCR_OCCITAN",label:"OCR_OCCITAN"},{type:"doc",id:"06-api-reference/variables/OCR_PALI",label:"OCR_PALI"},{type:"doc",id:"06-api-reference/variables/OCR_POLISH",label:"OCR_POLISH"},{type:"doc",id:"06-api-reference/variables/OCR_PORTUGUESE",label:"OCR_PORTUGUESE"},{type:"doc",id:"06-api-reference/variables/OCR_ROMANIAN",label:"OCR_ROMANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_RUSSIAN",label:"OCR_RUSSIAN"},{type:"doc",id:"06-api-reference/variables/OCR_SERBIAN_CYRILLIC",label:"OCR_SERBIAN_CYRILLIC"},{type:"doc",id:"06-api-reference/variables/OCR_SERBIAN_LATIN",label:"OCR_SERBIAN_LATIN"},{type:"doc",id:"06-api-reference/variables/OCR_SIMPLIFIED_CHINESE",label:"OCR_SIMPLIFIED_CHINESE"},{type:"doc",id:"06-api-reference/variables/OCR_SLOVAK",label:"OCR_SLOVAK"},{type:"doc",id:"06-api-reference/variables/OCR_SLOVENIAN",label:"OCR_SLOVENIAN"},{type:"doc",id:"06-api-reference/variables/OCR_SPANISH",label:"OCR_SPANISH"},{type:"doc",id:"06-api-reference/variables/OCR_SWAHILI",label:"OCR_SWAHILI"},{type:"doc",id:"06-api-reference/variables/OCR_SWEDISH",label:"OCR_SWEDISH"},{type:"doc",id:"06-api-reference/variables/OCR_TABASSARAN",label:"OCR_TABASSARAN"},{type:"doc",id:"06-api-reference/variables/OCR_TAGALOG",label:"OCR_TAGALOG"},{type:"doc",id:"06-api-reference/variables/OCR_TAJIK",label:"OCR_TAJIK"},{type:"doc",id:"06-api-reference/variables/OCR_TELUGU",label:"OCR_TELUGU"},{type:"doc",id:"06-api-reference/variables/OCR_TURKISH",label:"OCR_TURKISH"},{type:"doc",id:"06-api-reference/variables/OCR_UKRAINIAN",label:"OCR_UKRAINIAN"},{type:"doc",id:"06-api-reference/variables/OCR_UZBEK",label:"OCR_UZBEK"},{type:"doc",id:"06-api-reference/variables/OCR_VIETNAMESE",label:"OCR_VIETNAMESE"},{type:"doc",id:"06-api-reference/variables/OCR_WELSH",label:"OCR_WELSH"}]},{type:"category",label:"Other",items:[{type:"doc",id:"06-api-reference/enumerations/RnExecutorchErrorCode",label:"RnExecutorchErrorCode"},{type:"doc",id:"06-api-reference/classes/Logger",label:"Logger"},{type:"doc",id:"06-api-reference/classes/RnExecutorchError",label:"RnExecutorchError"},{type:"doc",id:"06-api-reference/variables/IMAGENET1K_MEAN",label:"IMAGENET1K_MEAN"},{type:"doc",id:"06-api-reference/variables/IMAGENET1K_STD",label:"IMAGENET1K_STD"}]},{type:"category",label:"TTS Supported Voices",items:[{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_HEART",label:"KOKORO_VOICE_AF_HEART"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_RIVER",label:"KOKORO_VOICE_AF_RIVER"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_SARAH",label:"KOKORO_VOICE_AF_SARAH"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_ADAM",label:"KOKORO_VOICE_AM_ADAM"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_MICHAEL",label:"KOKORO_VOICE_AM_MICHAEL"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_SANTA",label:"KOKORO_VOICE_AM_SANTA"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_BF_EMMA",label:"KOKORO_VOICE_BF_EMMA"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_BM_DANIEL",label:"KOKORO_VOICE_BM_DANIEL"}]},{type:"category",label:"Types",items:[{type:"doc",id:"06-api-reference/enumerations/CocoLabel",label:"CocoLabel"},{type:"doc",id:"06-api-reference/enumerations/DeeplabLabel",label:"DeeplabLabel"},{type:"doc",id:"06-api-reference/enumerations/DownloadStatus",label:"DownloadStatus"},{type:"doc",id:"06-api-reference/enumerations/HTTP_CODE",label:"HTTP_CODE"},{type:"doc",id:"06-api-reference/enumerations/ScalarType",label:"ScalarType"},{type:"doc",id:"06-api-reference/enumerations/SelfieSegmentationLabel",label:"SelfieSegmentationLabel"},{type:"doc",id:"06-api-reference/enumerations/SourceType",label:"SourceType"},{type:"doc",id:"06-api-reference/interfaces/Bbox",label:"Bbox"},{type:"doc",id:"06-api-reference/interfaces/ChatConfig",label:"ChatConfig"},{type:"doc",id:"06-api-reference/interfaces/ClassificationProps",label:"ClassificationProps"},{type:"doc",id:"06-api-reference/interfaces/ClassificationType",label:"ClassificationType"},{type:"doc",id:"06-api-reference/interfaces/ContextStrategy",label:"ContextStrategy"},{type:"doc",id:"06-api-reference/interfaces/DecodingOptions",label:"DecodingOptions"},{type:"doc",id:"06-api-reference/interfaces/Detection",label:"Detection"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchModuleProps",label:"ExecutorchModuleProps"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchModuleType",label:"ExecutorchModuleType"},{type:"doc",id:"06-api-reference/interfaces/GenerationConfig",label:"GenerationConfig"},{type:"doc",id:"06-api-reference/interfaces/ImageEmbeddingsProps",label:"ImageEmbeddingsProps"},{type:"doc",id:"06-api-reference/interfaces/ImageEmbeddingsType",label:"ImageEmbeddingsType"},{type:"doc",id:"06-api-reference/interfaces/ImageSegmentationProps",label:"ImageSegmentationProps"},{type:"doc",id:"06-api-reference/interfaces/ImageSegmentationType",label:"ImageSegmentationType"},{type:"doc",id:"06-api-reference/interfaces/KokoroConfig",label:"KokoroConfig"},{type:"doc",id:"06-api-reference/interfaces/KokoroVoiceExtras",label:"KokoroVoiceExtras"},{type:"doc",id:"06-api-reference/interfaces/LLMConfig",label:"LLMConfig"},{type:"doc",id:"06-api-reference/interfaces/LLMProps",label:"LLMProps"},{type:"doc",id:"06-api-reference/interfaces/LLMType",label:"LLMType"},{type:"doc",id:"06-api-reference/interfaces/Message",label:"Message"},{type:"doc",id:"06-api-reference/interfaces/ObjectDetectionProps",label:"ObjectDetectionProps"},{type:"doc",id:"06-api-reference/interfaces/ObjectDetectionType",label:"ObjectDetectionType"},{type:"doc",id:"06-api-reference/interfaces/OCRDetection",label:"OCRDetection"},{type:"doc",id:"06-api-reference/interfaces/OCRProps",label:"OCRProps"},{type:"doc",id:"06-api-reference/interfaces/OCRType",label:"OCRType"},{type:"doc",id:"06-api-reference/interfaces/Point",label:"Point"},{type:"doc",id:"06-api-reference/interfaces/Segment",label:"Segment"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextModelConfig",label:"SpeechToTextModelConfig"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextProps",label:"SpeechToTextProps"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextType",label:"SpeechToTextType"},{type:"doc",id:"06-api-reference/interfaces/StyleTransferProps",label:"StyleTransferProps"},{type:"doc",id:"06-api-reference/interfaces/StyleTransferType",label:"StyleTransferType"},{type:"doc",id:"06-api-reference/interfaces/TensorPtr",label:"TensorPtr"},{type:"doc",id:"06-api-reference/interfaces/TextEmbeddingsProps",label:"TextEmbeddingsProps"},{type:"doc",id:"06-api-reference/interfaces/TextEmbeddingsType",label:"TextEmbeddingsType"},{type:"doc",id:"06-api-reference/interfaces/TextToImageProps",label:"TextToImageProps"},{type:"doc",id:"06-api-reference/interfaces/TextToImageType",label:"TextToImageType"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechConfig",label:"TextToSpeechConfig"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechInput",label:"TextToSpeechInput"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechProps",label:"TextToSpeechProps"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechStreamingInput",label:"TextToSpeechStreamingInput"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechType",label:"TextToSpeechType"},{type:"doc",id:"06-api-reference/interfaces/TokenizerProps",label:"TokenizerProps"},{type:"doc",id:"06-api-reference/interfaces/TokenizerType",label:"TokenizerType"},{type:"doc",id:"06-api-reference/interfaces/ToolCall",label:"ToolCall"},{type:"doc",id:"06-api-reference/interfaces/ToolsConfig",label:"ToolsConfig"},{type:"doc",id:"06-api-reference/interfaces/TranscriptionResult",label:"TranscriptionResult"},{type:"doc",id:"06-api-reference/interfaces/TranscriptionSegment",label:"TranscriptionSegment"},{type:"doc",id:"06-api-reference/interfaces/VADProps",label:"VADProps"},{type:"doc",id:"06-api-reference/interfaces/VADType",label:"VADType"},{type:"doc",id:"06-api-reference/interfaces/VerticalOCRProps",label:"VerticalOCRProps"},{type:"doc",id:"06-api-reference/interfaces/VoiceConfig",label:"VoiceConfig"},{type:"doc",id:"06-api-reference/interfaces/Word",label:"Word"},{type:"doc",id:"06-api-reference/type-aliases/LabelEnum",label:"LabelEnum"},{type:"doc",id:"06-api-reference/type-aliases/LLMTool",label:"LLMTool"},{type:"doc",id:"06-api-reference/type-aliases/MessageRole",label:"MessageRole"},{type:"doc",id:"06-api-reference/type-aliases/ModelNameOf",label:"ModelNameOf"},{type:"doc",id:"06-api-reference/type-aliases/ModelSources",label:"ModelSources"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionConfig",label:"ObjectDetectionConfig"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionLabels",label:"ObjectDetectionLabels"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionModelName",label:"ObjectDetectionModelName"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionModelSources",label:"ObjectDetectionModelSources"},{type:"doc",id:"06-api-reference/type-aliases/OCRLanguage",label:"OCRLanguage"},{type:"doc",id:"06-api-reference/type-aliases/ResourceSource",label:"ResourceSource"},{type:"doc",id:"06-api-reference/type-aliases/SegmentationConfig",label:"SegmentationConfig"},{type:"doc",id:"06-api-reference/type-aliases/SegmentationLabels",label:"SegmentationLabels"},{type:"doc",id:"06-api-reference/type-aliases/SegmentationModelName",label:"SegmentationModelName"},{type:"doc",id:"06-api-reference/type-aliases/SpeechToTextLanguage",label:"SpeechToTextLanguage"},{type:"doc",id:"06-api-reference/type-aliases/TensorBuffer",label:"TensorBuffer"},{type:"doc",id:"06-api-reference/type-aliases/TextToSpeechLanguage",label:"TextToSpeechLanguage"},{type:"doc",id:"06-api-reference/type-aliases/Triple",label:"Triple"},{type:"doc",id:"06-api-reference/variables/SPECIAL_TOKENS",label:"SPECIAL_TOKENS"}]},{type:"category",label:"Typescript API",items:[{type:"doc",id:"06-api-reference/classes/ClassificationModule",label:"ClassificationModule"},{type:"doc",id:"06-api-reference/classes/ExecutorchModule",label:"ExecutorchModule"},{type:"doc",id:"06-api-reference/classes/ImageEmbeddingsModule",label:"ImageEmbeddingsModule"},{type:"doc",id:"06-api-reference/classes/ImageSegmentationModule",label:"ImageSegmentationModule"},{type:"doc",id:"06-api-reference/classes/LLMModule",label:"LLMModule"},{type:"doc",id:"06-api-reference/classes/ObjectDetectionModule",label:"ObjectDetectionModule"},{type:"doc",id:"06-api-reference/classes/OCRModule",label:"OCRModule"},{type:"doc",id:"06-api-reference/classes/SpeechToTextModule",label:"SpeechToTextModule"},{type:"doc",id:"06-api-reference/classes/StyleTransferModule",label:"StyleTransferModule"},{type:"doc",id:"06-api-reference/classes/TextEmbeddingsModule",label:"TextEmbeddingsModule"},{type:"doc",id:"06-api-reference/classes/TextToImageModule",label:"TextToImageModule"},{type:"doc",id:"06-api-reference/classes/TextToSpeechModule",label:"TextToSpeechModule"},{type:"doc",id:"06-api-reference/classes/TokenizerModule",label:"TokenizerModule"},{type:"doc",id:"06-api-reference/classes/VADModule",label:"VADModule"},{type:"doc",id:"06-api-reference/classes/VerticalOCRModule",label:"VerticalOCRModule"}]},{type:"category",label:"Utilities - General",items:[{type:"category",label:"ResourceFetcherUtils",items:[{type:"category",label:"Functions",items:[{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/calculateDownloadProgress",label:"calculateDownloadProgress"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/getFilenameFromUri",label:"getFilenameFromUri"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/hashObject",label:"hashObject"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/removeFilePrefix",label:"removeFilePrefix"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/triggerHuggingFaceDownloadCounter",label:"triggerHuggingFaceDownloadCounter"}]}],link:{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/index"}},{type:"doc",id:"06-api-reference/classes/ResourceFetcher",label:"ResourceFetcher"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchConfig",label:"ExecutorchConfig"},{type:"doc",id:"06-api-reference/interfaces/ResourceFetcherAdapter",label:"ResourceFetcherAdapter"},{type:"doc",id:"06-api-reference/functions/cleanupExecutorch",label:"cleanupExecutorch"},{type:"doc",id:"06-api-reference/functions/initExecutorch",label:"initExecutorch"}]},{type:"category",label:"Utilities - LLM",items:[{type:"doc",id:"06-api-reference/variables/DEFAULT_CHAT_CONFIG",label:"DEFAULT_CHAT_CONFIG"},{type:"doc",id:"06-api-reference/variables/DEFAULT_CONTEXT_BUFFER_TOKENS",label:"DEFAULT_CONTEXT_BUFFER_TOKENS"},{type:"doc",id:"06-api-reference/variables/DEFAULT_MESSAGE_HISTORY",label:"DEFAULT_MESSAGE_HISTORY"},{type:"doc",id:"06-api-reference/variables/DEFAULT_SYSTEM_PROMPT",label:"DEFAULT_SYSTEM_PROMPT"},{type:"doc",id:"06-api-reference/variables/parseToolCall",label:"parseToolCall"},{type:"doc",id:"06-api-reference/functions/DEFAULT_STRUCTURED_OUTPUT_PROMPT",label:"DEFAULT_STRUCTURED_OUTPUT_PROMPT"},{type:"doc",id:"06-api-reference/functions/fixAndValidateStructuredOutput",label:"fixAndValidateStructuredOutput"},{type:"doc",id:"06-api-reference/functions/getStructuredOutputPrompt",label:"getStructuredOutputPrompt"}]},{type:"category",label:"Utils",items:[{type:"doc",id:"06-api-reference/classes/MessageCountContextStrategy",label:"MessageCountContextStrategy"},{type:"doc",id:"06-api-reference/classes/NoopContextStrategy",label:"NoopContextStrategy"},{type:"doc",id:"06-api-reference/classes/SlidingWindowContextStrategy",label:"SlidingWindowContextStrategy"}]}]}; +const typedocSidebar = {items:[{type:"category",label:"Hooks",items:[{type:"doc",id:"06-api-reference/functions/useClassification",label:"useClassification"},{type:"doc",id:"06-api-reference/functions/useExecutorchModule",label:"useExecutorchModule"},{type:"doc",id:"06-api-reference/functions/useImageEmbeddings",label:"useImageEmbeddings"},{type:"doc",id:"06-api-reference/functions/useLLM",label:"useLLM"},{type:"doc",id:"06-api-reference/functions/useObjectDetection",label:"useObjectDetection"},{type:"doc",id:"06-api-reference/functions/useOCR",label:"useOCR"},{type:"doc",id:"06-api-reference/functions/useSemanticSegmentation",label:"useSemanticSegmentation"},{type:"doc",id:"06-api-reference/functions/useSpeechToText",label:"useSpeechToText"},{type:"doc",id:"06-api-reference/functions/useStyleTransfer",label:"useStyleTransfer"},{type:"doc",id:"06-api-reference/functions/useTextEmbeddings",label:"useTextEmbeddings"},{type:"doc",id:"06-api-reference/functions/useTextToImage",label:"useTextToImage"},{type:"doc",id:"06-api-reference/functions/useTextToSpeech",label:"useTextToSpeech"},{type:"doc",id:"06-api-reference/functions/useTokenizer",label:"useTokenizer"},{type:"doc",id:"06-api-reference/functions/useVAD",label:"useVAD"},{type:"doc",id:"06-api-reference/functions/useVerticalOCR",label:"useVerticalOCR"}]},{type:"category",label:"Interfaces",items:[{type:"doc",id:"06-api-reference/interfaces/ResourceSourceExtended",label:"ResourceSourceExtended"}]},{type:"category",label:"Models - Classification",items:[{type:"doc",id:"06-api-reference/variables/EFFICIENTNET_V2_S",label:"EFFICIENTNET_V2_S"}]},{type:"category",label:"Models - Image Embeddings",items:[{type:"doc",id:"06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE",label:"CLIP_VIT_BASE_PATCH32_IMAGE"}]},{type:"category",label:"Models - Image Generation",items:[{type:"doc",id:"06-api-reference/variables/BK_SDM_TINY_VPRED_256",label:"BK_SDM_TINY_VPRED_256"},{type:"doc",id:"06-api-reference/variables/BK_SDM_TINY_VPRED_512",label:"BK_SDM_TINY_VPRED_512"}]},{type:"category",label:"Models - LMM",items:[{type:"doc",id:"06-api-reference/variables/HAMMER2_1_0_5B",label:"HAMMER2_1_0_5B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_0_5B_QUANTIZED",label:"HAMMER2_1_0_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_1_5B",label:"HAMMER2_1_1_5B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_1_5B_QUANTIZED",label:"HAMMER2_1_1_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_3B",label:"HAMMER2_1_3B"},{type:"doc",id:"06-api-reference/variables/HAMMER2_1_3B_QUANTIZED",label:"HAMMER2_1_3B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LFM2_5_1_2B_INSTRUCT",label:"LFM2_5_1_2B_INSTRUCT"},{type:"doc",id:"06-api-reference/variables/LFM2_5_1_2B_INSTRUCT_QUANTIZED",label:"LFM2_5_1_2B_INSTRUCT_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B",label:"LLAMA3_2_1B"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B_QLORA",label:"LLAMA3_2_1B_QLORA"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_1B_SPINQUANT",label:"LLAMA3_2_1B_SPINQUANT"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B",label:"LLAMA3_2_3B"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B_QLORA",label:"LLAMA3_2_3B_QLORA"},{type:"doc",id:"06-api-reference/variables/LLAMA3_2_3B_SPINQUANT",label:"LLAMA3_2_3B_SPINQUANT"},{type:"doc",id:"06-api-reference/variables/PHI_4_MINI_4B",label:"PHI_4_MINI_4B"},{type:"doc",id:"06-api-reference/variables/PHI_4_MINI_4B_QUANTIZED",label:"PHI_4_MINI_4B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_0_5B",label:"QWEN2_5_0_5B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_0_5B_QUANTIZED",label:"QWEN2_5_0_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_1_5B",label:"QWEN2_5_1_5B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_1_5B_QUANTIZED",label:"QWEN2_5_1_5B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_3B",label:"QWEN2_5_3B"},{type:"doc",id:"06-api-reference/variables/QWEN2_5_3B_QUANTIZED",label:"QWEN2_5_3B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_0_6B",label:"QWEN3_0_6B"},{type:"doc",id:"06-api-reference/variables/QWEN3_0_6B_QUANTIZED",label:"QWEN3_0_6B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_1_7B",label:"QWEN3_1_7B"},{type:"doc",id:"06-api-reference/variables/QWEN3_1_7B_QUANTIZED",label:"QWEN3_1_7B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/QWEN3_4B",label:"QWEN3_4B"},{type:"doc",id:"06-api-reference/variables/QWEN3_4B_QUANTIZED",label:"QWEN3_4B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_1_7B",label:"SMOLLM2_1_1_7B"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_1_7B_QUANTIZED",label:"SMOLLM2_1_1_7B_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_135M",label:"SMOLLM2_1_135M"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_135M_QUANTIZED",label:"SMOLLM2_1_135M_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_360M",label:"SMOLLM2_1_360M"},{type:"doc",id:"06-api-reference/variables/SMOLLM2_1_360M_QUANTIZED",label:"SMOLLM2_1_360M_QUANTIZED"}]},{type:"category",label:"Models - Object Detection",items:[{type:"doc",id:"06-api-reference/variables/RF_DETR_NANO",label:"RF_DETR_NANO"},{type:"doc",id:"06-api-reference/variables/SSDLITE_320_MOBILENET_V3_LARGE",label:"SSDLITE_320_MOBILENET_V3_LARGE"}]},{type:"category",label:"Models - Semantic Segmentation",items:[{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE",label:"DEEPLAB_V3_MOBILENET_V3_LARGE"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED",label:"DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET101",label:"DEEPLAB_V3_RESNET101"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET101_QUANTIZED",label:"DEEPLAB_V3_RESNET101_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET50",label:"DEEPLAB_V3_RESNET50"},{type:"doc",id:"06-api-reference/variables/DEEPLAB_V3_RESNET50_QUANTIZED",label:"DEEPLAB_V3_RESNET50_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/FCN_RESNET101",label:"FCN_RESNET101"},{type:"doc",id:"06-api-reference/variables/FCN_RESNET101_QUANTIZED",label:"FCN_RESNET101_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/FCN_RESNET50",label:"FCN_RESNET50"},{type:"doc",id:"06-api-reference/variables/FCN_RESNET50_QUANTIZED",label:"FCN_RESNET50_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE",label:"LRASPP_MOBILENET_V3_LARGE"},{type:"doc",id:"06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE_QUANTIZED",label:"LRASPP_MOBILENET_V3_LARGE_QUANTIZED"},{type:"doc",id:"06-api-reference/variables/SELFIE_SEGMENTATION",label:"SELFIE_SEGMENTATION"}]},{type:"category",label:"Models - Speech To Text",items:[{type:"doc",id:"06-api-reference/variables/WHISPER_BASE",label:"WHISPER_BASE"},{type:"doc",id:"06-api-reference/variables/WHISPER_BASE_EN",label:"WHISPER_BASE_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_SMALL",label:"WHISPER_SMALL"},{type:"doc",id:"06-api-reference/variables/WHISPER_SMALL_EN",label:"WHISPER_SMALL_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY",label:"WHISPER_TINY"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY_EN",label:"WHISPER_TINY_EN"},{type:"doc",id:"06-api-reference/variables/WHISPER_TINY_EN_QUANTIZED",label:"WHISPER_TINY_EN_QUANTIZED"}]},{type:"category",label:"Models - Style Transfer",items:[{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_CANDY",label:"STYLE_TRANSFER_CANDY"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_MOSAIC",label:"STYLE_TRANSFER_MOSAIC"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_RAIN_PRINCESS",label:"STYLE_TRANSFER_RAIN_PRINCESS"},{type:"doc",id:"06-api-reference/variables/STYLE_TRANSFER_UDNIE",label:"STYLE_TRANSFER_UDNIE"}]},{type:"category",label:"Models - Text Embeddings",items:[{type:"doc",id:"06-api-reference/variables/ALL_MINILM_L6_V2",label:"ALL_MINILM_L6_V2"},{type:"doc",id:"06-api-reference/variables/ALL_MPNET_BASE_V2",label:"ALL_MPNET_BASE_V2"},{type:"doc",id:"06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT",label:"CLIP_VIT_BASE_PATCH32_TEXT"},{type:"doc",id:"06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1",label:"MULTI_QA_MINILM_L6_COS_V1"},{type:"doc",id:"06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1",label:"MULTI_QA_MPNET_BASE_DOT_V1"}]},{type:"category",label:"Models - Text to Speech",items:[{type:"doc",id:"06-api-reference/variables/KOKORO_MEDIUM",label:"KOKORO_MEDIUM"},{type:"doc",id:"06-api-reference/variables/KOKORO_SMALL",label:"KOKORO_SMALL"}]},{type:"category",label:"Models - Voice Activity Detection",items:[{type:"doc",id:"06-api-reference/variables/FSMN_VAD",label:"FSMN_VAD"}]},{type:"category",label:"OCR Supported Alphabets",items:[{type:"doc",id:"06-api-reference/variables/OCR_ABAZA",label:"OCR_ABAZA"},{type:"doc",id:"06-api-reference/variables/OCR_ADYGHE",label:"OCR_ADYGHE"},{type:"doc",id:"06-api-reference/variables/OCR_AFRIKAANS",label:"OCR_AFRIKAANS"},{type:"doc",id:"06-api-reference/variables/OCR_ALBANIAN",label:"OCR_ALBANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_AVAR",label:"OCR_AVAR"},{type:"doc",id:"06-api-reference/variables/OCR_AZERBAIJANI",label:"OCR_AZERBAIJANI"},{type:"doc",id:"06-api-reference/variables/OCR_BELARUSIAN",label:"OCR_BELARUSIAN"},{type:"doc",id:"06-api-reference/variables/OCR_BOSNIAN",label:"OCR_BOSNIAN"},{type:"doc",id:"06-api-reference/variables/OCR_BULGARIAN",label:"OCR_BULGARIAN"},{type:"doc",id:"06-api-reference/variables/OCR_CHECHEN",label:"OCR_CHECHEN"},{type:"doc",id:"06-api-reference/variables/OCR_CROATIAN",label:"OCR_CROATIAN"},{type:"doc",id:"06-api-reference/variables/OCR_CZECH",label:"OCR_CZECH"},{type:"doc",id:"06-api-reference/variables/OCR_DANISH",label:"OCR_DANISH"},{type:"doc",id:"06-api-reference/variables/OCR_DARGWA",label:"OCR_DARGWA"},{type:"doc",id:"06-api-reference/variables/OCR_DUTCH",label:"OCR_DUTCH"},{type:"doc",id:"06-api-reference/variables/OCR_ENGLISH",label:"OCR_ENGLISH"},{type:"doc",id:"06-api-reference/variables/OCR_ESTONIAN",label:"OCR_ESTONIAN"},{type:"doc",id:"06-api-reference/variables/OCR_FRENCH",label:"OCR_FRENCH"},{type:"doc",id:"06-api-reference/variables/OCR_GERMAN",label:"OCR_GERMAN"},{type:"doc",id:"06-api-reference/variables/OCR_HUNGARIAN",label:"OCR_HUNGARIAN"},{type:"doc",id:"06-api-reference/variables/OCR_ICELANDIC",label:"OCR_ICELANDIC"},{type:"doc",id:"06-api-reference/variables/OCR_INDONESIAN",label:"OCR_INDONESIAN"},{type:"doc",id:"06-api-reference/variables/OCR_INGUSH",label:"OCR_INGUSH"},{type:"doc",id:"06-api-reference/variables/OCR_IRISH",label:"OCR_IRISH"},{type:"doc",id:"06-api-reference/variables/OCR_ITALIAN",label:"OCR_ITALIAN"},{type:"doc",id:"06-api-reference/variables/OCR_JAPANESE",label:"OCR_JAPANESE"},{type:"doc",id:"06-api-reference/variables/OCR_KANNADA",label:"OCR_KANNADA"},{type:"doc",id:"06-api-reference/variables/OCR_KARBADIAN",label:"OCR_KARBADIAN"},{type:"doc",id:"06-api-reference/variables/OCR_KOREAN",label:"OCR_KOREAN"},{type:"doc",id:"06-api-reference/variables/OCR_KURDISH",label:"OCR_KURDISH"},{type:"doc",id:"06-api-reference/variables/OCR_LAK",label:"OCR_LAK"},{type:"doc",id:"06-api-reference/variables/OCR_LATIN",label:"OCR_LATIN"},{type:"doc",id:"06-api-reference/variables/OCR_LATVIAN",label:"OCR_LATVIAN"},{type:"doc",id:"06-api-reference/variables/OCR_LEZGHIAN",label:"OCR_LEZGHIAN"},{type:"doc",id:"06-api-reference/variables/OCR_LITHUANIAN",label:"OCR_LITHUANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_MALAY",label:"OCR_MALAY"},{type:"doc",id:"06-api-reference/variables/OCR_MALTESE",label:"OCR_MALTESE"},{type:"doc",id:"06-api-reference/variables/OCR_MAORI",label:"OCR_MAORI"},{type:"doc",id:"06-api-reference/variables/OCR_MONGOLIAN",label:"OCR_MONGOLIAN"},{type:"doc",id:"06-api-reference/variables/OCR_NORWEGIAN",label:"OCR_NORWEGIAN"},{type:"doc",id:"06-api-reference/variables/OCR_OCCITAN",label:"OCR_OCCITAN"},{type:"doc",id:"06-api-reference/variables/OCR_PALI",label:"OCR_PALI"},{type:"doc",id:"06-api-reference/variables/OCR_POLISH",label:"OCR_POLISH"},{type:"doc",id:"06-api-reference/variables/OCR_PORTUGUESE",label:"OCR_PORTUGUESE"},{type:"doc",id:"06-api-reference/variables/OCR_ROMANIAN",label:"OCR_ROMANIAN"},{type:"doc",id:"06-api-reference/variables/OCR_RUSSIAN",label:"OCR_RUSSIAN"},{type:"doc",id:"06-api-reference/variables/OCR_SERBIAN_CYRILLIC",label:"OCR_SERBIAN_CYRILLIC"},{type:"doc",id:"06-api-reference/variables/OCR_SERBIAN_LATIN",label:"OCR_SERBIAN_LATIN"},{type:"doc",id:"06-api-reference/variables/OCR_SIMPLIFIED_CHINESE",label:"OCR_SIMPLIFIED_CHINESE"},{type:"doc",id:"06-api-reference/variables/OCR_SLOVAK",label:"OCR_SLOVAK"},{type:"doc",id:"06-api-reference/variables/OCR_SLOVENIAN",label:"OCR_SLOVENIAN"},{type:"doc",id:"06-api-reference/variables/OCR_SPANISH",label:"OCR_SPANISH"},{type:"doc",id:"06-api-reference/variables/OCR_SWAHILI",label:"OCR_SWAHILI"},{type:"doc",id:"06-api-reference/variables/OCR_SWEDISH",label:"OCR_SWEDISH"},{type:"doc",id:"06-api-reference/variables/OCR_TABASSARAN",label:"OCR_TABASSARAN"},{type:"doc",id:"06-api-reference/variables/OCR_TAGALOG",label:"OCR_TAGALOG"},{type:"doc",id:"06-api-reference/variables/OCR_TAJIK",label:"OCR_TAJIK"},{type:"doc",id:"06-api-reference/variables/OCR_TELUGU",label:"OCR_TELUGU"},{type:"doc",id:"06-api-reference/variables/OCR_TURKISH",label:"OCR_TURKISH"},{type:"doc",id:"06-api-reference/variables/OCR_UKRAINIAN",label:"OCR_UKRAINIAN"},{type:"doc",id:"06-api-reference/variables/OCR_UZBEK",label:"OCR_UZBEK"},{type:"doc",id:"06-api-reference/variables/OCR_VIETNAMESE",label:"OCR_VIETNAMESE"},{type:"doc",id:"06-api-reference/variables/OCR_WELSH",label:"OCR_WELSH"}]},{type:"category",label:"Other",items:[{type:"doc",id:"06-api-reference/enumerations/RnExecutorchErrorCode",label:"RnExecutorchErrorCode"},{type:"doc",id:"06-api-reference/classes/Logger",label:"Logger"},{type:"doc",id:"06-api-reference/classes/RnExecutorchError",label:"RnExecutorchError"},{type:"doc",id:"06-api-reference/interfaces/Frame",label:"Frame"},{type:"doc",id:"06-api-reference/variables/IMAGENET1K_MEAN",label:"IMAGENET1K_MEAN"},{type:"doc",id:"06-api-reference/variables/IMAGENET1K_STD",label:"IMAGENET1K_STD"}]},{type:"category",label:"TTS Supported Voices",items:[{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_HEART",label:"KOKORO_VOICE_AF_HEART"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_RIVER",label:"KOKORO_VOICE_AF_RIVER"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AF_SARAH",label:"KOKORO_VOICE_AF_SARAH"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_ADAM",label:"KOKORO_VOICE_AM_ADAM"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_MICHAEL",label:"KOKORO_VOICE_AM_MICHAEL"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_AM_SANTA",label:"KOKORO_VOICE_AM_SANTA"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_BF_EMMA",label:"KOKORO_VOICE_BF_EMMA"},{type:"doc",id:"06-api-reference/variables/KOKORO_VOICE_BM_DANIEL",label:"KOKORO_VOICE_BM_DANIEL"}]},{type:"category",label:"Types",items:[{type:"doc",id:"06-api-reference/enumerations/CocoLabel",label:"CocoLabel"},{type:"doc",id:"06-api-reference/enumerations/DeeplabLabel",label:"DeeplabLabel"},{type:"doc",id:"06-api-reference/enumerations/DownloadStatus",label:"DownloadStatus"},{type:"doc",id:"06-api-reference/enumerations/HTTP_CODE",label:"HTTP_CODE"},{type:"doc",id:"06-api-reference/enumerations/ScalarType",label:"ScalarType"},{type:"doc",id:"06-api-reference/enumerations/SelfieSegmentationLabel",label:"SelfieSegmentationLabel"},{type:"doc",id:"06-api-reference/enumerations/SourceType",label:"SourceType"},{type:"doc",id:"06-api-reference/interfaces/Bbox",label:"Bbox"},{type:"doc",id:"06-api-reference/interfaces/ChatConfig",label:"ChatConfig"},{type:"doc",id:"06-api-reference/interfaces/ClassificationProps",label:"ClassificationProps"},{type:"doc",id:"06-api-reference/interfaces/ClassificationType",label:"ClassificationType"},{type:"doc",id:"06-api-reference/interfaces/ContextStrategy",label:"ContextStrategy"},{type:"doc",id:"06-api-reference/interfaces/DecodingOptions",label:"DecodingOptions"},{type:"doc",id:"06-api-reference/interfaces/Detection",label:"Detection"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchModuleProps",label:"ExecutorchModuleProps"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchModuleType",label:"ExecutorchModuleType"},{type:"doc",id:"06-api-reference/interfaces/GenerationConfig",label:"GenerationConfig"},{type:"doc",id:"06-api-reference/interfaces/ImageEmbeddingsProps",label:"ImageEmbeddingsProps"},{type:"doc",id:"06-api-reference/interfaces/ImageEmbeddingsType",label:"ImageEmbeddingsType"},{type:"doc",id:"06-api-reference/interfaces/KokoroConfig",label:"KokoroConfig"},{type:"doc",id:"06-api-reference/interfaces/KokoroVoiceExtras",label:"KokoroVoiceExtras"},{type:"doc",id:"06-api-reference/interfaces/LLMConfig",label:"LLMConfig"},{type:"doc",id:"06-api-reference/interfaces/LLMProps",label:"LLMProps"},{type:"doc",id:"06-api-reference/interfaces/LLMType",label:"LLMType"},{type:"doc",id:"06-api-reference/interfaces/Message",label:"Message"},{type:"doc",id:"06-api-reference/interfaces/ObjectDetectionProps",label:"ObjectDetectionProps"},{type:"doc",id:"06-api-reference/interfaces/ObjectDetectionType",label:"ObjectDetectionType"},{type:"doc",id:"06-api-reference/interfaces/OCRDetection",label:"OCRDetection"},{type:"doc",id:"06-api-reference/interfaces/OCRProps",label:"OCRProps"},{type:"doc",id:"06-api-reference/interfaces/OCRType",label:"OCRType"},{type:"doc",id:"06-api-reference/interfaces/PixelData",label:"PixelData"},{type:"doc",id:"06-api-reference/interfaces/Point",label:"Point"},{type:"doc",id:"06-api-reference/interfaces/Segment",label:"Segment"},{type:"doc",id:"06-api-reference/interfaces/SemanticSegmentationProps",label:"SemanticSegmentationProps"},{type:"doc",id:"06-api-reference/interfaces/SemanticSegmentationType",label:"SemanticSegmentationType"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextModelConfig",label:"SpeechToTextModelConfig"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextProps",label:"SpeechToTextProps"},{type:"doc",id:"06-api-reference/interfaces/SpeechToTextType",label:"SpeechToTextType"},{type:"doc",id:"06-api-reference/interfaces/StyleTransferProps",label:"StyleTransferProps"},{type:"doc",id:"06-api-reference/interfaces/StyleTransferType",label:"StyleTransferType"},{type:"doc",id:"06-api-reference/interfaces/TensorPtr",label:"TensorPtr"},{type:"doc",id:"06-api-reference/interfaces/TextEmbeddingsProps",label:"TextEmbeddingsProps"},{type:"doc",id:"06-api-reference/interfaces/TextEmbeddingsType",label:"TextEmbeddingsType"},{type:"doc",id:"06-api-reference/interfaces/TextToImageProps",label:"TextToImageProps"},{type:"doc",id:"06-api-reference/interfaces/TextToImageType",label:"TextToImageType"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechConfig",label:"TextToSpeechConfig"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechInput",label:"TextToSpeechInput"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechProps",label:"TextToSpeechProps"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechStreamingInput",label:"TextToSpeechStreamingInput"},{type:"doc",id:"06-api-reference/interfaces/TextToSpeechType",label:"TextToSpeechType"},{type:"doc",id:"06-api-reference/interfaces/TokenizerProps",label:"TokenizerProps"},{type:"doc",id:"06-api-reference/interfaces/TokenizerType",label:"TokenizerType"},{type:"doc",id:"06-api-reference/interfaces/ToolCall",label:"ToolCall"},{type:"doc",id:"06-api-reference/interfaces/ToolsConfig",label:"ToolsConfig"},{type:"doc",id:"06-api-reference/interfaces/TranscriptionResult",label:"TranscriptionResult"},{type:"doc",id:"06-api-reference/interfaces/TranscriptionSegment",label:"TranscriptionSegment"},{type:"doc",id:"06-api-reference/interfaces/VADProps",label:"VADProps"},{type:"doc",id:"06-api-reference/interfaces/VADType",label:"VADType"},{type:"doc",id:"06-api-reference/interfaces/VerticalOCRProps",label:"VerticalOCRProps"},{type:"doc",id:"06-api-reference/interfaces/VoiceConfig",label:"VoiceConfig"},{type:"doc",id:"06-api-reference/interfaces/Word",label:"Word"},{type:"doc",id:"06-api-reference/type-aliases/LabelEnum",label:"LabelEnum"},{type:"doc",id:"06-api-reference/type-aliases/LLMTool",label:"LLMTool"},{type:"doc",id:"06-api-reference/type-aliases/MessageRole",label:"MessageRole"},{type:"doc",id:"06-api-reference/type-aliases/ModelNameOf",label:"ModelNameOf"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionConfig",label:"ObjectDetectionConfig"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionLabels",label:"ObjectDetectionLabels"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionModelName",label:"ObjectDetectionModelName"},{type:"doc",id:"06-api-reference/type-aliases/ObjectDetectionModelSources",label:"ObjectDetectionModelSources"},{type:"doc",id:"06-api-reference/type-aliases/OCRLanguage",label:"OCRLanguage"},{type:"doc",id:"06-api-reference/type-aliases/ResourceSource",label:"ResourceSource"},{type:"doc",id:"06-api-reference/type-aliases/SegmentationLabels",label:"SegmentationLabels"},{type:"doc",id:"06-api-reference/type-aliases/SemanticSegmentationConfig",label:"SemanticSegmentationConfig"},{type:"doc",id:"06-api-reference/type-aliases/SemanticSegmentationModelName",label:"SemanticSegmentationModelName"},{type:"doc",id:"06-api-reference/type-aliases/SemanticSegmentationModelSources",label:"SemanticSegmentationModelSources"},{type:"doc",id:"06-api-reference/type-aliases/SpeechToTextLanguage",label:"SpeechToTextLanguage"},{type:"doc",id:"06-api-reference/type-aliases/TensorBuffer",label:"TensorBuffer"},{type:"doc",id:"06-api-reference/type-aliases/TextToSpeechLanguage",label:"TextToSpeechLanguage"},{type:"doc",id:"06-api-reference/type-aliases/Triple",label:"Triple"},{type:"doc",id:"06-api-reference/variables/SPECIAL_TOKENS",label:"SPECIAL_TOKENS"}]},{type:"category",label:"Typescript API",items:[{type:"doc",id:"06-api-reference/classes/ClassificationModule",label:"ClassificationModule"},{type:"doc",id:"06-api-reference/classes/ExecutorchModule",label:"ExecutorchModule"},{type:"doc",id:"06-api-reference/classes/ImageEmbeddingsModule",label:"ImageEmbeddingsModule"},{type:"doc",id:"06-api-reference/classes/LLMModule",label:"LLMModule"},{type:"doc",id:"06-api-reference/classes/ObjectDetectionModule",label:"ObjectDetectionModule"},{type:"doc",id:"06-api-reference/classes/OCRModule",label:"OCRModule"},{type:"doc",id:"06-api-reference/classes/SemanticSegmentationModule",label:"SemanticSegmentationModule"},{type:"doc",id:"06-api-reference/classes/SpeechToTextModule",label:"SpeechToTextModule"},{type:"doc",id:"06-api-reference/classes/StyleTransferModule",label:"StyleTransferModule"},{type:"doc",id:"06-api-reference/classes/TextEmbeddingsModule",label:"TextEmbeddingsModule"},{type:"doc",id:"06-api-reference/classes/TextToImageModule",label:"TextToImageModule"},{type:"doc",id:"06-api-reference/classes/TextToSpeechModule",label:"TextToSpeechModule"},{type:"doc",id:"06-api-reference/classes/TokenizerModule",label:"TokenizerModule"},{type:"doc",id:"06-api-reference/classes/VADModule",label:"VADModule"},{type:"doc",id:"06-api-reference/classes/VerticalOCRModule",label:"VerticalOCRModule"}]},{type:"category",label:"Utilities - General",items:[{type:"category",label:"ResourceFetcherUtils",items:[{type:"category",label:"Functions",items:[{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/calculateDownloadProgress",label:"calculateDownloadProgress"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/getFilenameFromUri",label:"getFilenameFromUri"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/hashObject",label:"hashObject"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/removeFilePrefix",label:"removeFilePrefix"},{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/functions/triggerHuggingFaceDownloadCounter",label:"triggerHuggingFaceDownloadCounter"}]}],link:{type:"doc",id:"06-api-reference/react-native-executorch/namespaces/ResourceFetcherUtils/index"}},{type:"doc",id:"06-api-reference/classes/ResourceFetcher",label:"ResourceFetcher"},{type:"doc",id:"06-api-reference/interfaces/ExecutorchConfig",label:"ExecutorchConfig"},{type:"doc",id:"06-api-reference/interfaces/ResourceFetcherAdapter",label:"ResourceFetcherAdapter"},{type:"doc",id:"06-api-reference/functions/cleanupExecutorch",label:"cleanupExecutorch"},{type:"doc",id:"06-api-reference/functions/initExecutorch",label:"initExecutorch"}]},{type:"category",label:"Utilities - LLM",items:[{type:"doc",id:"06-api-reference/variables/DEFAULT_CHAT_CONFIG",label:"DEFAULT_CHAT_CONFIG"},{type:"doc",id:"06-api-reference/variables/DEFAULT_CONTEXT_BUFFER_TOKENS",label:"DEFAULT_CONTEXT_BUFFER_TOKENS"},{type:"doc",id:"06-api-reference/variables/DEFAULT_MESSAGE_HISTORY",label:"DEFAULT_MESSAGE_HISTORY"},{type:"doc",id:"06-api-reference/variables/DEFAULT_SYSTEM_PROMPT",label:"DEFAULT_SYSTEM_PROMPT"},{type:"doc",id:"06-api-reference/variables/parseToolCall",label:"parseToolCall"},{type:"doc",id:"06-api-reference/functions/DEFAULT_STRUCTURED_OUTPUT_PROMPT",label:"DEFAULT_STRUCTURED_OUTPUT_PROMPT"},{type:"doc",id:"06-api-reference/functions/fixAndValidateStructuredOutput",label:"fixAndValidateStructuredOutput"},{type:"doc",id:"06-api-reference/functions/getStructuredOutputPrompt",label:"getStructuredOutputPrompt"}]},{type:"category",label:"Utils",items:[{type:"doc",id:"06-api-reference/classes/MessageCountContextStrategy",label:"MessageCountContextStrategy"},{type:"doc",id:"06-api-reference/classes/NoopContextStrategy",label:"NoopContextStrategy"},{type:"doc",id:"06-api-reference/classes/SlidingWindowContextStrategy",label:"SlidingWindowContextStrategy"}]}]}; module.exports = typedocSidebar.items; \ No newline at end of file diff --git a/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md b/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md index 9df9d5e64..dfd9f2891 100644 --- a/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md +++ b/docs/docs/06-api-reference/variables/ALL_MINILM_L6_V2.md @@ -2,7 +2,7 @@ > `const` **ALL_MINILM_L6_V2**: `object` -Defined in: [constants/modelUrls.ts:596](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L596) +Defined in: [constants/modelUrls.ts:695](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L695) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md b/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md index b996eeda7..cb51bee2d 100644 --- a/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md +++ b/docs/docs/06-api-reference/variables/ALL_MPNET_BASE_V2.md @@ -2,7 +2,7 @@ > `const` **ALL_MPNET_BASE_V2**: `object` -Defined in: [constants/modelUrls.ts:604](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L604) +Defined in: [constants/modelUrls.ts:703](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L703) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md index 01dd61729..bd2a89eba 100644 --- a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md +++ b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_256.md @@ -2,7 +2,7 @@ > `const` **BK_SDM_TINY_VPRED_256**: `object` -Defined in: [constants/modelUrls.ts:649](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L649) +Defined in: [constants/modelUrls.ts:748](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L748) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md index 6e4f4c0c4..ec21972f1 100644 --- a/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md +++ b/docs/docs/06-api-reference/variables/BK_SDM_TINY_VPRED_512.md @@ -2,7 +2,7 @@ > `const` **BK_SDM_TINY_VPRED_512**: `object` -Defined in: [constants/modelUrls.ts:638](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L638) +Defined in: [constants/modelUrls.ts:737](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L737) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md index 1ae2347d6..0dc8afce7 100644 --- a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md +++ b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_IMAGE.md @@ -2,7 +2,7 @@ > `const` **CLIP_VIT_BASE_PATCH32_IMAGE**: `object` -Defined in: [constants/modelUrls.ts:577](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L577) +Defined in: [constants/modelUrls.ts:676](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L676) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md index 99df027e5..94fc574b3 100644 --- a/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md +++ b/docs/docs/06-api-reference/variables/CLIP_VIT_BASE_PATCH32_TEXT.md @@ -2,7 +2,7 @@ > `const` **CLIP_VIT_BASE_PATCH32_TEXT**: `object` -Defined in: [constants/modelUrls.ts:628](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L628) +Defined in: [constants/modelUrls.ts:727](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L727) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE.md b/docs/docs/06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE.md index 3b4b97699..e2154c4ac 100644 --- a/docs/docs/06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE.md +++ b/docs/docs/06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE.md @@ -2,7 +2,7 @@ > `const` **DEEPLAB_V3_MOBILENET_V3_LARGE**: `object` -Defined in: [constants/modelUrls.ts:574](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L574) +Defined in: [constants/modelUrls.ts:584](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L584) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED.md b/docs/docs/06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED.md index bf0da7ea7..341d3f629 100644 --- a/docs/docs/06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED.md +++ b/docs/docs/06-api-reference/variables/DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED.md @@ -2,7 +2,7 @@ > `const` **DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED**: `object` -Defined in: [constants/modelUrls.ts:622](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L622) +Defined in: [constants/modelUrls.ts:632](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L632) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET101.md b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET101.md index 9b0294dff..e40e1170b 100644 --- a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET101.md +++ b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET101.md @@ -2,7 +2,7 @@ > `const` **DEEPLAB_V3_RESNET101**: `object` -Defined in: [constants/modelUrls.ts:566](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L566) +Defined in: [constants/modelUrls.ts:576](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L576) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET101_QUANTIZED.md b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET101_QUANTIZED.md index b2ce1f2d5..4d7cf7498 100644 --- a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET101_QUANTIZED.md +++ b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET101_QUANTIZED.md @@ -2,7 +2,7 @@ > `const` **DEEPLAB_V3_RESNET101_QUANTIZED**: `object` -Defined in: [constants/modelUrls.ts:614](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L614) +Defined in: [constants/modelUrls.ts:624](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L624) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50.md b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50.md index 09f2f1641..b71326026 100644 --- a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50.md +++ b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50.md @@ -2,7 +2,7 @@ > `const` **DEEPLAB_V3_RESNET50**: `object` -Defined in: [constants/modelUrls.ts:557](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L557) +Defined in: [constants/modelUrls.ts:568](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L568) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50_QUANTIZED.md b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50_QUANTIZED.md index 552e3ec18..c82d5bf26 100644 --- a/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50_QUANTIZED.md +++ b/docs/docs/06-api-reference/variables/DEEPLAB_V3_RESNET50_QUANTIZED.md @@ -2,7 +2,7 @@ > `const` **DEEPLAB_V3_RESNET50_QUANTIZED**: `object` -Defined in: [constants/modelUrls.ts:606](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L606) +Defined in: [constants/modelUrls.ts:616](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L616) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/FCN_RESNET101.md b/docs/docs/06-api-reference/variables/FCN_RESNET101.md index 83529c803..07f4783d3 100644 --- a/docs/docs/06-api-reference/variables/FCN_RESNET101.md +++ b/docs/docs/06-api-reference/variables/FCN_RESNET101.md @@ -2,7 +2,7 @@ > `const` **FCN_RESNET101**: `object` -Defined in: [constants/modelUrls.ts:598](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L598) +Defined in: [constants/modelUrls.ts:608](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L608) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/FCN_RESNET101_QUANTIZED.md b/docs/docs/06-api-reference/variables/FCN_RESNET101_QUANTIZED.md index 11a0bc65d..377dc667a 100644 --- a/docs/docs/06-api-reference/variables/FCN_RESNET101_QUANTIZED.md +++ b/docs/docs/06-api-reference/variables/FCN_RESNET101_QUANTIZED.md @@ -2,7 +2,7 @@ > `const` **FCN_RESNET101_QUANTIZED**: `object` -Defined in: [constants/modelUrls.ts:646](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L646) +Defined in: [constants/modelUrls.ts:656](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L656) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/FCN_RESNET50.md b/docs/docs/06-api-reference/variables/FCN_RESNET50.md index d47b40a2b..2546001f3 100644 --- a/docs/docs/06-api-reference/variables/FCN_RESNET50.md +++ b/docs/docs/06-api-reference/variables/FCN_RESNET50.md @@ -2,7 +2,7 @@ > `const` **FCN_RESNET50**: `object` -Defined in: [constants/modelUrls.ts:590](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L590) +Defined in: [constants/modelUrls.ts:600](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L600) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/FCN_RESNET50_QUANTIZED.md b/docs/docs/06-api-reference/variables/FCN_RESNET50_QUANTIZED.md index 3a1acaaf9..b24a1757e 100644 --- a/docs/docs/06-api-reference/variables/FCN_RESNET50_QUANTIZED.md +++ b/docs/docs/06-api-reference/variables/FCN_RESNET50_QUANTIZED.md @@ -2,7 +2,7 @@ > `const` **FCN_RESNET50_QUANTIZED**: `object` -Defined in: [constants/modelUrls.ts:638](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L638) +Defined in: [constants/modelUrls.ts:648](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L648) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/FSMN_VAD.md b/docs/docs/06-api-reference/variables/FSMN_VAD.md index f91c2cb97..646e2dbae 100644 --- a/docs/docs/06-api-reference/variables/FSMN_VAD.md +++ b/docs/docs/06-api-reference/variables/FSMN_VAD.md @@ -2,7 +2,7 @@ > `const` **FSMN_VAD**: `object` -Defined in: [constants/modelUrls.ts:663](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L663) +Defined in: [constants/modelUrls.ts:762](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L762) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE.md b/docs/docs/06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE.md index 93518b723..1ad79ff02 100644 --- a/docs/docs/06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE.md +++ b/docs/docs/06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE.md @@ -2,7 +2,7 @@ > `const` **LRASPP_MOBILENET_V3_LARGE**: `object` -Defined in: [constants/modelUrls.ts:582](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L582) +Defined in: [constants/modelUrls.ts:592](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L592) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE_QUANTIZED.md b/docs/docs/06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE_QUANTIZED.md index a95ad31b5..0f5e16f41 100644 --- a/docs/docs/06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE_QUANTIZED.md +++ b/docs/docs/06-api-reference/variables/LRASPP_MOBILENET_V3_LARGE_QUANTIZED.md @@ -2,7 +2,7 @@ > `const` **LRASPP_MOBILENET_V3_LARGE_QUANTIZED**: `object` -Defined in: [constants/modelUrls.ts:630](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L630) +Defined in: [constants/modelUrls.ts:640](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L640) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md b/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md index ebbed0cf4..a7378ea8a 100644 --- a/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md +++ b/docs/docs/06-api-reference/variables/MULTI_QA_MINILM_L6_COS_V1.md @@ -2,7 +2,7 @@ > `const` **MULTI_QA_MINILM_L6_COS_V1**: `object` -Defined in: [constants/modelUrls.ts:612](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L612) +Defined in: [constants/modelUrls.ts:711](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L711) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md b/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md index e689be26d..0c22cb2b9 100644 --- a/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md +++ b/docs/docs/06-api-reference/variables/MULTI_QA_MPNET_BASE_DOT_V1.md @@ -2,7 +2,7 @@ > `const` **MULTI_QA_MPNET_BASE_DOT_V1**: `object` -Defined in: [constants/modelUrls.ts:620](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L620) +Defined in: [constants/modelUrls.ts:719](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L719) ## Type Declaration diff --git a/docs/docs/06-api-reference/variables/RF_DETR_NANO.md b/docs/docs/06-api-reference/variables/RF_DETR_NANO.md index 857746ce0..7a27c149d 100644 --- a/docs/docs/06-api-reference/variables/RF_DETR_NANO.md +++ b/docs/docs/06-api-reference/variables/RF_DETR_NANO.md @@ -12,4 +12,4 @@ Defined in: [constants/modelUrls.ts:402](https://github.com/software-mansion/rea ### modelSource -> `readonly` **modelSource**: `"https://huggingface.co/software-mansion/react-native-executorch-rf-detr-nano-detector/resolve/v0.8.0/rf-detr-nano.pte"` = `RF_DETR_NANO_MODEL` +> `readonly` **modelSource**: `"https://huggingface.co/software-mansion/react-native-executorch-rfdetr-nano-detector/resolve/v0.8.0/rfdetr_detector.pte"` = `RF_DETR_NANO_MODEL` diff --git a/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md b/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md index 90b55917a..bc6f0eeef 100644 --- a/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md +++ b/docs/docs/06-api-reference/variables/SELFIE_SEGMENTATION.md @@ -2,7 +2,7 @@ > `const` **SELFIE_SEGMENTATION**: `object` -Defined in: [constants/modelUrls.ts:566](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L566) +Defined in: [constants/modelUrls.ts:665](https://github.com/software-mansion/react-native-executorch/blob/main/packages/react-native-executorch/src/constants/modelUrls.ts#L665) ## Type Declaration From 972e0c5e3d52157184740a1d0360a882ef5a20e5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mateusz=20S=C5=82uszniak?= Date: Tue, 3 Mar 2026 15:22:58 +0100 Subject: [PATCH 24/24] test: fix test script --- .../common/rnexecutorch/tests/run_tests.sh | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/packages/react-native-executorch/common/rnexecutorch/tests/run_tests.sh b/packages/react-native-executorch/common/rnexecutorch/tests/run_tests.sh index 360aa9d11..63d738eb3 100755 --- a/packages/react-native-executorch/common/rnexecutorch/tests/run_tests.sh +++ b/packages/react-native-executorch/common/rnexecutorch/tests/run_tests.sh @@ -29,7 +29,6 @@ TEST_EXECUTABLES=( "TokenizerModuleTests" "SpeechToTextTests" "LLMTests" - "ImageSegmentationTests" "TextToImageTests" "OCRTests" "VerticalOCRTests" @@ -47,7 +46,7 @@ TEST_ASSETS=( # Models to download (format: "filename|url") # ============================================================================ MODELS=( - "style_transfer_candy_xnnpack.pte|https://huggingface.co/software-mansion/react-native-executorch-style-transfer-candy/resolve/main/xnnpack/style_transfer_candy_xnnpack.pte" + "style_transfer_candy_xnnpack_fp32.pte|https://huggingface.co/software-mansion/react-native-executorch-style-transfer-candy/resolve/main/xnnpack/style_transfer_candy_xnnpack_fp32.pte" "efficientnet_v2_s_xnnpack.pte|https://huggingface.co/software-mansion/react-native-executorch-efficientnet-v2-s/resolve/v0.6.0/xnnpack/efficientnet_v2_s_xnnpack.pte" "ssdlite320-mobilenetv3-large.pte|https://huggingface.co/software-mansion/react-native-executorch-ssdlite320-mobilenet-v3-large/resolve/v0.6.0/ssdlite320-mobilenetv3-large.pte" "test_image.jpg|https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/1200px-Cat_November_2010-1a.jpg" @@ -55,8 +54,8 @@ MODELS=( "all-MiniLM-L6-v2_xnnpack.pte|https://huggingface.co/software-mansion/react-native-executorch-all-MiniLM-L6-v2/resolve/v0.6.0/all-MiniLM-L6-v2_xnnpack.pte" "tokenizer.json|https://huggingface.co/software-mansion/react-native-executorch-all-MiniLM-L6-v2/resolve/v0.6.0/tokenizer.json" "fsmn-vad_xnnpack.pte|https://huggingface.co/software-mansion/react-native-executorch-fsmn-vad/resolve/main/xnnpack/fsmn-vad_xnnpack.pte" - "whisper_tiny_en_encoder_xnnpack.pte|https://huggingface.co/software-mansion/react-native-executorch-whisper-tiny.en/resolve/main/xnnpack/whisper_tiny_en_encoder_xnnpack.pte" - "whisper_tiny_en_decoder_xnnpack.pte|https://huggingface.co/software-mansion/react-native-executorch-whisper-tiny.en/resolve/main/xnnpack/whisper_tiny_en_decoder_xnnpack.pte" + "whisper_tiny_en_encoder_xnnpack.pte|https://huggingface.co/software-mansion/react-native-executorch-whisper-tiny.en/resolve/v0.7.0/xnnpack/whisper_tiny_en_encoder_xnnpack.pte" + "whisper_tiny_en_decoder_xnnpack.pte|https://huggingface.co/software-mansion/react-native-executorch-whisper-tiny.en/resolve/v0.7.0/xnnpack/whisper_tiny_en_decoder_xnnpack.pte" "whisper_tokenizer.json|https://huggingface.co/software-mansion/react-native-executorch-whisper-tiny.en/resolve/v0.6.0/tokenizer.json" "smolLm2_135M_8da4w.pte|https://huggingface.co/software-mansion/react-native-executorch-smolLm-2/resolve/v0.6.0/smolLm-2-135M/quantized/smolLm2_135M_8da4w.pte" "smollm_tokenizer.json|https://huggingface.co/software-mansion/react-native-executorch-smolLm-2/resolve/v0.6.0/tokenizer.json"