diff --git a/README.md b/README.md index ef3223c..a2a2367 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,11 @@ # Ithaca capture node The software a Jetson runs to join an Ithaca RGBD capture rig: a Python bridge that -puts the node on the WebSocket mesh and answers Preview / Record, and a C++ preview +puts the node on the WebSocket mesh and answers Preview / Record, a C++ preview server that streams each camera's colour (the sensor's own JPEG) and depth (its own -16-bit millimetres, LZ4-compressed) straight to the Unity viewer. +16-bit millimetres, LZ4-compressed) straight to the Unity viewer, and a C++ recorder +that writes K4A-format MKV takes on Record — the same format the rest of Ithaca +reads. One checkout, one command, on any Jetson: @@ -44,6 +46,9 @@ bridge/ server/ ithaca_rgbd_server.cpp the preview server CMakeLists.txt links a prebuilt libOrbbecSDK, no GStreamer +recorder/ + depth_color_k4arec_pipeline.cpp the MKV recorder — FTF source, not a stock example + CMakeLists.txt built inside the SDK examples tree, links the K4A wrapper systemd/ ithaca-bridge.service.in templated with the user and repo path usbfs-memory.service raises usbfs_memory_mb for the camera bandwidth @@ -52,41 +57,67 @@ udev/ scripts/ detect_platform.sh L4T / JetPack / SoC, as KEY=value install_deps.sh apt + websockets - build_sdk.sh reuse or build OrbbecSDK v2 for this platform + build_k4a_wrapper.sh reuse or build the Orbbec K4A wrapper (libk4a) + build_sdk.sh reuse or build OrbbecSDK v2 + inject/build the recorder build_server.sh cmake + make the preview server ``` +## The three C++ pieces, and how each is built + +- **Preview server** (`server/`) — our source, built standalone against the + prebuilt `libOrbbecSDK.so`. Simple. +- **Recorder** (`recorder/`) — FTF's own source, a customised SDK example, so it is + injected into the SDK's `examples/` tree and built there (it uses the SDK's own + cmake targets). It writes K4A-format MKV, and at runtime dlopens `libk4a.so` / + `libk4arecord.so` — provided by the **Orbbec K4A wrapper**, *not* Microsoft's + Azure Kinect SDK. Found by rpath, so no `LD_LIBRARY_PATH`. +- **Orbbec K4A wrapper** — Orbbec's `libk4a`, backed by their own SDK, cloned and + built by `build_k4a_wrapper.sh`. This is what makes Record work while setting the + Microsoft SDK aside. + ## What install.sh does 1. Detects L4T / JetPack / SoC (`scripts/detect_platform.sh`). -2. Installs build and run dependencies (no GStreamer — the direct route encodes no - video). -3. Locates a working OrbbecSDK build, or clones and builds one. A node that already - has a good build (the Nano) is left untouched. -4. Builds `ithaca_rgbd_server` against it. -5. Generates `bridge/config.json` for the current user — this is where all the +2. Installs build and run dependencies (OpenCV for the recorder; no GStreamer — the + direct route encodes no video). +3. Builds (or reuses) the Orbbec K4A wrapper — `libk4a` for the recorder. +4. Builds (or reuses) the OrbbecSDK, injecting and building the recorder example + against the wrapper. A node that already has a good build (the Nano) is left + untouched; the recorder is added to it only if missing. +5. Builds `ithaca_rgbd_server` against the SDK. +6. Generates `bridge/config.json` for the current user — this is where all the machine-specific paths live, so `bridge.py` itself stays unmodified. -6. Installs the udev rules and the `usbfs-memory` service, and adds the user to the +7. Installs the udev rules and the `usbfs-memory` service, and adds the user to the `video` group. -7. Generates and starts the `ithaca-bridge` systemd service. +8. Generates and starts the `ithaca-bridge` systemd service. + +If the K4A wrapper is unavailable, install.sh still completes: the node then does +**preview only**, and says so. Record needs the wrapper. Re-run it after a `git pull`: it rebuilds, regenerates the unit and config, and restarts the service. It keeps an existing `config.json` (delete it to regenerate) and an existing SDK build. -## The SDK is the one thing to watch on a new JetPack +## The two source builds to watch on a new JetPack -`build_sdk.sh` reuses an existing build when it finds one. On a machine with none it -clones the official OrbbecSDK v2 and builds it — this is the step most likely to -need attention on a JetPack it has not been tried on. If it fails, build the SDK by -hand once and re-run with its location: +Both scripts reuse an existing build when they find one. On a machine with none they +clone and build from source — the steps most likely to need attention on a JetPack +they have not been tried on: -```sh -OB_SDK_ROOT=/path/to/OrbbecSDK_v2 ./install.sh -``` +- **Orbbec K4A wrapper** — `OB_K4A_REPO` / `OB_K4A_REF` pin the source (default + `v1.10.4`). If it fails, build it by hand once and re-run with its location: + ```sh + K4A_WRAPPER_DIR=/path/to/OrbbecSDK-K4A-Wrapper ./install.sh + ``` +- **OrbbecSDK v2** — `OB_SDK_REPO` / `OB_SDK_REF` pin the source (default `main`). + Likewise: + ```sh + OB_SDK_ROOT=/path/to/OrbbecSDK_v2 ./install.sh + ``` -Override the source with `OB_SDK_REPO` / `OB_SDK_REF` to pin a tag known to build on -your JetPack. +Both can be passed together. Neither is Microsoft's Azure Kinect SDK — the recorder +gets its `libk4a` from the Orbbec wrapper, which is buildable from source across +platforms. ## Operating the node diff --git a/install.sh b/install.sh index 90d107a..5047a30 100755 --- a/install.sh +++ b/install.sh @@ -34,13 +34,30 @@ fi # ---- 1. dependencies ------------------------------------------------------ "$REPO/scripts/install_deps.sh" -# ---- 2. the SDK, then the server ------------------------------------------ -# build_sdk.sh reuses an existing build (leaving a working node untouched) or -# builds one; build_server.sh compiles ithaca_rgbd_server against it. -echo ">> Localisation / compilation de l'OrbbecSDK…" +# ---- 2. the K4A wrapper, the SDK (with the recorder), then the server ------ +# Three compiled pieces, in dependency order: +# - the Orbbec K4A wrapper provides libk4a, which the recorder needs; +# - the SDK builds the library AND the recorder example against that wrapper; +# - the preview server links the built SDK. +# Each script reuses an existing build, so a working node (the Nano) is untouched. +echo ">> Localisation / compilation du wrapper K4A (libk4a, Orbbec)…" +eval "$("$REPO/scripts/build_k4a_wrapper.sh")" || true +if [[ -n "${K4A_WRAPPER_DIR:-}" ]]; then + echo " K4A_WRAPPER_DIR = $K4A_WRAPPER_DIR" + export K4A_WRAPPER_DIR +else + echo " !! wrapper K4A indisponible — le nœud fera la PREVIEW mais pas le RECORD." +fi + +echo ">> Localisation / compilation de l'OrbbecSDK (+ recorder)…" eval "$("$REPO/scripts/build_sdk.sh")" echo " OB_SDK_ROOT = $OB_SDK_ROOT" echo " OB_SDK_BUILD = $OB_SDK_BUILD" +if [[ -n "${RECORDER_BIN:-}" ]]; then + echo " RECORDER_BIN = $RECORDER_BIN" +else + echo " !! recorder non produit — RECORD indisponible sur ce nœud." +fi echo ">> Compilation du serveur de preview…" SERVER_BIN=$("$REPO/scripts/build_server.sh") diff --git a/recorder/CMakeLists.txt b/recorder/CMakeLists.txt new file mode 100644 index 0000000..8ec3066 --- /dev/null +++ b/recorder/CMakeLists.txt @@ -0,0 +1,115 @@ +# Copyright (c) Orbbec Inc. / French Touch Factory. All Rights Reserved. +# Licensed under the MIT License. + +cmake_minimum_required(VERSION 3.10) +project(ob_stream_depth_color_k4arec_pipeline) + +if(WIN32) + set(K4A_WRAPPER_DIR + "C:/Ithaca_Projects/RGBD/Clone K4A Depot V2/OrbbecSDK-K4A-Wrapper" + CACHE PATH "Path to OrbbecSDK-K4A-Wrapper root directory") + # Windows: the wrapper's own out-of-source build dir is conventionally "Build", + # producing MSVC import libs (k4a.lib/k4arecord.lib) alongside the DLLs in bin/. + set(K4A_WRAPPER_BUILD_SUBDIR "Build") +else() + set(K4A_WRAPPER_DIR + "$ENV{HOME}/Downloads/OrbbecSDK-K4A-Wrapper" + CACHE PATH "Path to OrbbecSDK-K4A-Wrapper root directory") + # Linux: built with a lowercase "build" dir; shared libs land in lib/, not bin/. + set(K4A_WRAPPER_BUILD_SUBDIR "build") +endif() + +if(NOT EXISTS "${K4A_WRAPPER_DIR}/include/k4a/k4a.h") + message(WARNING + "${PROJECT_NAME}: K4A wrapper not found at '${K4A_WRAPPER_DIR}'. " + "Set K4A_WRAPPER_DIR to the OrbbecSDK-K4A-Wrapper root. Skipping.") + return() +endif() + +add_executable(${PROJECT_NAME} depth_color_k4arec_pipeline.cpp) +# C++14: the source uses digit-separator literals (2'000'000ULL); C++11 rejects them. +set_property(TARGET ${PROJECT_NAME} PROPERTY CXX_STANDARD 14) + +# rightW/rightH (Spout) and pinCpu/pinCore (SetProcessAffinityMask) are only read +# inside #ifdef _WIN32 blocks, so non-Windows builds see them as set-but-unused — +# harmless, but this SDK's own linux_config.cmake enables -Werror globally. +if(NOT WIN32) + target_compile_options(${PROJECT_NAME} PRIVATE -Wno-unused-but-set-variable) +endif() + +# SYSTEM: the K4A wrapper's public headers have GCC-pedantic issues (stray ';' +# after struct/enum decls) that -Wpedantic -Werror (set globally by this SDK's +# own linux_config.cmake) would otherwise turn into hard errors. -isystem makes +# GCC/Clang suppress warnings from headers found there. +target_include_directories(${PROJECT_NAME} SYSTEM PRIVATE + "${K4A_WRAPPER_DIR}/include" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/src/orbbec/include" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/src/orbbec/record/sdk/include" +) + +if(WIN32) + set(K4A_LIB "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4a.lib") + set(K4ARECORD_LIB "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4arecord.lib") +endif() + +target_link_libraries(${PROJECT_NAME} + ob::${OB_SDK_LIB_NAME} + ob::examples::utils + ${OpenCV_LIBS} + $<$:${K4A_LIB}> + $<$:${K4ARECORD_LIB}> + $<$:SpoutDX> + # Linux: libk4a.so/libk4arecord.so are dlopen'd (RTLD_DEEPBIND) at runtime + # instead of link-time linked — see the dlopen comment block near the top + # of depth_color_k4arec_pipeline.cpp for why. Needs libdl for dlopen/dlsym. + $<$>:dl> +) + +set_target_properties(${PROJECT_NAME} PROPERTIES FOLDER "examples") +if(MSVC) + set_target_properties(${PROJECT_NAME} PROPERTIES + VS_DEBUGGER_WORKING_DIRECTORY "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}") +endif() + +if(NOT WIN32) + # Linux: libk4a.so/libk4arecord.so aren't on the system search path — rpath + # the wrapper's build/lib so the binary finds them without LD_LIBRARY_PATH. + # BUILD_RPATH (not INSTALL_RPATH+BUILD_WITH_INSTALL_RPATH) so this ADDS to + # CMake's automatic build-tree rpath instead of replacing it — the automatic + # one is what makes libOrbbecSDK.so (built earlier in this same tree) found. + set_target_properties(${PROJECT_NAME} PROPERTIES + BUILD_RPATH "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/lib") +endif() + +if(WIN32) + add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4a.dll" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4arecord.dll" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/depthengine_2_0.dll" + $ + COMMENT "Copying K4A wrapper DLLs" + ) +endif() + +if(WIN32 AND TARGET opencv_world) + add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different + "$" + $ + COMMENT "Copying OpenCV DLL" + ) +endif() + +install(TARGETS ${PROJECT_NAME} RUNTIME DESTINATION bin) +if(WIN32) + install(FILES + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4a.dll" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4arecord.dll" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/depthengine_2_0.dll" + DESTINATION bin + ) +endif() +if(WIN32 AND TARGET opencv_world) + install(FILES "$" DESTINATION bin) +endif() diff --git a/recorder/depth_color_k4arec_pipeline.cpp b/recorder/depth_color_k4arec_pipeline.cpp new file mode 100644 index 0000000..3ca4f13 --- /dev/null +++ b/recorder/depth_color_k4arec_pipeline.cpp @@ -0,0 +1,1139 @@ +// Copyright (c) Orbbec Inc. / French Touch Factory. All Rights Reserved. +// Licensed under the MIT License. +// +// ob_stream_depth_color + K4A MKV recording using ob::Pipeline. +// ob::Pipeline with enableFrameSync() delivers pre-matched color+depth(+IR) +// framesets — the same approach used by the K4A SDK internally. +// This eliminates manual timestamp matching and improves resilience to +// Femto Bolt 25-bit timestamp overflow events at the cost of ~100 ms +// pipeline sync latency vs the lowlat variant. +// 'R' toggles recording; Ctrl+C or ESC/Q to exit. +// +// Accepts the Azure Kinect k4arecorder option set (--color-mode, --depth-mode, +// --rate, --record-length, and the manual color-control flags) so it can act as +// a drop-in MKV recorder for Orbbec Femto Bolt / Mega devices. + +#include +#include "utils.hpp" +#include "utils_opencv.hpp" + +#include +#include + +#ifdef _WIN32 +#include +#include +#include +#else +// --------------------------------------------------------------------------- +// Linux only: dlopen libk4arecord_v2.so instead of linking it directly. +// +// libk4arecord_v2.so est notre bibliotheque d'ecriture MKV compatible K4A, batie +// sur les sources d'ecriture du wrapper (record.cpp + matroska_write.cpp) mais +// liee a un shim de donnees pures (k4a_shim.cpp) au lieu de libk4a.so. Elle ne +// depend d'AUCUN OrbbecSDK : ce process ne charge donc plus que la v2.8.6 liee +// directement ci-dessous (ob::Pipeline / ob::Context). +// +// Historique : on chargeait auparavant libk4a.so + libk4arecord.so, qui +// embarquent leur propre copie d'OrbbecSDK v1.10.18. Les deux SDK exportent les +// memes symboles C non versionnes (ex. ob_create_frame_from_buffer) avec des +// encodages d'enums differents, ce qui imposait RTLD_DEEPBIND pour que chaque +// wrapper reste appaire a sa copie. Cette v1 coutait aussi un ob_create_frameset() +// par frame -- soit une creation/destruction de Context avec reenumeration USB +// complete a chaque image. Plus de v1, plus de DEEPBIND, plus de churn. +#include + +namespace k4a_dl { + +static void *g_k4arec = nullptr; // libk4arecord_v2.so : record + shim k4a + +// One function pointer per K4A / K4A-record API this file calls. The +// decltype(&::fn) keeps each pointer's signature in lockstep with the real +// declaration in k4a.h / k4arecord/record.h without repeating it by hand. +static decltype(&::k4a_capture_create) k4a_capture_create = nullptr; +static decltype(&::k4a_capture_release) k4a_capture_release = nullptr; +static decltype(&::k4a_capture_set_color_image) k4a_capture_set_color_image = nullptr; +static decltype(&::k4a_capture_set_depth_image) k4a_capture_set_depth_image = nullptr; +static decltype(&::k4a_capture_set_ir_image) k4a_capture_set_ir_image = nullptr; +static decltype(&::k4a_image_create_from_buffer) k4a_image_create_from_buffer = nullptr; +static decltype(&::k4a_image_set_device_timestamp_usec) k4a_image_set_device_timestamp_usec = nullptr; +static decltype(&::k4a_image_release) k4a_image_release = nullptr; +static decltype(&::k4a_record_create) k4a_record_create = nullptr; +static decltype(&::k4a_record_add_tag) k4a_record_add_tag = nullptr; +static decltype(&::k4a_record_add_attachment) k4a_record_add_attachment = nullptr; +static decltype(&::k4a_record_write_header) k4a_record_write_header = nullptr; +static decltype(&::k4a_record_write_capture) k4a_record_write_capture = nullptr; +static decltype(&::k4a_record_flush) k4a_record_flush = nullptr; +static decltype(&::k4a_record_close) k4a_record_close = nullptr; + +template static void bind(void *handle, const char *name, T &fnOut) { + void *sym = dlsym(handle, name); + if(!sym) { + std::cerr << "Fatal: symbol '" << name << "' not found: " << dlerror() << std::endl; + std::exit(EXIT_FAILURE); + } + fnOut = reinterpret_cast(sym); +} + +static void *openLib(const char *soname) { + // RTLD_DEEPBIND n'est plus necessaire : il n'y a plus qu'un seul OrbbecSDK + // dans le process, donc plus de collision de symboles a arbitrer. + void *h = dlopen(soname, RTLD_NOW); + if(!h) { + std::cerr << "Fatal: dlopen(" << soname << ") failed: " << dlerror() << std::endl; + std::exit(EXIT_FAILURE); + } + return h; +} + +static void init() { + if(g_k4arec) return; // idempotent + g_k4arec = openLib("libk4arecord_v2.so"); + + bind(g_k4arec, "k4a_capture_create", k4a_capture_create); + bind(g_k4arec, "k4a_capture_release", k4a_capture_release); + bind(g_k4arec, "k4a_capture_set_color_image", k4a_capture_set_color_image); + bind(g_k4arec, "k4a_capture_set_depth_image", k4a_capture_set_depth_image); + bind(g_k4arec, "k4a_capture_set_ir_image", k4a_capture_set_ir_image); + bind(g_k4arec, "k4a_image_create_from_buffer", k4a_image_create_from_buffer); + bind(g_k4arec, "k4a_image_set_device_timestamp_usec", k4a_image_set_device_timestamp_usec); + bind(g_k4arec, "k4a_image_release", k4a_image_release); + + bind(g_k4arec, "k4a_record_create", k4a_record_create); + bind(g_k4arec, "k4a_record_add_tag", k4a_record_add_tag); + bind(g_k4arec, "k4a_record_add_attachment", k4a_record_add_attachment); + bind(g_k4arec, "k4a_record_write_header", k4a_record_write_header); + bind(g_k4arec, "k4a_record_write_capture", k4a_record_write_capture); + bind(g_k4arec, "k4a_record_flush", k4a_record_flush); + bind(g_k4arec, "k4a_record_close", k4a_record_close); +} + +} // namespace k4a_dl + +// Redirect every call below (textually, after this point) to the dlopen'd, +// DEEPBIND-loaded copies instead of the statically-linked symbols. +#define k4a_capture_create k4a_dl::k4a_capture_create +#define k4a_capture_release k4a_dl::k4a_capture_release +#define k4a_capture_set_color_image k4a_dl::k4a_capture_set_color_image +#define k4a_capture_set_depth_image k4a_dl::k4a_capture_set_depth_image +#define k4a_capture_set_ir_image k4a_dl::k4a_capture_set_ir_image +#define k4a_image_create_from_buffer k4a_dl::k4a_image_create_from_buffer +#define k4a_image_set_device_timestamp_usec k4a_dl::k4a_image_set_device_timestamp_usec +#define k4a_image_release k4a_dl::k4a_image_release +#define k4a_record_create k4a_dl::k4a_record_create +#define k4a_record_add_tag k4a_dl::k4a_record_add_tag +#define k4a_record_add_attachment k4a_dl::k4a_record_add_attachment +#define k4a_record_write_header k4a_dl::k4a_record_write_header +#define k4a_record_write_capture k4a_dl::k4a_record_write_capture +#define k4a_record_flush k4a_dl::k4a_record_flush +#define k4a_record_close k4a_dl::k4a_record_close +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// ---- Globals --------------------------------------------------------------- + +static std::mutex g_recMutex; +static k4a_record_t g_recording = nullptr; +static std::atomic g_isRecording { false }; +static std::atomic g_pendingReset{ false }; +static uint64_t g_frameCount = 0; + +// Color image format written to the MKV (set from --color-mode; matches the +// k4a_device_configuration_t color_format so k4a_record_write_capture accepts it). +static k4a_image_format_t g_k4aColorFormat = K4A_IMAGE_FORMAT_COLOR_MJPG; + +// Timestamp handling (same logic as lowlat variant). +static uint64_t g_tsOffset = 0; +static uint64_t g_lastWrittenTs = 0; +static uint64_t g_writeFailures = 0; + +// Femto Bolt 25-bit counter overflow compensation. +static constexpr uint64_t kTsOverflowPeriod = 1ULL << 25; +static constexpr uint64_t kTsOverflowHalfPeriod = kTsOverflowPeriod >> 1; +static uint64_t g_lastRawTs = 0; +static uint64_t g_rawTsEpoch = 0; + +static constexpr uint64_t kMaxJumpUs = 2'000'000ULL; +static constexpr uint64_t kResetThresholdUs = 1'000'000ULL; + +// Subordinate: how long to wait for the master's sync pulse before force-starting +// "anyway" (avoids an empty MKV if the cable is bad). With many cameras the whole +// rig can take >5 s to come up, so this is configurable via --sync-timeout. +// 0 = wait forever (never start unsynced — safest for a guaranteed-sync rig). +static int64_t g_pendingResetTimeoutMs = 30000; +static std::chrono::steady_clock::time_point g_pendingResetStart; + +static std::shared_ptr g_device; +static std::atomic g_exitRequested{ false }; +static k4a_wired_sync_mode_t g_syncMode = K4A_WIRED_SYNC_MODE_STANDALONE; + +// Display frames — updated from pipeline callback, read by main thread. +static std::mutex g_dispMutex; +static std::condition_variable g_newFrameCv; +static std::shared_ptr g_dispColor; +static std::shared_ptr g_dispDepth; +static std::shared_ptr g_dispIR; +static bool g_hasNewFrame = false; + +// ---- Stream-mode descriptors (mirror the K4A k4arecorder mode strings) ----- + +struct ColorModeInfo { + bool enabled = false; + int width = 0; + int height = 0; + OBFormat obFormat = OB_FORMAT_MJPG; + k4a_image_format_t k4aFormat = K4A_IMAGE_FORMAT_COLOR_MJPG; + k4a_color_resolution_t k4aRes = K4A_COLOR_RESOLUTION_OFF; +}; + +struct DepthModeInfo { + bool depthEnabled = false; // produces a DEPTH stream + bool irEnabled = false; // produces an IR stream + int width = 0; + int height = 0; + k4a_depth_mode_t k4aMode = K4A_DEPTH_MODE_OFF; +}; + +static std::string toUpper(std::string s) { + for(char &c : s) c = static_cast(::toupper(static_cast(c))); + return s; +} + +// 2160p / 1536p / 1440p / 1080p / 720p / 720p_NV12 / 720p_YUY2 / OFF +static bool parseColorMode(const std::string &in, ColorModeInfo &o) { + const std::string s = toUpper(in); + o = ColorModeInfo{}; + if(s == "OFF") { o.enabled = false; o.k4aRes = K4A_COLOR_RESOLUTION_OFF; return true; } + o.enabled = true; + o.obFormat = OB_FORMAT_MJPG; + o.k4aFormat = K4A_IMAGE_FORMAT_COLOR_MJPG; + if(s == "2160P") { o.width = 3840; o.height = 2160; o.k4aRes = K4A_COLOR_RESOLUTION_2160P; } + else if(s == "1536P") { o.width = 2048; o.height = 1536; o.k4aRes = K4A_COLOR_RESOLUTION_1536P; } + else if(s == "1440P") { o.width = 2560; o.height = 1440; o.k4aRes = K4A_COLOR_RESOLUTION_1440P; } + else if(s == "1080P") { o.width = 1920; o.height = 1080; o.k4aRes = K4A_COLOR_RESOLUTION_1080P; } + else if(s == "720P") { o.width = 1280; o.height = 720; o.k4aRes = K4A_COLOR_RESOLUTION_720P; } + else if(s == "720P_NV12") { o.width = 1280; o.height = 720; o.k4aRes = K4A_COLOR_RESOLUTION_720P; + o.obFormat = OB_FORMAT_NV12; o.k4aFormat = K4A_IMAGE_FORMAT_COLOR_NV12; } + else if(s == "720P_YUY2") { o.width = 1280; o.height = 720; o.k4aRes = K4A_COLOR_RESOLUTION_720P; + o.obFormat = OB_FORMAT_YUY2; o.k4aFormat = K4A_IMAGE_FORMAT_COLOR_YUY2; } + else return false; + return true; +} + +// NFOV_2X2BINNED / NFOV_UNBINNED / WFOV_2X2BINNED / WFOV_UNBINNED / PASSIVE_IR / OFF +static bool parseDepthMode(const std::string &in, DepthModeInfo &o) { + const std::string s = toUpper(in); + o = DepthModeInfo{}; + if(s == "OFF") { o.k4aMode = K4A_DEPTH_MODE_OFF; return true; } + if(s == "NFOV_2X2BINNED") { o.width = 320; o.height = 288; o.depthEnabled = true; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_NFOV_2X2BINNED; return true; } + if(s == "NFOV_UNBINNED") { o.width = 640; o.height = 576; o.depthEnabled = true; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_NFOV_UNBINNED; return true; } + if(s == "WFOV_2X2BINNED") { o.width = 512; o.height = 512; o.depthEnabled = true; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_WFOV_2X2BINNED; return true; } + if(s == "WFOV_UNBINNED") { o.width = 1024; o.height = 1024; o.depthEnabled = true; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_WFOV_UNBINNED; return true; } + if(s == "PASSIVE_IR") { o.width = 1024; o.height = 1024; o.depthEnabled = false; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_PASSIVE_IR; return true; } + return false; +} + +static bool parseRate(int fps, k4a_fps_t &o) { + switch(fps) { + case 30: o = K4A_FRAMES_PER_SECOND_30; return true; + case 25: o = K4A_FRAMES_PER_SECOND_25; return true; + case 15: o = K4A_FRAMES_PER_SECOND_15; return true; + case 5: o = K4A_FRAMES_PER_SECOND_5; return true; + default: return false; + } +} + +#ifdef _WIN32 +static BOOL WINAPI ctrlHandler(DWORD) { + g_exitRequested = true; + g_newFrameCv.notify_all(); + return TRUE; +} +#else +// Linux equivalent of ctrlHandler above. Without this, SIGINT/SIGTERM (e.g. a +// process supervisor like the WebSocket bridge asking this process to stop) +// fall through to the OS default action — an immediate kill with no MKV +// finalization (closeRecording()/pipeline->stop() never run) and the camera +// left in whatever state it was mid-stream. Only sets the atomic flag: the +// rest of the handler body above (notify_all on a condition_variable) is not +// async-signal-safe, so it's left to the main loop's wait_for() timeout +// (<=200 ms) to notice g_exitRequested instead. +static void posixSignalHandler(int) { + g_exitRequested = true; +} +#endif + +static std::string makeTimestampedFilename() { + std::time_t t = std::time(nullptr); + std::tm *tm = std::localtime(&t); + std::ostringstream ss; + ss << "capture_" << std::put_time(tm, "%Y%m%d_%H%M%S") << ".mkv"; + return ss.str(); +} + +static void noopBufferRelease(void* /*buf*/, void* /*ctx*/) {} + +struct CalibAccum { + std::vector buf; + uint32_t totalSize = 0; + bool done = false; +}; + +static void calibDataCallback(ob_data_tran_state state, ob_data_chunk *chunk, void *userData) { + if(!chunk) return; + auto *acc = static_cast(userData); + if(state == DATA_TRAN_STAT_TRANSFERRING && chunk->data && chunk->size > 0) { + if(chunk->fullDataSize > 0 && acc->buf.size() < chunk->fullDataSize) { + acc->buf.resize(chunk->fullDataSize); + acc->totalSize = chunk->fullDataSize; + } + size_t end = static_cast(chunk->offset) + chunk->size; + if(acc->buf.size() < end) acc->buf.resize(end); + std::memcpy(acc->buf.data() + chunk->offset, chunk->data, chunk->size); + } + else if(state == DATA_TRAN_STAT_DONE) { + if(acc->totalSize > 0 && acc->buf.size() > acc->totalSize) + acc->buf.resize(acc->totalSize); + if(!acc->buf.empty() && acc->buf.size() % 2 == 1 && acc->buf.back() != 0x7d) + acc->buf.back() = 0x7d; + acc->done = true; + } +} + +static cv::Mat decodeColorFrame(const std::shared_ptr &frame) { + if(!frame) return {}; + auto vf = frame->as(); + uint32_t w = vf->getWidth(), h = vf->getHeight(); + const uint8_t *data = static_cast(frame->getData()); + cv::Mat colorBGR; + switch(frame->getFormat()) { + case OB_FORMAT_RGB: { cv::Mat s(h, w, CV_8UC3, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_RGB2BGR); break; } + case OB_FORMAT_BGR: colorBGR = cv::Mat(h, w, CV_8UC3, (void*)data).clone(); break; + case OB_FORMAT_BGRA: { cv::Mat s(h, w, CV_8UC4, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_BGRA2BGR); break; } + case OB_FORMAT_RGBA: { cv::Mat s(h, w, CV_8UC4, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_RGBA2BGR); break; } + case OB_FORMAT_YUYV: + case OB_FORMAT_YUY2: { cv::Mat s(h, w, CV_8UC2, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_YUV2BGR_YUYV); break; } + case OB_FORMAT_NV12: { cv::Mat s(h * 3 / 2, w, CV_8UC1, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_YUV2BGR_NV12); break; } + case OB_FORMAT_MJPG: { + cv::Mat m(1, static_cast(vf->getDataSize()), CV_8UC1, (void*)data); + colorBGR = cv::imdecode(m, cv::IMREAD_COLOR); + break; + } + default: break; + } + return colorBGR; +} + +static void resetDeviceTimestamp() { + if(!g_device) return; + try { + OBDeviceTimestampResetConfig cfg{}; + cfg.enable = true; + cfg.timestamp_reset_delay_us = 0; + cfg.timestamp_reset_signal_output_enable = (g_syncMode == K4A_WIRED_SYNC_MODE_MASTER); + g_device->setTimestampResetConfig(cfg); + + if(g_syncMode == K4A_WIRED_SYNC_MODE_SUBORDINATE) { + std::cout << "Subordinate ready — awaiting timestamp reset pulse from master." << std::endl; + return; + } + if(g_device->isPropertySupported(OB_PROP_TIMER_RESET_SIGNAL_BOOL, OB_PERMISSION_WRITE)) { + g_device->setBoolProperty(OB_PROP_TIMER_RESET_SIGNAL_BOOL, true); + std::cout << "Device timestamp reset (prop 104) + pulse sent to subordinates." << std::endl; + } + else { + g_device->timestampReset(); + std::cout << "Device timestamp reset + pulse sent to subordinates." << std::endl; + } + } + catch(ob::Error &e) { + std::cerr << "Warning: timestamp reset failed: " << e.what() << std::endl; + } +} + +static void writeCapture(std::shared_ptr colorFrame, + std::shared_ptr depthFrame, + std::shared_ptr irFrame = nullptr) { + if(!g_isRecording) return; + std::lock_guard lock(g_recMutex); + if(!g_recording) return; + + uint64_t rawTs = colorFrame ? colorFrame->getTimeStampUs() + : depthFrame ? depthFrame->getTimeStampUs() + : irFrame ? irFrame->getTimeStampUs() : 0; + + // Timestamp drop compensation: fires when the device counter wraps (25-bit overflow, + // ~33.55 s) OR when resetDeviceTimestamp() resets a long-running counter back to near-0. + // IMPORTANT: always update g_lastRawTs to rawTs after detection — if we only update on + // rawTs > g_lastRawTs the condition stays true every subsequent frame, accumulating + // a spurious epoch increment per frame and making adjTs jump > kMaxJumpUs every time. + if(g_lastRawTs > 0 && rawTs + kTsOverflowHalfPeriod < g_lastRawTs) { + g_rawTsEpoch += kTsOverflowPeriod; + std::cout << "[INFO] Device ts overflow/reset (rawTs=" << rawTs + << " prev=" << g_lastRawTs << " epoch=" << g_rawTsEpoch << " us)\n"; + g_lastRawTs = rawTs; + } else if(rawTs > g_lastRawTs) { + g_lastRawTs = rawTs; + } + uint64_t effectiveRawTs = rawTs + g_rawTsEpoch; + + uint64_t adjTs; + if(g_syncMode == K4A_WIRED_SYNC_MODE_STANDALONE) { + if(g_tsOffset == 0) g_tsOffset = effectiveRawTs; + adjTs = (effectiveRawTs >= g_tsOffset) ? (effectiveRawTs - g_tsOffset) : 0; + } + else { + adjTs = effectiveRawTs; + } + + if(g_lastWrittenTs > 0 && adjTs > g_lastWrittenTs + kMaxJumpUs) return; + if(adjTs < g_lastWrittenTs) return; + + k4a_capture_t cap = nullptr; + if(K4A_FAILED(k4a_capture_create(&cap))) return; + + if(colorFrame) { + auto vf = colorFrame->as(); + int w = static_cast(vf->getWidth()); + int h = static_cast(vf->getHeight()); + size_t sz = colorFrame->getDataSize(); + const void *raw = colorFrame->getData(); + uint8_t *buf = static_cast(const_cast(raw)); + // Stride per K4A color format (0 = compressed/MJPG). + int stride = 0; + switch(g_k4aColorFormat) { + case K4A_IMAGE_FORMAT_COLOR_NV12: stride = w; break; + case K4A_IMAGE_FORMAT_COLOR_YUY2: stride = w * 2; break; + case K4A_IMAGE_FORMAT_COLOR_BGRA32: stride = w * 4; break; + default: stride = 0; break; // MJPG + } + k4a_image_t img = nullptr; + if(K4A_SUCCEEDED(k4a_image_create_from_buffer( + g_k4aColorFormat, w, h, stride, buf, sz, + noopBufferRelease, nullptr, &img))) { + k4a_image_set_device_timestamp_usec(img, adjTs); + k4a_capture_set_color_image(cap, img); + k4a_image_release(img); + } + } + if(depthFrame) { + auto vf = depthFrame->as(); + int w = static_cast(vf->getWidth()); + int h = static_cast(vf->getHeight()); + size_t sz = depthFrame->getDataSize(); + const void *raw = depthFrame->getData(); + uint8_t *buf = static_cast(const_cast(raw)); + k4a_image_t img = nullptr; + if(K4A_SUCCEEDED(k4a_image_create_from_buffer( + K4A_IMAGE_FORMAT_DEPTH16, w, h, w * 2, buf, sz, + noopBufferRelease, nullptr, &img))) { + k4a_image_set_device_timestamp_usec(img, adjTs); + k4a_capture_set_depth_image(cap, img); + k4a_image_release(img); + } + } + if(irFrame) { + auto vf = irFrame->as(); + int w = static_cast(vf->getWidth()); + int h = static_cast(vf->getHeight()); + size_t sz = irFrame->getDataSize(); + const void *raw = irFrame->getData(); + uint8_t *buf = static_cast(const_cast(raw)); + k4a_image_t img = nullptr; + if(K4A_SUCCEEDED(k4a_image_create_from_buffer( + K4A_IMAGE_FORMAT_IR16, w, h, w * 2, buf, sz, + noopBufferRelease, nullptr, &img))) { + k4a_image_set_device_timestamp_usec(img, adjTs); + k4a_capture_set_ir_image(cap, img); + k4a_image_release(img); + } + } + + if(K4A_FAILED(k4a_record_write_capture(g_recording, cap))) { + ++g_writeFailures; + if(g_writeFailures == 1 || g_writeFailures % 30 == 0) + std::cerr << "[WARN] k4a_record_write_capture failed (ts=" << adjTs + << " us, total=" << g_writeFailures << ")\n"; + } else { + g_lastWrittenTs = adjTs; + g_frameCount++; + } + k4a_capture_release(cap); +} + +// ---- Main ------------------------------------------------------------------ + +int main(int argc, char *argv[]) try { +#ifndef _WIN32 + k4a_dl::init(); // dlopen libk4arecord_v2.so — see comment near the top of this file. +#endif + + std::string outputPath; + int deviceIndex = 0; + bool showPreview = false; + bool useSpout = false; + bool recordIR = true; + uint32_t syncDelayUs = 0; + std::string argSerial; + std::string argFirmware; + bool pinCpu = false; // --pin-cpu : affinite process = coeur d'indice deviceIndex + // --pin-core N : coeur d'affinite explicite. Avant, l'affinite derivait de --device ; la + // selection se faisant maintenant par --serial, le coeur doit etre donne a part (sinon tous + // les process se retrouvent sur le coeur 0 = la contention MF que ce pinning evite). + int pinCore = -1; + + // k4arecorder-compatible capture parameters. + int recordLengthSec = 0; // 0 = infinite + std::string colorModeStr = "1080p"; + std::string depthModeStr = "NFOV_UNBINNED"; + int rateFps = 30; + int exposureUs = -1; // -1 = auto exposure + int brightness = -1; // -1 = auto/unset + int contrast = -1; + int saturation = -1; + int sharpness = -1; + int whitebalance = -1; // -1 = auto white balance + int gain = -1; // -1 = auto/unset + + for(int i = 1; i < argc; i++) { + std::string arg = argv[i]; + if(arg == "--help" || arg == "-h") { + std::cout << "Usage: " << argv[0] << " [options] [output.mkv]\n" + << "\n" + << "Capture options (Azure Kinect k4arecorder compatible):\n" + << " -l, --record-length N Limit the recording to N seconds (default: infinite)\n" + << " -c, --color-mode M Set the color sensor mode (default: 1080p). Options:\n" + << " 2160p, 1536p, 1440p, 1080p, 720p, 720p_NV12, 720p_YUY2, OFF\n" + << " -d, --depth-mode M Set the depth sensor mode (default: NFOV_UNBINNED). Options:\n" + << " NFOV_2X2BINNED, NFOV_UNBINNED, WFOV_2X2BINNED, WFOV_UNBINNED, PASSIVE_IR, OFF\n" + << " -r, --rate N Camera frame rate in FPS (default: 30). Options: 30, 25, 15, 5\n" + << " -e, --exposure-control N Manual RGB exposure 100..409500 us (default: auto)\n" + << " -b, --brightness N Manual brightness 0..100 (default: auto)\n" + << " -t, --contrast N Manual contrast 0..100 (default: auto)\n" + << " -s, --saturation N Manual saturation 0..100 (default: auto)\n" + << " -p, --sharpness N Manual sharpness 0..100 (default: auto)\n" + << " -w, --whitebalance N Manual white balance 2000..11000 (default: auto)\n" + << " -g, --gain N Manual gain 1..255 (default: auto)\n" + << "\n" + << "Device / output options:\n" + << " --device N Select device by index (default: 0)\n" + << " --preview Open preview window\n" + << " --spout Enable Spout output\n" + << " -m, --master Set device as sync master\n" + << " --subordinate Set device as sync subordinate\n" + << " --sub Alias for --subordinate\n" + << " --standalone Free-running, no sync (default)\n" + << " --sync-delay N Subordinate trigger delay in microseconds\n" + << " --delay N Alias for --sync-delay\n" + << " --sync-timeout N Subordinate: seconds to wait for the master pulse\n" + << " before starting anyway (default 30; 0 = wait forever)\n" + << " --pin-cpu Pin this process to CPU core (see --pin-core)\n" + << " --pin-core N Core index used by --pin-cpu (default: --device)\n" + << " --serial SN SELECT the device by serial; also MKV metadata\n" + << " --firmware N Override firmware version in MKV metadata\n" + << " --no-ir Disable IR recording (reduces disk write)\n" + << "\n" + << "Note: -d/-p/-s follow k4arecorder (depth-mode/sharpness/saturation);\n" + << " --device/--preview/--spout have no short form to avoid the clash.\n" + << "\n" + << "Uses ob::Pipeline with frame sync for robust color+depth(+IR) matching.\n" + << "If output.mkv is given, recording starts immediately.\n" + << "Without --preview the program runs headless; Ctrl+C to stop.\n"; + return 0; + } + else if(arg == "--device" && i + 1 < argc) { deviceIndex = std::stoi(argv[++i]); } + else if(arg == "--preview") { showPreview = true; } + else if(arg == "--spout") { useSpout = true; } + else if(arg == "--master" || arg == "-m") { g_syncMode = K4A_WIRED_SYNC_MODE_MASTER; } + else if(arg == "--subordinate" || arg == "--sub") { g_syncMode = K4A_WIRED_SYNC_MODE_SUBORDINATE; } + else if(arg == "--standalone") { g_syncMode = K4A_WIRED_SYNC_MODE_STANDALONE; } + else if((arg == "--sync-delay" || arg == "--delay") && i + 1 < argc) { syncDelayUs = static_cast(std::stoul(argv[++i])); } + else if(arg == "--sync-timeout" && i + 1 < argc) { int s = std::stoi(argv[++i]); g_pendingResetTimeoutMs = (s <= 0) ? 0 : static_cast(s) * 1000; } + else if(arg == "--pin-cpu") { pinCpu = true; } + else if(arg == "--pin-core" && i + 1 < argc) { pinCore = std::stoi(argv[++i]); } + else if(arg == "--serial" && i + 1 < argc) { argSerial = argv[++i]; } + else if(arg == "--firmware" && i + 1 < argc) { argFirmware = argv[++i]; } + else if(arg == "--no-ir") { recordIR = false; } + else if((arg == "--record-length" || arg == "-l") && i + 1 < argc) { recordLengthSec = std::stoi(argv[++i]); } + else if((arg == "--color-mode" || arg == "-c") && i + 1 < argc) { colorModeStr = argv[++i]; } + else if((arg == "--depth-mode" || arg == "-d") && i + 1 < argc) { depthModeStr = argv[++i]; } + else if((arg == "--rate" || arg == "-r") && i + 1 < argc) { rateFps = std::stoi(argv[++i]); } + else if((arg == "--exposure-control" || arg == "-e") && i + 1 < argc) { exposureUs = std::stoi(argv[++i]); } + else if((arg == "--brightness" || arg == "-b") && i + 1 < argc) { brightness = std::stoi(argv[++i]); } + else if((arg == "--contrast" || arg == "-t") && i + 1 < argc) { contrast = std::stoi(argv[++i]); } + else if((arg == "--saturation" || arg == "-s") && i + 1 < argc) { saturation = std::stoi(argv[++i]); } + else if((arg == "--sharpness" || arg == "-p") && i + 1 < argc) { sharpness = std::stoi(argv[++i]); } + else if((arg == "--whitebalance" || arg == "-w") && i + 1 < argc) { whitebalance = std::stoi(argv[++i]); } + else if((arg == "--gain" || arg == "-g") && i + 1 < argc) { gain = std::stoi(argv[++i]); } + else if(arg[0] != '-' && outputPath.empty()) { outputPath = arg; } + } + + // ---- Optional CPU pinning: affinity = core index deviceIndex ----------- + // Pins the whole process (all its threads) to a single core = --device. + // Simple first step so the N camera processes don't fight over the same cores. + // NB: 1 core per process is restrictive (this process has several threads: + // depth engine, pipeline, writer, callbacks) — extend to a block of cores + // later if a single core saturates. +#ifdef _WIN32 + if(pinCpu) { + SYSTEM_INFO si; GetSystemInfo(&si); + int nCores = static_cast(si.dwNumberOfProcessors); + int core = (pinCore >= 0) ? pinCore : deviceIndex; // --pin-core wins, --device is the legacy default + if(core >= 0 && core < nCores && core < 64) { + DWORD_PTR mask = (static_cast(1) << core); + if(SetProcessAffinityMask(GetCurrentProcess(), mask)) + std::cout << "CPU affinity: pinned to core " << core << std::endl; + else + std::cerr << "Warning: SetProcessAffinityMask failed for core " << core << std::endl; + } else { + std::cerr << "Warning: --pin-cpu: core index " << core + << " out of range (" << nCores << " cores) — not pinned." << std::endl; + } + } +#endif + + // ---- Resolve capture modes -------------------------------------------- + ColorModeInfo color; + DepthModeInfo depth; + k4a_fps_t k4aFps; + if(!parseColorMode(colorModeStr, color)) { std::cerr << "Invalid --color-mode '" << colorModeStr << "'.\n"; return EXIT_FAILURE; } + if(!parseDepthMode(depthModeStr, depth)) { std::cerr << "Invalid --depth-mode '" << depthModeStr << "'.\n"; return EXIT_FAILURE; } + if(!parseRate(rateFps, k4aFps)) { std::cerr << "Invalid --rate '" << rateFps << "' (use 30, 25, 15 or 5).\n"; return EXIT_FAILURE; } + + bool hasColor = color.enabled; + bool hasDepth = depth.depthEnabled; + bool hasIR = depth.irEnabled && recordIR; + if(!hasColor && !hasDepth && !hasIR) { + std::cerr << "Nothing to stream: --color-mode and --depth-mode are both OFF (or IR disabled).\n"; + return EXIT_FAILURE; + } + g_k4aColorFormat = color.k4aFormat; + + // ---- Device selection -------------------------------------------------- + ob::Context ctx; + auto deviceList = ctx.queryDeviceList(); + uint32_t deviceCount = deviceList->deviceCount(); + std::cout << "Found " << deviceCount << " device(s)." << std::endl; + if(deviceCount == 0) { std::cerr << "No device connected.\n"; return EXIT_FAILURE; } + + // --serial SELECTS the device inside this SDK's own enumeration. Indexes are per-SDK, so the + // host cannot map them across camera brands - the serial is the only stable key it has. No + // fallback on failure: silently opening --device (0 by default) would start two processes on + // the same camera in a multi-camera rig. + if(!argSerial.empty()) { + bool found = false; + for(uint32_t i = 0; i < deviceCount; i++) { + const char *sn = deviceList->getSerialNumber(i); + if(sn && argSerial == sn) { deviceIndex = static_cast(i); found = true; break; } + } + if(!found) { + std::cerr << "Serial '" << argSerial << "' not found among " << deviceCount + << " device(s).\n"; + return EXIT_FAILURE; + } + } + if(static_cast(deviceIndex) >= deviceCount) { + std::cerr << "Device index " << deviceIndex << " out of range.\n"; return EXIT_FAILURE; + } + + std::string deviceSerial, deviceFirmwareCOLOR, deviceFirmwareDEPTH, deviceName; + { + const char *sn = deviceList->getSerialNumber(static_cast(deviceIndex)); + const char *nm = deviceList->getName(static_cast(deviceIndex)); + if(sn && *sn) deviceSerial = sn; + if(nm && *nm) deviceName = nm; + } + if(!argSerial.empty()) deviceSerial = argSerial; + if(!argFirmware.empty()) { deviceFirmwareCOLOR = argFirmware; deviceFirmwareDEPTH = argFirmware; } + std::cout << "Device: " << deviceName << " S/N: '" << deviceSerial << "'" << std::endl; + std::cout << "Config: color=" << (hasColor ? colorModeStr : std::string("OFF")) + << " depth=" << depthModeStr + << " ir=" << (hasIR ? "on" : "off") + << " rate=" << rateFps << " fps" << std::endl; + + g_device = deviceList->getDevice(static_cast(deviceIndex)); + + // ---- Multi-device sync mode -------------------------------------------- + // Les TROIS modes ecrivent OB_STRUCT_MULTI_DEVICE_SYNC_CONFIG. Auparavant ce + // bloc etait garde par `g_syncMode != STANDALONE` : --standalone ne + // configurait donc rien et laissait la NVRAM du device telle quelle. Une + // camera restee en OB_SYNC_MODE_SECONDARY attendait alors un pulse materiel + // qui ne venait jamais -- flux bloques en STARTING, zero frame, et pas la + // moindre erreur pour le signaler. Le mode doit etre ecrit explicitement, + // sinon "standalone" ne veut rien dire au niveau du capteur. + { + ob_error *syncErr = nullptr; + OBDeviceSyncConfig obSync{}; + uint32_t syncLen = 0; + ob_device_get_structured_data(g_device->getImpl(), + OB_STRUCT_MULTI_DEVICE_SYNC_CONFIG, + reinterpret_cast(&obSync), + &syncLen, &syncErr); + if(syncErr) { ob_delete_error(syncErr); syncErr = nullptr; } + + // Mode herite, avant ecrasement : rend visible une camera laissee en + // secondary par un run precedent (ou par un autre outil). + const int previousMode = static_cast(obSync.syncMode); + + const char *modeName = "standalone"; + if(g_syncMode == K4A_WIRED_SYNC_MODE_MASTER) { + obSync.syncMode = OB_SYNC_MODE_PRIMARY_MCU_TRIGGER; + modeName = "master"; + } + else if(g_syncMode == K4A_WIRED_SYNC_MODE_SUBORDINATE) { + obSync.syncMode = OB_SYNC_MODE_SECONDARY; + obSync.irTriggerSignalInDelay = static_cast(syncDelayUs); + obSync.rgbTriggerSignalInDelay = static_cast(syncDelayUs); + modeName = "subordinate"; + } + else { + // Free-running : le capteur ne doit attendre aucun signal externe, et + // les retards de declenchement herites n'ont plus de sens. + obSync.syncMode = OB_SYNC_MODE_STANDALONE; + obSync.irTriggerSignalInDelay = 0; + obSync.rgbTriggerSignalInDelay = 0; + } + + ob_device_set_structured_data(g_device->getImpl(), + OB_STRUCT_MULTI_DEVICE_SYNC_CONFIG, + reinterpret_cast(&obSync), + sizeof(obSync), &syncErr); + if(syncErr) { + std::cerr << "Warning: sync config failed: " << ob_error_get_message(syncErr) << std::endl; + ob_delete_error(syncErr); + } else { + std::cout << "Sync mode: " << modeName + << " (device etait en mode " << previousMode << ")"; + if(g_syncMode == K4A_WIRED_SYNC_MODE_SUBORDINATE && syncDelayUs > 0) + std::cout << " (delay=" << syncDelayUs << " us)"; + std::cout << std::endl; + } + } + + // ---- Calibration JSON -------------------------------------------------- + std::vector calibJson; + { + ob_error *err = nullptr; + CalibAccum acc; + ob_device_get_raw_data(g_device->getImpl(), + static_cast(OB_RAW_DATA_CAMERA_CALIB_JSON_FILE), + calibDataCallback, &acc, &err); + if(err) { std::cerr << "Warning: calibration read failed: " << ob_error_get_message(err) << std::endl; ob_delete_error(err); } + else if(!acc.done) std::cerr << "Warning: calibration transfer incomplete.\n"; + else if(acc.buf.empty()) std::cerr << "Warning: calibration JSON empty.\n"; + else { calibJson = std::move(acc.buf); std::cout << "Calibration JSON: " << calibJson.size() << " bytes\n"; } + } + + // ---- K4A stream configuration ------------------------------------------ + k4a_device_configuration_t k4aCfg = K4A_DEVICE_CONFIG_INIT_DISABLE_ALL; + k4aCfg.color_format = color.k4aFormat; + k4aCfg.color_resolution = color.k4aRes; // OFF when color disabled + k4aCfg.depth_mode = depth.k4aMode; + k4aCfg.camera_fps = k4aFps; + k4aCfg.wired_sync_mode = g_syncMode; + if(g_syncMode == K4A_WIRED_SYNC_MODE_SUBORDINATE) + k4aCfg.subordinate_delay_off_master_usec = syncDelayUs; + + auto openRecording = [&](const std::string &path) -> bool { + k4a_record_t rec = nullptr; + bool ok = K4A_SUCCEEDED(k4a_record_create(path.c_str(), nullptr, k4aCfg, &rec)); + if(ok) { + if(!deviceSerial.empty()) k4a_record_add_tag(rec, "K4A_DEVICE_SERIAL_NUMBER", deviceSerial.c_str()); + if(!deviceFirmwareCOLOR.empty()) k4a_record_add_tag(rec, "K4A_COLOR_FIRMWARE_VERSION", deviceFirmwareCOLOR.c_str()); + if(!deviceFirmwareDEPTH.empty()) k4a_record_add_tag(rec, "K4A_DEPTH_FIRMWARE_VERSION", deviceFirmwareDEPTH.c_str()); + if(!deviceName.empty()) k4a_record_add_tag(rec, "K4A_DEVICE_NAME", deviceName.c_str()); + } + if(ok && !calibJson.empty()) { + k4a_record_add_tag(rec, "K4A_CALIBRATION_FILE", "calibration.json"); + k4a_record_add_attachment(rec, "calibration.json", calibJson.data(), calibJson.size()); + } + if(ok) ok = K4A_SUCCEEDED(k4a_record_write_header(rec)); + if(ok) { + std::lock_guard lock(g_recMutex); + g_recording = rec; g_tsOffset = 0; g_lastWrittenTs = 0; + g_writeFailures = 0; g_lastRawTs = 0; g_rawTsEpoch = 0; + return true; + } + std::cerr << "Warning: k4a_record_create failed for " << path << std::endl; + if(rec) k4a_record_close(rec); + return false; + }; + + auto closeRecording = [&](const std::string &path) { + std::lock_guard lock(g_recMutex); + if(g_recording) { + k4a_record_flush(g_recording); + k4a_record_close(g_recording); + g_recording = nullptr; + std::cout << "Saved: " << path << " (" << g_frameCount << " frames)" << std::endl; + } + }; + + std::string recPath; + if(!outputPath.empty()) { + recPath = outputPath; + if(!openRecording(recPath)) std::cerr << "Streaming only (no MKV).\n"; + else std::cout << "MKV ready: " << recPath << std::endl; + } + + // ---- Pipeline setup ---------------------------------------------------- + // ob::Pipeline delivers pre-synchronized color+depth(+IR) framesets, + // avoiding manual timestamp matching entirely. + auto pipeline = std::make_shared(g_device); + auto config = std::make_shared(); + + if(hasColor) { + try { + auto p = pipeline->getStreamProfileList(OB_SENSOR_COLOR) + ->getVideoStreamProfile(color.width, color.height, color.obFormat, rateFps); + config->enableStream(p); + } catch(ob::Error &e) { + std::cerr << "Error: color mode " << colorModeStr << " (" << color.width << "x" << color.height + << " @ " << rateFps << " fps) not supported by device: " << e.what() << std::endl; + return EXIT_FAILURE; + } + } + if(hasDepth) { + try { + auto p = pipeline->getStreamProfileList(OB_SENSOR_DEPTH) + ->getVideoStreamProfile(depth.width, depth.height, OB_FORMAT_Y16, rateFps); + config->enableStream(p); + } catch(ob::Error &e) { + std::cerr << "Error: depth mode " << depthModeStr << " (" << depth.width << "x" << depth.height + << " @ " << rateFps << " fps) not supported by device: " << e.what() << std::endl; + return EXIT_FAILURE; + } + } + if(hasIR) { + try { + auto p = pipeline->getStreamProfileList(OB_SENSOR_IR) + ->getVideoStreamProfile(depth.width, depth.height, OB_FORMAT_Y16, rateFps); + config->enableStream(p); + } catch(ob::Error &e) { + std::cerr << "Warning: IR stream (" << depth.width << "x" << depth.height << " @ " << rateFps + << " fps) not available, continuing without IR: " << e.what() << std::endl; + hasIR = false; + } + } + + // Frame sync: ob::Pipeline matches color+depth(+IR) by timestamp internally. + pipeline->enableFrameSync(); + + // Auto-record: reset device timestamp before pipeline starts. + if(!outputPath.empty() && g_recording) { + g_frameCount = 0; + if(g_syncMode != K4A_WIRED_SYNC_MODE_STANDALONE) { + resetDeviceTimestamp(); + g_pendingReset = true; + g_pendingResetStart = std::chrono::steady_clock::now(); + std::cout << "Timestamp reset — waiting for sync before recording to: " << recPath << std::endl; + } else { + g_isRecording = true; + std::cout << "Auto-recording to: " << recPath << std::endl; + } + } + + // ---- Pipeline callback ------------------------------------------------- + pipeline->start(config, [&](std::shared_ptr frameset) { + if(!frameset) return; + + auto colorFrame = hasColor ? frameset->getFrame(OB_FRAME_COLOR) : nullptr; + auto depthFrame = hasDepth ? frameset->getFrame(OB_FRAME_DEPTH) : nullptr; + auto irFrame = hasIR ? frameset->getFrame(OB_FRAME_IR) : nullptr; + + // Require every enabled primary stream to be present in the synced set. + if(hasColor && !colorFrame) return; + if(hasDepth && !depthFrame) return; + if(!hasColor && !hasDepth && hasIR && !irFrame) return; // IR-only / PASSIVE_IR + + // Canonical frame for display + timestamp gating. + auto primary = colorFrame ? colorFrame : (depthFrame ? depthFrame : irFrame); + if(!primary) return; + + // Update display frames for main thread. + { + std::lock_guard lk(g_dispMutex); + g_dispColor = colorFrame; + g_dispDepth = depthFrame; + g_dispIR = irFrame; + g_hasNewFrame = true; + } + g_newFrameCv.notify_one(); + + // Sync mode: wait for the timestamp reset to propagate. + if(g_pendingReset) { + uint64_t pts = primary->getTimeStampUs(); + bool tsOk = pts < kResetThresholdUs; + bool timedOut = !tsOk && g_pendingResetTimeoutMs > 0 && std::chrono::duration_cast( + std::chrono::steady_clock::now() - g_pendingResetStart).count() > g_pendingResetTimeoutMs; + if(tsOk || timedOut) { + if(timedOut) + std::cerr << "[WARN] pendingReset timeout (ts=" << pts + << " us) — sync pulse not received, starting anyway\n"; + g_pendingReset = false; + g_isRecording = true; + std::cout << "Recording started (ts=" << pts << " us)\n"; + } + return; + } + + if(g_isRecording) + writeCapture(colorFrame, depthFrame, irFrame); + }); + + // ---- Apply manual color controls (after the color sensor is streaming) -- + auto setColorInt = [&](OBPropertyID id, int value, const char *name) { + try { + if(!g_device->isPropertySupported(id, OB_PERMISSION_WRITE)) { + std::cerr << "Warning: " << name << " not supported by device.\n"; return; + } + OBIntPropertyRange range = g_device->getIntPropertyRange(id); + if(value < range.min || value > range.max) { + int clamped = value < range.min ? range.min : range.max; // std::min/max clash with windows.h macros + std::cerr << "Warning: " << name << " " << value << " out of range [" + << range.min << ".." << range.max << "], clamping to " << clamped << ".\n"; + value = clamped; + } + g_device->setIntProperty(id, value); + std::cout << "Set " << name << " = " << value << std::endl; + } catch(ob::Error &e) { + std::cerr << "Warning: set " << name << " failed: " << e.what() << std::endl; + } + }; + auto setColorBool = [&](OBPropertyID id, bool value, const char *name) { + try { + if(g_device->isPropertySupported(id, OB_PERMISSION_WRITE)) + g_device->setBoolProperty(id, value); + } catch(ob::Error &e) { + std::cerr << "Warning: set " << name << " failed: " << e.what() << std::endl; + } + }; + + if(hasColor) { + if(exposureUs >= 0) { + setColorBool(OB_PROP_COLOR_AUTO_EXPOSURE_BOOL, false, "auto-exposure(off)"); + setColorInt (OB_PROP_COLOR_EXPOSURE_INT, exposureUs, "exposure(us)"); + } + if(whitebalance >= 0) { + setColorBool(OB_PROP_COLOR_AUTO_WHITE_BALANCE_BOOL, false, "auto-white-balance(off)"); + setColorInt (OB_PROP_COLOR_WHITE_BALANCE_INT, whitebalance, "white-balance"); + } + if(gain >= 0) setColorInt(OB_PROP_COLOR_GAIN_INT, gain, "gain"); + if(brightness >= 0) setColorInt(OB_PROP_COLOR_BRIGHTNESS_INT, brightness, "brightness"); + if(contrast >= 0) setColorInt(OB_PROP_COLOR_CONTRAST_INT, contrast, "contrast"); + if(saturation >= 0) setColorInt(OB_PROP_COLOR_SATURATION_INT, saturation, "saturation"); + if(sharpness >= 0) setColorInt(OB_PROP_COLOR_SHARPNESS_INT, sharpness, "sharpness"); + } + + // ---- Spout + signal handling ------------------------------------------- +#ifdef _WIN32 + spoutDX spoutColor, spoutDepth; + if(useSpout) { + spoutColor.SetSenderName("OrbbecColor"); + spoutDepth.SetSenderName("OrbbecDepth"); + } + SetConsoleCtrlHandler(ctrlHandler, TRUE); +#else + std::signal(SIGINT, posixSignalHandler); + std::signal(SIGTERM, posixSignalHandler); +#endif + + // Arrêt via EOF sur stdin — UNIQUEMENT si stdin est un vrai pipe (l'hôte l'a redirigé pour piloter + // l'arrêt en le fermant). Sinon (console interactive, ou CREATE_NEW_CONSOLE depuis un parent GUI + // comme Unity), stdin peut renvoyer EOF immédiatement → l'enregistrement s'arrêterait après 1 frame. + // En mode non-pipe, on n'active pas ce watcher ; l'arrêt passe par Ctrl+C/Ctrl+Break (ctrlHandler). + { + bool stdinIsPipe = false; +#ifdef _WIN32 + stdinIsPipe = (GetFileType(GetStdHandle(STD_INPUT_HANDLE)) == FILE_TYPE_PIPE); +#endif + if(stdinIsPipe) { + std::thread([]{ + int ch; + do { ch = std::getchar(); } while(ch != EOF && ch != 'q' && ch != 'Q'); + g_exitRequested = true; + g_newFrameCv.notify_all(); + }).detach(); + } + } + + if(showPreview) { + cv::namedWindow("K4A Rec (pipeline)", cv::WINDOW_NORMAL); + cv::resizeWindow("K4A Rec (pipeline)", 1280, 480); + std::cout << "R=start/stop recording T=timestamp ESC/Q=quit Ctrl+C=quit" << std::endl; + } else { + std::cout << "Headless mode. Ctrl+C to stop." << std::endl; + } + if(recordLengthSec > 0) + std::cout << "Recording limited to " << recordLengthSec << " s." << std::endl; + + // ---- Main loop --------------------------------------------------------- + bool showTimestamp = false; + const bool needDisplay = showPreview || useSpout; + std::chrono::steady_clock::time_point recStartTp{}; + + while(true) { + std::shared_ptr colorForDisplay, depthForDisplay, irForDisplay; + { + std::unique_lock lk(g_dispMutex); + g_newFrameCv.wait_for(lk, std::chrono::milliseconds(needDisplay ? 33 : 200), + [&]{ return (needDisplay && g_hasNewFrame) || g_exitRequested.load(); }); + if(g_exitRequested) break; + colorForDisplay = g_dispColor; + depthForDisplay = g_dispDepth; + irForDisplay = g_dispIR; + g_hasNewFrame = false; + } + + // ---- Record-length limit: stop + exit after N seconds of recording -- + if(recordLengthSec > 0) { + if(g_isRecording.load()) { + if(recStartTp == std::chrono::steady_clock::time_point{}) + recStartTp = std::chrono::steady_clock::now(); + else if(std::chrono::duration_cast( + std::chrono::steady_clock::now() - recStartTp).count() >= recordLengthSec) { + std::cout << "Record length (" << recordLengthSec << " s) reached — stopping." << std::endl; + break; + } + } else { + recStartTp = std::chrono::steady_clock::time_point{}; + } + } + + if(showPreview) { + int key = cv::waitKey(1); + if(key == 27 || key == 'q' || key == 'Q') break; + if(cv::getWindowProperty("K4A Rec (pipeline)", cv::WND_PROP_VISIBLE) < 1.0) break; + if(key == 'r' || key == 'R') { + if(!g_isRecording && !g_pendingReset) { + recPath = makeTimestampedFilename(); + if(openRecording(recPath)) { + g_frameCount = 0; + resetDeviceTimestamp(); + g_pendingReset = true; + g_pendingResetStart = std::chrono::steady_clock::now(); + std::cout << "Recording to: " << recPath << " ..." << std::endl; + } + } else if(g_isRecording) { + g_isRecording = false; + closeRecording(recPath); + } + } + if(key == 't' || key == 'T') showTimestamp = !showTimestamp; + } + + if(!needDisplay) continue; + if(!colorForDisplay && !depthForDisplay && !irForDisplay) continue; + + cv::Mat colorBGR = decodeColorFrame(colorForDisplay); + + // Right panel: colorized depth, or normalized IR when there is no depth. + cv::Mat depthBGR; + int rightW = 0, rightH = 0; + if(depthForDisplay) { + auto df = depthForDisplay->as(); + uint32_t w = df->getWidth(), h = df->getHeight(); + rightW = static_cast(w); rightH = static_cast(h); + float scale = df->getValueScale(); + const uint16_t *data = reinterpret_cast(df->getData()); + cv::Mat depth16(h, w, CV_16UC1, (void*)data); + cv::Mat depth8; + depth16.convertTo(depth8, CV_8UC1, scale * 255.0 / 5000.0); + cv::applyColorMap(depth8, depthBGR, cv::COLORMAP_JET); + } else if(irForDisplay) { + auto vf = irForDisplay->as(); + uint32_t w = vf->getWidth(), h = vf->getHeight(); + rightW = static_cast(w); rightH = static_cast(h); + const uint16_t *data = reinterpret_cast(irForDisplay->getData()); + cv::Mat ir16(h, w, CV_16UC1, (void*)data); + cv::Mat ir8; + cv::normalize(ir16, ir8, 0, 255, cv::NORM_MINMAX, CV_8UC1); + cv::cvtColor(ir8, depthBGR, cv::COLOR_GRAY2BGR); + } + +#ifdef _WIN32 + if(useSpout) { + if(!colorBGR.empty()) { + auto vf = colorForDisplay->as(); + cv::Mat bgra; + cv::cvtColor(colorBGR, bgra, cv::COLOR_RGB2RGBA); + spoutColor.SendImage(bgra.data, vf->getWidth(), vf->getHeight(), vf->getWidth() * 4); + } + if(!depthBGR.empty() && rightW > 0) { + cv::Mat bgra; + cv::cvtColor(depthBGR, bgra, cv::COLOR_RGB2RGBA); + spoutDepth.SendImage(bgra.data, rightW, rightH, rightW * 4); + } + } +#endif + + if(showPreview) { + cv::Mat cs, ds; + if(!colorBGR.empty()) cv::resize(colorBGR, cs, cv::Size(640, 360)); + else cs = cv::Mat::zeros(360, 640, CV_8UC3); + if(!depthBGR.empty()) cv::resize(depthBGR, ds, cv::Size(640, 360)); + else ds = cv::Mat::zeros(360, 640, CV_8UC3); + + cv::Mat combined; + cv::hconcat(cs, ds, combined); + if(g_isRecording) { + cv::circle(combined, cv::Point(20, 20), 8, cv::Scalar(0, 0, 255), -1); + cv::putText(combined, "REC " + std::to_string(g_frameCount), + cv::Point(35, 27), cv::FONT_HERSHEY_SIMPLEX, 0.6, cv::Scalar(0, 0, 255), 1); + } + if(showTimestamp) { + char buf[64]; + uint64_t cts = colorForDisplay ? colorForDisplay->getTimeStampUs() : 0; + uint64_t dts = depthForDisplay ? depthForDisplay->getTimeStampUs() + : irForDisplay ? irForDisplay->getTimeStampUs() : 0; + int y0 = combined.rows - 42, y1 = combined.rows - 18; + cv::rectangle(combined, cv::Point(0, y0 - 16), cv::Point(280, combined.rows), cv::Scalar(0,0,0), cv::FILLED); + snprintf(buf, sizeof(buf), "C: %.3f ms", cts / 1000.0); + cv::putText(combined, buf, cv::Point(6, y0), cv::FONT_HERSHEY_SIMPLEX, 0.6, cv::Scalar(0,220,220), 1); + snprintf(buf, sizeof(buf), "D: %.3f ms", dts / 1000.0); + cv::putText(combined, buf, cv::Point(6, y1), cv::FONT_HERSHEY_SIMPLEX, 0.6, cv::Scalar(0,220,220), 1); + } + cv::imshow("K4A Rec (pipeline)", combined); + } + } + + // ---- Cleanup ----------------------------------------------------------- + g_isRecording = false; + closeRecording(recPath); + pipeline->stop(); + g_device.reset(); + +#ifdef _WIN32 + if(useSpout) { spoutColor.ReleaseSender(); spoutDepth.ReleaseSender(); } +#endif + return 0; +} +catch(ob::Error &e) { + std::cerr << "function:" << e.getFunction() << "\nargs:" << e.getArgs() + << "\nmessage:" << e.what() << "\nstatus:" << e.getStatus() + << "\ntype:" << e.getExceptionType() << std::endl; + std::cout << "\nPress any key to exit."; + // Bounded timeout: with no timeout this blocks forever when stdin isn't + // interactive (SSH, a script, a service supervisor, ...), leaving the + // process running and the camera claimed. 3s is plenty for a human to + // notice on a double-clicked Windows console; headless runs just move on. + ob_smpl::waitForKeyPressed(3000); + exit(EXIT_FAILURE); +} diff --git a/scripts/build_k4a_wrapper.sh b/scripts/build_k4a_wrapper.sh new file mode 100755 index 0000000..ad2aaaa --- /dev/null +++ b/scripts/build_k4a_wrapper.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# Makes sure the Orbbec K4A wrapper is built for THIS platform, and prints where +# it is as K4A_WRAPPER_DIR=... for build_sdk.sh (the recorder example needs its +# headers) and for the recorder's runtime rpath (its libk4a.so/libk4arecord.so). +# +# This is the libk4a provider — Orbbec's own, NOT Microsoft's Azure Kinect SDK. +# It bundles its own OrbbecSDK internally, so it builds standalone. The recorder +# dlopens the libk4a it produces (RTLD_DEEPBIND) rather than link-time linking it, +# which is why only its build/lib path matters at runtime, not link time. +# +# Reuses an existing build (leaving a working node untouched) or clones + builds. +set -euo pipefail + +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +eval "$("$here/detect_platform.sh")" + +REPO_URL="${OB_K4A_REPO:-https://github.com/orbbec/OrbbecSDK-K4A-Wrapper}" +# The tag the Nano runs. A caller can override to one known to build on a newer +# JetPack. +REF="${OB_K4A_REF:-v1.10.4}" + +find_wrapper() { + local d="$1" + [[ -f "$d/include/k4a/k4a.h" ]] || return 1 + [[ -f "$d/build/lib/libk4a.so" ]] || return 1 + echo "$d" +} + +for candidate in "${K4A_WRAPPER_DIR:-}" \ + "$HOME/OrbbecSDK-K4A-Wrapper" \ + "$HOME/Downloads/OrbbecSDK-K4A-Wrapper"; do + [[ -z "$candidate" ]] && continue + if find_wrapper "$candidate" >/dev/null 2>&1; then + echo ">> Wrapper K4A deja bati, reutilise: $candidate" >&2 + echo "K4A_WRAPPER_DIR=$candidate" + exit 0 + fi +done + +dir="${K4A_WRAPPER_DIR:-$HOME/OrbbecSDK-K4A-Wrapper}" +echo ">> Wrapper K4A absent — clone et compilation (JetPack ${JETPACK}, ${SOC})" >&2 +echo ">> ${REPO_URL} @ ${REF} -> ${dir}" >&2 + +if [[ ! -d "$dir/.git" ]]; then + # --recursive: the wrapper carries its OrbbecSDK and depth engine as submodules. + git clone --recursive --depth 1 --branch "$REF" "$REPO_URL" "$dir" >&2 +fi + +mkdir -p "$dir/build" +( + cd "$dir/build" + cmake .. -DCMAKE_BUILD_TYPE=Release >&2 + make -j"$(nproc)" k4a k4arecord >&2 +) + +if ! find_wrapper "$dir" >/dev/null 2>&1; then + cat >&2 <> Wrapper K4A bati: $dir" >&2 +echo "K4A_WRAPPER_DIR=$dir" diff --git a/scripts/build_sdk.sh b/scripts/build_sdk.sh index 5681220..28b5499 100755 --- a/scripts/build_sdk.sh +++ b/scripts/build_sdk.sh @@ -1,71 +1,108 @@ #!/usr/bin/env bash -# Makes sure an OrbbecSDK v2 build exists for THIS platform, and prints where it -# is as OB_SDK_ROOT=... / OB_SDK_BUILD=... / OB_SDK_GENERATED=... for build_server.sh. +# Makes sure an OrbbecSDK v2 build exists for THIS platform — INCLUDING the custom +# MKV recorder example — and prints where it all is: +# +# OB_SDK_ROOT / OB_SDK_BUILD / OB_SDK_GENERATED the SDK +# RECORDER_BIN the recorder binary, or empty # # Never ships a prebuilt binary between Jetsons: CUDA and glibc differ by -# generation, so the SDK is built from source on the machine that will run it. -# The heavy artifact stays out of git. +# generation, so everything is built from source on the machine that runs it. The +# heavy artifacts stay out of git. # -# Order of preference: -# 1. An existing build pointed to by $OB_SDK_ROOT — reused as-is. This is what -# keeps a working node (the Nano already has one) untouched. -# 2. A build at one of the known locations, likewise reused. -# 3. A fresh clone + build. On a new JetPack this is the step to watch; if it -# fails, build the SDK by hand once and re-run install.sh with OB_SDK_ROOT set. +# The recorder is FTF's own source (recorder/ in this repo), not a stock SDK +# example, so it is injected into the SDK's examples tree before the build and +# compiled there — it depends on the SDK's own cmake targets (ob::examples::utils). +# It needs the Orbbec K4A wrapper's headers; build_k4a_wrapper.sh provides them. +# +# Order of preference for the SDK itself: +# 1. $OB_SDK_ROOT if it already holds a build — reused, which keeps a working +# node (the Nano) untouched. +# 2. A build at a known location, likewise reused. +# 3. A fresh clone + build. set -euo pipefail here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +repo=$(cd "$here/.." && pwd) eval "$("$here/detect_platform.sh")" -# The official v2 source. Pinned loosely: a caller can override both to match a -# tag known to build on their JetPack. SDK_REPO="${OB_SDK_REPO:-https://github.com/orbbec/OrbbecSDK_v2.git}" SDK_REF="${OB_SDK_REF:-main}" find_build() { - local root="$1" + local root="$1" b [[ -f "$root/include/libobsensor/ObSensor.hpp" ]] || return 1 - local b b=$(ls -d "$root"/build/linux_* 2>/dev/null | head -1) [[ -z "$b" ]] && b="$root/build" [[ -f "$b/lib/libOrbbecSDK.so" ]] || return 1 echo "$b" } +recorder_bin_in() { # echoes the recorder path under a build dir if present + local b="$1" + [[ -x "$b/bin/ob_stream_depth_color_k4arec_pipeline" ]] \ + && echo "$b/bin/ob_stream_depth_color_k4arec_pipeline" +} + emit() { local root="$1" build="$2" echo "OB_SDK_ROOT=$root" echo "OB_SDK_BUILD=$build" echo "OB_SDK_GENERATED=$root/build/src/generated" + echo "RECORDER_BIN=$(recorder_bin_in "$build")" } -# 1 & 2: reuse whatever is already built. +# Drops the FTF recorder into the SDK examples tree and registers it, so a plain +# example build picks it up. Idempotent. +inject_recorder() { + local root="$1" + local dst="$root/examples/1.stream.depth_color.k4arec.pipeline" + mkdir -p "$dst" + cp "$repo/recorder/depth_color_k4arec_pipeline.cpp" "$dst/" + cp "$repo/recorder/CMakeLists.txt" "$dst/" + local ex_cmake="$root/examples/CMakeLists.txt" + if [[ -f "$ex_cmake" ]] \ + && ! grep -q "1.stream.depth_color.k4arec.pipeline" "$ex_cmake"; then + echo "add_subdirectory(1.stream.depth_color.k4arec.pipeline)" >> "$ex_cmake" + fi +} + +# 1 & 2: reuse an existing SDK build. Only inject/rebuild the recorder if it is +# missing there — never disturb the library build. for candidate in "${OB_SDK_ROOT:-}" \ "$HOME/OrbbecSDK_v2" \ "$HOME/OrbbecSDK_v2_jetson"; do [[ -z "$candidate" ]] && continue if b=$(find_build "$candidate" 2>/dev/null); then echo ">> SDK deja bati, reutilise: $candidate" >&2 + if [[ -z "$(recorder_bin_in "$b")" && -n "${K4A_WRAPPER_DIR:-}" ]]; then + echo ">> Recorder absent de ce build — injection et compilation…" >&2 + inject_recorder "$candidate" + ( cd "$b" && cmake .. -DOB_BUILD_EXAMPLES=ON \ + -DK4A_WRAPPER_DIR="$K4A_WRAPPER_DIR" >&2 \ + && make -j"$(nproc)" ob_stream_depth_color_k4arec_pipeline >&2 ) || true + fi emit "$candidate" "$b" exit 0 fi done -# 3: clone + build from source. +# 3: clone + build from source, with examples and the recorder. root="${OB_SDK_ROOT:-$HOME/OrbbecSDK_v2}" echo ">> Aucun SDK bati — clone et compilation (JetPack ${JETPACK}, ${SOC})" >&2 -echo ">> ${SDK_REPO} @ ${SDK_REF} -> ${root}" >&2 +[[ -d "$root/.git" ]] || git clone --depth 1 --branch "$SDK_REF" "$SDK_REPO" "$root" >&2 -if [[ ! -d "$root/.git" ]]; then - git clone --depth 1 --branch "$SDK_REF" "$SDK_REPO" "$root" >&2 -fi +inject_recorder "$root" + +# The recorder needs the wrapper; without it the SDK still builds (its CMakeLists +# skips the recorder with a warning), giving preview-only. +cmake_k4a=() +[[ -n "${K4A_WRAPPER_DIR:-}" ]] && cmake_k4a=(-DK4A_WRAPPER_DIR="$K4A_WRAPPER_DIR") build="$root/build/linux_$(uname -m)" mkdir -p "$build" ( cd "$build" - # -DOB_BUILD_EXAMPLES=ON gives the k4a recorder sample used as recorder_bin. - cmake ../.. -DCMAKE_BUILD_TYPE=Release -DOB_BUILD_EXAMPLES=ON >&2 + cmake ../.. -DCMAKE_BUILD_TYPE=Release -DOB_BUILD_EXAMPLES=ON "${cmake_k4a[@]}" >&2 make -j"$(nproc)" >&2 ) @@ -74,9 +111,11 @@ if ! b=$(find_build "$root" 2>/dev/null); then !! La compilation du SDK ne s'est pas terminee comme attendu. !! Bâtis-le une fois a la main, puis relance install.sh en pointant dessus : -!! OB_SDK_ROOT= ./install.sh +!! OB_SDK_ROOT=$root ./install.sh EOF exit 1 fi echo ">> SDK bati: $b" >&2 +[[ -z "$(recorder_bin_in "$b")" ]] && \ + echo ">> NOTE: recorder non produit (wrapper K4A absent) — preview seule." >&2 emit "$root" "$b" diff --git a/scripts/install_deps.sh b/scripts/install_deps.sh index b71093c..16562dc 100755 --- a/scripts/install_deps.sh +++ b/scripts/install_deps.sh @@ -17,9 +17,12 @@ echo ">> Dependances pour JetPack ${JETPACK} (L4T ${L4T}, ${SOC})" # back to pip when the distro is too old to carry it. sudo apt-get update -qq sudo apt-get install -y --no-install-recommends \ - build-essential cmake pkg-config \ + build-essential cmake pkg-config git \ libusb-1.0-0-dev libudev-dev liblz4-dev libjpeg-turbo8-dev \ + libopencv-dev \ python3 python3-pip +# libopencv-dev: the MKV recorder links OpenCV (image handling). liblz4/libjpeg: +# the preview server. libusb/libudev: the SDK and the K4A wrapper. if ! python3 -c 'import websockets' 2>/dev/null; then echo ">> websockets absent — installation via pip"