From 23b30ac1e8bdbbad6b8352d2409eb98c1f5437c7 Mon Sep 17 00:00:00 2001 From: AngePierre Date: Thu, 10 Sep 2026 16:21:44 +0200 Subject: [PATCH] Add the MKV recorder so a node can Record, not only preview MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The recorder was the gap: it is FTF's own customised SDK example (a 1139-line K4A MKV writer), not a stock sample, and it lived only in one machine's build tree. Bring its source into the repo and build it as part of install. It writes K4A-format MKV — the format the rest of Ithaca reads — so it stays, and gets its libk4a from the Orbbec K4A wrapper rather than Microsoft's Azure Kinect SDK, which is set aside. build_k4a_wrapper.sh builds that wrapper; build_sdk.sh injects the recorder into the SDK examples tree and builds it there against the wrapper (it depends on the SDK's own cmake targets). Found by rpath at runtime, so no LD_LIBRARY_PATH. install.sh now builds wrapper -> SDK+recorder -> preview server, each reusing an existing build so the Nano is untouched. If the wrapper is unavailable the install still completes preview-only and says so. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 73 +- install.sh | 25 +- recorder/CMakeLists.txt | 115 +++ recorder/depth_color_k4arec_pipeline.cpp | 1139 ++++++++++++++++++++++ scripts/build_k4a_wrapper.sh | 67 ++ scripts/build_sdk.sh | 85 +- scripts/install_deps.sh | 5 +- 7 files changed, 1460 insertions(+), 49 deletions(-) create mode 100644 recorder/CMakeLists.txt create mode 100644 recorder/depth_color_k4arec_pipeline.cpp create mode 100755 scripts/build_k4a_wrapper.sh diff --git a/README.md b/README.md index ef3223c..a2a2367 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,11 @@ # Ithaca capture node The software a Jetson runs to join an Ithaca RGBD capture rig: a Python bridge that -puts the node on the WebSocket mesh and answers Preview / Record, and a C++ preview +puts the node on the WebSocket mesh and answers Preview / Record, a C++ preview server that streams each camera's colour (the sensor's own JPEG) and depth (its own -16-bit millimetres, LZ4-compressed) straight to the Unity viewer. +16-bit millimetres, LZ4-compressed) straight to the Unity viewer, and a C++ recorder +that writes K4A-format MKV takes on Record — the same format the rest of Ithaca +reads. One checkout, one command, on any Jetson: @@ -44,6 +46,9 @@ bridge/ server/ ithaca_rgbd_server.cpp the preview server CMakeLists.txt links a prebuilt libOrbbecSDK, no GStreamer +recorder/ + depth_color_k4arec_pipeline.cpp the MKV recorder — FTF source, not a stock example + CMakeLists.txt built inside the SDK examples tree, links the K4A wrapper systemd/ ithaca-bridge.service.in templated with the user and repo path usbfs-memory.service raises usbfs_memory_mb for the camera bandwidth @@ -52,41 +57,67 @@ udev/ scripts/ detect_platform.sh L4T / JetPack / SoC, as KEY=value install_deps.sh apt + websockets - build_sdk.sh reuse or build OrbbecSDK v2 for this platform + build_k4a_wrapper.sh reuse or build the Orbbec K4A wrapper (libk4a) + build_sdk.sh reuse or build OrbbecSDK v2 + inject/build the recorder build_server.sh cmake + make the preview server ``` +## The three C++ pieces, and how each is built + +- **Preview server** (`server/`) — our source, built standalone against the + prebuilt `libOrbbecSDK.so`. Simple. +- **Recorder** (`recorder/`) — FTF's own source, a customised SDK example, so it is + injected into the SDK's `examples/` tree and built there (it uses the SDK's own + cmake targets). It writes K4A-format MKV, and at runtime dlopens `libk4a.so` / + `libk4arecord.so` — provided by the **Orbbec K4A wrapper**, *not* Microsoft's + Azure Kinect SDK. Found by rpath, so no `LD_LIBRARY_PATH`. +- **Orbbec K4A wrapper** — Orbbec's `libk4a`, backed by their own SDK, cloned and + built by `build_k4a_wrapper.sh`. This is what makes Record work while setting the + Microsoft SDK aside. + ## What install.sh does 1. Detects L4T / JetPack / SoC (`scripts/detect_platform.sh`). -2. Installs build and run dependencies (no GStreamer — the direct route encodes no - video). -3. Locates a working OrbbecSDK build, or clones and builds one. A node that already - has a good build (the Nano) is left untouched. -4. Builds `ithaca_rgbd_server` against it. -5. Generates `bridge/config.json` for the current user — this is where all the +2. Installs build and run dependencies (OpenCV for the recorder; no GStreamer — the + direct route encodes no video). +3. Builds (or reuses) the Orbbec K4A wrapper — `libk4a` for the recorder. +4. Builds (or reuses) the OrbbecSDK, injecting and building the recorder example + against the wrapper. A node that already has a good build (the Nano) is left + untouched; the recorder is added to it only if missing. +5. Builds `ithaca_rgbd_server` against the SDK. +6. Generates `bridge/config.json` for the current user — this is where all the machine-specific paths live, so `bridge.py` itself stays unmodified. -6. Installs the udev rules and the `usbfs-memory` service, and adds the user to the +7. Installs the udev rules and the `usbfs-memory` service, and adds the user to the `video` group. -7. Generates and starts the `ithaca-bridge` systemd service. +8. Generates and starts the `ithaca-bridge` systemd service. + +If the K4A wrapper is unavailable, install.sh still completes: the node then does +**preview only**, and says so. Record needs the wrapper. Re-run it after a `git pull`: it rebuilds, regenerates the unit and config, and restarts the service. It keeps an existing `config.json` (delete it to regenerate) and an existing SDK build. -## The SDK is the one thing to watch on a new JetPack +## The two source builds to watch on a new JetPack -`build_sdk.sh` reuses an existing build when it finds one. On a machine with none it -clones the official OrbbecSDK v2 and builds it — this is the step most likely to -need attention on a JetPack it has not been tried on. If it fails, build the SDK by -hand once and re-run with its location: +Both scripts reuse an existing build when they find one. On a machine with none they +clone and build from source — the steps most likely to need attention on a JetPack +they have not been tried on: -```sh -OB_SDK_ROOT=/path/to/OrbbecSDK_v2 ./install.sh -``` +- **Orbbec K4A wrapper** — `OB_K4A_REPO` / `OB_K4A_REF` pin the source (default + `v1.10.4`). If it fails, build it by hand once and re-run with its location: + ```sh + K4A_WRAPPER_DIR=/path/to/OrbbecSDK-K4A-Wrapper ./install.sh + ``` +- **OrbbecSDK v2** — `OB_SDK_REPO` / `OB_SDK_REF` pin the source (default `main`). + Likewise: + ```sh + OB_SDK_ROOT=/path/to/OrbbecSDK_v2 ./install.sh + ``` -Override the source with `OB_SDK_REPO` / `OB_SDK_REF` to pin a tag known to build on -your JetPack. +Both can be passed together. Neither is Microsoft's Azure Kinect SDK — the recorder +gets its `libk4a` from the Orbbec wrapper, which is buildable from source across +platforms. ## Operating the node diff --git a/install.sh b/install.sh index 90d107a..5047a30 100755 --- a/install.sh +++ b/install.sh @@ -34,13 +34,30 @@ fi # ---- 1. dependencies ------------------------------------------------------ "$REPO/scripts/install_deps.sh" -# ---- 2. the SDK, then the server ------------------------------------------ -# build_sdk.sh reuses an existing build (leaving a working node untouched) or -# builds one; build_server.sh compiles ithaca_rgbd_server against it. -echo ">> Localisation / compilation de l'OrbbecSDK…" +# ---- 2. the K4A wrapper, the SDK (with the recorder), then the server ------ +# Three compiled pieces, in dependency order: +# - the Orbbec K4A wrapper provides libk4a, which the recorder needs; +# - the SDK builds the library AND the recorder example against that wrapper; +# - the preview server links the built SDK. +# Each script reuses an existing build, so a working node (the Nano) is untouched. +echo ">> Localisation / compilation du wrapper K4A (libk4a, Orbbec)…" +eval "$("$REPO/scripts/build_k4a_wrapper.sh")" || true +if [[ -n "${K4A_WRAPPER_DIR:-}" ]]; then + echo " K4A_WRAPPER_DIR = $K4A_WRAPPER_DIR" + export K4A_WRAPPER_DIR +else + echo " !! wrapper K4A indisponible — le nœud fera la PREVIEW mais pas le RECORD." +fi + +echo ">> Localisation / compilation de l'OrbbecSDK (+ recorder)…" eval "$("$REPO/scripts/build_sdk.sh")" echo " OB_SDK_ROOT = $OB_SDK_ROOT" echo " OB_SDK_BUILD = $OB_SDK_BUILD" +if [[ -n "${RECORDER_BIN:-}" ]]; then + echo " RECORDER_BIN = $RECORDER_BIN" +else + echo " !! recorder non produit — RECORD indisponible sur ce nœud." +fi echo ">> Compilation du serveur de preview…" SERVER_BIN=$("$REPO/scripts/build_server.sh") diff --git a/recorder/CMakeLists.txt b/recorder/CMakeLists.txt new file mode 100644 index 0000000..8ec3066 --- /dev/null +++ b/recorder/CMakeLists.txt @@ -0,0 +1,115 @@ +# Copyright (c) Orbbec Inc. / French Touch Factory. All Rights Reserved. +# Licensed under the MIT License. + +cmake_minimum_required(VERSION 3.10) +project(ob_stream_depth_color_k4arec_pipeline) + +if(WIN32) + set(K4A_WRAPPER_DIR + "C:/Ithaca_Projects/RGBD/Clone K4A Depot V2/OrbbecSDK-K4A-Wrapper" + CACHE PATH "Path to OrbbecSDK-K4A-Wrapper root directory") + # Windows: the wrapper's own out-of-source build dir is conventionally "Build", + # producing MSVC import libs (k4a.lib/k4arecord.lib) alongside the DLLs in bin/. + set(K4A_WRAPPER_BUILD_SUBDIR "Build") +else() + set(K4A_WRAPPER_DIR + "$ENV{HOME}/Downloads/OrbbecSDK-K4A-Wrapper" + CACHE PATH "Path to OrbbecSDK-K4A-Wrapper root directory") + # Linux: built with a lowercase "build" dir; shared libs land in lib/, not bin/. + set(K4A_WRAPPER_BUILD_SUBDIR "build") +endif() + +if(NOT EXISTS "${K4A_WRAPPER_DIR}/include/k4a/k4a.h") + message(WARNING + "${PROJECT_NAME}: K4A wrapper not found at '${K4A_WRAPPER_DIR}'. " + "Set K4A_WRAPPER_DIR to the OrbbecSDK-K4A-Wrapper root. Skipping.") + return() +endif() + +add_executable(${PROJECT_NAME} depth_color_k4arec_pipeline.cpp) +# C++14: the source uses digit-separator literals (2'000'000ULL); C++11 rejects them. +set_property(TARGET ${PROJECT_NAME} PROPERTY CXX_STANDARD 14) + +# rightW/rightH (Spout) and pinCpu/pinCore (SetProcessAffinityMask) are only read +# inside #ifdef _WIN32 blocks, so non-Windows builds see them as set-but-unused — +# harmless, but this SDK's own linux_config.cmake enables -Werror globally. +if(NOT WIN32) + target_compile_options(${PROJECT_NAME} PRIVATE -Wno-unused-but-set-variable) +endif() + +# SYSTEM: the K4A wrapper's public headers have GCC-pedantic issues (stray ';' +# after struct/enum decls) that -Wpedantic -Werror (set globally by this SDK's +# own linux_config.cmake) would otherwise turn into hard errors. -isystem makes +# GCC/Clang suppress warnings from headers found there. +target_include_directories(${PROJECT_NAME} SYSTEM PRIVATE + "${K4A_WRAPPER_DIR}/include" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/src/orbbec/include" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/src/orbbec/record/sdk/include" +) + +if(WIN32) + set(K4A_LIB "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4a.lib") + set(K4ARECORD_LIB "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4arecord.lib") +endif() + +target_link_libraries(${PROJECT_NAME} + ob::${OB_SDK_LIB_NAME} + ob::examples::utils + ${OpenCV_LIBS} + $<$:${K4A_LIB}> + $<$:${K4ARECORD_LIB}> + $<$:SpoutDX> + # Linux: libk4a.so/libk4arecord.so are dlopen'd (RTLD_DEEPBIND) at runtime + # instead of link-time linked — see the dlopen comment block near the top + # of depth_color_k4arec_pipeline.cpp for why. Needs libdl for dlopen/dlsym. + $<$>:dl> +) + +set_target_properties(${PROJECT_NAME} PROPERTIES FOLDER "examples") +if(MSVC) + set_target_properties(${PROJECT_NAME} PROPERTIES + VS_DEBUGGER_WORKING_DIRECTORY "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}") +endif() + +if(NOT WIN32) + # Linux: libk4a.so/libk4arecord.so aren't on the system search path — rpath + # the wrapper's build/lib so the binary finds them without LD_LIBRARY_PATH. + # BUILD_RPATH (not INSTALL_RPATH+BUILD_WITH_INSTALL_RPATH) so this ADDS to + # CMake's automatic build-tree rpath instead of replacing it — the automatic + # one is what makes libOrbbecSDK.so (built earlier in this same tree) found. + set_target_properties(${PROJECT_NAME} PROPERTIES + BUILD_RPATH "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/lib") +endif() + +if(WIN32) + add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4a.dll" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4arecord.dll" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/depthengine_2_0.dll" + $ + COMMENT "Copying K4A wrapper DLLs" + ) +endif() + +if(WIN32 AND TARGET opencv_world) + add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different + "$" + $ + COMMENT "Copying OpenCV DLL" + ) +endif() + +install(TARGETS ${PROJECT_NAME} RUNTIME DESTINATION bin) +if(WIN32) + install(FILES + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4a.dll" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/k4arecord.dll" + "${K4A_WRAPPER_DIR}/${K4A_WRAPPER_BUILD_SUBDIR}/bin/depthengine_2_0.dll" + DESTINATION bin + ) +endif() +if(WIN32 AND TARGET opencv_world) + install(FILES "$" DESTINATION bin) +endif() diff --git a/recorder/depth_color_k4arec_pipeline.cpp b/recorder/depth_color_k4arec_pipeline.cpp new file mode 100644 index 0000000..3ca4f13 --- /dev/null +++ b/recorder/depth_color_k4arec_pipeline.cpp @@ -0,0 +1,1139 @@ +// Copyright (c) Orbbec Inc. / French Touch Factory. All Rights Reserved. +// Licensed under the MIT License. +// +// ob_stream_depth_color + K4A MKV recording using ob::Pipeline. +// ob::Pipeline with enableFrameSync() delivers pre-matched color+depth(+IR) +// framesets — the same approach used by the K4A SDK internally. +// This eliminates manual timestamp matching and improves resilience to +// Femto Bolt 25-bit timestamp overflow events at the cost of ~100 ms +// pipeline sync latency vs the lowlat variant. +// 'R' toggles recording; Ctrl+C or ESC/Q to exit. +// +// Accepts the Azure Kinect k4arecorder option set (--color-mode, --depth-mode, +// --rate, --record-length, and the manual color-control flags) so it can act as +// a drop-in MKV recorder for Orbbec Femto Bolt / Mega devices. + +#include +#include "utils.hpp" +#include "utils_opencv.hpp" + +#include +#include + +#ifdef _WIN32 +#include +#include +#include +#else +// --------------------------------------------------------------------------- +// Linux only: dlopen libk4arecord_v2.so instead of linking it directly. +// +// libk4arecord_v2.so est notre bibliotheque d'ecriture MKV compatible K4A, batie +// sur les sources d'ecriture du wrapper (record.cpp + matroska_write.cpp) mais +// liee a un shim de donnees pures (k4a_shim.cpp) au lieu de libk4a.so. Elle ne +// depend d'AUCUN OrbbecSDK : ce process ne charge donc plus que la v2.8.6 liee +// directement ci-dessous (ob::Pipeline / ob::Context). +// +// Historique : on chargeait auparavant libk4a.so + libk4arecord.so, qui +// embarquent leur propre copie d'OrbbecSDK v1.10.18. Les deux SDK exportent les +// memes symboles C non versionnes (ex. ob_create_frame_from_buffer) avec des +// encodages d'enums differents, ce qui imposait RTLD_DEEPBIND pour que chaque +// wrapper reste appaire a sa copie. Cette v1 coutait aussi un ob_create_frameset() +// par frame -- soit une creation/destruction de Context avec reenumeration USB +// complete a chaque image. Plus de v1, plus de DEEPBIND, plus de churn. +#include + +namespace k4a_dl { + +static void *g_k4arec = nullptr; // libk4arecord_v2.so : record + shim k4a + +// One function pointer per K4A / K4A-record API this file calls. The +// decltype(&::fn) keeps each pointer's signature in lockstep with the real +// declaration in k4a.h / k4arecord/record.h without repeating it by hand. +static decltype(&::k4a_capture_create) k4a_capture_create = nullptr; +static decltype(&::k4a_capture_release) k4a_capture_release = nullptr; +static decltype(&::k4a_capture_set_color_image) k4a_capture_set_color_image = nullptr; +static decltype(&::k4a_capture_set_depth_image) k4a_capture_set_depth_image = nullptr; +static decltype(&::k4a_capture_set_ir_image) k4a_capture_set_ir_image = nullptr; +static decltype(&::k4a_image_create_from_buffer) k4a_image_create_from_buffer = nullptr; +static decltype(&::k4a_image_set_device_timestamp_usec) k4a_image_set_device_timestamp_usec = nullptr; +static decltype(&::k4a_image_release) k4a_image_release = nullptr; +static decltype(&::k4a_record_create) k4a_record_create = nullptr; +static decltype(&::k4a_record_add_tag) k4a_record_add_tag = nullptr; +static decltype(&::k4a_record_add_attachment) k4a_record_add_attachment = nullptr; +static decltype(&::k4a_record_write_header) k4a_record_write_header = nullptr; +static decltype(&::k4a_record_write_capture) k4a_record_write_capture = nullptr; +static decltype(&::k4a_record_flush) k4a_record_flush = nullptr; +static decltype(&::k4a_record_close) k4a_record_close = nullptr; + +template static void bind(void *handle, const char *name, T &fnOut) { + void *sym = dlsym(handle, name); + if(!sym) { + std::cerr << "Fatal: symbol '" << name << "' not found: " << dlerror() << std::endl; + std::exit(EXIT_FAILURE); + } + fnOut = reinterpret_cast(sym); +} + +static void *openLib(const char *soname) { + // RTLD_DEEPBIND n'est plus necessaire : il n'y a plus qu'un seul OrbbecSDK + // dans le process, donc plus de collision de symboles a arbitrer. + void *h = dlopen(soname, RTLD_NOW); + if(!h) { + std::cerr << "Fatal: dlopen(" << soname << ") failed: " << dlerror() << std::endl; + std::exit(EXIT_FAILURE); + } + return h; +} + +static void init() { + if(g_k4arec) return; // idempotent + g_k4arec = openLib("libk4arecord_v2.so"); + + bind(g_k4arec, "k4a_capture_create", k4a_capture_create); + bind(g_k4arec, "k4a_capture_release", k4a_capture_release); + bind(g_k4arec, "k4a_capture_set_color_image", k4a_capture_set_color_image); + bind(g_k4arec, "k4a_capture_set_depth_image", k4a_capture_set_depth_image); + bind(g_k4arec, "k4a_capture_set_ir_image", k4a_capture_set_ir_image); + bind(g_k4arec, "k4a_image_create_from_buffer", k4a_image_create_from_buffer); + bind(g_k4arec, "k4a_image_set_device_timestamp_usec", k4a_image_set_device_timestamp_usec); + bind(g_k4arec, "k4a_image_release", k4a_image_release); + + bind(g_k4arec, "k4a_record_create", k4a_record_create); + bind(g_k4arec, "k4a_record_add_tag", k4a_record_add_tag); + bind(g_k4arec, "k4a_record_add_attachment", k4a_record_add_attachment); + bind(g_k4arec, "k4a_record_write_header", k4a_record_write_header); + bind(g_k4arec, "k4a_record_write_capture", k4a_record_write_capture); + bind(g_k4arec, "k4a_record_flush", k4a_record_flush); + bind(g_k4arec, "k4a_record_close", k4a_record_close); +} + +} // namespace k4a_dl + +// Redirect every call below (textually, after this point) to the dlopen'd, +// DEEPBIND-loaded copies instead of the statically-linked symbols. +#define k4a_capture_create k4a_dl::k4a_capture_create +#define k4a_capture_release k4a_dl::k4a_capture_release +#define k4a_capture_set_color_image k4a_dl::k4a_capture_set_color_image +#define k4a_capture_set_depth_image k4a_dl::k4a_capture_set_depth_image +#define k4a_capture_set_ir_image k4a_dl::k4a_capture_set_ir_image +#define k4a_image_create_from_buffer k4a_dl::k4a_image_create_from_buffer +#define k4a_image_set_device_timestamp_usec k4a_dl::k4a_image_set_device_timestamp_usec +#define k4a_image_release k4a_dl::k4a_image_release +#define k4a_record_create k4a_dl::k4a_record_create +#define k4a_record_add_tag k4a_dl::k4a_record_add_tag +#define k4a_record_add_attachment k4a_dl::k4a_record_add_attachment +#define k4a_record_write_header k4a_dl::k4a_record_write_header +#define k4a_record_write_capture k4a_dl::k4a_record_write_capture +#define k4a_record_flush k4a_dl::k4a_record_flush +#define k4a_record_close k4a_dl::k4a_record_close +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// ---- Globals --------------------------------------------------------------- + +static std::mutex g_recMutex; +static k4a_record_t g_recording = nullptr; +static std::atomic g_isRecording { false }; +static std::atomic g_pendingReset{ false }; +static uint64_t g_frameCount = 0; + +// Color image format written to the MKV (set from --color-mode; matches the +// k4a_device_configuration_t color_format so k4a_record_write_capture accepts it). +static k4a_image_format_t g_k4aColorFormat = K4A_IMAGE_FORMAT_COLOR_MJPG; + +// Timestamp handling (same logic as lowlat variant). +static uint64_t g_tsOffset = 0; +static uint64_t g_lastWrittenTs = 0; +static uint64_t g_writeFailures = 0; + +// Femto Bolt 25-bit counter overflow compensation. +static constexpr uint64_t kTsOverflowPeriod = 1ULL << 25; +static constexpr uint64_t kTsOverflowHalfPeriod = kTsOverflowPeriod >> 1; +static uint64_t g_lastRawTs = 0; +static uint64_t g_rawTsEpoch = 0; + +static constexpr uint64_t kMaxJumpUs = 2'000'000ULL; +static constexpr uint64_t kResetThresholdUs = 1'000'000ULL; + +// Subordinate: how long to wait for the master's sync pulse before force-starting +// "anyway" (avoids an empty MKV if the cable is bad). With many cameras the whole +// rig can take >5 s to come up, so this is configurable via --sync-timeout. +// 0 = wait forever (never start unsynced — safest for a guaranteed-sync rig). +static int64_t g_pendingResetTimeoutMs = 30000; +static std::chrono::steady_clock::time_point g_pendingResetStart; + +static std::shared_ptr g_device; +static std::atomic g_exitRequested{ false }; +static k4a_wired_sync_mode_t g_syncMode = K4A_WIRED_SYNC_MODE_STANDALONE; + +// Display frames — updated from pipeline callback, read by main thread. +static std::mutex g_dispMutex; +static std::condition_variable g_newFrameCv; +static std::shared_ptr g_dispColor; +static std::shared_ptr g_dispDepth; +static std::shared_ptr g_dispIR; +static bool g_hasNewFrame = false; + +// ---- Stream-mode descriptors (mirror the K4A k4arecorder mode strings) ----- + +struct ColorModeInfo { + bool enabled = false; + int width = 0; + int height = 0; + OBFormat obFormat = OB_FORMAT_MJPG; + k4a_image_format_t k4aFormat = K4A_IMAGE_FORMAT_COLOR_MJPG; + k4a_color_resolution_t k4aRes = K4A_COLOR_RESOLUTION_OFF; +}; + +struct DepthModeInfo { + bool depthEnabled = false; // produces a DEPTH stream + bool irEnabled = false; // produces an IR stream + int width = 0; + int height = 0; + k4a_depth_mode_t k4aMode = K4A_DEPTH_MODE_OFF; +}; + +static std::string toUpper(std::string s) { + for(char &c : s) c = static_cast(::toupper(static_cast(c))); + return s; +} + +// 2160p / 1536p / 1440p / 1080p / 720p / 720p_NV12 / 720p_YUY2 / OFF +static bool parseColorMode(const std::string &in, ColorModeInfo &o) { + const std::string s = toUpper(in); + o = ColorModeInfo{}; + if(s == "OFF") { o.enabled = false; o.k4aRes = K4A_COLOR_RESOLUTION_OFF; return true; } + o.enabled = true; + o.obFormat = OB_FORMAT_MJPG; + o.k4aFormat = K4A_IMAGE_FORMAT_COLOR_MJPG; + if(s == "2160P") { o.width = 3840; o.height = 2160; o.k4aRes = K4A_COLOR_RESOLUTION_2160P; } + else if(s == "1536P") { o.width = 2048; o.height = 1536; o.k4aRes = K4A_COLOR_RESOLUTION_1536P; } + else if(s == "1440P") { o.width = 2560; o.height = 1440; o.k4aRes = K4A_COLOR_RESOLUTION_1440P; } + else if(s == "1080P") { o.width = 1920; o.height = 1080; o.k4aRes = K4A_COLOR_RESOLUTION_1080P; } + else if(s == "720P") { o.width = 1280; o.height = 720; o.k4aRes = K4A_COLOR_RESOLUTION_720P; } + else if(s == "720P_NV12") { o.width = 1280; o.height = 720; o.k4aRes = K4A_COLOR_RESOLUTION_720P; + o.obFormat = OB_FORMAT_NV12; o.k4aFormat = K4A_IMAGE_FORMAT_COLOR_NV12; } + else if(s == "720P_YUY2") { o.width = 1280; o.height = 720; o.k4aRes = K4A_COLOR_RESOLUTION_720P; + o.obFormat = OB_FORMAT_YUY2; o.k4aFormat = K4A_IMAGE_FORMAT_COLOR_YUY2; } + else return false; + return true; +} + +// NFOV_2X2BINNED / NFOV_UNBINNED / WFOV_2X2BINNED / WFOV_UNBINNED / PASSIVE_IR / OFF +static bool parseDepthMode(const std::string &in, DepthModeInfo &o) { + const std::string s = toUpper(in); + o = DepthModeInfo{}; + if(s == "OFF") { o.k4aMode = K4A_DEPTH_MODE_OFF; return true; } + if(s == "NFOV_2X2BINNED") { o.width = 320; o.height = 288; o.depthEnabled = true; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_NFOV_2X2BINNED; return true; } + if(s == "NFOV_UNBINNED") { o.width = 640; o.height = 576; o.depthEnabled = true; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_NFOV_UNBINNED; return true; } + if(s == "WFOV_2X2BINNED") { o.width = 512; o.height = 512; o.depthEnabled = true; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_WFOV_2X2BINNED; return true; } + if(s == "WFOV_UNBINNED") { o.width = 1024; o.height = 1024; o.depthEnabled = true; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_WFOV_UNBINNED; return true; } + if(s == "PASSIVE_IR") { o.width = 1024; o.height = 1024; o.depthEnabled = false; o.irEnabled = true; o.k4aMode = K4A_DEPTH_MODE_PASSIVE_IR; return true; } + return false; +} + +static bool parseRate(int fps, k4a_fps_t &o) { + switch(fps) { + case 30: o = K4A_FRAMES_PER_SECOND_30; return true; + case 25: o = K4A_FRAMES_PER_SECOND_25; return true; + case 15: o = K4A_FRAMES_PER_SECOND_15; return true; + case 5: o = K4A_FRAMES_PER_SECOND_5; return true; + default: return false; + } +} + +#ifdef _WIN32 +static BOOL WINAPI ctrlHandler(DWORD) { + g_exitRequested = true; + g_newFrameCv.notify_all(); + return TRUE; +} +#else +// Linux equivalent of ctrlHandler above. Without this, SIGINT/SIGTERM (e.g. a +// process supervisor like the WebSocket bridge asking this process to stop) +// fall through to the OS default action — an immediate kill with no MKV +// finalization (closeRecording()/pipeline->stop() never run) and the camera +// left in whatever state it was mid-stream. Only sets the atomic flag: the +// rest of the handler body above (notify_all on a condition_variable) is not +// async-signal-safe, so it's left to the main loop's wait_for() timeout +// (<=200 ms) to notice g_exitRequested instead. +static void posixSignalHandler(int) { + g_exitRequested = true; +} +#endif + +static std::string makeTimestampedFilename() { + std::time_t t = std::time(nullptr); + std::tm *tm = std::localtime(&t); + std::ostringstream ss; + ss << "capture_" << std::put_time(tm, "%Y%m%d_%H%M%S") << ".mkv"; + return ss.str(); +} + +static void noopBufferRelease(void* /*buf*/, void* /*ctx*/) {} + +struct CalibAccum { + std::vector buf; + uint32_t totalSize = 0; + bool done = false; +}; + +static void calibDataCallback(ob_data_tran_state state, ob_data_chunk *chunk, void *userData) { + if(!chunk) return; + auto *acc = static_cast(userData); + if(state == DATA_TRAN_STAT_TRANSFERRING && chunk->data && chunk->size > 0) { + if(chunk->fullDataSize > 0 && acc->buf.size() < chunk->fullDataSize) { + acc->buf.resize(chunk->fullDataSize); + acc->totalSize = chunk->fullDataSize; + } + size_t end = static_cast(chunk->offset) + chunk->size; + if(acc->buf.size() < end) acc->buf.resize(end); + std::memcpy(acc->buf.data() + chunk->offset, chunk->data, chunk->size); + } + else if(state == DATA_TRAN_STAT_DONE) { + if(acc->totalSize > 0 && acc->buf.size() > acc->totalSize) + acc->buf.resize(acc->totalSize); + if(!acc->buf.empty() && acc->buf.size() % 2 == 1 && acc->buf.back() != 0x7d) + acc->buf.back() = 0x7d; + acc->done = true; + } +} + +static cv::Mat decodeColorFrame(const std::shared_ptr &frame) { + if(!frame) return {}; + auto vf = frame->as(); + uint32_t w = vf->getWidth(), h = vf->getHeight(); + const uint8_t *data = static_cast(frame->getData()); + cv::Mat colorBGR; + switch(frame->getFormat()) { + case OB_FORMAT_RGB: { cv::Mat s(h, w, CV_8UC3, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_RGB2BGR); break; } + case OB_FORMAT_BGR: colorBGR = cv::Mat(h, w, CV_8UC3, (void*)data).clone(); break; + case OB_FORMAT_BGRA: { cv::Mat s(h, w, CV_8UC4, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_BGRA2BGR); break; } + case OB_FORMAT_RGBA: { cv::Mat s(h, w, CV_8UC4, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_RGBA2BGR); break; } + case OB_FORMAT_YUYV: + case OB_FORMAT_YUY2: { cv::Mat s(h, w, CV_8UC2, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_YUV2BGR_YUYV); break; } + case OB_FORMAT_NV12: { cv::Mat s(h * 3 / 2, w, CV_8UC1, (void*)data); cv::cvtColor(s, colorBGR, cv::COLOR_YUV2BGR_NV12); break; } + case OB_FORMAT_MJPG: { + cv::Mat m(1, static_cast(vf->getDataSize()), CV_8UC1, (void*)data); + colorBGR = cv::imdecode(m, cv::IMREAD_COLOR); + break; + } + default: break; + } + return colorBGR; +} + +static void resetDeviceTimestamp() { + if(!g_device) return; + try { + OBDeviceTimestampResetConfig cfg{}; + cfg.enable = true; + cfg.timestamp_reset_delay_us = 0; + cfg.timestamp_reset_signal_output_enable = (g_syncMode == K4A_WIRED_SYNC_MODE_MASTER); + g_device->setTimestampResetConfig(cfg); + + if(g_syncMode == K4A_WIRED_SYNC_MODE_SUBORDINATE) { + std::cout << "Subordinate ready — awaiting timestamp reset pulse from master." << std::endl; + return; + } + if(g_device->isPropertySupported(OB_PROP_TIMER_RESET_SIGNAL_BOOL, OB_PERMISSION_WRITE)) { + g_device->setBoolProperty(OB_PROP_TIMER_RESET_SIGNAL_BOOL, true); + std::cout << "Device timestamp reset (prop 104) + pulse sent to subordinates." << std::endl; + } + else { + g_device->timestampReset(); + std::cout << "Device timestamp reset + pulse sent to subordinates." << std::endl; + } + } + catch(ob::Error &e) { + std::cerr << "Warning: timestamp reset failed: " << e.what() << std::endl; + } +} + +static void writeCapture(std::shared_ptr colorFrame, + std::shared_ptr depthFrame, + std::shared_ptr irFrame = nullptr) { + if(!g_isRecording) return; + std::lock_guard lock(g_recMutex); + if(!g_recording) return; + + uint64_t rawTs = colorFrame ? colorFrame->getTimeStampUs() + : depthFrame ? depthFrame->getTimeStampUs() + : irFrame ? irFrame->getTimeStampUs() : 0; + + // Timestamp drop compensation: fires when the device counter wraps (25-bit overflow, + // ~33.55 s) OR when resetDeviceTimestamp() resets a long-running counter back to near-0. + // IMPORTANT: always update g_lastRawTs to rawTs after detection — if we only update on + // rawTs > g_lastRawTs the condition stays true every subsequent frame, accumulating + // a spurious epoch increment per frame and making adjTs jump > kMaxJumpUs every time. + if(g_lastRawTs > 0 && rawTs + kTsOverflowHalfPeriod < g_lastRawTs) { + g_rawTsEpoch += kTsOverflowPeriod; + std::cout << "[INFO] Device ts overflow/reset (rawTs=" << rawTs + << " prev=" << g_lastRawTs << " epoch=" << g_rawTsEpoch << " us)\n"; + g_lastRawTs = rawTs; + } else if(rawTs > g_lastRawTs) { + g_lastRawTs = rawTs; + } + uint64_t effectiveRawTs = rawTs + g_rawTsEpoch; + + uint64_t adjTs; + if(g_syncMode == K4A_WIRED_SYNC_MODE_STANDALONE) { + if(g_tsOffset == 0) g_tsOffset = effectiveRawTs; + adjTs = (effectiveRawTs >= g_tsOffset) ? (effectiveRawTs - g_tsOffset) : 0; + } + else { + adjTs = effectiveRawTs; + } + + if(g_lastWrittenTs > 0 && adjTs > g_lastWrittenTs + kMaxJumpUs) return; + if(adjTs < g_lastWrittenTs) return; + + k4a_capture_t cap = nullptr; + if(K4A_FAILED(k4a_capture_create(&cap))) return; + + if(colorFrame) { + auto vf = colorFrame->as(); + int w = static_cast(vf->getWidth()); + int h = static_cast(vf->getHeight()); + size_t sz = colorFrame->getDataSize(); + const void *raw = colorFrame->getData(); + uint8_t *buf = static_cast(const_cast(raw)); + // Stride per K4A color format (0 = compressed/MJPG). + int stride = 0; + switch(g_k4aColorFormat) { + case K4A_IMAGE_FORMAT_COLOR_NV12: stride = w; break; + case K4A_IMAGE_FORMAT_COLOR_YUY2: stride = w * 2; break; + case K4A_IMAGE_FORMAT_COLOR_BGRA32: stride = w * 4; break; + default: stride = 0; break; // MJPG + } + k4a_image_t img = nullptr; + if(K4A_SUCCEEDED(k4a_image_create_from_buffer( + g_k4aColorFormat, w, h, stride, buf, sz, + noopBufferRelease, nullptr, &img))) { + k4a_image_set_device_timestamp_usec(img, adjTs); + k4a_capture_set_color_image(cap, img); + k4a_image_release(img); + } + } + if(depthFrame) { + auto vf = depthFrame->as(); + int w = static_cast(vf->getWidth()); + int h = static_cast(vf->getHeight()); + size_t sz = depthFrame->getDataSize(); + const void *raw = depthFrame->getData(); + uint8_t *buf = static_cast(const_cast(raw)); + k4a_image_t img = nullptr; + if(K4A_SUCCEEDED(k4a_image_create_from_buffer( + K4A_IMAGE_FORMAT_DEPTH16, w, h, w * 2, buf, sz, + noopBufferRelease, nullptr, &img))) { + k4a_image_set_device_timestamp_usec(img, adjTs); + k4a_capture_set_depth_image(cap, img); + k4a_image_release(img); + } + } + if(irFrame) { + auto vf = irFrame->as(); + int w = static_cast(vf->getWidth()); + int h = static_cast(vf->getHeight()); + size_t sz = irFrame->getDataSize(); + const void *raw = irFrame->getData(); + uint8_t *buf = static_cast(const_cast(raw)); + k4a_image_t img = nullptr; + if(K4A_SUCCEEDED(k4a_image_create_from_buffer( + K4A_IMAGE_FORMAT_IR16, w, h, w * 2, buf, sz, + noopBufferRelease, nullptr, &img))) { + k4a_image_set_device_timestamp_usec(img, adjTs); + k4a_capture_set_ir_image(cap, img); + k4a_image_release(img); + } + } + + if(K4A_FAILED(k4a_record_write_capture(g_recording, cap))) { + ++g_writeFailures; + if(g_writeFailures == 1 || g_writeFailures % 30 == 0) + std::cerr << "[WARN] k4a_record_write_capture failed (ts=" << adjTs + << " us, total=" << g_writeFailures << ")\n"; + } else { + g_lastWrittenTs = adjTs; + g_frameCount++; + } + k4a_capture_release(cap); +} + +// ---- Main ------------------------------------------------------------------ + +int main(int argc, char *argv[]) try { +#ifndef _WIN32 + k4a_dl::init(); // dlopen libk4arecord_v2.so — see comment near the top of this file. +#endif + + std::string outputPath; + int deviceIndex = 0; + bool showPreview = false; + bool useSpout = false; + bool recordIR = true; + uint32_t syncDelayUs = 0; + std::string argSerial; + std::string argFirmware; + bool pinCpu = false; // --pin-cpu : affinite process = coeur d'indice deviceIndex + // --pin-core N : coeur d'affinite explicite. Avant, l'affinite derivait de --device ; la + // selection se faisant maintenant par --serial, le coeur doit etre donne a part (sinon tous + // les process se retrouvent sur le coeur 0 = la contention MF que ce pinning evite). + int pinCore = -1; + + // k4arecorder-compatible capture parameters. + int recordLengthSec = 0; // 0 = infinite + std::string colorModeStr = "1080p"; + std::string depthModeStr = "NFOV_UNBINNED"; + int rateFps = 30; + int exposureUs = -1; // -1 = auto exposure + int brightness = -1; // -1 = auto/unset + int contrast = -1; + int saturation = -1; + int sharpness = -1; + int whitebalance = -1; // -1 = auto white balance + int gain = -1; // -1 = auto/unset + + for(int i = 1; i < argc; i++) { + std::string arg = argv[i]; + if(arg == "--help" || arg == "-h") { + std::cout << "Usage: " << argv[0] << " [options] [output.mkv]\n" + << "\n" + << "Capture options (Azure Kinect k4arecorder compatible):\n" + << " -l, --record-length N Limit the recording to N seconds (default: infinite)\n" + << " -c, --color-mode M Set the color sensor mode (default: 1080p). Options:\n" + << " 2160p, 1536p, 1440p, 1080p, 720p, 720p_NV12, 720p_YUY2, OFF\n" + << " -d, --depth-mode M Set the depth sensor mode (default: NFOV_UNBINNED). Options:\n" + << " NFOV_2X2BINNED, NFOV_UNBINNED, WFOV_2X2BINNED, WFOV_UNBINNED, PASSIVE_IR, OFF\n" + << " -r, --rate N Camera frame rate in FPS (default: 30). Options: 30, 25, 15, 5\n" + << " -e, --exposure-control N Manual RGB exposure 100..409500 us (default: auto)\n" + << " -b, --brightness N Manual brightness 0..100 (default: auto)\n" + << " -t, --contrast N Manual contrast 0..100 (default: auto)\n" + << " -s, --saturation N Manual saturation 0..100 (default: auto)\n" + << " -p, --sharpness N Manual sharpness 0..100 (default: auto)\n" + << " -w, --whitebalance N Manual white balance 2000..11000 (default: auto)\n" + << " -g, --gain N Manual gain 1..255 (default: auto)\n" + << "\n" + << "Device / output options:\n" + << " --device N Select device by index (default: 0)\n" + << " --preview Open preview window\n" + << " --spout Enable Spout output\n" + << " -m, --master Set device as sync master\n" + << " --subordinate Set device as sync subordinate\n" + << " --sub Alias for --subordinate\n" + << " --standalone Free-running, no sync (default)\n" + << " --sync-delay N Subordinate trigger delay in microseconds\n" + << " --delay N Alias for --sync-delay\n" + << " --sync-timeout N Subordinate: seconds to wait for the master pulse\n" + << " before starting anyway (default 30; 0 = wait forever)\n" + << " --pin-cpu Pin this process to CPU core (see --pin-core)\n" + << " --pin-core N Core index used by --pin-cpu (default: --device)\n" + << " --serial SN SELECT the device by serial; also MKV metadata\n" + << " --firmware N Override firmware version in MKV metadata\n" + << " --no-ir Disable IR recording (reduces disk write)\n" + << "\n" + << "Note: -d/-p/-s follow k4arecorder (depth-mode/sharpness/saturation);\n" + << " --device/--preview/--spout have no short form to avoid the clash.\n" + << "\n" + << "Uses ob::Pipeline with frame sync for robust color+depth(+IR) matching.\n" + << "If output.mkv is given, recording starts immediately.\n" + << "Without --preview the program runs headless; Ctrl+C to stop.\n"; + return 0; + } + else if(arg == "--device" && i + 1 < argc) { deviceIndex = std::stoi(argv[++i]); } + else if(arg == "--preview") { showPreview = true; } + else if(arg == "--spout") { useSpout = true; } + else if(arg == "--master" || arg == "-m") { g_syncMode = K4A_WIRED_SYNC_MODE_MASTER; } + else if(arg == "--subordinate" || arg == "--sub") { g_syncMode = K4A_WIRED_SYNC_MODE_SUBORDINATE; } + else if(arg == "--standalone") { g_syncMode = K4A_WIRED_SYNC_MODE_STANDALONE; } + else if((arg == "--sync-delay" || arg == "--delay") && i + 1 < argc) { syncDelayUs = static_cast(std::stoul(argv[++i])); } + else if(arg == "--sync-timeout" && i + 1 < argc) { int s = std::stoi(argv[++i]); g_pendingResetTimeoutMs = (s <= 0) ? 0 : static_cast(s) * 1000; } + else if(arg == "--pin-cpu") { pinCpu = true; } + else if(arg == "--pin-core" && i + 1 < argc) { pinCore = std::stoi(argv[++i]); } + else if(arg == "--serial" && i + 1 < argc) { argSerial = argv[++i]; } + else if(arg == "--firmware" && i + 1 < argc) { argFirmware = argv[++i]; } + else if(arg == "--no-ir") { recordIR = false; } + else if((arg == "--record-length" || arg == "-l") && i + 1 < argc) { recordLengthSec = std::stoi(argv[++i]); } + else if((arg == "--color-mode" || arg == "-c") && i + 1 < argc) { colorModeStr = argv[++i]; } + else if((arg == "--depth-mode" || arg == "-d") && i + 1 < argc) { depthModeStr = argv[++i]; } + else if((arg == "--rate" || arg == "-r") && i + 1 < argc) { rateFps = std::stoi(argv[++i]); } + else if((arg == "--exposure-control" || arg == "-e") && i + 1 < argc) { exposureUs = std::stoi(argv[++i]); } + else if((arg == "--brightness" || arg == "-b") && i + 1 < argc) { brightness = std::stoi(argv[++i]); } + else if((arg == "--contrast" || arg == "-t") && i + 1 < argc) { contrast = std::stoi(argv[++i]); } + else if((arg == "--saturation" || arg == "-s") && i + 1 < argc) { saturation = std::stoi(argv[++i]); } + else if((arg == "--sharpness" || arg == "-p") && i + 1 < argc) { sharpness = std::stoi(argv[++i]); } + else if((arg == "--whitebalance" || arg == "-w") && i + 1 < argc) { whitebalance = std::stoi(argv[++i]); } + else if((arg == "--gain" || arg == "-g") && i + 1 < argc) { gain = std::stoi(argv[++i]); } + else if(arg[0] != '-' && outputPath.empty()) { outputPath = arg; } + } + + // ---- Optional CPU pinning: affinity = core index deviceIndex ----------- + // Pins the whole process (all its threads) to a single core = --device. + // Simple first step so the N camera processes don't fight over the same cores. + // NB: 1 core per process is restrictive (this process has several threads: + // depth engine, pipeline, writer, callbacks) — extend to a block of cores + // later if a single core saturates. +#ifdef _WIN32 + if(pinCpu) { + SYSTEM_INFO si; GetSystemInfo(&si); + int nCores = static_cast(si.dwNumberOfProcessors); + int core = (pinCore >= 0) ? pinCore : deviceIndex; // --pin-core wins, --device is the legacy default + if(core >= 0 && core < nCores && core < 64) { + DWORD_PTR mask = (static_cast(1) << core); + if(SetProcessAffinityMask(GetCurrentProcess(), mask)) + std::cout << "CPU affinity: pinned to core " << core << std::endl; + else + std::cerr << "Warning: SetProcessAffinityMask failed for core " << core << std::endl; + } else { + std::cerr << "Warning: --pin-cpu: core index " << core + << " out of range (" << nCores << " cores) — not pinned." << std::endl; + } + } +#endif + + // ---- Resolve capture modes -------------------------------------------- + ColorModeInfo color; + DepthModeInfo depth; + k4a_fps_t k4aFps; + if(!parseColorMode(colorModeStr, color)) { std::cerr << "Invalid --color-mode '" << colorModeStr << "'.\n"; return EXIT_FAILURE; } + if(!parseDepthMode(depthModeStr, depth)) { std::cerr << "Invalid --depth-mode '" << depthModeStr << "'.\n"; return EXIT_FAILURE; } + if(!parseRate(rateFps, k4aFps)) { std::cerr << "Invalid --rate '" << rateFps << "' (use 30, 25, 15 or 5).\n"; return EXIT_FAILURE; } + + bool hasColor = color.enabled; + bool hasDepth = depth.depthEnabled; + bool hasIR = depth.irEnabled && recordIR; + if(!hasColor && !hasDepth && !hasIR) { + std::cerr << "Nothing to stream: --color-mode and --depth-mode are both OFF (or IR disabled).\n"; + return EXIT_FAILURE; + } + g_k4aColorFormat = color.k4aFormat; + + // ---- Device selection -------------------------------------------------- + ob::Context ctx; + auto deviceList = ctx.queryDeviceList(); + uint32_t deviceCount = deviceList->deviceCount(); + std::cout << "Found " << deviceCount << " device(s)." << std::endl; + if(deviceCount == 0) { std::cerr << "No device connected.\n"; return EXIT_FAILURE; } + + // --serial SELECTS the device inside this SDK's own enumeration. Indexes are per-SDK, so the + // host cannot map them across camera brands - the serial is the only stable key it has. No + // fallback on failure: silently opening --device (0 by default) would start two processes on + // the same camera in a multi-camera rig. + if(!argSerial.empty()) { + bool found = false; + for(uint32_t i = 0; i < deviceCount; i++) { + const char *sn = deviceList->getSerialNumber(i); + if(sn && argSerial == sn) { deviceIndex = static_cast(i); found = true; break; } + } + if(!found) { + std::cerr << "Serial '" << argSerial << "' not found among " << deviceCount + << " device(s).\n"; + return EXIT_FAILURE; + } + } + if(static_cast(deviceIndex) >= deviceCount) { + std::cerr << "Device index " << deviceIndex << " out of range.\n"; return EXIT_FAILURE; + } + + std::string deviceSerial, deviceFirmwareCOLOR, deviceFirmwareDEPTH, deviceName; + { + const char *sn = deviceList->getSerialNumber(static_cast(deviceIndex)); + const char *nm = deviceList->getName(static_cast(deviceIndex)); + if(sn && *sn) deviceSerial = sn; + if(nm && *nm) deviceName = nm; + } + if(!argSerial.empty()) deviceSerial = argSerial; + if(!argFirmware.empty()) { deviceFirmwareCOLOR = argFirmware; deviceFirmwareDEPTH = argFirmware; } + std::cout << "Device: " << deviceName << " S/N: '" << deviceSerial << "'" << std::endl; + std::cout << "Config: color=" << (hasColor ? colorModeStr : std::string("OFF")) + << " depth=" << depthModeStr + << " ir=" << (hasIR ? "on" : "off") + << " rate=" << rateFps << " fps" << std::endl; + + g_device = deviceList->getDevice(static_cast(deviceIndex)); + + // ---- Multi-device sync mode -------------------------------------------- + // Les TROIS modes ecrivent OB_STRUCT_MULTI_DEVICE_SYNC_CONFIG. Auparavant ce + // bloc etait garde par `g_syncMode != STANDALONE` : --standalone ne + // configurait donc rien et laissait la NVRAM du device telle quelle. Une + // camera restee en OB_SYNC_MODE_SECONDARY attendait alors un pulse materiel + // qui ne venait jamais -- flux bloques en STARTING, zero frame, et pas la + // moindre erreur pour le signaler. Le mode doit etre ecrit explicitement, + // sinon "standalone" ne veut rien dire au niveau du capteur. + { + ob_error *syncErr = nullptr; + OBDeviceSyncConfig obSync{}; + uint32_t syncLen = 0; + ob_device_get_structured_data(g_device->getImpl(), + OB_STRUCT_MULTI_DEVICE_SYNC_CONFIG, + reinterpret_cast(&obSync), + &syncLen, &syncErr); + if(syncErr) { ob_delete_error(syncErr); syncErr = nullptr; } + + // Mode herite, avant ecrasement : rend visible une camera laissee en + // secondary par un run precedent (ou par un autre outil). + const int previousMode = static_cast(obSync.syncMode); + + const char *modeName = "standalone"; + if(g_syncMode == K4A_WIRED_SYNC_MODE_MASTER) { + obSync.syncMode = OB_SYNC_MODE_PRIMARY_MCU_TRIGGER; + modeName = "master"; + } + else if(g_syncMode == K4A_WIRED_SYNC_MODE_SUBORDINATE) { + obSync.syncMode = OB_SYNC_MODE_SECONDARY; + obSync.irTriggerSignalInDelay = static_cast(syncDelayUs); + obSync.rgbTriggerSignalInDelay = static_cast(syncDelayUs); + modeName = "subordinate"; + } + else { + // Free-running : le capteur ne doit attendre aucun signal externe, et + // les retards de declenchement herites n'ont plus de sens. + obSync.syncMode = OB_SYNC_MODE_STANDALONE; + obSync.irTriggerSignalInDelay = 0; + obSync.rgbTriggerSignalInDelay = 0; + } + + ob_device_set_structured_data(g_device->getImpl(), + OB_STRUCT_MULTI_DEVICE_SYNC_CONFIG, + reinterpret_cast(&obSync), + sizeof(obSync), &syncErr); + if(syncErr) { + std::cerr << "Warning: sync config failed: " << ob_error_get_message(syncErr) << std::endl; + ob_delete_error(syncErr); + } else { + std::cout << "Sync mode: " << modeName + << " (device etait en mode " << previousMode << ")"; + if(g_syncMode == K4A_WIRED_SYNC_MODE_SUBORDINATE && syncDelayUs > 0) + std::cout << " (delay=" << syncDelayUs << " us)"; + std::cout << std::endl; + } + } + + // ---- Calibration JSON -------------------------------------------------- + std::vector calibJson; + { + ob_error *err = nullptr; + CalibAccum acc; + ob_device_get_raw_data(g_device->getImpl(), + static_cast(OB_RAW_DATA_CAMERA_CALIB_JSON_FILE), + calibDataCallback, &acc, &err); + if(err) { std::cerr << "Warning: calibration read failed: " << ob_error_get_message(err) << std::endl; ob_delete_error(err); } + else if(!acc.done) std::cerr << "Warning: calibration transfer incomplete.\n"; + else if(acc.buf.empty()) std::cerr << "Warning: calibration JSON empty.\n"; + else { calibJson = std::move(acc.buf); std::cout << "Calibration JSON: " << calibJson.size() << " bytes\n"; } + } + + // ---- K4A stream configuration ------------------------------------------ + k4a_device_configuration_t k4aCfg = K4A_DEVICE_CONFIG_INIT_DISABLE_ALL; + k4aCfg.color_format = color.k4aFormat; + k4aCfg.color_resolution = color.k4aRes; // OFF when color disabled + k4aCfg.depth_mode = depth.k4aMode; + k4aCfg.camera_fps = k4aFps; + k4aCfg.wired_sync_mode = g_syncMode; + if(g_syncMode == K4A_WIRED_SYNC_MODE_SUBORDINATE) + k4aCfg.subordinate_delay_off_master_usec = syncDelayUs; + + auto openRecording = [&](const std::string &path) -> bool { + k4a_record_t rec = nullptr; + bool ok = K4A_SUCCEEDED(k4a_record_create(path.c_str(), nullptr, k4aCfg, &rec)); + if(ok) { + if(!deviceSerial.empty()) k4a_record_add_tag(rec, "K4A_DEVICE_SERIAL_NUMBER", deviceSerial.c_str()); + if(!deviceFirmwareCOLOR.empty()) k4a_record_add_tag(rec, "K4A_COLOR_FIRMWARE_VERSION", deviceFirmwareCOLOR.c_str()); + if(!deviceFirmwareDEPTH.empty()) k4a_record_add_tag(rec, "K4A_DEPTH_FIRMWARE_VERSION", deviceFirmwareDEPTH.c_str()); + if(!deviceName.empty()) k4a_record_add_tag(rec, "K4A_DEVICE_NAME", deviceName.c_str()); + } + if(ok && !calibJson.empty()) { + k4a_record_add_tag(rec, "K4A_CALIBRATION_FILE", "calibration.json"); + k4a_record_add_attachment(rec, "calibration.json", calibJson.data(), calibJson.size()); + } + if(ok) ok = K4A_SUCCEEDED(k4a_record_write_header(rec)); + if(ok) { + std::lock_guard lock(g_recMutex); + g_recording = rec; g_tsOffset = 0; g_lastWrittenTs = 0; + g_writeFailures = 0; g_lastRawTs = 0; g_rawTsEpoch = 0; + return true; + } + std::cerr << "Warning: k4a_record_create failed for " << path << std::endl; + if(rec) k4a_record_close(rec); + return false; + }; + + auto closeRecording = [&](const std::string &path) { + std::lock_guard lock(g_recMutex); + if(g_recording) { + k4a_record_flush(g_recording); + k4a_record_close(g_recording); + g_recording = nullptr; + std::cout << "Saved: " << path << " (" << g_frameCount << " frames)" << std::endl; + } + }; + + std::string recPath; + if(!outputPath.empty()) { + recPath = outputPath; + if(!openRecording(recPath)) std::cerr << "Streaming only (no MKV).\n"; + else std::cout << "MKV ready: " << recPath << std::endl; + } + + // ---- Pipeline setup ---------------------------------------------------- + // ob::Pipeline delivers pre-synchronized color+depth(+IR) framesets, + // avoiding manual timestamp matching entirely. + auto pipeline = std::make_shared(g_device); + auto config = std::make_shared(); + + if(hasColor) { + try { + auto p = pipeline->getStreamProfileList(OB_SENSOR_COLOR) + ->getVideoStreamProfile(color.width, color.height, color.obFormat, rateFps); + config->enableStream(p); + } catch(ob::Error &e) { + std::cerr << "Error: color mode " << colorModeStr << " (" << color.width << "x" << color.height + << " @ " << rateFps << " fps) not supported by device: " << e.what() << std::endl; + return EXIT_FAILURE; + } + } + if(hasDepth) { + try { + auto p = pipeline->getStreamProfileList(OB_SENSOR_DEPTH) + ->getVideoStreamProfile(depth.width, depth.height, OB_FORMAT_Y16, rateFps); + config->enableStream(p); + } catch(ob::Error &e) { + std::cerr << "Error: depth mode " << depthModeStr << " (" << depth.width << "x" << depth.height + << " @ " << rateFps << " fps) not supported by device: " << e.what() << std::endl; + return EXIT_FAILURE; + } + } + if(hasIR) { + try { + auto p = pipeline->getStreamProfileList(OB_SENSOR_IR) + ->getVideoStreamProfile(depth.width, depth.height, OB_FORMAT_Y16, rateFps); + config->enableStream(p); + } catch(ob::Error &e) { + std::cerr << "Warning: IR stream (" << depth.width << "x" << depth.height << " @ " << rateFps + << " fps) not available, continuing without IR: " << e.what() << std::endl; + hasIR = false; + } + } + + // Frame sync: ob::Pipeline matches color+depth(+IR) by timestamp internally. + pipeline->enableFrameSync(); + + // Auto-record: reset device timestamp before pipeline starts. + if(!outputPath.empty() && g_recording) { + g_frameCount = 0; + if(g_syncMode != K4A_WIRED_SYNC_MODE_STANDALONE) { + resetDeviceTimestamp(); + g_pendingReset = true; + g_pendingResetStart = std::chrono::steady_clock::now(); + std::cout << "Timestamp reset — waiting for sync before recording to: " << recPath << std::endl; + } else { + g_isRecording = true; + std::cout << "Auto-recording to: " << recPath << std::endl; + } + } + + // ---- Pipeline callback ------------------------------------------------- + pipeline->start(config, [&](std::shared_ptr frameset) { + if(!frameset) return; + + auto colorFrame = hasColor ? frameset->getFrame(OB_FRAME_COLOR) : nullptr; + auto depthFrame = hasDepth ? frameset->getFrame(OB_FRAME_DEPTH) : nullptr; + auto irFrame = hasIR ? frameset->getFrame(OB_FRAME_IR) : nullptr; + + // Require every enabled primary stream to be present in the synced set. + if(hasColor && !colorFrame) return; + if(hasDepth && !depthFrame) return; + if(!hasColor && !hasDepth && hasIR && !irFrame) return; // IR-only / PASSIVE_IR + + // Canonical frame for display + timestamp gating. + auto primary = colorFrame ? colorFrame : (depthFrame ? depthFrame : irFrame); + if(!primary) return; + + // Update display frames for main thread. + { + std::lock_guard lk(g_dispMutex); + g_dispColor = colorFrame; + g_dispDepth = depthFrame; + g_dispIR = irFrame; + g_hasNewFrame = true; + } + g_newFrameCv.notify_one(); + + // Sync mode: wait for the timestamp reset to propagate. + if(g_pendingReset) { + uint64_t pts = primary->getTimeStampUs(); + bool tsOk = pts < kResetThresholdUs; + bool timedOut = !tsOk && g_pendingResetTimeoutMs > 0 && std::chrono::duration_cast( + std::chrono::steady_clock::now() - g_pendingResetStart).count() > g_pendingResetTimeoutMs; + if(tsOk || timedOut) { + if(timedOut) + std::cerr << "[WARN] pendingReset timeout (ts=" << pts + << " us) — sync pulse not received, starting anyway\n"; + g_pendingReset = false; + g_isRecording = true; + std::cout << "Recording started (ts=" << pts << " us)\n"; + } + return; + } + + if(g_isRecording) + writeCapture(colorFrame, depthFrame, irFrame); + }); + + // ---- Apply manual color controls (after the color sensor is streaming) -- + auto setColorInt = [&](OBPropertyID id, int value, const char *name) { + try { + if(!g_device->isPropertySupported(id, OB_PERMISSION_WRITE)) { + std::cerr << "Warning: " << name << " not supported by device.\n"; return; + } + OBIntPropertyRange range = g_device->getIntPropertyRange(id); + if(value < range.min || value > range.max) { + int clamped = value < range.min ? range.min : range.max; // std::min/max clash with windows.h macros + std::cerr << "Warning: " << name << " " << value << " out of range [" + << range.min << ".." << range.max << "], clamping to " << clamped << ".\n"; + value = clamped; + } + g_device->setIntProperty(id, value); + std::cout << "Set " << name << " = " << value << std::endl; + } catch(ob::Error &e) { + std::cerr << "Warning: set " << name << " failed: " << e.what() << std::endl; + } + }; + auto setColorBool = [&](OBPropertyID id, bool value, const char *name) { + try { + if(g_device->isPropertySupported(id, OB_PERMISSION_WRITE)) + g_device->setBoolProperty(id, value); + } catch(ob::Error &e) { + std::cerr << "Warning: set " << name << " failed: " << e.what() << std::endl; + } + }; + + if(hasColor) { + if(exposureUs >= 0) { + setColorBool(OB_PROP_COLOR_AUTO_EXPOSURE_BOOL, false, "auto-exposure(off)"); + setColorInt (OB_PROP_COLOR_EXPOSURE_INT, exposureUs, "exposure(us)"); + } + if(whitebalance >= 0) { + setColorBool(OB_PROP_COLOR_AUTO_WHITE_BALANCE_BOOL, false, "auto-white-balance(off)"); + setColorInt (OB_PROP_COLOR_WHITE_BALANCE_INT, whitebalance, "white-balance"); + } + if(gain >= 0) setColorInt(OB_PROP_COLOR_GAIN_INT, gain, "gain"); + if(brightness >= 0) setColorInt(OB_PROP_COLOR_BRIGHTNESS_INT, brightness, "brightness"); + if(contrast >= 0) setColorInt(OB_PROP_COLOR_CONTRAST_INT, contrast, "contrast"); + if(saturation >= 0) setColorInt(OB_PROP_COLOR_SATURATION_INT, saturation, "saturation"); + if(sharpness >= 0) setColorInt(OB_PROP_COLOR_SHARPNESS_INT, sharpness, "sharpness"); + } + + // ---- Spout + signal handling ------------------------------------------- +#ifdef _WIN32 + spoutDX spoutColor, spoutDepth; + if(useSpout) { + spoutColor.SetSenderName("OrbbecColor"); + spoutDepth.SetSenderName("OrbbecDepth"); + } + SetConsoleCtrlHandler(ctrlHandler, TRUE); +#else + std::signal(SIGINT, posixSignalHandler); + std::signal(SIGTERM, posixSignalHandler); +#endif + + // Arrêt via EOF sur stdin — UNIQUEMENT si stdin est un vrai pipe (l'hôte l'a redirigé pour piloter + // l'arrêt en le fermant). Sinon (console interactive, ou CREATE_NEW_CONSOLE depuis un parent GUI + // comme Unity), stdin peut renvoyer EOF immédiatement → l'enregistrement s'arrêterait après 1 frame. + // En mode non-pipe, on n'active pas ce watcher ; l'arrêt passe par Ctrl+C/Ctrl+Break (ctrlHandler). + { + bool stdinIsPipe = false; +#ifdef _WIN32 + stdinIsPipe = (GetFileType(GetStdHandle(STD_INPUT_HANDLE)) == FILE_TYPE_PIPE); +#endif + if(stdinIsPipe) { + std::thread([]{ + int ch; + do { ch = std::getchar(); } while(ch != EOF && ch != 'q' && ch != 'Q'); + g_exitRequested = true; + g_newFrameCv.notify_all(); + }).detach(); + } + } + + if(showPreview) { + cv::namedWindow("K4A Rec (pipeline)", cv::WINDOW_NORMAL); + cv::resizeWindow("K4A Rec (pipeline)", 1280, 480); + std::cout << "R=start/stop recording T=timestamp ESC/Q=quit Ctrl+C=quit" << std::endl; + } else { + std::cout << "Headless mode. Ctrl+C to stop." << std::endl; + } + if(recordLengthSec > 0) + std::cout << "Recording limited to " << recordLengthSec << " s." << std::endl; + + // ---- Main loop --------------------------------------------------------- + bool showTimestamp = false; + const bool needDisplay = showPreview || useSpout; + std::chrono::steady_clock::time_point recStartTp{}; + + while(true) { + std::shared_ptr colorForDisplay, depthForDisplay, irForDisplay; + { + std::unique_lock lk(g_dispMutex); + g_newFrameCv.wait_for(lk, std::chrono::milliseconds(needDisplay ? 33 : 200), + [&]{ return (needDisplay && g_hasNewFrame) || g_exitRequested.load(); }); + if(g_exitRequested) break; + colorForDisplay = g_dispColor; + depthForDisplay = g_dispDepth; + irForDisplay = g_dispIR; + g_hasNewFrame = false; + } + + // ---- Record-length limit: stop + exit after N seconds of recording -- + if(recordLengthSec > 0) { + if(g_isRecording.load()) { + if(recStartTp == std::chrono::steady_clock::time_point{}) + recStartTp = std::chrono::steady_clock::now(); + else if(std::chrono::duration_cast( + std::chrono::steady_clock::now() - recStartTp).count() >= recordLengthSec) { + std::cout << "Record length (" << recordLengthSec << " s) reached — stopping." << std::endl; + break; + } + } else { + recStartTp = std::chrono::steady_clock::time_point{}; + } + } + + if(showPreview) { + int key = cv::waitKey(1); + if(key == 27 || key == 'q' || key == 'Q') break; + if(cv::getWindowProperty("K4A Rec (pipeline)", cv::WND_PROP_VISIBLE) < 1.0) break; + if(key == 'r' || key == 'R') { + if(!g_isRecording && !g_pendingReset) { + recPath = makeTimestampedFilename(); + if(openRecording(recPath)) { + g_frameCount = 0; + resetDeviceTimestamp(); + g_pendingReset = true; + g_pendingResetStart = std::chrono::steady_clock::now(); + std::cout << "Recording to: " << recPath << " ..." << std::endl; + } + } else if(g_isRecording) { + g_isRecording = false; + closeRecording(recPath); + } + } + if(key == 't' || key == 'T') showTimestamp = !showTimestamp; + } + + if(!needDisplay) continue; + if(!colorForDisplay && !depthForDisplay && !irForDisplay) continue; + + cv::Mat colorBGR = decodeColorFrame(colorForDisplay); + + // Right panel: colorized depth, or normalized IR when there is no depth. + cv::Mat depthBGR; + int rightW = 0, rightH = 0; + if(depthForDisplay) { + auto df = depthForDisplay->as(); + uint32_t w = df->getWidth(), h = df->getHeight(); + rightW = static_cast(w); rightH = static_cast(h); + float scale = df->getValueScale(); + const uint16_t *data = reinterpret_cast(df->getData()); + cv::Mat depth16(h, w, CV_16UC1, (void*)data); + cv::Mat depth8; + depth16.convertTo(depth8, CV_8UC1, scale * 255.0 / 5000.0); + cv::applyColorMap(depth8, depthBGR, cv::COLORMAP_JET); + } else if(irForDisplay) { + auto vf = irForDisplay->as(); + uint32_t w = vf->getWidth(), h = vf->getHeight(); + rightW = static_cast(w); rightH = static_cast(h); + const uint16_t *data = reinterpret_cast(irForDisplay->getData()); + cv::Mat ir16(h, w, CV_16UC1, (void*)data); + cv::Mat ir8; + cv::normalize(ir16, ir8, 0, 255, cv::NORM_MINMAX, CV_8UC1); + cv::cvtColor(ir8, depthBGR, cv::COLOR_GRAY2BGR); + } + +#ifdef _WIN32 + if(useSpout) { + if(!colorBGR.empty()) { + auto vf = colorForDisplay->as(); + cv::Mat bgra; + cv::cvtColor(colorBGR, bgra, cv::COLOR_RGB2RGBA); + spoutColor.SendImage(bgra.data, vf->getWidth(), vf->getHeight(), vf->getWidth() * 4); + } + if(!depthBGR.empty() && rightW > 0) { + cv::Mat bgra; + cv::cvtColor(depthBGR, bgra, cv::COLOR_RGB2RGBA); + spoutDepth.SendImage(bgra.data, rightW, rightH, rightW * 4); + } + } +#endif + + if(showPreview) { + cv::Mat cs, ds; + if(!colorBGR.empty()) cv::resize(colorBGR, cs, cv::Size(640, 360)); + else cs = cv::Mat::zeros(360, 640, CV_8UC3); + if(!depthBGR.empty()) cv::resize(depthBGR, ds, cv::Size(640, 360)); + else ds = cv::Mat::zeros(360, 640, CV_8UC3); + + cv::Mat combined; + cv::hconcat(cs, ds, combined); + if(g_isRecording) { + cv::circle(combined, cv::Point(20, 20), 8, cv::Scalar(0, 0, 255), -1); + cv::putText(combined, "REC " + std::to_string(g_frameCount), + cv::Point(35, 27), cv::FONT_HERSHEY_SIMPLEX, 0.6, cv::Scalar(0, 0, 255), 1); + } + if(showTimestamp) { + char buf[64]; + uint64_t cts = colorForDisplay ? colorForDisplay->getTimeStampUs() : 0; + uint64_t dts = depthForDisplay ? depthForDisplay->getTimeStampUs() + : irForDisplay ? irForDisplay->getTimeStampUs() : 0; + int y0 = combined.rows - 42, y1 = combined.rows - 18; + cv::rectangle(combined, cv::Point(0, y0 - 16), cv::Point(280, combined.rows), cv::Scalar(0,0,0), cv::FILLED); + snprintf(buf, sizeof(buf), "C: %.3f ms", cts / 1000.0); + cv::putText(combined, buf, cv::Point(6, y0), cv::FONT_HERSHEY_SIMPLEX, 0.6, cv::Scalar(0,220,220), 1); + snprintf(buf, sizeof(buf), "D: %.3f ms", dts / 1000.0); + cv::putText(combined, buf, cv::Point(6, y1), cv::FONT_HERSHEY_SIMPLEX, 0.6, cv::Scalar(0,220,220), 1); + } + cv::imshow("K4A Rec (pipeline)", combined); + } + } + + // ---- Cleanup ----------------------------------------------------------- + g_isRecording = false; + closeRecording(recPath); + pipeline->stop(); + g_device.reset(); + +#ifdef _WIN32 + if(useSpout) { spoutColor.ReleaseSender(); spoutDepth.ReleaseSender(); } +#endif + return 0; +} +catch(ob::Error &e) { + std::cerr << "function:" << e.getFunction() << "\nargs:" << e.getArgs() + << "\nmessage:" << e.what() << "\nstatus:" << e.getStatus() + << "\ntype:" << e.getExceptionType() << std::endl; + std::cout << "\nPress any key to exit."; + // Bounded timeout: with no timeout this blocks forever when stdin isn't + // interactive (SSH, a script, a service supervisor, ...), leaving the + // process running and the camera claimed. 3s is plenty for a human to + // notice on a double-clicked Windows console; headless runs just move on. + ob_smpl::waitForKeyPressed(3000); + exit(EXIT_FAILURE); +} diff --git a/scripts/build_k4a_wrapper.sh b/scripts/build_k4a_wrapper.sh new file mode 100755 index 0000000..ad2aaaa --- /dev/null +++ b/scripts/build_k4a_wrapper.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# Makes sure the Orbbec K4A wrapper is built for THIS platform, and prints where +# it is as K4A_WRAPPER_DIR=... for build_sdk.sh (the recorder example needs its +# headers) and for the recorder's runtime rpath (its libk4a.so/libk4arecord.so). +# +# This is the libk4a provider — Orbbec's own, NOT Microsoft's Azure Kinect SDK. +# It bundles its own OrbbecSDK internally, so it builds standalone. The recorder +# dlopens the libk4a it produces (RTLD_DEEPBIND) rather than link-time linking it, +# which is why only its build/lib path matters at runtime, not link time. +# +# Reuses an existing build (leaving a working node untouched) or clones + builds. +set -euo pipefail + +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +eval "$("$here/detect_platform.sh")" + +REPO_URL="${OB_K4A_REPO:-https://github.com/orbbec/OrbbecSDK-K4A-Wrapper}" +# The tag the Nano runs. A caller can override to one known to build on a newer +# JetPack. +REF="${OB_K4A_REF:-v1.10.4}" + +find_wrapper() { + local d="$1" + [[ -f "$d/include/k4a/k4a.h" ]] || return 1 + [[ -f "$d/build/lib/libk4a.so" ]] || return 1 + echo "$d" +} + +for candidate in "${K4A_WRAPPER_DIR:-}" \ + "$HOME/OrbbecSDK-K4A-Wrapper" \ + "$HOME/Downloads/OrbbecSDK-K4A-Wrapper"; do + [[ -z "$candidate" ]] && continue + if find_wrapper "$candidate" >/dev/null 2>&1; then + echo ">> Wrapper K4A deja bati, reutilise: $candidate" >&2 + echo "K4A_WRAPPER_DIR=$candidate" + exit 0 + fi +done + +dir="${K4A_WRAPPER_DIR:-$HOME/OrbbecSDK-K4A-Wrapper}" +echo ">> Wrapper K4A absent — clone et compilation (JetPack ${JETPACK}, ${SOC})" >&2 +echo ">> ${REPO_URL} @ ${REF} -> ${dir}" >&2 + +if [[ ! -d "$dir/.git" ]]; then + # --recursive: the wrapper carries its OrbbecSDK and depth engine as submodules. + git clone --recursive --depth 1 --branch "$REF" "$REPO_URL" "$dir" >&2 +fi + +mkdir -p "$dir/build" +( + cd "$dir/build" + cmake .. -DCMAKE_BUILD_TYPE=Release >&2 + make -j"$(nproc)" k4a k4arecord >&2 +) + +if ! find_wrapper "$dir" >/dev/null 2>&1; then + cat >&2 <> Wrapper K4A bati: $dir" >&2 +echo "K4A_WRAPPER_DIR=$dir" diff --git a/scripts/build_sdk.sh b/scripts/build_sdk.sh index 5681220..28b5499 100755 --- a/scripts/build_sdk.sh +++ b/scripts/build_sdk.sh @@ -1,71 +1,108 @@ #!/usr/bin/env bash -# Makes sure an OrbbecSDK v2 build exists for THIS platform, and prints where it -# is as OB_SDK_ROOT=... / OB_SDK_BUILD=... / OB_SDK_GENERATED=... for build_server.sh. +# Makes sure an OrbbecSDK v2 build exists for THIS platform — INCLUDING the custom +# MKV recorder example — and prints where it all is: +# +# OB_SDK_ROOT / OB_SDK_BUILD / OB_SDK_GENERATED the SDK +# RECORDER_BIN the recorder binary, or empty # # Never ships a prebuilt binary between Jetsons: CUDA and glibc differ by -# generation, so the SDK is built from source on the machine that will run it. -# The heavy artifact stays out of git. +# generation, so everything is built from source on the machine that runs it. The +# heavy artifacts stay out of git. # -# Order of preference: -# 1. An existing build pointed to by $OB_SDK_ROOT — reused as-is. This is what -# keeps a working node (the Nano already has one) untouched. -# 2. A build at one of the known locations, likewise reused. -# 3. A fresh clone + build. On a new JetPack this is the step to watch; if it -# fails, build the SDK by hand once and re-run install.sh with OB_SDK_ROOT set. +# The recorder is FTF's own source (recorder/ in this repo), not a stock SDK +# example, so it is injected into the SDK's examples tree before the build and +# compiled there — it depends on the SDK's own cmake targets (ob::examples::utils). +# It needs the Orbbec K4A wrapper's headers; build_k4a_wrapper.sh provides them. +# +# Order of preference for the SDK itself: +# 1. $OB_SDK_ROOT if it already holds a build — reused, which keeps a working +# node (the Nano) untouched. +# 2. A build at a known location, likewise reused. +# 3. A fresh clone + build. set -euo pipefail here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +repo=$(cd "$here/.." && pwd) eval "$("$here/detect_platform.sh")" -# The official v2 source. Pinned loosely: a caller can override both to match a -# tag known to build on their JetPack. SDK_REPO="${OB_SDK_REPO:-https://github.com/orbbec/OrbbecSDK_v2.git}" SDK_REF="${OB_SDK_REF:-main}" find_build() { - local root="$1" + local root="$1" b [[ -f "$root/include/libobsensor/ObSensor.hpp" ]] || return 1 - local b b=$(ls -d "$root"/build/linux_* 2>/dev/null | head -1) [[ -z "$b" ]] && b="$root/build" [[ -f "$b/lib/libOrbbecSDK.so" ]] || return 1 echo "$b" } +recorder_bin_in() { # echoes the recorder path under a build dir if present + local b="$1" + [[ -x "$b/bin/ob_stream_depth_color_k4arec_pipeline" ]] \ + && echo "$b/bin/ob_stream_depth_color_k4arec_pipeline" +} + emit() { local root="$1" build="$2" echo "OB_SDK_ROOT=$root" echo "OB_SDK_BUILD=$build" echo "OB_SDK_GENERATED=$root/build/src/generated" + echo "RECORDER_BIN=$(recorder_bin_in "$build")" } -# 1 & 2: reuse whatever is already built. +# Drops the FTF recorder into the SDK examples tree and registers it, so a plain +# example build picks it up. Idempotent. +inject_recorder() { + local root="$1" + local dst="$root/examples/1.stream.depth_color.k4arec.pipeline" + mkdir -p "$dst" + cp "$repo/recorder/depth_color_k4arec_pipeline.cpp" "$dst/" + cp "$repo/recorder/CMakeLists.txt" "$dst/" + local ex_cmake="$root/examples/CMakeLists.txt" + if [[ -f "$ex_cmake" ]] \ + && ! grep -q "1.stream.depth_color.k4arec.pipeline" "$ex_cmake"; then + echo "add_subdirectory(1.stream.depth_color.k4arec.pipeline)" >> "$ex_cmake" + fi +} + +# 1 & 2: reuse an existing SDK build. Only inject/rebuild the recorder if it is +# missing there — never disturb the library build. for candidate in "${OB_SDK_ROOT:-}" \ "$HOME/OrbbecSDK_v2" \ "$HOME/OrbbecSDK_v2_jetson"; do [[ -z "$candidate" ]] && continue if b=$(find_build "$candidate" 2>/dev/null); then echo ">> SDK deja bati, reutilise: $candidate" >&2 + if [[ -z "$(recorder_bin_in "$b")" && -n "${K4A_WRAPPER_DIR:-}" ]]; then + echo ">> Recorder absent de ce build — injection et compilation…" >&2 + inject_recorder "$candidate" + ( cd "$b" && cmake .. -DOB_BUILD_EXAMPLES=ON \ + -DK4A_WRAPPER_DIR="$K4A_WRAPPER_DIR" >&2 \ + && make -j"$(nproc)" ob_stream_depth_color_k4arec_pipeline >&2 ) || true + fi emit "$candidate" "$b" exit 0 fi done -# 3: clone + build from source. +# 3: clone + build from source, with examples and the recorder. root="${OB_SDK_ROOT:-$HOME/OrbbecSDK_v2}" echo ">> Aucun SDK bati — clone et compilation (JetPack ${JETPACK}, ${SOC})" >&2 -echo ">> ${SDK_REPO} @ ${SDK_REF} -> ${root}" >&2 +[[ -d "$root/.git" ]] || git clone --depth 1 --branch "$SDK_REF" "$SDK_REPO" "$root" >&2 -if [[ ! -d "$root/.git" ]]; then - git clone --depth 1 --branch "$SDK_REF" "$SDK_REPO" "$root" >&2 -fi +inject_recorder "$root" + +# The recorder needs the wrapper; without it the SDK still builds (its CMakeLists +# skips the recorder with a warning), giving preview-only. +cmake_k4a=() +[[ -n "${K4A_WRAPPER_DIR:-}" ]] && cmake_k4a=(-DK4A_WRAPPER_DIR="$K4A_WRAPPER_DIR") build="$root/build/linux_$(uname -m)" mkdir -p "$build" ( cd "$build" - # -DOB_BUILD_EXAMPLES=ON gives the k4a recorder sample used as recorder_bin. - cmake ../.. -DCMAKE_BUILD_TYPE=Release -DOB_BUILD_EXAMPLES=ON >&2 + cmake ../.. -DCMAKE_BUILD_TYPE=Release -DOB_BUILD_EXAMPLES=ON "${cmake_k4a[@]}" >&2 make -j"$(nproc)" >&2 ) @@ -74,9 +111,11 @@ if ! b=$(find_build "$root" 2>/dev/null); then !! La compilation du SDK ne s'est pas terminee comme attendu. !! Bâtis-le une fois a la main, puis relance install.sh en pointant dessus : -!! OB_SDK_ROOT= ./install.sh +!! OB_SDK_ROOT=$root ./install.sh EOF exit 1 fi echo ">> SDK bati: $b" >&2 +[[ -z "$(recorder_bin_in "$b")" ]] && \ + echo ">> NOTE: recorder non produit (wrapper K4A absent) — preview seule." >&2 emit "$root" "$b" diff --git a/scripts/install_deps.sh b/scripts/install_deps.sh index b71093c..16562dc 100755 --- a/scripts/install_deps.sh +++ b/scripts/install_deps.sh @@ -17,9 +17,12 @@ echo ">> Dependances pour JetPack ${JETPACK} (L4T ${L4T}, ${SOC})" # back to pip when the distro is too old to carry it. sudo apt-get update -qq sudo apt-get install -y --no-install-recommends \ - build-essential cmake pkg-config \ + build-essential cmake pkg-config git \ libusb-1.0-0-dev libudev-dev liblz4-dev libjpeg-turbo8-dev \ + libopencv-dev \ python3 python3-pip +# libopencv-dev: the MKV recorder links OpenCV (image handling). liblz4/libjpeg: +# the preview server. libusb/libudev: the SDK and the K4A wrapper. if ! python3 -c 'import websockets' 2>/dev/null; then echo ">> websockets absent — installation via pip"