arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.02648cs.MM
收藏

HoloLens 2传感器流

HoloLens 2 Sensor Streaming

Juan C. Dibene, Enrique Dunn

摘要:

我们提出了一个HoloLens 2服务器应用程序,用于通过TCP实时的传输设备数据。服务器可以从四个灰度相机、深度传感器、IMU、前置RGB相机、麦克风、头部跟踪、眼动追踪和手部跟踪流式传输数据。每个发送的数据帧具有时间戳,并且可选地具有设备在3D空间中的瞬时姿态。该服务器允许下载设备校准数据,如相机内部数据,并可以作为插件集成到Unity项目中,支持基本的上游功能。为了实现全帧速率的实时的视频流,我们利用了HoloLens 2的视频编码功能。最后,我们提供了一个用于接收和解码数据的Python库,其中包括便于将数据传递到其他库的实用程序。源代码、Python演示和预编译的二进制文件可以在https://github.com/jdibenes/hl2ss上获得。

英文摘要:

We present a HoloLens 2 server application for streaming device data via TCP in real time. The server can stream data from the four grayscale cameras, depth sensor, IMU, front RGB camera, microphone, head tracking, eye tracking, and hand tracking. Each sent data frame has a timestamp and, optionally, the instantaneous pose of the device in 3D space. The server allows downloading device calibration data, such as camera intrinsics, and can be integrated into Unity projects as a plugin, with support for basic upstream capabilities. To achieve real time video streaming at full frame rate, we leverage the video encoding capabilities of the HoloLens 2. Finally, we present a Python library for receiving and decoding the data, which includes utilities that facilitate passing the data to other libraries. The source code, Python demos, and precompiled binaries are available at https://github.com/jdibenes/hl2ss.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑