PixelGen：重新构想嵌入式相机系统
PixelGen: Rethinking Embedded Camera Systems
AI总结：
PixelGen重新设计嵌入式相机系统，用低功耗低分辨率传感器结合Transformer模型生成高清图像及声波等新颖环境表征，支持扩展现实可视化，开启研究新途径。
摘要：
嵌入式相机系统无处不在，代表了无线嵌入式系统中部署最广泛的实例。它们捕获世界的表征——由可见光或红外光照亮的周围环境。尽管使用广泛，嵌入式相机系统的架构却一直保持不变，这导致了诸多局限。它们仅能可视化世界的极小部分。此外，它们能耗高，导致电池寿命有限。我们提出了PixelGen，它重新构想了嵌入式相机系统。具体而言，PixelGen结合了传感器、收发器以及低分辨率图像和红外视觉传感器，以捕获更广泛的世界表征。这些组件被特意选择，因其简单性、低比特率和低功耗，最终形成一个节能平台。我们表明，尽管简单，捕获的数据可通过基于Transformer的图像和语言模型进行处理，以生成环境的新颖表征。例如，我们证明，它能够生成高清图像，而相机使用的是低功耗、低分辨率的单色相机。此外，PixelGen的能力超越了传统摄影，能够可视化传统相机无法观察到的现象，如声波。PixelGen能够支持众多新颖应用，我们证明它实现了对周围环境的独特可视化，并可投射到扩展现实头显上。我们相信，PixelGen超越了传统相机，为研究和摄影开辟了新的途径。
英文摘要：
Embedded camera systems are ubiquitous, representing the most widely deployed example of a wireless embedded system. They capture a representation of the world - the surroundings illuminated by visible or infrared light. Despite their widespread usage, the architecture of embedded camera systems has remained unchanged, which leads to limitations. They visualize only a tiny portion of the world. Additionally, they are energy-intensive, leading to limited battery lifespan. We present PixelGen, which re-imagines embedded camera systems. Specifically, PixelGen combines sensors, transceivers, and low-resolution image and infrared vision sensors to capture a broader world representation. They are deliberately chosen for their simplicity, low bitrate, and power consumption, culminating in an energy-efficient platform. We show that despite the simplicity, the captured data can be processed using transformer-based image and language models to generate novel representations of the environment. For example, we demonstrate that it can allow the generation of high-definition images, while the camera utilises low-power, low-resolution monochrome cameras. Furthermore, the capabilities of PixelGen extend beyond traditional photography, enabling visualization of phenomena invisible to conventional cameras, such as sound waves. PixelGen can enable numerous novel applications, and we demonstrate that it enables unique visualization of the surroundings that are then projected on extended reality headsets. We believe, PixelGen goes beyond conventional cameras and opens new avenues for research and photography.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
