Modern agricultural applications require knowledge about the position and size of fruits on plants. However, occlusions from leaves typically make obtaining this information difficult. We present a novel viewpoint planning approach that builds up an octree of plants with labeled regions of interest (ROIs), i.e., fruits. Our method uses this octree to sample viewpoint candidates that increase the information around the fruit regions and evaluates them using a heuristic utility function that takes into account the expected information gain. Our system automatically switches between ROI targeted sampling and exploration sampling, which considers general frontier voxels, depending on the estimated utility. When the plants have been sufficiently covered with the RGB-D sensor, our system clusters the ROI voxels and estimates the position and size of the detected fruits. We evaluated our approach in simulated scenarios and compared the resulting fruit estimations with the ground truth. The results demonstrate that our combined approach outperforms a sampling method that does not explicitly consider the ROIs to generate viewpoints in terms of the number of discovered ROI cells. Furthermore, we show the real-world applicability by testing our framework on a robotic arm equipped with an RGB-D camera installed on an automated pipe-rail trolley in a capsicum glasshouse.
Hierarchical Tri-manual Planning for Vision-assisted Fruit Harvesting with Quadrupedal Robots
Institute for Integrative & Innovative Research (I 3 R), University of Arkansas, USA(整合与创新研究所(I 3 R)、亚拉巴马大学);Department of Electrical and Computer Engineering, University of California, Riverside, USA(电气与计算机工程系、加州大学河滨分校)
Fast Heuristic Scheduling and Trajectory Planning for Robotic Fruit Harvesters with Multiple Cartesian Arms
Department of Mechanical & Aerospace Engineering University of California, Davis(机械与航空航天工程系加州大学戴维斯分校);Department of Biological and Agricultural Engineering University of California, Davis(生物与农业工程系加州大学戴维斯分校)
GO-VMP: Global Optimization for View Motion Planning in Fruit Mapping
Bonn-Rhein-Sieg University of Applied Sciences(波恩-莱茵-锡格应用科学大学);University of Bonn(波恩大学);Humanoid Robots Lab(人形机器人实验室);Fraunhofer Institute for Intelligent Analysis and Information Systems(弗劳恩霍夫智能分析与信息系统研究所);Lamarr Institute for Machine Learning and Artificial Intelligence(拉玛尔机器学习与人工智能研究所);Center for Robotics, University of Bonn(波恩大学机器人中心)
GS-NBV: a Geometry-based, Semantics-aware Viewpoint Planning Algorithm for Avocado Harvesting under Occlusions
Autonomous Robots and Control Systems Lab, University of California, Riverside(加州大学河滨分校自主机器人与控制系统实验室);Perception, Robotics, AI, Sensing Lab, University of Colorado, Boulder(科罗拉多大学博尔德分校感知、机器人、人工智能与传感实验室)
Safe Leaf Manipulation for Accurate Shape and Pose Estimation of Occluded Fruits
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Bonn(波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所);Center for Robotics(机器人中心);Robotics Institute Germany(德国机器人研究所)
A Review on Viewpoints and Path-planning for UAV-based 3D Reconstruction
Institute of Geodesy and Photogrammetry, Technische Universität Braunschweig(布伦瑞克工业大学测量与摄影测量研究所);University of Tehran(德黑兰大学);Intelligent Robotics Research Division, ETRI(韩国电子通信研究院智能机器人研究部);New York University(纽约大学);University of Edinburgh(爱丁堡大学)
Perceptual Factors for Environmental Modeling in Robotic Active Perception
Instituto de Investigación en Ingeniería de Aragón, Universidad de Zaragoza(萨拉戈萨大学阿拉贡工程研究院);Institute of Geodesy and Geoinformation, University of Bonn(波恩大学大地测量与地理信息研究所)
Development and evaluation of automated localisation and reconstruction of all fruits on tomato plants in a greenhouse based on multi-view perception and 3D multi-object tracking
ActiveSplat: High-Fidelity Scene Reconstruction through Active Gaussian Splatting
School of Optics and Photonics, Beijing Institute of Technology(光学与光子学学院,北京理工大学);Institute for AI Industry Research (AIR), Tsinghua University(人工智能产业研究院(AIR),清华大学);School of Vehicle and Mobility, Tsinghua University(车辆与移动学院,清华大学)
VF-Plan: Bridging the Art Gallery Problem and Static LiDAR Scanning with Visibility Field Optimization
Wuhan University of Technology(武汉理工大学);NED University of Engineering and Technology(奈及利亚工程与技术大学);Zhejiang University(浙江大学);The Advanced Laser Technology Laboratory of Anhui Province(安徽省先进激光技术实验室)
Investigating Sensors and Methods in Grasp State Classification in Agricultural Manipulation
Department of Mechanical Engineering, University of Illinois at Urbana-Champaign(机械工程系,伊利诺伊大学厄巴纳-香槟分校);Department of Industrial and Enterprise Systems Engineering, University of Illinois at Urbana-Champaign(工业与企业系统工程系,伊利诺伊大学厄巴纳-香槟分校)