UI Layers Group Detector: Grouping UI Layers via Text Fusion and Box Attention
Zhejiang University（浙江大学）；Alibaba Group（阿里巴巴集团）；Chongqing University of Posts and Telecommunications（重庆邮电大学）；Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies（阿里巴巴-浙江大学前沿技术联合研究院）
Graphic User Interface (GUI) is facing great demand with the popularization and prosperity of mobile apps. Automatic UI code generation from UI design draft dramatically simplifies the development process. However, the nesting layer structure in the design draft affects the quality and usability of the generated code. Few existing GUI automated techniques detect and group the nested layers to improve the accessibility of generated code. In this paper, we proposed our UI Layers Group Detector as a vision-based method that automatically detects images (i.e., basic shapes and visual elements) and text layers that present the same semantic meanings. We propose two plug-in components, text fusion and box attention, that utilize text information from design drafts as a priori information for group localization. We construct a large-scale UI dataset for training and testing, and present a data augmentation approach to boost the detection performance. The experiment shows that the proposed method achieves a decent accuracy regarding layers grouping.
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）
In recent years, estimating the 6D pose of object instances with convolutional neural network (CNN) has received considerable attention. Depending on whether intermediate cues are used, the relevant literature can be roughly divided into two broad categories: direct methods and two stage pipelines. For the latter, intermediate cues, such as 3D object coordinates, semantic keypoints, or virtual control points instead of pose parameters are regressed by CNN in the first stage. Object pose can then be solved by correspondence constraints constructed with these intermediate cues. In this paper, we…
Fusion and Grouping Strategies in Deep Learning for Local Climate Zone Classification of Multimodal Remote Sensing Data
Graphics-Visualization-Computing Lab, International Institute of Information Technology Bangalore, Karnataka 560100, India（图形可视化计算实验室，国际信息学院班加罗尔，卡纳塔克邦560100，印度）
The Chinese University of Hong Kong(Shenzhen)（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Future Network of Intelligence Institute（未来网络智能研究院）；The Third Affiliated Hospital of Shenzhen University(Luohu Hospital Group)（深圳大学第三附属医院（罗湖医院集团））；Sun Yat-Sen University（中山大学）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Accurate polyp segmentation is of great importance for colorectal cancer diagnosis and treatment. However, due to the high cost of producing accurate mask annotations, existing polyp segmentation methods suffer from severe data shortage and impaired model generalization. Reversely, coarse polyp bounding box annotations are more accessible. Thus, in this paper, we propose a boosted BoxPolyp model to make full use of both accurate mask and extra coarse box annotations. In practice, box annotations are applied to alleviate the over-fitting issue of previous polyp segmentation models, which genera…
We present PointFusion, a generic 3D object detection method that leverages both image and 3D point cloud information. Unlike existing methods that either use multi-stage pipelines or hold sensor and dataset-specific assumptions, PointFusion is conceptually simple and application-agnostic. The image data and the raw point cloud data are independently processed by a CNN and a PointNet architecture, respectively. The resulting outputs are then combined by a novel fusion network, which predicts multiple 3D box hypotheses and their confidences, using the input 3D points as spatial anchors. We eval…
NetEase（网易）；Huazhong University of Science and Technology（华中科技大学）
Recently, transformer-based methods have achieved promising progresses in object detection, as they can eliminate the post-processes like NMS and enrich the deep representations. However, these methods cannot well cope with scene text due to its extreme variance of scales and aspect ratios. In this paper, we present a simple yet effective transformer-based architecture for scene text detection. Different from previous approaches that learn robust deep representations of scene text in a holistic manner, our method performs scene text detection based on a few representative features, which avoid…
Kazan Federal University（喀山联邦大学）；Czech Technical University in Prague（捷克布拉格技术大学）
We propose ALFA - a novel late fusion algorithm for object detection. ALFA is based on agglomerative clustering of object detector predictions taking into consideration both the bounding box locations and the class scores. Each cluster represents a single object hypothesis whose location is a weighted combination of the clustered bounding boxes. ALFA was evaluated using combinations of a pair (SSD and DeNet) and a triplet (SSD, DeNet and Faster R-CNN) of recent object detectors that are close to the state-of-the-art. ALFA achieves state of the art results on PASCAL VOC 2007 and PASCAL VOC 2012…
FlatFusion: Delving into Details of Sparse Transformer-based Camera-LiDAR Fusion for Autonomous Driving
School of Computer Science and School of Artificial Intelligence, Shanghai Jiao Tong University（计算机科学学院和人工智能学院，上海交通大学）；Carnegie Mellon University（卡内基梅隆大学）
Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）；Leopoldo Américo Miguez de Mello Research and Development Center (Cenpes), Petrobras（巴西石油公司莱奥波尔多·亚美利哥·米格斯·德梅洛研发中心（Cenpes））
Deep learning-based models, such as convolutional neural networks, have advanced various segments of computer vision. However, this technology is rarely applied to seismic shot gather noise localization problem. This letter presents an investigation on the effectiveness of a multi-scale feature-fusion-based network for seismic shot-gather noise localization. Herein, we describe the following: (1) the construction of a real-world dataset of seismic noise localization based on 6,500 seismograms; (2) a multi-scale feature-fusion-based detector that uses the MobileNet combined with the Feature Pyr…
This paper presents Multi-view Labelling Object Detector (MLOD). The detector takes an RGB image and a LIDAR point cloud as input and follows the two-stage object detection framework. A Region Proposal Network (RPN) generates 3D proposals in a Bird's Eye View (BEV) projection of the point cloud. The second stage projects the 3D proposal bounding boxes to the image and BEV feature maps and sends the corresponding map crops to a detection header for classification and bounding-box regression. Unlike other multi-view based methods, the cropped image features are not directly fed to the detection …
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Royal Brompton Hospital（皇家布朗普顿医院）；Imperial Institute of Advanced Technology（帝国理工高等研究院）；Hangzhou Ocean’s Smart Boya Co., Ltd（杭州海洋智能博雅有限公司）；University of California, San Diego（加州大学圣地亚哥分校）；Shenzhen Second People’s Hospital（深圳市第二人民医院）
Explainable Artificial Intelligence (XAI) is an emerging research topic of machine learning aimed at unboxing how AI systems' black-box choices are made. This research field inspects the measures and models involved in decision-making and seeks solutions to explain them explicitly. Many of the machine learning algorithms can not manifest how and why a decision has been cast. This is particularly true of the most popular deep neural network approaches currently in use. Consequently, our confidence in AI systems can be hindered by the lack of explainability in these black-box models. The XAI bec…
Take Fake as Real: Realistic-like Robust Black-box Adversarial Attack to Evade AIGC Detection
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education（教育部航空航天信息安全与可信计算重点实验室）；School of Cyber Science and Engineering, Wuhan University（武汉大学网络安全学院）
NumbOD: A Spatial-Frequency Fusion Attack Against Object Detectors
National Engineering Research Center for Big Data Technology and System（国家大数据技术与系统工程研究中心）；Hubei Engineering Research Center on Big Data Security（湖北省大数据安全工程研究中心）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Cyber Science and Engineering, Huazhong University of Science and Technology（华中科技大学网络空间安全学院）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）
Microsoft（微软）；University of California Los Angeles（加利福尼亚大学洛杉矶分校）；New York University（纽约大学）
Vision-language (VL) pre-training has recently received considerable attention. However, most existing end-to-end pre-training approaches either only aim to tackle VL tasks such as image-text retrieval, visual question answering (VQA) and image captioning that test high-level understanding of images, or only target region-level understanding for tasks such as phrase grounding and object detection. We present FIBER (Fusion-In-the-Backbone-based transformER), a new VL model architecture that can seamlessly handle both these types of tasks. Instead of having dedicated transformer layers for fusio…
Mississippi State University（密西西比州立大学）；University of Missouri-Columbia（密苏里大学哥伦比亚分校）
A significant challenge in object detection is accurate identification of an object's position in image space, whereas one algorithm with one set of parameters is usually not enough, and the fusion of multiple algorithms and/or parameters can lead to more robust results. Herein, a new computational intelligence fusion approach based on the dynamic analysis of agreement among object detection outputs is proposed. Furthermore, we propose an online versus just in training image augmentation strategy. Experiments comparing the results both with and without fusion are presented. We demonstrate that…
A novel integrated method of detection-grasping for specific object based on the box coordinate matching
National Research Base of Intelligent Manufacturing, Chongqing Technology and Business University（重庆工商大学智能制造国家研究基地）；School of Mechanical Engineering, Chongqing Technology and Business University（重庆工商大学机械工程学院）；School of Electrical and Mechanical Engineering, University of Adelaide（阿德莱德大学机电工程学院）
The thighbone is the largest bone supporting the lower body. If the thighbone fracture is not treated in time, it will lead to lifelong inability to walk. Correct diagnosis of thighbone disease is very important in orthopedic medicine. Deep learning is promoting the development of fracture detection technology. However, the existing computer aided diagnosis (CAD) methods baesd on deep learning rely on a large number of manually labeled data, and labeling these data costs a lot of time and energy. Therefore, we develop a object detection method with limited labeled image quantity and apply it t…
School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；Alibaba Group, Inc.（阿里巴巴集团）
Learning subtle representation about object parts plays a vital role in fine-grained visual recognition (FGVR) field. The vision transformer (ViT) achieves promising results on computer vision due to its attention mechanism. Nonetheless, with the fixed size of patches in ViT, the class token in deep layer focuses on the global receptive field and cannot generate multi-granularity features for FGVR. To capture region attention without box annotations and compensate for ViT shortcomings in FGVR, we propose a novel method named Adaptive attention multi-scale Fusion Transformer (AFTrans). The Sele…
University of California, Berkeley（加利福尼亚大学伯克利分校）；Department of Mechanical Engineering（机械工程系）
We propose a new method for fusing a LIDAR point cloud and camera-captured images in the deep convolutional neural network (CNN). The proposed method constructs a new layer called non-homogeneous pooling layer to transform features between bird view map and front view map. The sparse LIDAR point cloud is used to construct the mapping between the two maps. The pooling layer allows efficient fusion of the bird view and front view features at any stage of the network. This is favorable for the 3D-object detection using camera-LIDAR fusion in autonomous driving scenarios. A corresponding deep CNN …
To advance models of multimodal context, we introduce a simple yet powerful neural architecture for data that combines vision and natural language. The "Bounding Boxes in Text Transformer" (B2T2) also leverages referential information binding words to portions of the image in a single unified architecture. B2T2 is highly effective on the Visual Commonsense Reasoning benchmark (https://visualcommonsense.com), achieving a new state-of-the-art with a 25% relative reduction in error rate compared to published baselines and obtaining the best performance to date on the public leaderboard (as of May…
