In this short paper, we outline nine classical benchmarks submitted to the first hierarchical planning track of the International Planning competition in 2020. All of these benchmarks are based on the HDDL language. The choice of the benchmarks was based on a questionnaire sent to the HTN community. They are the following: Barman, Childsnack, Rover, Satellite, Blocksworld, Depots, Gripper, and Hiking. In the rest of the paper we give a short description of these benchmarks. All are totally ordered.
University of Maryland（马里兰大学）；Institute for Systems Research（系统研究学院）；IBM Watson Research Center（IBM沃森研究中心）
The SHOP2 planning system received one of the awards for distinguished performance in the 2002 International Planning Competition. This paper describes the features of SHOP2 which enabled it to excel in the competition, especially those aspects of SHOP2 that deal with temporal and metric planning domains.
Many planning techniques have been developed to allow autonomous systems to act and make decisions based on their perceptions of the environment. Among these techniques, HTN ({\it Hierarchical Task Network}) planning is one of the most used in practice. Unlike classical approaches of planning. HTN operates by decomposing task into sub-tasks until each of these sub-tasks can be achieved an action. This hierarchical representation provide a richer representation of planning problems and allows to better guide the plan search and provides more knowledge to the underlying algorithms. In this paper…
In this paper, we outline the implementation of the TFD (Totally Ordered Fast Downward) and the PFD (Partially ordered Fast Downward) hierarchical planners that participated in the first HTN IPC competition in 2020. These two planners are based on forward-chaining task decomposition coupled with a compact grounding of actions, methods, tasks and HTN problems.
This volume contains the system description of the 18 solvers submitted to the First International Competition on Computational Models of Argumentation (ICCMA'15) and therefore gives an overview on state-of-the-art of computational approaches to abstract argumentation problems. Further information on the results of the competition and the performance of the individual solvers can be found on at http://argumentationcompetition.org/2015/.
University of Udine（乌迪内大学）；KU Leuven（鲁汶大学）；Vives University College（维夫斯大学学院）
In this paper, we provide all information to participate to the Second International Nurse Rostering Competition (INRC-II). First, we describe the problem formulation, which, differently from INRC-I, is a multi-stage procedure. Second, we illustrate all the necessary infrastructure do be used together with the participant's solver, including the testbed, the file formats, and the validation/simulation tools. Finally, we state the rules of the competition. All update-to-date information about the competition is available at http://mobiz.vives.be/inrc2/.
INRIA Saclay（法国国家信息与自动化研究所萨克雷中心）；LRI（计算机研究实验室）；Université Paris-Sud（巴黎南大学）；ONERA-DCSD（法国航空航天实验室-DCSD）；THALES Research & Technology（泰雷兹研究与技术公司）
All standard AI planners to-date can only handle a single objective, and the only way for them to take into account multiple objectives is by aggregation of the objectives. Furthermore, and in deep contrast with the single objective case, there exists no benchmark problems on which to test the algorithms for multi-objective planning. Divide and Evolve (DAE) is an evolutionary planner that won the (single-objective) deterministic temporal satisficing track in the last International Planning Competition. Even though it uses intensively the classical (and hence single-objective) planner YAHSP, it…
This paper reports the outcome of the third in the series of biennial international planning competitions, held in association with the International Conference on AI Planning and Scheduling (AIPS) in 2002. In addition to describing the domains, the planners and the objectives of the competition, the paper includes analysis of the results. The results are analysed from several perspectives, in order to address the questions of comparative performance between planners, comparative difficulty of domains, the degree of agreement between planners about the relative difficulty of individual problem…
TU Dresden（德累斯顿工业大学）；TU Wien（维也纳工业大学）；Università di Genova（热那亚大学）
Argumentation is a major topic in the study of Artificial Intelligence. Since the first edition in 2015, advancements in solving (abstract) argumentation frameworks are assessed in competition events, similar to other closely related problem solving technologies. In this paper, we report about the design and results of the Second International Competition on Computational Models of Argumentation, which has been jointly organized by TU Dresden (Germany), TU Wien (Austria), and the University of Genova (Italy), in affiliation with the 2017 International Workshop on Theory and Applications of For…
Polytechnic School of Pernambuco - University of Pernambuco（伯南布哥大学伯南布哥理工学院）；PRHLT Research Center, Universitat Politécnica de Valéncia（瓦伦西亚理工大学PRHLT研究中心）；University of Bari（巴里大学）
This paper describes the short-term competition on the Components Segmentation Task of Document Photos that was prepared in the context of the 16th International Conference on Document Analysis and Recognition (ICDAR 2021). This competition aims to bring together researchers working in the field of identification document image processing and provides them a suitable benchmark to compare their techniques on the component segmentation task of document images. Three challenge tasks were proposed entailing different segmentation assignments to be performed on a provided dataset. The collected dat…
CRIL, University of Artois and CNRS（法国亚眠大学CRIL实验室及法国国家科学研究中心）；LIPADE, University of Paris（巴黎大学LIPADE实验室）
Since 2015, the International Competition on Computational Models of Argumentation (ICCMA) provides a systematic comparison of the different algorithms for solving some classical reasoning problems in the domain of abstract argumentation. This paper discusses the design of the Fourth International Competition on Computational Models of Argumentation. We describe the rules of the competition and the benchmark selection method that we used. After a brief presentation of the competitors, we give an overview of the results.
The 6th International Verification of Neural Networks Competition (VNN-COMP 2025): Summary and Results
RWTH Aachen University（亚琛工业大学）；Technical University of Munich（慕尼黑技术大学）；Stony Brook University（石溪大学）；George Mason University（乔治·梅奥大学）；Maynooth University（梅诺特大学）；Vanderbilt University（范德比尔特大学）；University of Manchester（曼彻斯特大学）；Amherst College（阿默斯特学院）；VMware Research by Broadcom（Broadcom VMware 研究）
ETH Zurich（苏黎世联邦理工学院）；RWTH Aachen University（亚琛工业大学）；Stony Brook University（石溪大学）；Carnegie Mellon University（卡内基梅隆大学）；Vanderbilt University（范德堡大学）
This report summarizes the 3rd International Verification of Neural Networks Competition (VNN-COMP 2022), held as a part of the 5th Workshop on Formal Methods for ML-Enabled Autonomous Systems (FoMLAS), which was collocated with the 34th International Conference on Computer-Aided Verification (CAV). VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools, encourage the standardization of tool interfaces, and bring together the neural network verification community. To this end, standardized formats for networks (ONNX) and …
Eindhoven University of Technology（埃因霍温理工大学）；Delft University of Technology（代尔夫特理工大学）；McGill University（麦吉尔大学）；Bielefeld University（比勒费尔德大学）；General Electric（通用电气）；Shopify（Shopify（电商平台））；Université de Montréal（蒙特利尔大学）；Polytechnic Institute of Viseu（维塞乌理工学院）；Rockets of Awesome（Rockets of Awesome（游戏公司））；TH Köln（科隆应用技术大学）；University of Málaga（马拉加大学）；Telecom Paris（巴黎电信学院（巴黎高等电信学院））
This paper reports on the first international competition on AI for the traveling salesman problem (TSP) at the International Joint Conference on Artificial Intelligence 2021 (IJCAI-21). The TSP is one of the classical combinatorial optimization problems, with many variants inspired by real-world applications. This first competition asked the participants to develop algorithms to solve a time-dependent orienteering problem with stochastic weights and time windows (TD-OPSWTW). It focused on two types of learning approaches: surrogate-based optimization and deep reinforcement learning. In this p…
University Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；Ulm University（乌尔姆大学）；The MITRE Corporation（MITRE公司）
Over the last year, the amount of research in hierarchical planning has increased, leading to significant improvements in the performance of planners. However, the research is diverging and planners are somewhat hard to compare against each other. This is mostly caused by the fact that there is no standard set of benchmark domains, nor even a common description language for hierarchical planning problems. As a consequence, the available planners support a widely varying set of features and (almost) none of them can solve (or even parse) any problem developed for another planner. With this pape…
The Optiplan planning system is the first integer programming-based planner that successfully participated in the international planning competition. This engineering note describes the architecture of Optiplan and provides the integer programming formulation that enabled it to perform reasonably well in the competition. We also touch upon some recent developments that make integer programming encodings significantly more competitive.
Max-Planck-Institut für Informatik（马克斯·普朗克信息学研究所）；Universität Dortmund（多特蒙德大学）
We provide an overview of the organization and results of the deterministic part of the 4th International Planning Competition, i.e., of the part concerned with evaluating systems doing deterministic planning. IPC-4 attracted even more competing systems than its already large predecessors, and the competition event was revised in several important respects. After giving an introduction to the IPC, we briefly explain the main differences between the deterministic part of IPC-4 and its predecessors. We then introduce formally the language used, called PDDL2.2 that extends PDDL2.1 by derived pred…
TALplanner is a forward-chaining planner that relies on domain knowledge in the shape of temporal logic formulas in order to prune irrelevant parts of the search space. TALplanner recently participated in the third International Planning Competition, which had a clear emphasis on increasing the complexity of the problem domains being used as benchmark tests and the expressivity required to represent these domains in a planning system. Like many other planners, TALplanner had support for some but not all aspects of this increase in expressivity, and a number of changes to the planner were requi…
Despite recent progress in AI planning, many benchmarks remain challenging for current planners. In many domains, the performance of a planner can greatly be improved by discovering and exploiting information about the domain structure that is not explicitly encoded in the initial PDDL formulation. In this paper we present and compare two automated methods that learn relevant information from previous experience in a domain and use it to solve new problem instances. Our methods share a common four-step strategy. First, a domain is analyzed and structural information is extracted, then macro-op…
The article presents a local search approach for the solution of timetabling problems in general, with a particular implementation for competition track 3 of the International Timetabling Competition 2007 (ITC 2007). The heuristic search procedure is based on Threshold Accepting to overcome local optima. A stochastic neighborhood is proposed and implemented, randomly removing and reassigning events from the current solution. The overall concept has been incrementally obtained from a series of experiments, which we describe in each (sub)section of the paper. In result, we successfully derived a…
PDDL was originally conceived and constructed as a lingua franca for the International Planning Competition. PDDL2.1 embodies a set of extensions intended to support the expression of something closer to real planning problems. This objective has only been partially achieved, due in large part to a deliberate focus on not moving too far from classical planning models and solution methods.
The hm admissible heuristics for (sequential and temporal) regression planning are defined by a parameterized relaxation of the optimal cost function in the regression search space, where the parameter m offers a trade-off between the accuracy and computational cost of theheuristic. Existing methods for computing the hm heuristic require time exponential in m, limiting them to small values (m andlt= 2). The hm heuristic can also be viewed as the optimal cost function in a relaxation of the search space: this paper presents relaxed search, a method for computing this function partially by searc…
Planning with numeric state variables has been a challenge for many years, and was a part of the 3rd International Planning Competition (IPC-3). Currently one of the most popular and successful algorithmic techniques in STRIPS planning is to guide search by a heuristic function, where the heuristic is based on relaxing the planning task by ignoring the delete lists of the available actions. We present a natural extension of ``ignoring delete lists'' to numeric state variables, preserving the relevant theoretical properties of the STRIPS relaxation under the condition that the numeric task at h…
Meituan（美团）；University of Chinese Academy of Sciences（中国科学院大学）；Harbin Institute of Technology（哈尔滨工业大学）
We present AMO-Bench, an Advanced Mathematical reasoning benchmark with Olympiad level or even higher difficulty, comprising 50 human-crafted problems. Existing benchmarks have widely leveraged high school math competitions for evaluating mathematical reasoning capabilities of large language models (LLMs). However, many existing math competitions are becoming less effective for assessing top-tier LLMs due to performance saturation (e.g., AIME24/25). To address this, AMO-Bench introduces more rigorous challenges by ensuring all 50 problems are (1) cross-validated by experts to meet at least the…
RealAI（瑞莱智慧）；Tsinghua University（清华大学）；Beijing Institute of Technology（北京理工大学）；Shanghai Jiao Tong University（上海交通大学）；China Nanhu Academy of Electronics and Information Technology（中国电子科技南湖研究院）；Xi’an Jiaotong University（西安交通大学）；Tencent YouTu Lab（腾讯优图实验室）；China Construction Bank Fintech（中国建设银行金融科技公司）；RippleInfo（涟漪信息科技）
The security of artificial intelligence (AI) is an important research area towards safe, reliable, and trustworthy AI systems. To accelerate the research on AI security, the Artificial Intelligence Security Competition (AISC) was organized by the Zhongguancun Laboratory, China Industrial Control Systems Cyber Emergency Response Team, Institute for Artificial Intelligence, Tsinghua University, and RealAI as part of the Zhongguancun International Frontier Technology Innovation Competition (https://www.zgc-aisc.com/en). The competition consists of three tracks, including Deepfake Security Competi…
CITlab（CITlab实验室）；Institute of Mathematics（数学研究所）；University of Rostock（罗斯托克大学）
We describe CITlab's recognition system for the HTRtS competition attached to the 13. International Conference on Document Analysis and Recognition, ICDAR 2015. The task comprises the recognition of historical handwritten documents. The core algorithms of our system are based on multi-dimensional recurrent neural networks (MDRNN) and connectionist temporal classification (CTC). The software modules behind that as well as the basic utility technologies are essentially powered by PLANET's ARGUS framework for intelligent text recognition and image processing.
