Tsinghua University, together with top teams such as Alibaba Security and RealAI, released the first fair and comprehensive AI against offense and defense benchmark platform. Is the AI model safe, what are the attack and defense capabilities? Just submit to the platform, you can see the ability ranking.
From the perspective of development, artificial intelligence is changing from the first generation of knowledge-driven and the second-generation data-driven to the third-generation multi-driven, with knowledge, data, algorithms and computing power becoming the four major factors. Safety and controllability has also become the core development goal of the third-generation artificial intelligence, and data and algorithm safety has become one of the research topics that academia and industry people focus on. Among them, at the data security level, data leakage and poisoning are two important sources of data security risks; at the algorithm security level, the security components of adversarial samples for artificial intelligence applications such as face recognition, identity authentication, and face-swiping gates a huge challenge.
In recent years, we have seen typical cases of AI algorithms being broken in many scenarios. Since 2016, the Autopilot automatic assisted driving system on Tesla Model S, Model X and vehicles has been breached by Tencent Keen Security Lab. High-risk security loopholes and defects in AI algorithms have put vehicles in a dangerous state and seriously threatened Personal and property safety; in 2021, 19 domestic Android phones using 2D face recognition technology will be successfully unlocked by RealAI using special glasses with anti-attack capabilities, which has raised concerns about face payment, online identity verification, etc. .
As AI models and algorithms face various challenges, how to accurately detect the offensive and defensive capabilities of each AI offensive and defensive model becomes more and more important. At this time, if there is a platform that can rank the offensive and defensive capabilities of AI models and algorithms, then we can adjust and improve in a timely manner, and take preventive measures in a targeted manner, which can also reduce the security risks in the process of technology implementation.
At the 2021 Beijing Zhiyuan Conference, Tsinghua University, together with Alibaba Security and RealAI, released the industry’s latest Adversarial Robustness Benchmark based on a deep learning model. This benchmark can measure different AI more fairly and comprehensively. The effect of the attack and defense algorithm provides an easy-to-use robustness test tool to comprehensively measure the attack and defense capabilities of the AI attack and defense model. Users can obtain the ranking of offensive and defensive capabilities by submitting models.
The Necessity of Building a Fair and Comprehensive AI Against Offensive and Defense Benchmarking Platform
In-depth study of potential attack algorithms against machine learning models is of great significance to improve the security and reliability of machine learning. In the past, when researchers measured the defense performance of the model, they basically only tested under one attack algorithm, which is obviously not comprehensive enough. Attack algorithms change frequently, and it is necessary to consider the defense capabilities of the model under multiple attack algorithms and stronger attacks, so that the defense capabilities of the AI model can be evaluated systematically.
At the same time, the various “attack algorithm rankings” previously proposed by the industry only include some scattered algorithms, and the environment for measuring attack algorithms only includes a single defense algorithm. There are not many data sets for evaluation, and there is no suitable Statistics and metrics.
Therefore, the AI confrontation security benchmark launched this time basically includes the current mainstream AI confrontation attack and defense models, covering dozens of typical attack and defense algorithms. In the process of comparing different algorithms, the same experimental settings and consistent measurement standards are used as much as possible, so as to ensure the fairness of the comparison to the greatest extent.
Examples of attack results and defense results rankings of AI algorithms. The left is the defense algorithm ranking, and the right is the attack algorithm ranking.
Benchmark test platform website: http://ml.cs.tsinghua.edu.cn/adv-bench
By ranking the attack results and defense results of AI algorithms, and comparing the performance of different algorithms, it has important academic significance for establishing AI security benchmarks, which can more fairly and comprehensively measure the effects of different algorithms.
Xue Hui, technical director of Alibaba’s security department, said, “Participating in the promotion of this research work is not only to help AI models conduct scientific assessments of security, but also to promote the AI industry to further build ‘strong’ AI.
Development and Significance of AI Offensive and Defense Benchmark Platform
In recent years, international competitions on AI against offense and defense have emerged, such as the NIPS 2017 Adversarial Sample Offensive and Defense Competition, and the 2018 DEFCON CAAD CTF Adversarial Offensive and Defense Competition, organized by Ian Goodfellow, the father of Generative Adversarial Networks. Among them, in the NIPS 2017 adversarial sample attack and defense competition, Professor Zhu Jun’s team won the championship in all three projects.
In 2020, the Artificial Intelligence Research Institute of Tsinghua University developed and open-sourced the AI confrontation security algorithm platform ARES (Adversarial Robustness Evaluation for Safety). This is a Python library for adversarial machine learning research dedicated to accurate and comprehensive benchmarking of the adversarial robustness of different models on image classification tasks. This algorithm platform is also the main support for the AI confrontation robust evaluation benchmark released this time.
GitHub project address: https://github.com/thu-ml/ares
Paper address: https://arxiv.org/pdf/1912.11852.pdf
In this benchmark, we use 16 defense models (half each on the CIFAR-10 and ImageNet datasets) and 15 attack methods for adversarial robustness evaluation. The figure below (top) is the defense model, and the figure (bottom) is the attack method (where FGSM, BIM, and MIM use white-box and migration-based attacks, respectively). This benchmark brings together the current mainstream and representative adversarial attack and defense algorithms, and the paper is also selected for the CVPR 2020 Oral.
In addition to dozens of typical attack and defense algorithms, the AI security ranking released this time also includes the attack algorithms of the top 5 teams born in the just-concluded CVPR 2021 artificial intelligence attack and defense competition. This competition attracted more than 2,000 teams from around the world to submit the latest algorithms. The contestants submitted attack algorithms based on the ARES platform, and performed accurate robustness tests on the existing adversarial defense models, which further improved the scientificity and credibility of the security benchmark. .
The top 5 teams in the “Track 1 Defense Model White Box Adversarial Attack” in the CVPR 2021 AI Offense and Defense Competition.
Therefore, based on the previous research results and the algorithms submitted in the CVPR 2021 artificial intelligence attack and defense competition, Tsinghua University, Alibaba Security and RealAI released the latest AI confrontation robustness evaluation benchmark platform. The full timeline is as follows:
Tang Jiayu, vice president of RealAI, said: “The benchmark evaluation platform uses typical offensive and defensive algorithms and multiple algorithms with superior performance accumulated in the CVPR 2021 competition to conduct mutual evaluation, representing the current international standards for security and stability measurement.”
Tsinghua University, Alibaba Security and RealAI all emphasized that the benchmark evaluation platform is not a platform built by a certain institution or company, and requires the joint participation of industry and academia to build it into a truly recognized comprehensive and authoritative AI security platform. Evaluation platform. Therefore, the three parties will join forces to continuously inject new attack and defense algorithms into the leaderboard, and welcome teams from academia and industry to submit new attack and defense models through the ARES platform.
The release of the platform will have a positive impact on both industry and academia. For example, the industry can use the platform to evaluate the security of current AI services and discover model security vulnerabilities. At the same time, it can also provide a comprehensive, objective, fair and scientific industry standard for the academic community, and promote the rapid development of the entire academic community in the field of AI countermeasures.