Driven by clear demand, Internet giants such as Baidu, Alibaba, and Tencent have joined the cloud AI chip market. Chip companies must also know where their customers are. In addition, in addition to the hardware itself, chip companies also need to pay attention to system-wide design, consider economic issues, apply more latest machine learning technologies in the field of software architecture, and build an open source ecosystem and community, so as to better serve users and seize customers.

Author: Yao Song

Yao Song is the current senior director of Xilinx’s artificial intelligence business, responsible for the company’s artificial intelligence business expansion and ecological construction in the global field.

Driven by clear demand, Internet giants such as Baidu, Alibaba, and Tencent have joined the cloud AI chip market. Chip companies must also know where their customers are. In addition, in addition to the hardware itself, chip companies also need to pay attention to system-wide design, consider economic issues, apply more latest machine learning technologies in the field of software architecture, and build an open source ecosystem and community, so as to better serve users and seize customers.

In the future, it is not yet known who can become the source of “destructive innovation” in analog computing, in-memory computing, optical computing, and quantum computing.

Trend One

-The number of chips developed by Internet companies has increased significantly

Baidu has Kunlun, and Ali has Hanguang. Where are your customers in the future and where is the company’s way out? How much space is left for startups?

Baidu is a frequent visitor of Hot Chips: under the leadership of chief architect Ouyang Jian, its Software-Defined Accelerator (SDA, software-defined accelerator) developed based on FPGA and XPU for various practical applications have all been reported on Hot Chips . This time Baidu showcased the Kunlun chip that was taped out using the 14nm process at Samsung last year.

Alibaba has also invested heavily in infrastructure and chip development in recent years. Last year, Dr. Zhang Jiansong of the Alibaba Dharma Academy team released the FPGA-based new generation of speech synthesis chip Ouroboros at Hot Chips. This year, Alibaba has two reports selected, including Hanguang developed by Jiao Yang (Jiao D). 800 NPU.

Both are AI chips developed by Internet companies, Baidu Kunlun and Ali Hanguang 800 are very different, but each has its own advantages. Baidu Kunlun emphasizes versatility. Although its peak performance of 256 INT8 TOPS/150W is not so prominent, it can support multiple models such as search engines, computer vision, NLP, and speech recognition. Hanguang 800 has outstanding performance indicators. Using TSMC’s 12nm process, it achieves an energy efficiency ratio of 825 INT8 TOPS/280W. However, because there is no external storage, the models that can be used are limited. The data currently disclosed are basically for ResNet-50 .

Regardless of the difference between the two chips, it is worth noting that Ali and Baidu, including many Internet companies, are deploying cloud AI chips:

On the one hand, Tencent strategically invested in Suiyuan Technology, which was founded by brother Zhao Lidong, who had worked for AMD for many years and also served as the CEO of RDA (RDA); on the other hand, Tencent is also quietly recruiting troops to deploy its own AI chips;

ByteDance made a very low-key strategic investment in an AI chip company, which also received a Series A financing from Sequoia China at the beginning of this year. So far, the old Internet giants BAT and the largest bytebeat among the emerging Internet companies have already had their own AI chip lineage forces. And any of these companies’ own demand for AI chips, the ratio of cost and benefit is not enough to support the development of a 7nm chip.

In the cloud AI chip market, in addition to monopolists such as Nvidia, there are also veteran giant players such as Intel and Xilinx. In addition to the aforementioned Internet companies self-researched or closely related companies in China, there are also Cambrian, Biren, and Denglin companies participating in the competition. Although Internet companies such as Kuaishou, Pinduoduo, Meituan, and Didi, which have been rapidly rising in recent years, have not seen the public AI chip layout, it is not ruled out that they will deploy through strategic investment in the future. The market for cloud AI chips in China is rapidly being carved up. In the future, even if there are opportunities for third-party independent AI chip companies, the entire market may only support one such company-all related companies must think about their own future Where are the customers and where is the company?

Trend Two

-The next generation of computing technology gets more attention

It is difficult to achieve simple, convenient, and low-cost simple structural changes. Analog computing, in-storage computing, quantum computing, photon computing, who will be the source of “destructive innovation”?

After 2018, the development of AI chips based on traditional digital integrated circuits has begun to slow down, and it has been difficult to see particularly fresh ideas and unprecedented micro-architectures. On the one hand, this is because the iteration range of in-depth algorithms for vision and speech recognition and other fields has begun to become smaller, and there are not so many new problems that need to be solved. On the other hand, AI chips are nothing more than ASICs in a special field. Many ideas for solving past problems can be migrated, and good gold mines have also been dug out first. Therefore, in the past two years, we have seen that the advancement of digital AI chips is more integrated with business (such as Internet company core manufacturing, Tesla’s autopilot chips), or using some new platform technologies (such as HBM, Chiplet) , Wafer-Scale Chip), the micro-architecture is also being upgraded, but there are no major changes that are particularly prominent.

The more essential problem is that people originally hoped that the opportunity for “disruptive innovation” or “disruptive innovation” in the field of AI chips would be difficult to achieve. As shown in Figure 1, disruptive innovation refers to a new type of technology that is simple, convenient, and low-cost. Although the performance cannot meet customer needs in the early stage, it will gradually improve until it meets customer needs and forms a subversion of traditional technology. . In the chip field, the cost of tape-out is getting higher and higher, reaching more than 100 million US dollars at the 7nm node. If there is no large number of orders to support the diluted cost, the average enterprise simply cannot afford it. Therefore, in the field of AI chips, it becomes: the larger the company, the more chips are sold, and the cheaper the chips. It is difficult to achieve simple, convenient, low-cost, simple structural changes, and it is difficult to become a source of “disruptive innovation”.

Hot Chips can let us see which product trends
Figure 1. Disruptive innovation model (source: “Innovator’s Answer”)

In this case, more and more new technology routes have attracted attention, such as analog computing, in-memory computing, optical computing, and so on. In Hot Chips last year, two in-store calculation reports from a French startup company Upmem and Princeton University’s Dr. Jia Hongyang were accepted as strong evidence.

This Hot Chips Tutorial chose an extremely important calculation route in the future: quantum computing. Two of the four reports came from the Google quantum computing team that achieved “Quantum Supremacy” last year, and two reports were from the IBM team and the Intel team. However, although quantum computing has received a lot of attention, it is still very far away from practicality. I personally think that it will be at least 10 years. As shown in Figure 2, James S. Clarke from Intel’s quantum computing team said that 50 qubits have been implemented, but the proof of concept has been completed. Commercial use in the future, such as password cracking, needs to achieve more than 1 million qubits. There is a long way to go.

Hot Chips can let us see which product trends

Figure 2. The scale of the quantum computing system needed to solve different problems (Source: James S. Clarke)

The photon computing solution introduced by the Lightmatter team incubated at MIT is closer to practicality in comparison. Using a Mach Zehnder interferometer (MZI for short) manufactured by a MEMS process, photon calculation transforms the traditional multiplication into phase modulation and interference of the optical path, and the calculation can be completed with almost no energy consumption. However, MZI still has a certain amount of signal loss, because if the optical path passes through multiple cascaded MZIs, the degree of optical path loss may make the final result wrong, and therefore it is impossible to achieve a particularly large calculation array. At the same time, a problem that still exists in photon computing is that it still needs to solve the memory wall, because MZI only replaces the multiplier.

It is worth mentioning that the MIT photonic computing team actually incubated two entrepreneurial companies. In addition to Lightmatter, there is also Lightelligence (also known as LightAI, Xizhi Technology) founded by the Chinese team. I also hope Lightelligence can do better and better.

Trend Three

-Keynote returns to AI algorithms and applications again

What is the next step for AI to enter large-scale applications, robots, AR, or brain-computer interfaces?

Hot Chips is the pinnacle event in the chip industry, and its Keynote selection also represents the current direction that everyone in the industry is most concerned about.

In 2017, Hot Chips, Google’s well-known architect, has many legends and jokes “God of programmers” Jeff Dean, at the conference did “Recent Advances in Artificial Intelligence via Machine Learning and the Implications for Computer System Design (based on machine Learning the latest developments in artificial intelligence and its impact on computer system design)”, enough to show that AI has truly entered mainstream applications, and everyone is beginning to pay attention to the latest developments in AI and how to design targeted systems.

Several Keynotes from 2018 to 2019 have nothing to do with AI. The major news in the chip industry in 2018 was that two major vulnerabilities in the Intel CPU, Spectre and Meltdown, were discovered. Therefore, Keynote invited Professor John Hennessy to explain the two vulnerabilities and the security of the processor; in the same year, Xilinx’s new president and CEO Victor Peng took office and launched the ACAP (Adaptive Computing Acceleration Platform, Adaptive Computing Acceleration Platform) architecture, which also received great attention, so he was also invited to the conference to do Keynote. In 2019, everyone’s most concerned question is whether Moore’s Law can be continued. Therefore, AMD CEO Lisa Su and Professor Huang Hansen, a professor at Stanford University and the then VP of TSMC Research, were invited to introduce their views.

In this year’s Keynote, Hot Chips invited Dan Belov, a distinguished engineer from DeepMind, to give a report titled “AI Research at Scale – Opportunities on the Road Ahead” to introduce the future to everyone New opportunities that AI research may bring. Dan Belov’s report did not mention at all the algorithms that have entered the stage of large-scale applications such as computer vision and speech recognition, but focused on introducing reinforcement learning and its application in the field of robotics, Go, graphics and other fields. Dan pointed out that from AlexNet in 2012 to the present, the efficiency of the algorithm has increased by 44 times (that is, the amount of calculation to achieve the same accuracy), and the total calculation volume of the algorithm has increased by 300,000 times. We still have a lot of work to do to fill this. Nearly 10,000 times the gap, so we also need to pay attention to system-wide design, consider economic issues, and apply more of the latest machine learning technologies in the field of software architecture.

Hot Chips can let us see which product trends

Figure 3. Professor Huang Hansen’s Keynote report at Hot Chips 2019 (Source: Yao Song)

Going back to a more essential question, as Professor Huang Hansen said in last year’s Hot Chips Keynote (see Figure 3), the development of semiconductor technology is largely driven by important applications, because we need to understand and predict future new Application-This is also the reason why DeepMind, an algorithm company, is invited to a chip industry summit to do Keynote. From radio in the 1940s, to computers in the 1970s, to PCs and the Internet in the 1990s, to mobile phones and mobile devices in the first 20 years of the 21st century, and at this point in time, the mobile market has become saturated, and AI has begun. Entering large-scale applications, of course everyone is thinking, what is the next step? Is it a robot, is it AR, or is it a brain-computer interface?

Concluding remarks

-AI chip is not a technology game

The experience of starting a business gave me the opportunity to meet entrepreneurs and investors from all walks of life. I once heard an investor share the logic and barriers of Internet trading platforms: For this type of platform, one side is connected to the supply side, and the other side is connected to the demand side. After development, the number of both sides of the supply and demand reaches a certain critical value. , The platform does not need to pay a particularly large cost and can obtain a natural increase in the scale of users, as shown in Figure 4. The platform can almost be said to be the business model with the strongest model and the highest revenue, such as the well-known Taobao, WeChat, Dianping, Meituan Waimai, Pinduoduo, Douyin, and Kuaishou, all of which are in this category.

Hot Chips can let us see which product trends
Figure 4. When the supply and demand sides reach a certain scale, Internet applications begin to grow naturally (Source: Yao Song)

As for AI chips, everyone always discusses technology, but in fact, we have to do much more than technology. Since 2017, I have emphasized the importance of software in all public reports, emphasizing not only to make chips perform well, but also to allow users to use new chips extremely simply. And I often quote Youzan’s founder Bai Ya’s pyramid model of products, mentioning that for AI chip products, to make users “inseparable”, the most important thing is the open source ecology and the community. Up to now, I feel more and more that the most fundamental of AI chip competition is the ecological competition similar to the Internet platform. As shown in Figure 5, when there are enough open source projects, when users first come into contact with AI chips , There is a higher chance of using your chip, and then he may continue to contribute more open source projects to achieve a positive cycle. Because of this, Xilinx pays more and more attention to the software ecology and developers. It has launched a software platform for all types of developers such as Vitis, and has continued to advance the developer ecology as a key area.

Hot Chips can let us see which product trends

Figure 5. The similarity between AI chip ecological development and Internet platform (Source: Yao Song)

At present, the leader of the developer ecology in the AI ​​chip field is NVIDIA: According to Huang Renxun, the founder and CEO of NVIDIA at the latest press conference, there are 1.8 million AI developers using NVIDIA GPUs; and AI beginners, I believe that more than 99% will Choose to buy a GPU and download the open source code to try. Are there any opportunities for AI chip startups, other large chip companies? Two points popped out of my mind: First, in the promotion of piano, I heard a saying “A Lang Lang is worth 10,000 piano teachers”. Second, the same platform, the degree of stability of different platforms is different: For example, Taobao, each merchant has its own supply chain, has its own investment costs in marketing, so it will be more stable; while Didi, connect The individual driver who enters does not have much cost on switching platforms, so wherever the subsidy is high, the driver will go to which platform to take the order.

The Links:   BG160160AFPEBB-01 1MBI600U4-120 MY LCD