Dr. Hang Li: Artificial Intelligence Needs New Computing Paradigms and Theories
On the morning of November 17, 2022, Dr. Hang Li, the head of the research department at ByteDance Technology Co., Ltd., an ACL Fellow, IEEE Fellow, and ACM Distinguished Member, was invited by Peking University to deliver an exciting online presentation at the "Peking University Frontiers of Intelligence Science Forum." The title of the presentation was "Artificial Intelligence Needs New Computing Paradigms and Theories".
Dr. Li first started with the latest developments in artificial intelligence and introduced several major technologies developed in the industry to discuss the computing paradigms and theories that artificial intelligence needs.
In the forefront of current research in the field of artificial intelligence, the differences among processing various media such as images, speech, and language lie more in the data format. For example, models based on Transformer can now be applied to many different forms of data. Pre-training, self-supervised learning, and other methods are also becoming increasingly important. In addition, artificial intelligence is gradually expanding into new fields such as AI for Science.
AI Lab in industry has developed effective models adapted to various forms of data, including non-autoregressive model DA-Transformer, end-to-end speech-to-text translation ConST, multi-granularity visual-language model X-VLM, image and text unified generation model DaVinci, and language understanding model Neural Symbolic Processor, etc.
Dr. Li pointed out that the field of artificial intelligence may require new computing paradigms, among which brain-inspired computing may be a valuable direction. Brain-inspired computing refers to building machine learning and reasoning intelligence systems based on the mechanisms of the human brain, focusing on addressing issues such as sample efficiency, energy efficiency, and logical reasoning. The human brain operates in separate regions, including areas for analogical reasoning and logical reasoning. In the future, we hope that artificial intelligence models can not only perform analogical reasoning based on existing datasets but also have innate abilities for reasonable logical reasoning.
In terms of the computational theory of artificial intelligence, the main theoretical research problems currently include generalization ability, optimization ability, robustness, and interpretability. Dr. Li introduced the information bottleneck theory and the principle of maximum encoding rate reduction and pointed out that the popular Transformer structure today is an integration of self-attention networks. Using only this module would lead to model degradation. Future research can be conducted in directions such as information flow, data structure, and function representation.
In the end, Dr. Li patiently answered questions from teachers and students online about the content of the presentation. Professor Zhouchen Lin summarized and expressed gratitude for the excellent presentation shared by Dr. Li, marking the successful conclusion of this event.
