Prof. Dacheng Tao: ViTAE - Complexity in Diversity, Simplicity in Essence
On the morning of November 24, 2022, Prof. Dacheng Tao, the Dean of JD EXPLORE ACADEMY and Senior Vice President of JD.Com, Inc., was invited by Peking University School of Intelligence Science and Technology to deliver an exciting online presentation at the "Peking University Frontiers of Intelligence Science Forum." The title of the presentation was "ViTAE - Complexity in Diversity, Simplicity in Essence", which attracted the participation of many teachers and students from the School of Intelligence Science and Technology, Institute for Artificial Intelligence, Wangxuan Institute of Computer Technology, and School of Computer Science.
First, Prof. Tao reviewed the development of artificial intelligence and neural networks, briefly showing an important breakthrough timeline of artificial intelligence and neural networks from their inception to the present in the form of a diagram. Subsequently, Prof. Tao provided a comprehensive description of the various challenges that artificial intelligence faces today and in the future, such as the relationship between machines and humans, ethical issues in artificial intelligence, and social issues related to artificial intelligence, such as "price discrimination" in big data.
Next, Prof. Tao introduced in more detail how the field of artificial neural networks has evolved from a simple neuron to large-scale models. Starting from individual artificial neurons, to multilayer perceptrons with the introduction of the backpropagation algorithm, and then to more complex neural network structures proposed in recent years, such as AlexNet and ResNet, and finally to the currently widely studied large-scale models, Prof. Tao explained this progression from various perspectives. He not only described these works from mathematical derivations and network model perspectives but also took a broad view of the historical development of this discipline, explaining these works from multiple dimensions such as datasets, computational power, and theoretical advancements, broadening the students' horizons.
Furthermore, Prof. Tao focused on large-scale models and provided a more specific introduction to the students. First, he explained the structure and parameters of transformers from a theoretical perspective. Then, he led the students to explore how to expand network models in a deeper, wider, and larger direction. Finally, Prof. Tao discussed the application and performance of transformers, as well as the design philosophy and structure of ViTAE. Experimental results showed that the current models achieved excellent performance in image recognition, image classification, semantic segmentation, remote sensing scene recognition, and other fields.
In the final part of the presentation, Professor Zhouchen Lin and Professor Hongbin Zha respectively raised valuable questions to Prof. Tao from the perspectives of network design and the connection between large-scale models and the human brain. Prof. Tao provided answers to these two questions based on the current research status and his own understanding. Prof. Zha summarized and expressed gratitude for the excellent presentation shared by Prof. Tao, marking the successful conclusion of this event.
