AIGC大模型：开创工程与科学双重革命时代

What is the Big Model of Artificial Intelligence

Artificial intelligence macromodels are huge neural network models constructed using large-scale data and complex network structures in the field of machine learning and deep learning.

This development is an engineering revolution, not just a scientific revolution. The amount of data in large language models has grown exponentially over the past decade or so, and this trend exists for large models in other domains as well. As the amount of data in large models increases, the generalization will also change qualitatively.

In practical applications, it is not the larger the parameter scale that is better, but rather a number of factors need to be taken into account to determine the most suitable model scale. Targeted knowledge enhancement for different industries will play an important role. With the evolution of technology, the big model industry will move toward advancing the automation process of building and deploying models to lower the threshold for industry users to acquire AI capabilities.

Overall performance of large domestic and international models

The following figures are based on data measured by SuperCLUE (Chinese Language Understanding Evaluation Benchmark), which is a continuation and development of the CLUE benchmark in the era of big models, focusing on the comprehensive evaluation of generalized big models.

There is a clear gap between the performance of large models at home and abroadThe total score of GPT4-Turbo is 90.63 points far ahead, higher than other domestic and foreign big models. The best domestic model, Wenxin Yiyin 4.0 (API), has a total score of 79.02, 11.61 points away from GPT4-Turbo and 4.9 points away from GPT4 (web).

It is worth noting that domestic big models have made great progress in the past year, with 11 models exceeding GPT3.5 and Gemini-Pro in terms of their comprehensive capabilities. e.g., Baidu's Wenxin Yiyin 4.0, AliCloud's Tongyi Qianqian 2.0 and Qwen-72B-Chat, OPPO's AndesGPT, Tsinghua & Zhispectrum Al's Zhispectrum Qingyin, and Byte Jump's Lark Big Model, etc. all performed relatively well.

In addition.Domestic open-source models outperform foreign open-source models in Chinese, for example, Baichuan Intelligence's Baichuan2-13B-Chat, Aliyun's Qwen-72B and Yi-34B-Chat are superior to Llama2-13B-Chat.

Fig. 1 Benchmark scores of SuperCLUE for large domestic and foreign models

Classification of Large Models

Universal Large ModelIt refers to a large neural network model that can handle multiple natural language tasks, characterized by powerful language understanding and generation capabilities, and capable of handling a variety of natural language tasks, such as text categorization, sentiment analysis, and question and answer systems. Generalized large models include BERT developed by Google, GPT-2 developed by OpenAI, RoBERTa developed by Facebook, and so on.

large-scale model of draping class (botany)It is a large neural network model optimized for a specific domain or task, which is characterized by higher accuracy and efficiency, and can be better adapted to the needs of a specific domain. Pendant large models include BioBERT in the medical field, FinBERT in the financial field, LegalBERT in the legal field, and so on.

Fig. 2 Panoramic view of Chinese big model

Large Model ServiceIt refers to the application of large neural network models to actual business scenarios and the provision of corresponding services and solutions, which are characterized by a high degree of customization and flexibility to meet the needs of different customers. Large model services include intelligent customer service, intelligent recommendation, intelligent risk control, etc.

Fig. 3 Big model architecture diagram

Industrial Efficiency Revolution Driven by Big Models

Big Models Will Spark a Revolution in Industrial Efficiency.. Through deep learning and training on large-scale data, large models enable intelligent interactions that are multimodal, generative, interpretable, and conversational.

When targeting more specific domains and scenarios, combining technologies such as knowledge graph, transfer learning and joint learning to efficiently combine the expertise of different vertical domains and build specialized big models with domain expertise and business logic.. Such models can provide intelligent solutions to specific scenarios and problems in various industries, fundamentally reducing the cost and threshold of downstream application of big models, so that more enterprises and organizations can conveniently apply the powerful capabilities of big models to improve their efficiency and innovation.

Deep integration of the real economy with the digital economy driven by big models as a key driver to promote the strengthening, optimization and expansion of the real economy. For example, automobile manufacturing, energy, transportation and other industries can innovate in the fields of intelligent customer service, supply chain, system scheduling and other areas through big models to promote the digital transformation and intelligent enhancement of the industry.

Figure 4 AI data industry mapping

In addition, large models have the following advantages over traditional AI models::

Solves the problem of AI fragmentation and diversification and improves the generalizability of modelsTraditional AI models require customized development, adjustment and optimization. Traditional AI models require customized development, tuning and optimization, which increases human investment, while the big model adopts a "pre-training + fine-tuning" approach, storing a large amount of information and fine-tuning, which greatly improves the general usability.
Self-supervised learning capability reduces training R&D costs. With self-supervised learning, the need for data labeling is reduced, allowing even large amounts of unlabeled data to be fully utilized, reducing labor costs and enabling small sample training.
Freedom from the limitations imposed by structural change opens the upper limit of model accuracy. In the past, to improve model accuracy relied primarily on changes to the network structure, but this became difficult as structural design techniques matured. It has been shown that larger data sizes can increase the upper bound of model accuracy.

Large modeling trends

In 2023, the AI large model market experienced the preparation period, the growth period and eventually reached the outbreak of a hundred barges, in which the growth period representative model has Baidu released Wenxin Yiyin, the second half of the outbreak of the second half of the period, such as the GPT-4 release of Turbo.

Figure 5 2023 Large Model Development Timeline

Among them, the GPT is rapidly iterating from GPT 1.0 into the 3.5 era GPT is a large-scale unsupervised language model, including GPT-1, GPT-2, and GPT-3. GPT-1 utilizes unsupervised pre-training and supervised fine-tuning, and has good generalization ability; however, GPT-2 employs a multi-tasking mode to improve the generalization ability, which verifies that the larger the model capacity and the amount of data, the larger the potential; GPT-3 outperforms the traditional based on the huge amount of parameters and training data. techniques and performs well in multiple tasks.GPT3.5 introduced human feedback reinforcement learning, and its variant code-davinci-002 was fine-tuned to spawn ChatGPT, which uses a version of human feedback-based reinforcement learning to fine-tune the model with instructions.

Figure 6 Iterative diagram of the development of the large model

The rapid expansion of the artificial intelligence large model market is trending, and since 2020, large pre-trained models have demonstrated superior performance in areas such as natural language processing, computer vision, speech recognition, and recommender systems, triggering widespread attention in the industry.

Meanwhile, government support and investment as well as the promotion of technology enterprises have strengthened the cultivation and introduction of talents and promoted the development of China's big model industry. With further technological breakthroughs and innovations, China is expected to achieve more results in the field of big modeling and promote the development and application of AI big modeling together with the world's leading countries.