Sci-Tech

Constructing a prevention and governance system for the illusion of large models and their value risks

2025-04-11   

Currently, big models are rapidly transforming human society with their powerful ability to generate text, images, and videos, bringing about an efficiency revolution and paradigm innovation in people's learning and work. More and more people are applying big models to daily practical activities. However, in this process, some users, due to a lack of necessary media literacy and information verification habits, are easily deceived and deceived by the surface tight artificial intelligence illusion. They unconditionally trust all the answers output by the big model, which in turn has a negative impact on their life, study, or work, leading to risks such as decision misguidance and cognitive bias. In the process of promoting the widespread application of big models, it is necessary to be vigilant and address the problem of big model illusion and its value risks, and accelerate the construction of a prevention and governance system that integrates technological optimization, legal regulation, and ethical adjustment. The reason for the generation of illusion in large models lies in the field of artificial intelligence. Illusion specifically refers to the phenomenon where the content generated by large models appears grammatically correct and logically rigorous, but in reality there are factual errors or unverifiable facts, characterized by "serious nonsense" and inability to confirm the basis of reality. The illusion of big models includes two types: factual illusion and fidelity illusion. The former refers to the inconsistency or fabrication of facts, while the latter refers to the inconsistency between generated content and user instructions, context, or logic. Essentially, the illusion problem of large models is a product of the combined effects of their technical architecture, training, and generative patterns, characterized by generality, contingency, randomness, and difficulty in avoiding or overcoming. In terms of generation mechanism, the core causes of the illusion of large models come from probability driven technical architecture, limitations of training data, and multiple couplings of human-machine interaction generation logic. Firstly, there is a capability gap in the technical architecture. At present, large-scale models mainly adopt the GPT paradigm based on the converter structure, which can significantly improve the accuracy and efficiency of natural language processing. However, there may be capability gaps in pre training, supervised fine-tuning, inference, and other stages, resulting in illusion problems that are inconsistent with facts, instructions, or context. For example, in the pre training stage, the large model uses autoregressive generation to predict the output word by word based on the probability distribution of historical markers. This mechanism naturally lacks the ability to control contextual semantic consistency, and is prone to prioritizing vocabulary combinations with higher probabilities but inconsistent with facts and logic, resulting in the illusion of "grammatically correct but content distorted" output. Secondly, there are natural flaws in the training data. The big model conducts in-depth learning based on massive data on the Internet. However, because the Internet corpus has not been strictly tested and processed, or because of error labeling, there are inevitably factual errors or logical contradictions. The big model lacks the ability to identify the authenticity of data, and is easy to capture or generate answers based on wrong data. For example, when asked to tell the story of "Lin Daiyu pulling down a willow tree" using the GPT 4 model, the model cannot distinguish the traps involved. Instead, it directly splices content from massive textual data without factual verification, and fabricates a ridiculous plot. Finally, there is the stereotype of human-machine interaction. The human feedback reinforcement learning adopted by large models can easily lead to problems such as making unfounded statements and falsifying facts in the process of deliberately catering to human needs. After misunderstanding the concept of "special refund", the chatbot of Air Canada continuously generated fictional refund conditions and time limits, ultimately leading to legal disputes. The unique technical architecture and generation logic of large models lead to the danger of self reinforcing illusions. The value risk of big model illusion lies in its random occurrence and unavoidable nature. There are also value risks such as weakening trust between humans and machines, leading to information polarization, disrupting social order, and even triggering ideological security crises. It is urgent to strengthen prevention and governance. The most direct harm of the illusion of big models is the misleading of user decisions, especially in fields such as healthcare, health, and finance. The authoritative expression style and smooth narrative logic of large models make erroneous information extremely confusing. If users rely too much on large models to generate information to assist decision-making, they are likely to be misled and have serious consequences. For example, believing in the wrong treatment plan provided by the big model may lead to uncontrollable or even further deterioration of the disease. Over time, this may weaken the trust relationship between humans and machines. What is even more worrying is that the value risk caused by the illusion of big models presents a diffusion path from individual decision misguidance to group cognitive bias and social order impact. In the field of public decision-making, illusions may distort policy cognition. If we do not strengthen the screening and control of the output information of large models, there is a high possibility of misinterpreting policies, making discriminatory remarks, and other illusions. This not only weakens the credibility of the government, but also endangers social public safety. In the field of ideological security, related threats are more covert. For example, relevant research has monitored that certain overseas models view the achievements and institutional advantages of socialism with Chinese characteristics through ideological colored glasses, deliberately mixing in false facts or incorrect evaluations, and forming outputs that are different from mainstream discourse. This kind of illusionary content, packaged in ideology and infused with value through knowledge Q&A, is far more misleading than traditional false information. The governance measures for preventing and managing the illusion of big models should establish a three-dimensional governance system of technological correction, legal regulation, and ethical adjustment. The problem of illusion should be eliminated through technological optimization, responsibility boundaries should be clarified through legal regulation, and value rationality should be cultivated through ethical adjustment, making big models a more reliable partner for humanity. Build a multi-level prevention and control system. 'Technology for technology' is the preferred path to solve the problem of illusion in large models. The "value sensitive design" or value alignment strategy of artificial intelligence ethics also relies on technological innovation and breakthroughs. This not only requires artificial intelligence enterprises and experts to improve the performance of large models by improving the quality of training data, strengthening external verification and fact checking, improving model inference ability, enhancing transparency and interpretability, but also encourages philosophy and social science experts to work together with artificial intelligence experts to help large models improve question answering accuracy, eliminate potential illusion problems and value risks through knowledge base optimization, training corpus error correction, value alignment monitoring, and other means. Establish an adaptive governance framework. Faced with the widespread application of large-scale models, agile, flexible, and standardized legislative governance is imperative. The National Cyberspace Administration and seven other departments have issued and implemented the Interim Measures for the Management of Generative Artificial Intelligence Services, which provide clear legal regulations and risk prevention requirements for training data selection, model generation and optimization, and service provision. This is conducive to promoting "intelligence for good" and facilitating the compliant application of large-scale models. The EU Artificial Intelligence Act requires large models to fulfill their disclosure obligations, ensure the robustness and reliability of technical solutions, and other regulatory requirements, forming an effective institutional constraint and accountability framework for the application of large models, which is worth referencing and learning from. Improve the value benchmark for technology development and application. The more meaningful governance innovation for the illusion of big models lies in establishing technological values, integrating ethical values such as responsible innovation and controllable creativity into the minds of engineers, and embedding them into the code of big models. For example, advocating the principles of not generating controversial conclusions, not generating untraceable information, and not generating content beyond the cognitive boundaries of the model, promoting the transformation of large models from pursuing smooth generation to ensuring content reliability; For example, establishing a hierarchical confidence prompt system for answering factual questions in large models, categorizing and annotating conclusions based on high credibility, verifiability, and speculative nature, and enhancing the transparency and interpretability of output content. For users, they should further improve their information literacy in the scientific and correct application of large models, and become commanders in guiding content generation and discriminators of illusion problems. Research has shown that training on usage habits through artificial intelligence cross validation can significantly reduce the probability of users being misled by illusions. People need to keep up with the times to enhance their comprehensive abilities in distinguishing illusions, mastering common sense, and critical thinking. In the process of using large models to retrieve information and generate content, they should adhere to the value principles of fact checking, logical verification, professional identification, minimum necessity, and scene control, and minimize illusion problems and value risks to the greatest extent possible. The reliability construction of artificial intelligence often lags behind its ability expansion. The ultimate goal of managing the illusion of big models is not to completely eliminate technological uncertainty, but to establish a risk controllable human-machine collaboration mechanism. In this human-machine collaborative cyber evolution, maintaining technological humility and ethical clarity is the rightful way to dispel the illusion of big models. (Author: Li Ling, Associate Researcher at the Institute of Marxism at Fudan University) (New Press)

Edit:He Chuanning Responsible editor:Su Suiyue

Source:Guang Ming Daily

Special statement: if the pictures and texts reproduced or quoted on this site infringe your legitimate rights and interests, please contact this site, and this site will correct and delete them in time. For copyright issues and website cooperation, please contact through outlook new era email:lwxsd@liaowanghn.com

Recommended Reading Change it

Links