Outline

Ingegneria Sismica

Ingegneria Sismica

RC-CoSA: Controllable secure alignment architecture for large language models based on risk-constrained inference search

Author(s): Linghao Meng1
1School of Electrical Automation and Information Engineering, Tianjin University, Tianjin, China, 300072
Meng, Linghao . “RC-CoSA: Controllable secure alignment architecture for large language models based on risk-constrained inference search.” Ingegneria Sismica Volume 43 Issue 3: 1-20, doi:10.65102/is20261283.

Abstract

The current secure alignment of large language models (LLMs) generally adopts a static paradigm, that is, training a single model through predefined general principles. However, this approach lacks flexibility in the face of diverse security needs in different cultural backgrounds, geographical norms, and specific application scenarios. At the same time, re-aligning models for each segment requirement will bring high computing costs and engineering overhead. To this end, we propose a risk-constrained controllable safety alignment architecture (RC-CoSA), which aims to adapt the model to diverse and intertwined safety requirements in the inference stage without updating the underlying model parameters. Compared with existing methods that rely on single-sample autoregressive generation, RC-CoSA improves the robustness and controllability of response generation under complex security configurations through compliance-first best-of-N candidate screening, structured security completion for partial-compliance scenarios, and decoupling multi-stage reasoning-evaluation process. The experimental results show that the actual benefits of RC-CoSA have a certain base dependence: on the DeepSeek base, the proposed method significantly reduces the Helpful + Unsafe ratio from 11.0% to 0.5%, and increases the CoSA-Score to 0.596, and improves the overall information validity. On the GPT-4o base, RC-CoSAlign also increased the CoSA-Score from 0.288 to 0.349 and the Helpful + Safe from 50.8% to 61.9%, but its compression of the risk of violations is relatively limited. On the Llama3.1-8B-INST base, although the inference period enhancement can improve the comprehensive control performance, its inhibition stability against the risk of violation is still affected by the characteristics of the base model. The above results show that RC-CoSA, as an inference-period execution control framework, can effectively improve the comprehensive controllability of the model under complex security configurations, but its benefit intensity is still affected by the original security boundary, generation distribution and instruction compliance ability of the base model.

Keywords
Large Language Models, Risk-Constrained Controllable Safety Alignment, Best-of-N Optimization, Inference-Time Adaptation

Related Articles

Zhihao Jiang1,2, Limi Chen1,2, Jing Yang1
1Hainan Vocational University of Science and Technology, Haikou 571126, China
2Institute for Mathematical Research, Universiti Putra Malaysia, Serdang 43400, Malaysia
Limi Chen1,2, Zhihao Jiang1,2, Jing Yang1
1Hainan Vocational University of Science and Technology, Haikou 571126, China
2Institute for Mathematical Research, Universiti Putra Malaysia, Serdang 43400, Malaysia
Hui Yuan1, Minjie Chai2, Siqing Xu1, Jinsong Li1, Jinwan Zheng1
1Electric Power Research Institute, State Grid Shanxi Electric Power Co., Ltd., Taiyuan, 030001, Shanxi, China
2Jincheng Power Supply Branch, State Grid Shanxi Electric Power Co., Ltd., Jincheng, 048000, Shanxi, China
Yanhan Zhu1,2
1China Academy of Cultural Heritage, Chaoyang District, 100029, Beijing, China
2Beijing University of Civil Engineering and Architecture, Xicheng District, 100044, Beijing, China
Ken Wang1, Jinhan Shu2, Kan Yuan1
1School of Digital Media, Shenzhen Polytechnic University, Shenzhen 518055, Guangdong, China
2Postdoctoral Mobile Station of Journalism and communication, Fudan University, Shanghai 200433, Shanghai, China