SIGMA is a data generation and training pipeline that enables large models to improve their safety alignment autonomously when only a Model Spec describing the desired behavior is available. First, the model leverages its reasoning ability to synthesize tasks guided by the spec: acting as a task designer, it creates diverse alignment dilemma scenarios and converts them into training tasks that stress‑test its understanding of the spec. Next, the model conducts self‑judged alignment training on these tasks, combining supervised fine‑tuning with rubric‑based reinforcement learning, where the model itself serves as the reward model. Although training uses only single‑turn chat data, safety improves markedly in multi‑turn agentic settings: AgentHarm harmfulness drops from 22.6 to 14.8 and Agentic Misalignment falls from 79.1 to 3.8; SIGMA also outperforms Deliberative Alignment and Constitutional AI baselines while preserving general capability. Analyses reveal that a spec balancing harmlessness and helpfulness, test‑time safety reasoning, and high‑quality rubrics from the task‑designer agent are essential for effective self‑improvement.
Review: SIGMA demonstrates a viable path for models to achieve alignment without external supervision by generating their own tasks and reward signals, offering valuable insights for future safe self‑improvement research.