Adversarial Attack Methods for Large Language Models

Large language models have seen rapid real-world deployment following ChatGPT's launch, spurring research into their vulnerabilities. Despite alignment training efforts such as RLHF, these models remain susceptible to adversarial attacks and jailbreak prompts designed to elicit harmful outputs. Adversarial attack research on images has been extensive, but attacking text-based models presents distinct challenges due to the discrete nature of language and the absence of direct gradient signals. Current research examines attacks at inference time with fixed model weights, as well as more sophisticated approaches targeting training data extraction or model poisoning. Researchers have developed multiple attack strategies. Token manipulation methods apply simple operations like synonym replacement in black-box settings. Gradient-based approaches, available only with full model access, leverage techniques such as Gumbel-Softmax to optimize attacks directly. Alternative methods include HotFlip, which computes loss derivatives relative to input vectors, and Universal Adversarial Triggers, short token sequences that function regardless of input content. The literature underscores advancing attack sophistication and the ongoing requirement for robust defenses in large language models.