Language Scanner

This scanner identifies and assesses the authenticity of the language used in prompts.

Attack scenario

With the rise of sophisticated LLMs, there has been an increase in attempts to manipulate or "confuse" these models. Some common tactics employed by users to attack LLMs include:

• Jailbreaks and Prompt Injections in different languages. For example, by utilizing unique aspects of the Japanese language to try and confuse the model. Paper: Multilingual Jailbreak Challenges in Large Language Models
• Encapsulation & Overloading: Using excessive code or surrounding prompts with a plethora of special characters to overload or trick the model.

The Language Scanner is designed to identify such attempts, assess the authenticity of the language used.

How it works

At its core, the scanner leverages the capabilities of papluca/xlm-roberta-base-language-detection model. The primary function of the scanner is to analyze the input prompt, determine its language, and check if it's in the list.

It supports the 22 languages:

arabic (ar), bulgarian (bg), german (de), modern greek (el), english (en), spanish (es), french (fr), hindi (hi), italian (it), japanese (ja), dutch (nl), polish (pl), portuguese (pt), russian (ru), swahili (sw), thai (th), turkish (tr), urdu (ur), vietnamese (vi), and chinese (zh)

Usage

from llm_guard.input_scanners import Language
from llm_guard.input_scanners.language import MatchType

scanner = Language(valid_languages=["en"], match_type=MatchType.FULL)  # Add other valid language codes (ISO 639-1) as needed
sanitized_prompt, is_valid, risk_score = scanner.scan(prompt)

Optimization Strategies

Read more

Benchmarks

Test setup:

• Platform: Amazon Linux 2
• Python Version: 3.11.6
• Input length: 1362
• Test times: 5

Run the following script:

python benchmarks/run.py input Language

Results: