Language Scanner

This scanner identifies and assesses the authenticity of the language used in outputs.

Attack scenario

With the rise of sophisticated LLMs, there has been an increase in attempts to manipulate or "confuse" these models. For example, model might produce an output in unexpected language.

The Language Scanner is designed to identify such attempts, assess the authenticity of the language used.

How it works

At its core, the scanner leverages the capabilities of papluca/xlm-roberta-base-language-detection model. The primary function of the scanner is to analyze the model's output, determine its language, and check if it's in the list.

It supports the 22 languages:

arabic (ar), bulgarian (bg), german (de), modern greek (el), english (en), spanish (es), french (fr), hindi (hi), italian (it), japanese (ja), dutch (nl), polish (pl), portuguese (pt), russian (ru), swahili (sw), thai (th), turkish (tr), urdu (ur), vietnamese (vi), and chinese (zh)

Usage

from llm_guard.output_scanners import Language
from llm_guard.input_scanners.language import MatchType

scanner = Language(valid_languages=["en", ...], match_type=MatchType.FULL)  # Add other valid language codes (ISO 639-1) as needed
sanitized_output, is_valid, risk_score = scanner.scan(prompt, model_output)

Optimization Strategies

Read more

Benchmarks

Test setup:

• Platform: Amazon Linux 2
• Python Version: 3.11.6
• Input length: 14
• Test times: 5

Run the following script:

python benchmarks/run.py output Language

Results: