Bias Detection Scanner

This scanner is designed to inspect the outputs generated by Language Learning Models (LLMs) to detect and evaluate potential biases. Its primary function is to ensure that LLM outputs remain neutral and don't exhibit unwanted or predefined biases.

Attack scenario

In the age of AI, it's pivotal that machine-generated content adheres to neutrality. Biases, whether intentional or inadvertent, in LLM outputs can be misrepresentative, misleading, or offensive. The Bias scanner serves to address this by detecting and quantifying biases in generated content.

How it works

The scanner utilizes a model from HuggingFace: valurank/distilroberta-bias. This model is specifically trained to detect biased statements in text. By examining a text's classification and score against a predefined threshold, the scanner determines whether it's biased.

Usage

from llm_guard.output_scanners import Bias
from llm_guard.output_scanners.bias import MatchType

scanner = Bias(threshold=0.5, match_type=MatchType.FULL)
sanitized_output, is_valid, risk_score = scanner.scan(prompt, model_output)

Optimization Strategies

Read more

Benchmarks

Test setup:

• Platform: Amazon Linux 2
• Python Version: 3.11.6
• Input length: 128
• Test times: 5

Run the following script:

python benchmarks/run.py output Bias

Results: