Ban Topics Scanner

This scanner is designed to restrict specific topics, such as religion, violence, from being introduced in the prompt using Zero-Shot classifier.

This ensures that interactions remain within acceptable boundaries and avoids potentially sensitive or controversial discussions.

Attack scenario

Certain topics, when used as prompts for Language Learning Models, can lead to outputs that might be deemed sensitive, controversial, or inappropriate. By banning these topics, service providers can maintain the quality of interactions and reduce the risk of generating responses that could lead to misunderstandings or misinterpretations.

How it works

It relies on the capabilities of the following models to perform zero-shot classification:

Collection on HuggingFace

Usage

from llm_guard.input_scanners import BanTopics

scanner = BanTopics(topics=["violence"], threshold=0.5)
sanitized_prompt, is_valid, risk_score = scanner.scan(prompt)

How to configure topics

The topics to be banned can be chosen based on the use-case and the potential risks associated with it.

The dataset, which was used to train the zero-shot classifier model can be found here. It will give you an idea of the topics that the model can classify.

Additionally, we recommend experimenting with the formulating of the topics to choose the longer options (Read more).

Optimization Strategies

Read more

Benchmarks

Test setup:

• Platform: Amazon Linux 2
• Python Version: 3.11.6
• Input Length: 100
• Test Times: 5

Run the following script:

python benchmarks/run.py input BanTopics

Results: