Documentation

NAME

Mail::MIMEDefang::ML - provider-neutral ham/spam/phishing verdicts from machine-learning models.

SYNOPSIS

use Mail::MIMEDefang::ML;

# Pick a backend, once, at the top of mimedefang-filter:
$Mail::MIMEDefang::ML::Config{backend} = 'gliclass';   # or 'laya', 'openai'

my $state = ml_build_state(
    From    => $Sender,
    Subject => $subject,                    # RFC 2047-decoded
    Body    => md_get_plain_text_body($entity),
    SPF     => $SPFResult,
    DKIM    => $DKIMResult,
    DMARC   => $DMARCResult,
    SAScore => $hits,                       # optional, see ml_build_state()
    SARules => $names,
);

my $verdict = ml_classify($state);
# $verdict->{is_spam}{answer}      -> 1, 0, or undef (no opinion)
# $verdict->{is_spam}{confidence}  -> 0.0 .. 1.0
# $verdict->{is_phishing}{answer}
# $verdict->{is_phishing}{confidence}
# $verdict->{backend}              -> name of the backend that answered
# $verdict->{error}                -> set, and the rest absent, on failure

DESCRIPTION

Mail::MIMEDefang::ML is a single interface to several machine-learning classification backends, so that a filter keeps the same code whichever model gives the verdict. Laya and GLiClass are encoder classifiers, small and fast because they classify in one forward pass instead of generating text; the openai backend talks to a generative LLM.

Each backend lives in Mail::MIMEDefang::ML::<Name>. All backends are loaded together with this module, i.e. when the filter is loaded, before MIMEDefang changes into the per-message work directory.

This module never blocks mail flow on a backend's availability: any failure to reach the backend, or a low-confidence verdict, is treated as "no opinion" so callers can fall back to their existing SpamAssassin/rspamd-only path.

The stock models are not trained on your mail and their accuracy varies a lot from site to site: for good results, train them first, see "TRAINING A MODEL".

Backends are self-hosted model servers; see "SECURITY" for how to keep them safe. mimedefang-laya-server and mimedefang-gliclass-server, with instructions for installing their Python modules and models, are in the script/ml-servers/ directory of the MIMEDefang sources. Both servers are also available as Docker images, mimedefang/laya and mimedefang/gliclass on Docker Hub.

The body truncation limits below are the defaults for every backend; each backend's section can override them.

CONFIGURATION

Site admins override %Mail::MIMEDefang::ML::Config from mimedefang-filter. The keys, with their defaults:

# Common settings
$Mail::MIMEDefang::ML::Config{enabled}         = 1;       # 0 turns ml_classify() into a no-op
$Mail::MIMEDefang::ML::Config{backend}         = 'laya';  # 'laya', 'gliclass' or 'openai'
$Mail::MIMEDefang::ML::Config{timeout}         = 30;      # seconds per HTTP request
$Mail::MIMEDefang::ML::Config{connect_retries} = 1;       # retries when the server can't be
                                                          # reached, never after a timeout
$Mail::MIMEDefang::ML::Config{min_confidence}  = 0.55;    # below this -> no opinion
$Mail::MIMEDefang::ML::Config{questions}       = [qw(is_spam is_phishing)];
                                                          # questions asked, e.g.
                                                          # ['is_phishing'] alone
$Mail::MIMEDefang::ML::Config{question_texts}  = {};      # question wording, see below

# What ml_build_state() puts in the state, and what backends get of it
$Mail::MIMEDefang::ML::Config{body_head_chars} = 1500;    # body kept: first N chars ...
$Mail::MIMEDefang::ML::Config{body_tail_chars} = 500;     # ... plus last N chars
$Mail::MIMEDefang::ML::Config{spam_top_rules}  = 8;       # rules kept per engine
$Mail::MIMEDefang::ML::Config{url_max_chars}   = 80;      # longer URLs are cut, 0: never
$Mail::MIMEDefang::ML::Config{max_link_domains} = 5;      # link domains listed in the signals
$Mail::MIMEDefang::ML::Config{max_attachments}  = 5;      # attachments listed in the signals

# Laya (mimedefang-laya-server)
$Mail::MIMEDefang::ML::Config{laya}{server_url}     = 'http://127.0.0.1:8687';
$Mail::MIMEDefang::ML::Config{laya}{send_signals}   = 0;

# GLiClass (mimedefang-gliclass-server)
$Mail::MIMEDefang::ML::Config{gliclass}{server_url} = 'http://127.0.0.1:8688';
$Mail::MIMEDefang::ML::Config{gliclass}{body_head_chars} = 6000;
$Mail::MIMEDefang::ML::Config{gliclass}{body_tail_chars} = 1500;

# Self-hosted generative model (Ollama, llama.cpp, vLLM, LM Studio)
$Mail::MIMEDefang::ML::Config{openai}{base_url}     = 'http://127.0.0.1:11434/v1';
$Mail::MIMEDefang::ML::Config{openai}{model}        = undef;   # required, e.g. 'qwen3:8b' (>= 7-8B parameters)
$Mail::MIMEDefang::ML::Config{openai}{max_tokens}   = 100;
$Mail::MIMEDefang::ML::Config{openai}{body_head_chars} = 6000;
$Mail::MIMEDefang::ML::Config{openai}{body_tail_chars} = 1500;

Model latency grows faster than linearly with the text length: the defaults assume a GPU, or a fast CPU. On slower hardware lower the per-backend budgets, or raise timeout; ml_classify can wait up to timeout seconds (connect_retries only retries requests that could not connect, which fail at once), keep that below the multiplexor's busy timeout (see "TIMEOUT" in Mail::MIMEDefang::ML::OpenAI).

questions lists the questions the backend is asked, is_spam and/or is_phishing. Asking only one of them makes the request lighter (fewer GLiClass labels, a shorter prompt and reply for generative models), e.g. to use the model only as a phishing detector and leave spam to SpamAssassin or rspamd. It can also be set per call, see "ml_classify($state, %opts)".

question_texts rewords the questions put to the backends that take free-text instructions (Laya, generative models), keyed by question; a question not in it keeps its default text from %Mail::MIMEDefang::ML::QUESTIONS. Each of these backends can have its own, which wins over the global one: Laya has a small input budget and wants short texts, a generative model can take longer ones. A backend may also have default texts of its own, used when neither sets one: Laya asks short questions by default, see Mail::MIMEDefang::ML::Laya. In order, a question's text comes from the backend's question_texts, the global question_texts, the backend's default, and %Mail::MIMEDefang::ML::QUESTIONS.

$Mail::MIMEDefang::ML::Config{question_texts}{is_phishing} =
    'Is this email phishing?  ...';
$Mail::MIMEDefang::ML::Config{laya}{question_texts}{is_spam} =
    'Is this email unsolicited bulk mail or a scam?';

GLiClass scores labels instead of answering questions: set them with $Mail::MIMEDefang::ML::Config{gliclass}{labels} and {ham_labels}, see "CONFIGURATION" in Mail::MIMEDefang::ML::GLiClass.

See each backend's documentation for its remaining keys.

INTEGRATION EXAMPLE

A filter_end in mimedefang-filter that runs SpamAssassin as usual and then asks the model for a second, independent opinion. The model is additive, not a replacement for the existing spam path: it only acts on confident verdicts, and any error falls back to SpamAssassin alone. This assumes $SPFResult, $DKIMResult and $DMARCResult were set earlier, in filter_sender and filter_begin; see examples/example-filter-with-ml in the MIMEDefang sources for the complete filter.

use Mail::MIMEDefang::ML;

$Mail::MIMEDefang::ML::Config{backend} = 'gliclass';

sub filter_end {
    my ($entity) = @_;
    return if message_rejected();

    my ($hits, $req, $names, $report);
    ($hits, $req, $names, $report) = spam_assassin_check()
        if $Features{"SpamAssassin"};

    # The SpamAssassin results are not sent here, so the verdict stays
    # independent: a model that sees them tends to follow them and
    # amplify their false positives.  Uncomment SAScore/SARules to send
    # them anyway (or pass RspamdScore/RspamdSymbols from rspamd_check()).
    my $verdict = ml_classify(ml_build_state(
        Entity   => $entity,     # headers, subject and body
        MailFrom => $Sender,
        SPF      => $SPFResult,
        DKIM     => $DKIMResult,
        DMARC    => $DMARCResult,
        # SAScore => $hits,
        # SARules => $names,
    ));

    if ($verdict->{error}) {
        md_syslog('info', "ml: no verdict ($verdict->{error}), "
                         . "falling back to SpamAssassin only");
        return;
    }

    my $phishing = $verdict->{is_phishing}{answer}
                && $verdict->{is_phishing}{confidence} >= 0.90;
    my $spam     = $verdict->{is_spam}{answer}
                && $verdict->{is_spam}{confidence} >= 0.80;

    if ($phishing) {
        md_syslog('info', sprintf('ml(%s): phishing, confidence %.2f',
            $verdict->{backend}, $verdict->{is_phishing}{confidence}));
        action_quarantine_entire_message("Suspected phishing");
    } elsif ($spam) {
        action_change_header('Subject', "[SPAM] $Subject");
    }

    # Per question: a signed score (positive leans yes, negative leans
    # no, 0.00 for no opinion) and yes/no/unsure.
    my %acted = (is_spam => $spam, is_phishing => $phishing);
    foreach my $q (['is_spam', 'Spam'], ['is_phishing', 'Phishing']) {
        my ($key, $name) = @$q;
        next unless grep { $_ eq $key } @{ $verdict->{questions} };
        my $answer = $verdict->{$key}{answer};
        my $sign = !defined $answer ? 0 : $answer ? 1 : -1;
        action_change_header("X-MIMEDefang-ML-$name-Score",
            sprintf('%.2f', $sign * $verdict->{$key}{confidence}));
        action_change_header("X-MIMEDefang-ML-$name-Answer",
            $acted{$key} ? 'yes' : $sign < 0 ? 'no' : 'unsure');
    }
    action_change_header('X-MIMEDefang-ML-Backend', $verdict->{backend});
}

To use the model only against phishing, ask just that question; the is_spam answer is then always undef:

$Mail::MIMEDefang::ML::Config{questions} = ['is_phishing'];

or, for one call only:

my $verdict = ml_classify($state, questions => ['is_phishing']);

Change the confidence thresholds, the actions (action_quarantine_entire_message, action_change_header, action_bounce, action_discard, ...) and where in the filter this runs to match your site's policy. Phishing uses a higher threshold than spam because legitimate bulk marketing mail (tracking links, redirects) can look like phishing to a model.

TRAINING A MODEL

The stock models are zero-shot: they classify mail they were never trained on, from the label texts or the questions alone. Out of the box their accuracy is modest and varies a lot from site to site, especially on mail that is not in English; do not act on their verdicts before checking them against your own mail. To get good results, train the model on a corpus of your own ham and spam (a few hundred labelled messages at least, ideally thousands), and retrain it as your mail changes.

  • gliclass

    contrib/ml-benchmark/gliclass-train fine-tunes a GLiClass model on a directory of ham and one of spam (and, optionally, one of phishing with --phishing-dir), using the exact text and labels this module sends. It reports false positives and false negatives on a held-out part of the corpus before and after training. Serve the result with mimedefang-gliclass-server --model DIR, with the labels it was trained on (see Mail::MIMEDefang::ML::GLiClass). Training needs a GPU in practice.

  • laya

    The laya Python package contains no training code; a Laya model trained with the Laya project's own tools can be served with mimedefang-laya-server --checkpoint.

  • openai

    Fine-tuning a generative model (LoRA training with tools such as Unsloth or LLaMA-Factory, then importing the result into Ollama) is outside the scope of MIMEDefang.

contrib/ml-benchmark/ml-benchmark measures the false positive and false negative rates of every backend on a directory of ham and one of spam: use it before and after training, and to choose the confidence thresholds used in the filter. See "Training a model" in script/ml-servers/README.md for details.

SECURITY

The model servers (mimedefang-laya-server, mimedefang-gliclass-server, Ollama, llama.cpp, vLLM, ...) have no authentication of their own, and every request carries message content. Run them on the MIMEDefang host or on a trusted network, and keep them secure by limiting access with a firewall so that only the MIMEDefang host(s) can reach their ports. Never expose them to the Internet. Any host name or IP address can be configured as a backend URL; this module does not restrict where requests go.

FUNCTIONS

  • ml_build_state(%args)

    Assembles the truncated, structured state hash the backends are given. Callers supply already-decoded/HTML-stripped text; this function does not touch MIME parsing or auth-result computation.

    Pass the message as Entity (the MIME::Entity given to filter_end) and the state is filled from it: From, ReplyTo and Subject from their headers, Body from md_get_plain_text_body, plus the headers listed in $Config{headers} (decoded, unfolded and capped at header_max_chars each) under headers. Arguments passed explicitly override the values read from the entity.

    The text is normalized for the models: HTML entities are decoded, invisible (zero-width, bidi-control, soft-hyphen) characters removed, blanks and separator lines collapsed and URLs longer than url_max_chars cut. The whole body is kept in the state (up to 64k characters); each backend cuts it to its own body_head_chars and body_tail_chars when it sends it.

    signals summarizes, as short lines of text, the domains the body links to (up to max_link_domains, with the number of links to each) and the names and types of the attachments (up to max_attachments). Everything else about the message (authentication, sender reputation, header anomalies, ...) is left to SpamAssassin, whose results can be passed in as SAScore/SARules.

    Args: Entity, From, ReplyTo, Subject, Body, MailFrom (the envelope sender, $Sender), SPF, DKIM, DMARC.

    Optional spam-filter results: SAScore, SARules, RspamdScore, RspamdSymbols. Rules/symbols may be an arrayref or the comma- or space-separated string returned by spam_assassin_check / rspamd_check, and are capped at spam_top_rules entries each. They are put in the state, and so sent to the model, whenever they are passed (rules only together with their score); whether to pass them is up to the caller. Leaving them out keeps the model's verdict independent of SpamAssassin/Rspamd, since a model that sees them tends to follow them and amplify their false positives; passing them can help a model that would otherwise lack context, at the cost of repeating their mistakes.

  • ml_classify($state, %opts)

    Hands the state to the configured backend and returns a verdict hashref. %opts may hold questions, overriding $Config{questions} for this call, e.g. questions => ['is_phishing']. Questions not asked are returned with answer set to undef and confidence 0, as for no opinion; questions in the verdict lists the ones that were asked. Returns { error = $reason }> on any failure, callers should treat that the same as "no opinion" and fall back to SA/rspamd-only handling.

    Answers whose confidence is below min_confidence are returned with answer set to undef, whichever backend produced them.

    When the backend reports its raw scores (see "BACKEND API"), they are copied to scores, for diagnostics; don't base filtering decisions on them, their meaning is backend specific.

BACKEND API

A backend is a module Mail::MIMEDefang::ML::<Name> with a classify($state, \%Config) function returning either { error => $reason } or { is_spam => { answer => 0|1, confidence => 0..1 }, is_phishing => {...} }. Confidence gating is done here, not in the backend. A backend may also return scores => { name => number, ... } with its raw scores, passed on as they are for diagnostics. Backends may use the helpers http_post_json($url, $payload, \%headers), state_body($state, \%backend_cfg) (the body cut to the backend's body_head_chars/body_tail_chars), state_to_text($state, \%backend_cfg) (the state as text, with the signals and that body), active_questions(\%Config) (the questions to answer, in order) and question_text(\%Config, $question, \%backend_cfg, \%defaults) (the question's text, see question_texts; \%defaults holds the backend's own default texts, optional) from this package. A backend asks the model only the questions active_questions returns; answers to other questions are ignored.

SEE ALSO

Mail::MIMEDefang::ML::Laya, Mail::MIMEDefang::ML::GLiClass, Mail::MIMEDefang::ML::OpenAI, mimedefang-laya-server, mimedefang-gliclass-server, script/ml-servers/README.md in the MIMEDefang sources.