Understanding Cloudflare Clef-omni: Architecture, Multimodal Workflows, and Performance Benchmarks

The Shift from Cascading Pipelines to Native Multimodal Decisions

Traditional agentic workflows that evaluate rich media often rely on cascading pipelines. A standard setup requires splitting an incoming payload into separate processing streams: running speech-to-text (STT) models to transcribe audio files, extraction algorithms to pull discrete visual frames from video files, and downstream Large Language Models (LLMs) to synthesize text outputs. This multi-stage pipeline introduces substantial latency, accumulates error across stages, and adds complex infrastructure overhead.

Cloudflare’s Clef-omni introduces a native multimodal decision model designed to process text, image, audio (such as .wav or .mp3), and video (such as .mp4 or .webm) payloads within a single, unified pipeline. Unlike generative LLMs, decision models in the Clef family do not generate autoregressive output tokens. Instead, they run schema-constrained evaluation over candidate options, dramatically reducing token-generation overhead while simultaneously scoring multiple input modalities.

Under the Hood of Clef-omni: MoE Architecture and Attention Routing

Clef-omni is built upon a Qwen3-Omni-30B-A3B-Instruct mixture-of-experts (MoE) foundation. To adapt this foundation specifically for high-speed decision tasks, the architecture retains the primary multimodal comprehension backbone while discarding the underlying text-to-speech (TTS) output components.

When a request is submitted, Clef-omni processes the data using a specific execution sequence:

• Unified Sequence Mapping: Media elements are placed directly into a single unified sequence. Audio streams and video frames are synchronized temporally with visual elements and text inputs without requiring prior transcription or captioning stages.
• Single Prefill Pass: The model executes a single prefill pass across the complete payload, evaluating all modalities and valid schema parameter options in parallel.
• Two-Stage Attention Routing: Candidate values are evaluated directly from internal embeddings. In the first stage, each valid option gathers key evidence across all provided media types (text snippets, image features, or audio frequencies). In the second stage, field vectors cross-attend across the entire context window to compute explicit confidence scores.
• Lexical Grammar Binding: A built-in lexical grammar enforces schema constraints, preserving option semantics across varying prompt structures and field orderings.

The post-training process mirrors the core Clef pipeline: the Qwen3 backbone is frozen while Low-Rank Adapters (LoRA) are trained. Model output calibration relies on combining label-smoothed cross-entropy loss with Brier score calibration, ensuring output probabilities accurately reflect decision confidence.

API Integration and Payload Structure

Clef-omni is integrated into Cloudflare’s Workers AI infrastructure and remains compatible with existing Jev-style decision API requests. It supports direct Base64-encoded binary media streams passed alongside context descriptions and field instruction schemas.

Below is a production single-call POST request sending text, base64-encoded PNG image data, base64 MP3 audio data, and base64 MP4 video data to evaluate hardware installation status:

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/cloudflare/clef-omni 
  -X POST 
  -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" 
  -H "Content-Type: application/json" 
  -d '{
    "model": "clef-omni",
    "state": "Review the installation: a photo of the unit, an audio recording of it running, and a video of the fan.",
    "images": ["data:image/png;base64,<base64-png>"],
    "audio": ["data:audio/mpeg;base64,<base64-mp3>"],
    "videos": ["data:video/mp4;base64,<base64-mp4>"],
    "questions": {
      "label_visible": {
        "type": "noul",
        "instructions": "Is the model and serial number label visible in the photo?"
      },
      "sounds_normal": {
        "type": "noul",
        "instructions": "Does the unit sound like it is running smoothly, without rattling or grinding?"
      },
      "fan_running": {
        "type": "noul",
        "instructions": "Is the fan running in the video?"
      }
    }
  }'

Latency Profiles by Input Modality

Because Clef-omni evaluates internal embeddings rather than decoding sequence text, processing latency scales predictably across input modalities:

• Text-only decisions: ~130 ms median latency.
• Image payloads: ~150 ms median latency.
• Audio inputs: A few hundred milliseconds depending on clip length.
• Full Video Payloads: A complete 21-second video clip with synchronized audio is evaluated in ~1.5 seconds in a single API pass.

Benchmark Evaluation: Clef-omni vs. Clef, Clef-flash, and Jev

Evaluating decision performance across structured benchmarks demonstrates how Clef-omni maintains strong performance relative to text-only decision models despite its broader multimodal architecture.

On TypeSafe agentic workflow evaluations, Clef-omni scores competitively across dynamic task execution metrics:

• Invoice Processing (Exact Actions): Clef-omni (60.2) vs. Clef (64.7), Clef-flash (57.1), Jev (61.8).
• Invoice Processing (Primary Action): Clef-omni (82.0) vs. Clef (86.2), Clef-flash (73.3), Jev (83.1).
• Customer Service (Exact Actions): Clef-omni (71.6) vs. Clef (76.3), Clef-flash (77.0), Jev (76.0).
• Security Incidents (Exact Actions): Clef-omni (61.7) vs. Clef (62.9), Clef-flash (61.7), Jev (61.7).
• Agent Trace Observability (Primary Action): Clef-omni (65.8) vs. Clef (68.5), Clef-flash (69.8), Jev (71.6).

Price Restructuring and Context Limits in Clef-flash

Cloudflare updated pricing structures across the Clef model family to lower execution costs for production agentic systems:

• Clef-flash: Reduced from $0.09 per M input tokens to $0.038 per M input tokens (making it cheaper than Jev).
• Clef-omni: Introduced at $0.15 per M input tokens.
• Clef: Retained at $0.24 per M input tokens.

Context Window Engineering Trade-off

To support the price reduction on hosted Workers AI, Cloudflare adjusted the hosted context window for Clef-flash from 64k tokens to 24k tokens. Operational telemetry across Workers AI indicated that only 0.24% of production decision requests exceeded 24k tokens. Reducing the hosted context allocation allows higher hardware density per serving node without impacting 99.76% of incoming workloads.

For self-hosted deployments, the open-weight release of Clef-flash on Hugging Face remains unchanged, retaining support for up to a 256k context window. Managed workflows requiring context windows larger than 24k tokens can route to the standard Clef model, which maintains a 64k token context window on Workers AI.

Serving Layer Optimizations: SGLang Integration and Latency Improvements

In addition to releasing Clef-omni, Cloudflare optimized the serving infrastructure for the standard Clef model on Workers AI. By transitioning serving backend execution to SGLang (via upstream PR #42721 included in SGLang release 0.5.22), inference speed improved up to 2.0x without modifying base model weights.

Developers self-hosting Clef can pull the updated SGLang launch commands from the official Hugging Face repository or SGLang cookbooks to replicate these throughput gains in custom deployments.

Production Deployments at Cloudflare

Deploying decision models directly at the model layer replaces domain-specific zero-shot classifiers or custom-trained ML models. Cloudflare uses Clef models across several internal production systems:

• Automated Spam Remediation: Cloudflare’s public GitHub documentation repository uses Clef to classify and automatically close spam issues in real time.
• CMS Plugin Moderation: The EmDash content management platform monitors plugin submissions using Clef to identify phishing risks.
• Data Loss Prevention (DLP): Cloudflare DLP pipelines evaluate structured and unstructured data payloads to prevent exposure of personally identifiable information (PII), such as government identification numbers.
• Threat Intelligence: Security systems utilize Clef to score domain characteristics and identify malicious hostnames across edge networks.

Primary reference: Review the original announcement for exact release details. This article is an independent explanation and does not reproduce the source text.