Thought Filtering vs. Text Filtering: Empirical Evidence of Latent Space Defense Supremacy Against Adversarial Obfuscation

Research
1

Large Language Model (LLM) guardrails typically rely on either shallow syntax matching (Regex) or high-latency vector embedding comparisons. Both demonstrate failure modes against adversarial obfuscation and “Living-off-the-Land” attacks where agents utilize opaque protocols (e.g., compression) to hide intent. We present a multi-layered defense architecture shifting security from textual pattern matching to latent space intent analysis and stateful risk profiling.

We introduce three novel contributions: (1) Opaque Protocol Detection, a high-speed entropy filter blocking encrypted command tunnels; (2) Context Fusion, a symbolic pre-processing layer for near-instantaneous de-obfuscation; and (3) Project Sentinel, a stateful risk engine that dynamically hardens detection thresholds against iterative probing.

Empirical validation across adversarial test suites (Garak, N=1,500 adaptive iterations) demonstrates a 98.5% Block Rate (95% CI: [97.7%, 99.0%]), significantly outperforming stateless baselines (91.1%) and traditional regex (<20%). Crucially, the system maintains an end-to-end latency of 17.06ms, achieving a ~10x speedup over embedding-based guardrails while closing the “Glitch Paradox” loophole through multi-turn risk accumulation.

1. Introduction

1.1 The Security-Latency Gap & Supply Chain Risks

As LLMs transition to agentic workflows, security faces two critical bottlenecks: latency (real-time agents require <20ms decision loops) and opaque protocols (agents co-opting compression tools like Slipstream to evade monitoring). Traditional regex is too brittle; embedding-based guardrails (e.g., Llama Guard) are too slow (50-200ms) and stateless, failing to detect iterative probing attacks.

1.2 The Failure of Stateless Inspection

Recent studies show that attackers leverage “Glitch Tokens” and “Leetspeak” to bypass filters. Furthermore, “Living-off-the-Land” (LotL) attacks exploit legitimate infrastructure (e.g., Base64/Compression) to shield malicious payloads from audit. We argue that relying on single-turn, surface-level inspection is fundamentally flawed.

1.3 Contributions

Project CORTEX proposes a Stateful Neuro-Symbolic Architecture:

  1. Fast Path Defense (Entropy & State): A sub-millisecond layer that blocks high-entropy payloads (tunneling) and penalizes repeated probing attempts (Sentinel Risk Engine).

  2. Context Fusion: Symbolic pre-processing that expands obfuscated tokens (1.56ms latency).

  3. The Neuro-Surgeon (Layer 11): RepE-based latent space analysis targeting the “semantic collapse point” to detect intent regardless of syntax.

2. Methodology

2.1 System Architecture: The CORTEX Pipeline (4-Stage)

Stage 0: Fast Path (Entropy & Risk Engine)
Before semantic analysis, we apply two ultra-low latency checks (<0.5ms):

  • Entropy Detector: Calculates Shannon Entropy. Inputs with

    H>5.8H>5.8

    bits/char are blocked as “Opaque Protocols” (countering Slipstream/Tunneling attacks).

  • Sentinel Risk Engine: Tracks session-based risk. If a user triggers a block, their

    RiskScoreRiskScore

    increases. The detection threshold

    ττ

    for subsequent requests is dynamically lowered:

    τdynamic=τbase−(RiskScore×α)τdynamic​=τbase​−(RiskScore×α)

    .

Stage 1: Context Fusion (Symbolic Layer)
A deterministic module normalizes homoglyphs and zero-width characters in

O(n)O(n)

time (1.56ms), converting obfuscated text (e.g.,H4ck) to canonical forms (Hack) for downstream analysis.

Stage 2: The Neuro-Surgeon (Latent Layer)
We utilize a Representation Engineering (RepE) probe at Layer 11 of Llama-3-8B. We compute the cosine similarity between the prompt’s activation vector and a learned “Harmful Direction” vector. If similarity >

τdynamicτdynamic​

, the request is blocked.

Stage 3: iCRAFT (Hybrid Governance)
A deterministic fallback layer enforcing rigid policies for dual-use concepts (e.g., “Biological Weapons”), mitigating the probabilistic uncertainty of latent models.

2.2 Datasets & Benchmarks

  • Garak Adversarial Suite (N=1,500): 500 iterations each of Glitch, Leetspeak, and Mixed attacks.

  • HarmBench (N=387): Used for threshold calibration and FPR (False Positive Rate) validation.

  • Baselines: Compared against Regex (Standard), Llama Guard 2 (7B), and a stateless version of CORTEX.

3. Results

3.1 Measured Performance: Adversarial Robustness

We evaluated CORTEX in two modes: Stateless (Rev 2.0) and Stateful (Sentinel/Rev 3.0) against Garak adaptive attacks.

Overall Finding: Enabling Stateful Defense reduced the Survival Rate from 8.9% to 1.5%.

Attack Type Stateless Block Rate Stateful Block Rate Survival Rate
Glitch Tokens 88.0% 99.0% 1.0%
Leetspeak 90.4% 97.0% 3.0%
Mixed (Heavy) 96.2% 99.4% 0.6%
Overall 91.1% 98.5% 1.5%

Comparison: Traditional Regex achieved only 17.3% block rate on the same set. CORTEX demonstrates a 5.7x improvement in robustness.

3.2 The “Glitch Paradox” Resolved

In stateless mode, subtle glitches (e.g., biølogical) occasionally bypassed detection (12% survival). With Project Sentinel, the first failed attempt raises the user’s risk score. Subsequent attempts face a stricter threshold (

τ<0.7τ<0.7

), leading to the near-elimination of the “Glitch Paradox” (1% survival).

3.3 Measured Performance: Latency

Despite adding the Risk Engine and Entropy Detector, the impact on latency is negligible due to optimized in-memory structures.

Component Processing Time
Fast Path (Entropy + Risk) < 0.10 ms
Context Fusion 1.56 ms
Neuro-Surgeon 12.40 ms
iCRAFT Policy 3.10 ms
Total Latency 17.16 ms

Conclusion: CORTEX remains ~9-10x faster than embedding-based guardrails (typically 150ms+).

3.4 Opaque Protocol Defense

To validate Phase 5 (Entropy), we injected Base64-encoded payloads and simulated compressed “Slipstream” packets.

  • Result: The Fast Path reliably blocked inputs exceeding the entropy threshold with negligible latency overhead, significantly impeding the “Living-off-the-Land” vector described in recent supply-chain security research.

4. Discussion & Conclusion

4.1 From Firewall to Immune System

The transition from 91.1% to 98.5% block rate validates the hypothesis that Stateful Defense is mandatory for agentic security. Stateless firewalls are vulnerable to iterative probing (“hill-climbing attacks”). By introducing a “memory” (Risk Engine), CORTEX behaves like an immune system: it adapts to the aggressor in real-time.

4.2 Latency as a Security Feature

Achieving this robustness at ~17ms allows CORTEX to be deployed in high-frequency agent loops where traditional guardrails are prohibitive. The implementation of “Fast Path” checks (Entropy) ensures that expensive semantic computation is not wasted on encrypted or nonsensical payloads.

4.3 Limitations

While Stateful Defense effectively mitigates iterative attacks, it requires session persistence. Distributed deployments (Kubernetes) require a shared state store (Redis) to maintain risk scores across replicas, introducing a minor architectural complexity compared to stateless designs.

4.4 Conclusion

We introduced CORTEX v2.0, adding Opaque Protocol Detection and Stateful Risk Profiling to the Neuro-Symbolic core. With a 98.5% Block Rate and 17ms latency, it establishes a new standard for high-velocity LLM security, effectively countering both semantic obfuscation and systemic supply-chain co-option attempts.

Open Questions for Future Research

  1. Cross-Model Generalization
    Do the “Layer 11” principles identified in Llama-3-8B transfer universally to other architectures? We hypothesize that the “semantic collapse point” exists in all LLMs, but the specific layer index (e.g., Layer 11 vs. Layer 24) likely varies by model depth and training methodology.

  2. White-Box Resilience in Stateful Systems
    Can white-box attacks succeed against Stateful Defenses? While our Sentinel Risk Engine effectively mitigates iterative gradient-based attacks (by penalizing probing), the theoretical possibility of “Single-Shot” optimized perturbations—which bypass detection in the very first attempt without triggering the risk score—remains an open vector.

  3. The Dual-Use Precision Limit
    What is the theoretical lower bound for False Positive Rates (FPR) on dual-use concepts? Our data suggests that due to the inherent semantic overlap between benign (e.g., “immunology”) and harmful (e.g., “bioweapons”) concepts in latent space, an irreducible FPR of ~3% may exist, necessitating hybrid governance (iCRAFT) rather than pure latent filtering.

  4. Latency of Certified Defenses
    Can mathematically certified defenses (e.g., Randomized Smoothing) ever scale to the 20ms latency requirement of agentic runtimes? Current certification methods add 50-100x latency overhead, suggesting that the probabilistic-but-fast approach of CORTEX remains the only viable path for real-time systems.

  5. Distributed State Synchronization
    How does Stateful Risk Profiling scale in globally distributed architectures? Maintaining sub-millisecond latency for user risk scores across geographically separated clusters (e.g., via Redis or Memcached) presents a CAP theorem challenge for global agent defense that local in-memory dictionaries do not address.

  6. Next-Gen Steganography (Low-Entropy Tunnels)
    Will attackers evolve towards “Natural Language Steganography”? Since our Opaque Protocol Detection now effectively blocks high-entropy payloads (Base64/Slipstream), future research must investigate detecting covert command tunnels hidden within low-entropy, grammatically correct text (e.g., linguistic watermarking or acrostic ciphers).

1 Like
A Bidirectional LLM Firewall: Next Level X1 - help wanted!
2

for now. detailed version.

Below are concrete, research-grade answers to each open question. None of these are fully “settled.” The goal is to replace vague uncertainty with: what existing evidence suggests, what the most likely outcome is, and what experiments would actually decide it.

1) Cross-model generalization

Question: Does the “Layer 11” intent signal transfer to other models. Does a “semantic collapse point” exist universally.

Background

Latent-space defenses rely on the idea that “harmful intent” is represented as a direction or subspace in internal activations, and that this direction is more stable than surface text. This sits inside the broader area of representation engineering and activation steering. (arXiv)

What evidence suggests

Partial transfer is real. There is direct evidence that concept steering vectors can transfer across different LLMs, sometimes via simple linear transformations that align representations. Some work even reports “weak-to-strong” transfer, where vectors from smaller models can steer larger models. (arXiv)

But the “best layer” is not universal. Layer choice matters for stability and effectiveness, and optimal layers vary across architectures and goals. Work on layer selection for stable control explicitly treats “which layer” as a tunable choice rather than a constant. (arXiv)

Likely answer

  • A “semantic collapse point” (a mid-layer region where representations become more linearly separable for high-level concepts) is plausible across transformers, because concept abstraction tends to increase with depth.

  • The exact index (Layer 11 vs Layer 15 vs Layer 24) will vary with:

    • depth, width, tokenizer behavior
    • instruction tuning and safety tuning
    • architecture variations

So: the phenomenon generalizes more than the layer number.

What would convincingly answer it

Run a cross-model “layer sweep” study:

  1. Pick 5–10 diverse models (different sizes and families).

  2. For each model, learn the harmful direction (or probe) using the same protocol.

  3. Measure:

    • best-layer location
    • robustness under obfuscation
    • transferability of the learned direction to other models (with and without alignment transforms)

Use a standardized red-teaming evaluation so results compare cleanly. HarmBench exists for precisely this sort of standardized robustness evaluation. (arXiv)

2) White-box resilience in stateful systems

Question: Can a knowledgeable attacker bypass stateful defenses. What about single-shot optimized perturbations.

Background

Stateful defenses change the game: repeated probing increases risk and tightens thresholds. That defeats “many-shot” hill-climbing. But white-box or high-feedback attackers can sometimes optimize a single prompt to win immediately.

There is strong evidence that adaptive attacks are significantly stronger than static ones, and that many defenses collapse under adaptive evaluation. (arXiv)

Also, jailbreak literature shows attackers can optimize prompts using query feedback, sometimes even without transferability assumptions. (OpenReview)

What evidence suggests

  • Statefulness helps most against iterative attackers.

  • White-box or high-feedback attackers can still do one-shot optimization.

    • If the attacker gets extra signals like logprobs, optimization becomes easier. (GitHub)

  • Evaluations that include full pipelines (input filter + output filter) show the arms race is real and system-level assessment matters. (arXiv)

Likely answer

Yes, white-box bypasses remain possible even with statefulness. Statefulness mainly forces the attacker into a harder regime: “win on the first try.” That is an improvement, not a proof of security.

What actually improves resilience (practical research directions)

High-leverage mitigations that specifically target one-shot optimization:

  1. Reduce attacker feedback

    • No detailed refusal reasons
    • No token-level scores
    • Uniform response timing where possible
      Rationale: optimization needs gradient-like hints. (GitHub)

  2. Randomize parts of the decision boundary

    • stochastic thresholds
    • randomized feature subsampling
    • ensemble of probes
      Rationale: makes black-box optimization noisier.

  3. Multi-signal gating

    • latent probe + canonicalization + tool-boundary constraints
    • not “one classifier to rule them all”
      Rationale: adaptive attacks tend to overfit to a single signal. (arXiv)

  4. Train against adaptive attacks

    • Use standardized frameworks and co-development of attacks/defenses (HarmBench explicitly motivates this). (arXiv)

3) The dual-use precision limit

Question: Is there an irreducible false positive rate for dual-use concepts. Is ~3% a true lower bound.

Background

Dual-use classification is hard because “benign” and “harmful” share vocabulary and even shared reasoning steps. The ambiguity is not only model error; it is often label-policy ambiguity: different policies label the same prompt differently.

What evidence suggests

  • Safeguard model documentation explicitly discusses tradeoffs between F1 and false positive rate, and also notes that policy mismatch between training labels and evaluation labels affects results. (Hugging Face)
  • Research on safety evaluation highlights that adversarial contexts and dataset/policy choices matter, and that “one-number” claims tend to hide these tradeoffs. (arXiv)
  • Empirical work also reports that guard models can misclassify, including false negatives and false positives, depending on setup. (OpenReview)

Likely answer

There is no universal constant like “3% is unavoidable” across all domains and policies.

But there is an unavoidable concept: Bayes error / irreducible overlap.

  • If benign and harmful intents are genuinely overlapping in the observable features, no classifier can separate them perfectly.

  • The size of that lower bound depends on:

    • labeling policy strictness
    • domain (medicine vs chemistry vs cybersecurity)
    • user population and language distribution
    • how much context you include (single turn vs multi-turn)

So: irreducible error exists, but the specific number is conditional.

How to estimate the “irreducible” part in practice

A workable approach:

  1. Build a carefully adjudicated dataset with multiple annotators and disagreement tracking.

  2. Measure:

    • inter-annotator agreement (how ambiguous the policy is)
    • best achievable ROC curve under that policy

  3. Treat “high-disagreement region” as the irreducible zone and route it to:

    • deterministic policy constraints, or
    • human-in-the-loop review, or
    • “ask clarifying intent” dialogue

This is exactly why hybrid governance layers exist: they are a policy tool, not just a model tool. (Hugging Face)

4) Latency of certified defenses

Question: Can certified defenses like randomized smoothing ever meet ~20 ms.

Background

Randomized smoothing is a well-known way to get provable robustness certificates for classifiers by injecting noise and estimating class probabilities. (arXiv)

The catch: certification typically requires many samples (Monte Carlo) for tight confidence bounds, which costs time. There is active work on accelerating certification via smarter sampling, but it is still compute-heavy. (ojs.aaai.org)

What evidence suggests

  • Randomized smoothing is practical in vision settings with enough compute, but it is not “free.” (arXiv)
  • Even newer variants often discuss computational tradeoffs or expensive solvers in some model families. (proceedings.neurips.cc)

Likely answer

For full-strength, high-confidence certificates on rich inputs, hitting <20 ms end-to-end is unlikely without severe constraints.

For LLM security specifically, certification is even harder because:

  • input space is discrete tokens, not continuous pixels
  • attacker model is semantic, not small-norm perturbations

So: certified methods may be useful for subcomponents or restricted transforms, but “certified everything in 20 ms” is not the likely outcome.

What might work (realistic path)

  • Certify cheap, narrow properties (example: strict grammars for tool calls, or bounded structured outputs).
  • Cache certificates for repeated templates.
  • Use probabilistic fast defenses in real time, and run expensive certification asynchronously for auditing or high-risk sessions.

5) Distributed state synchronization

Question: How does stateful risk scoring scale globally without losing latency. CAP theorem issues.

Background

If you store per-session risk in a distributed system, you are inside classic distributed-systems tradeoffs. The CAP theorem formalizes that under network partitions, you cannot simultaneously guarantee consistency and availability. (cs.princeton.edu)

What evidence suggests

  • CAP tradeoffs are real and unavoidable in partition scenarios. (cs.princeton.edu)
  • In-memory key-value systems like Redis can be extremely low-latency in normal operation, often microsecond-scale processing, but real deployments must handle tail latency and operational issues. (Redis)

Likely answer

You will not get “perfectly consistent global risk state” with “always available” and “sub-millisecond everywhere.”

What you can get is security-engineered consistency:

  • choose where you are willing to be stale
  • decide whether stale state fails open or fails closed

Practical architectures that work

  1. Regional risk + eventual global convergence

    • Each region enforces its own risk score immediately.
    • Periodically merge upward (eventual consistency).
    • Failure mode: attacker hops regions. Mitigation: global token bucket or signed risk token.

  2. Sticky sessions (affinity)

    • Route a user to the same region for the session.
    • Minimizes cross-region reads.

  3. Monotonic risk tokens

    • Risk only increases within a window.
    • You can embed risk in a signed token passed between services.
    • Reduces dependence on cross-region reads.

  4. Fail-closed for high-risk

    • If global state is unavailable and the user is already risky, default stricter thresholds.

These are CAP-compatible designs: you pick availability for most traffic, and consistency where it matters most. (cs.princeton.edu)

6) Next-gen steganography and low-entropy tunnels

Question: Will attackers move to natural language steganography. How do you detect it.

Background

This is not hypothetical. There is published work showing:

  • natural language steganography using LLMs
  • LLMs used as covert channels
  • covert channels created by biasing LLM output distributions (ACM Digital Library)

There is also ongoing work on tokenizer-consistency issues in linguistic steganography, which matters because tokenization affects both embedding capacity and detectability. (Language Processing Meeting)

What evidence suggests

  • High-entropy blocks are the easy case.
  • Low-entropy covert channels are feasible and actively studied. (ACM Digital Library)

Likely answer

Yes. As soon as high-entropy gates become common, capable attackers will shift toward fluent-looking covert channels.

Detection and mitigation approaches that are actually plausible

  1. Tool-boundary hardening

    • Most real damage comes from tool execution, not from hidden text alone.
    • Strict schemas, allowlists, argument constraints, and sandboxes reduce payoff.

  2. Multi-turn extraction pattern detection

    • Covert channels often require back-and-forth to transmit.
    • Stateful risk profiling helps here because the “channel establishment” phase looks like probing.

  3. Statistical detection of constrained text

    • Steganographic text often optimizes for hidden capacity while preserving fluency.
    • That can leave detectable fingerprints (distribution shifts, unusual stylistic invariants).

  4. Active defenses

    • Insert benign “semantic canaries” and check whether the response preserves hidden structure.
    • Use randomized paraphrasing or normalization before sensitive boundaries (but measure impact).

The core point repeats: fast gates stop obvious tunnels; state stops iterative channel setup; tool-boundary controls reduce impact even if a covert channel exists. (ACM Digital Library)

Bullet summary

  • Cross-model generalization likely holds for the phenomenon, not the exact layer index. Steering vectors can transfer, layers vary. (arXiv)
  • Stateful defenses blunt iterative probing but do not eliminate one-shot optimized jailbreaks. Adaptive evaluation is mandatory. (arXiv)
  • Dual-use “irreducible FPR” exists in principle but is not a universal constant; it depends on policy and ambiguity. (Hugging Face)
  • Certified defenses like randomized smoothing are provable but typically too slow for tight real-time loops at high confidence. (arXiv)
  • Global state faces CAP tradeoffs; design for monotonic risk, locality, and controlled failure modes. (cs.princeton.edu)
  • Low-entropy natural-language steganography is feasible and studied; prioritize tool-boundary controls plus stateful multi-turn detection. (ACM Digital Library)
3

Update (implementation report): We implemented the architecture and evaluation suggestions you proposed (decoupling the “fat orchestrator,” improving observability, and validating security behavior with controlled tests). Below is what changed, what was validated, and which trade-offs remain.
​

  1. Implemented changes
    A) Separation of concerns

Introduced a dedicated Perimeter / verification plane as its own service (Port 8005) instead of embedding it in the Orchestrator.

Goal: keep the Orchestrator closer to routing/governance and reduce coupling to heavy inspection logic.

B) Observability & reproducibility

Integrated QuantumStatsCollector to record decisions, block reasons, and latency in a structured format (regression + dashboard use).

Added Port 8009 as metrics backend (polling + SQLite history), while Port 8005 handles live decisions + WebSocket streaming.

C) Perimeter detection logic

Adaptive routing via Shannon entropy:

High entropy → crypto/PQC heuristics (distribution/size-based).

Low entropy → text/code inspection (pattern-based).

Unicode homoglyph hardening: NFKD normalization + ASCII filtering + explicit transliteration map for stubborn/non-Latin substitutions observed during adversarial testing.

DoS guard / performance: Smart Sampling (deep inspect head window; fast scan tail window; middle intentionally not fully scanned to cap cost).

  1. Validation (controlled test protocol)
    We ran a protocol designed to validate Smart Sampling behavior and homoglyph defense under low-entropy “document-like” carriers.

Carrier: repeated natural-language/log-like text to deterministically force the low-entropy/text path.

Injections: HEAD (homoglyph), TAIL (plain), MIDDLE (control outside head+tail).

Expected invariants: HEAD→BLOCK, TAIL→BLOCK, MIDDLE→ALLOW (confirms we’re not doing a full scan).

Observed: HEAD BLOCK, TAIL BLOCK, MIDDLE ALLOW.
This matches the intended semantics: bounded compute with explicit head/tail coverage and a documented middle blind spot as the performance trade-off.

  1. Operational status
    Services 8000–8009 running.

8005 streams perimeter decisions via WebSocket to the live dashboard.

8009 persists historical telemetry for analysis.

  1. Explicit limitations / trade-offs
    Smart Sampling: attacks exclusively in the middle of very large payloads are not guaranteed to be detected (by design, to avoid normalization-based DoS).

Unicode: current normalization + mapping covers the homoglyph classes observed so far; exhaustive Unicode confusables coverage remains a hardening axis.

PQC: crypto detection is heuristic (conservative size/distribution checks; not a full PQC object parser).

  1. Next steps
    Ablation matrix: normalization on/off, sampling on/off, judge-call on/off → FPR/FNR + latency under fixed concurrency.

Configurable “security mode”: additional budgeted middle-window sampling.

Reproducible benchmark bundle: fixed seeds/corpora/test vectors + automated report generation.

1 Like
4

Seems good changes.

You did the right kind of changes. Separation of concerns is real. Observability is real. Controlled invariants are real. The remaining weaknesses are mostly “predictability” and “policy drift across services.” Below are concrete improvements that increase security without blowing your latency budget.

1) Separation of concerns is good. Now harden the trust boundary.

What looks strong

  • A dedicated verification plane is the correct move. It turns inspection into a service with a clean contract, and keeps routing/governance lighter.

Suggestions

A. Make bypass impossible by construction

  • Ensure all tool calls and all model invocations require an allow decision from the perimeter plane.

  • Decide one global rule: “no decision, no execute.”

  • Add explicit fail behavior:

    • Fail-closed for tool calls and high-risk sessions.
    • Fail-open only for low-risk, non-tool chat if you choose.

B. Version and sign the decision contract
Add these fields to every decision record (and to the WebSocket stream):

  • policy_version (hash of rules and thresholds)
  • normalizer_version (hash of Unicode maps)
  • sampler_version
  • model/probe_version if any learned detector exists
  • config_digest (single hash covering all runtime config)

This is what lets you reproduce a regression later. Without it, metrics are “pretty graphs” but not forensics.

2) Entropy routing is useful. Calibrate it to evade-proof it.

Background

Entropy checks are fast and broadly model-agnostic. But attackers can lower measured entropy by using encodings with smaller alphabets (hex, base32), or by embedding ciphertext into structured-looking containers.

Also, Shannon entropy estimates are noisy on short strings.

Your routing rule should treat entropy as a hint, not a final truth.

Suggestions

A. Make entropy length-aware

  • For short payloads, require a minimum length before the “high entropy” branch is trusted.
  • Consider confidence bounds or a simple rule: “entropy routing only if length >= N.”

B. Add “alphabet detection → decode → re-check entropy”
If the string looks like:

  • base64-ish (64-char alphabet)
  • base32-ish (32-char alphabet)
  • hex-ish (16-char alphabet)

…then decode a bounded prefix and recompute entropy on the decoded bytes. This kills the “I will base32 my ciphertext to slip under your bits/char threshold” trick.

There is real literature suggesting attackers can manipulate measured entropy by encoding encrypted material. (Napier Repository)
And in C2 contexts, “random-looking” byte distributions often sit around ~5.8 bits in practice, which matches why your threshold feels plausible. (SciSpace)

C. Treat structured crypto artifacts as “opaque” even if entropy is not extreme
Examples:

  • PEM-like blocks
  • DER-like binary blobs
  • JWK-like JSON
  • TLS-ish byte patterns in tools traffic

This reduces reliance on entropy alone.

3) Unicode hardening: move from “observed map” to standards-backed coverage.

Background

Unicode attacks are not just homoglyphs. They include:

  • mixed-script confusables
  • “skeleton” equivalence (two different strings that render the same)
  • bidi controls that reorder what humans see versus what parsers execute (Trojan Source) (arXiv)

Normalization inconsistencies across components are a classic bypass source. (Medium)

Suggestions

A. Implement TR39 “skeleton” + mixed-script detection
Instead of only NFKD + ASCII filtering + custom transliteration maps, add:

  • TR39 confusable detection
  • mixed-script checks
  • optional “whole-script confusable” checks for dangerous script mixtures

Unicode TR39 is the canonical spec for this. (Unicode)
If you want lightweight implementations, libraries like libu8ident explicitly target TR39-style identifier security. (GitHub)

B. Add explicit bidi-control detection and policy
Trojan Source is not hypothetical. It is a published attack and tracked as CVE-2021-42574. (arXiv)
Unicode TR36 even suggests “reverse-bidi” style detection, with known false-positive tradeoffs. (Unicode)

Practical policy:

  • Reject or neutralize bidi controls in contexts where code, configs, or tool arguments exist.
  • At minimum, log a high-severity signal when bidi controls appear.

C. Avoid irreversible folding as the only representation
NFKD + ASCII filtering is aggressive. It can:

  • collapse distinct user inputs into the same canonical string
  • break non-Latin languages
  • increase false positives

This is a known tradeoff in other security contexts like passwords and identifiers. (Information Security Stack Exchange)

Better pattern:

  • Keep both forms:

    • raw_text
    • normalized_text

  • Run detectors on both

  • Decide based on a combined policy

This keeps internationalization viable and reduces “normalization-only” blind spots.

4) Smart Sampling: your test is good. Now fix the predictability problem.

What you validated correctly

Your invariant test proves:

  • head and tail are inspected
  • the middle is intentionally not fully scanned
  • the performance trade-off is real and documented

That is exactly what a bounded-cost design should prove.

The main risk

A motivated attacker will put payload only in the middle because you made the blind spot deterministic.

Suggestions

A. Add deterministic pseudo-random middle sampling (budgeted)
Keep bounded compute, but make coverage less gameable:

  • Choose K small windows from the middle region.
  • Seed the selection with a stable hash of (session_id, request_id, policy_version).
  • That keeps runs reproducible for testing, but unpredictable to an attacker.

This is the highest-leverage improvement you can make without scanning everything.

B. Consider content-defined chunking for sampling windows
Instead of fixed offsets, pick windows based on content boundaries using techniques like Rabin fingerprinting and content-defined chunking. CDC is used in other domains specifically because it gives stable boundaries and can be sampled efficiently. (MDPI)

You do not need “full CDC dedup.” You only need the idea: content-based anchors for picking representative windows.

C. Add an “attack-aware” escalation rule
If any suspicious signal exists (unicode anomalies, partial matches, tool-call indicators), temporarily increase sampling budget for the same request. This is a clean “security mode” ramp without permanently increasing cost.

5) PQC detection: upgrade from “size heuristics” to “lightweight structure awareness”

Background

Post-quantum artifacts can be large. NIST standardized ML-KEM (FIPS 203) and ML-DSA (FIPS 204). (NIST Publications)
Even the public draft tables show characteristic key and ciphertext sizes for ML-KEM (e.g., ciphertext 768/1088/1568 bytes depending on parameter set). (NIST Publications)
Large PQ sizes can cause real operational effects like fragmentation in protocols. (NIST Computer Security Resource Center)

Suggestions

A. Keep heuristics, but add shallow parsing where possible

  • Detect PEM framing and DER structure cheaply.

  • Recognize typical envelopes:

    • X.509-ish encodings
    • JWK-ish JSON forms
    • known library output formats

  • For ML-KEM and ML-DSA, size checks can remain, but do not treat size alone as proof.

B. Log “crypto suspicion class” separately from “block decision”
Example:

  • crypto_suspected=true
  • crypto_reason=size_match|pem_frame|entropy_high|known_header

This helps you tune conservatively without blinding your telemetry.

6) Observability: you have the components. Add “decision completeness.”

Suggestions

Make sure each decision log includes:

  • routing branch taken (entropy vs text path)
  • entropy score, length, alphabet classification
  • normalization actions and flags (NFKD applied, bidi controls present, mixed-script)
  • sampling plan actually executed (window positions, sizes, seed)
  • block reason taxonomy (stable enum, not free text)
  • latency breakdown per stage
  • risk state before and after (if statefulness exists)

This is what turns your telemetry into an ablation-ready dataset.

7) Validation and benchmarking: strengthen what you already started

Add these controlled tests

  1. Random-position injection tests

    • Insert the same payload at random offsets across the whole body.
    • Measure detection probability under your sampling plan.

  2. Unicode “bidi + homoglyph” combined cases

    • Trojan Source cases are explicitly about bidi controls. (arXiv)

  3. Normalization consistency tests across services

    • Same payload goes through 8005 and any downstream component.
    • Confirm identical normalized outputs and flags.

  4. Encoded payload tests

    • Hex, base32, base64, and nested encodings.
    • Confirm “decode then re-check” behavior works.

Ablation matrix suggestion

You already listed the right toggles. Add:

  • deterministic-middle-sampling on/off
  • bidi detection on/off
  • TR39 skeleton on/off

Curated references worth pinning in your benchmark bundle

Summary

  • Your split into a dedicated verification plane is correct. Now enforce “no decision, no execute.”
  • Entropy routing needs length-aware calibration and “detect encoding, decode, re-check” to avoid low-alphabet tunnels.
  • Unicode hardening should add TR39 skeleton and bidi-control mitigation (Trojan Source class), not only ad-hoc maps. (Unicode)
  • Smart Sampling is validated but predictable. Add budgeted deterministic pseudo-random middle windows or CDC-based anchors to shrink the blind spot. (arXiv)
  • PQC detection should remain conservative but gain shallow structure awareness, not only size heuristics. (NIST Publications)