Entity subscriptions in the Federal Regulatory Data Hub: per-entity change monitoring across 30+ enforcement lists

The naive approach to regulatory change monitoring is to subscribe to everything: one webhook feed for OFAC, another for SAM.gov exclusions, a third for SEC enforcement actions, and so on. A compliance team screening 200 active supplier relationships against 30 lists would receive thousands of events per week, most of them about entities they have never touched and never will. The signal-to-noise ratio collapses.

The entity subscription model inverts this. Instead of asking “what changed across all lists today?” the question becomes “did anything change for these specific counterparties?” That inversion is only possible because the cross-agency entity bridge joins OFAC SDN designations, SAM.gov exclusions, SEC enforcement actions, EPA ECHO violations, DOJ press releases, FDA warning letters, CFTC orders, FINRA sanctions, OIG exclusions, and 25+ other lists under a single entity_master_id. Once an entity is resolved into that shared identifier, a single subscription covers every list simultaneously.

The entity monitoring problem

A procurement team onboarding a new supplier runs a one-time screening check at contract signing. What it often does not do is re-screen that supplier every week for the duration of the relationship. That gap is where enforcement exposure accumulates: OFAC designations land mid-contract, SAM debarments are issued after a federal audit, EPA consent decrees follow years of violation accumulation. The problem is not knowing the supplier was screened clean at onboarding — it is not knowing when that status changed.

Entity subscriptions solve continuous monitoring without requiring the compliance team to own the polling infrastructure, manage list formats across 30 agencies, or maintain the entity resolution logic that links a company's trade name on an OFAC XML record to its legal name on a SAM exclusion to its CIK on an SEC enforcement action. The hub owns all of that. The subscriber provides an entity_master_id (or a name pattern to resolve one) and a delivery endpoint, and the hub fires when anything changes for that entity anywhere across the monitored list corpus.

EntitySubscription data model

Each subscription captures who is watching, what they are watching, and how to reach them. The data model supports two targeting modes and three delivery channels:

interface EntitySubscription {
  subscription_id: string;        // uuid v4
  subscriber_id: string;          // API key hash
  entity_master_id?: string;      // specific resolved entity (optional)
  entity_patterns?: string[];     // fuzzy name patterns ['Acme Corp*', 'Acme%']
  list_types?: string[];          // filter to specific lists
                                  // ['ofac_sdn', 'sam_exclusions', 'sec_enforcement']
  min_severity?: number;          // 0–10 (OFAC=10, SAM debarment=8, EPA notice=4)
  delivery: DeliveryConfig;
  created_at: string;
  updated_at: string;
}

interface DeliveryConfig {
  channel: 'webhook' | 'email' | 'rss';
  webhook_url?: string;
  webhook_secret?: string;        // HMAC-SHA256 signing key
  email?: string;
  include_entity_context: boolean; // include full cross-agency profile in payload
}

The list_types filter is optional. Omitting it means the subscription covers all 30+ monitored lists. Supplying it narrows delivery to a specific subset — for a financial institution that only cares about sanctions-related lists, that might be ['ofac_sdn', 'ofac_consolidated', 'fincen_section314a', 'bis_entity_list', 'uflpa_entity_list']. The min_severity filter provides a complementary coarse gate: events below the threshold are dropped before delivery, regardless of which list they came from.

The include_entity_context flag controls whether the delivery payload includes the full cross-agency profile for the entity — all current list appearances across all shards, not just the one that triggered the event. Compliance teams investigating a new OFAC designation often want to know immediately whether the entity also appears on SAM exclusions or the BIS Entity List. Setting this flag to true bundles that context into a single payload rather than requiring a follow-up /entity/{entity_master_id} lookup.

Subscription creation: two targeting modes

Subscriptions are created via POST /v1/subscriptions. The request body determines which of the two targeting modes the hub uses:

// Mode 1: entity-pinned — subscriber resolved entity_master_id from a prior lookup
POST /v1/subscriptions
{
  "entity_master_id": "em_7f3a9c",
  "min_severity": 6,
  "delivery": {
    "channel": "webhook",
    "webhook_url": "https://compliance.example.com/regulatory-alerts",
    "webhook_secret": "whsec_...",
    "include_entity_context": true
  }
}

// Mode 2: pattern-based — hub resolves at ingest time using FTS5 + fuzzy match
POST /v1/subscriptions
{
  "entity_patterns": ["Acme Corp*", "Acme Trading%", "ACM Enterprises"],
  "list_types": ["ofac_sdn", "sam_exclusions", "sec_enforcement"],
  "min_severity": 7,
  "delivery": {
    "channel": "webhook",
    "webhook_url": "https://compliance.example.com/regulatory-alerts",
    "webhook_secret": "whsec_...",
    "include_entity_context": true
  }
}

Entity-pinned subscriptions are the preferred mode for known counterparties. The subscriber performs a GET /v1/entity?q=Acme+Corp&jurisdiction=US at onboarding time, receives an entity_master_id, and stores it alongside the supplier record. Subsequent subscriptions reference that ID directly. The lookup cost is paid once; every future event matches in O(1) time against the entity bridge index.

Pattern-based subscriptions are for monitoring scenarios where the exact entity has not been pre-resolved — watchlist monitoring for broad name families, or early-stage due diligence where only a trade name is known. The pattern set is cached in Cloudflare KV with a 5-minute TTL. Every ingest batch applies the pattern set against newly arrived records using FTS5 MATCH as the first pass, followed by Jaro-Winkler fuzzy confirmation. The confidence threshold for pattern subscriptions is 0.75 (versus 0.90 for pinned subscriptions, where the looser gate is appropriate because the entity has not been pre-validated by the subscriber).

The pattern resolution pipeline runs approximately 12ms per ingest cycle for a subscriber with 50 active patterns — dominated by FTS5 index scan time, not the fuzzy pass, which only runs on candidates that pass the full-text gate.

Cross-list fan-out pipeline

When a new OFAC SDN designation arrives and the ingest pipeline detects it as an added record, the fan-out sequence is:

// Step 1: entity bridge lookup
// Map the OFAC record's UID to entity_master_id
const entity = await db.prepare(
  'SELECT entity_master_id FROM entity_bridge WHERE source_list = ? AND source_record_id = ?'
).bind('ofac_sdn', 'SDN-14827').first();
// entity_master_id = 'em_7f3a9c'

// Step 2: subscription filter lookup
// Find all subscriptions that cover this entity_master_id
const sql = 'SELECT s.* FROM entity_subscriptions s'
  + ' LEFT JOIN json_each(s.list_types)'
  + ' WHERE (s.entity_master_id = ?1 OR s.entity_master_id IS NULL)'
  + '   AND (s.list_types IS NULL OR json_each.value = ?2)'
  + '   AND (s.min_severity IS NULL OR s.min_severity <= ?3)'
  + '   AND s.is_active = 1';
const subscriptions = await db.prepare(sql).bind('em_7f3a9c', 'ofac_sdn', 10).all();

// Step 3: cross-agency context fetch (if include_entity_context = true)
const crossAgencyProfile = await fetchEntityProfile('em_7f3a9c', env);

// Step 4: build EntityChangeEvent
const event: EntityChangeEvent = buildEvent(record, entity, crossAgencyProfile);

// Step 5: enqueue per-subscription delivery
for (const sub of subscriptions.results) {
  await env.DELIVERY_QUEUE.send({ subscription: sub, event });
}

The cross-agency context fetch in step 3 queries all shards of the entity index — because a single entity may appear in OFAC data stored in one D1 shard and EPA ECHO data stored in another. This is the most latency-sensitive part of the fan-out for subscriptions with include_entity_context: true. The profile is fetched once per triggering event and attached to all subscription payloads for that entity, not once per subscriber.

EntityChangeEvent payload

The payload delivered to subscribers carries both the specific triggering event and, optionally, the full cross-agency context for the entity:

{
  "event_id": "evt_abc123",
  "event_type": "entity_list_addition",
  "entity_master_id": "em_7f3a9c",
  "entity_name": "Acme Trading LLC",
  "list_name": "ofac_sdn",
  "list_record_id": "SDN-14827",
  "severity": 10,
  "effective_date": "2026-04-26",
  "idempotency_key": "em_7f3a9c_ofac_sdn_20260426",
  "cross_agency_context": {
    "total_list_appearances": 3,
    "lists": [
      {
        "list": "ofac_sdn",
        "record_id": "SDN-14827",
        "added": "2026-04-26"
      },
      {
        "list": "bis_entity_list",
        "record_id": "BIS-EL-4421",
        "added": "2025-11-10"
      },
      {
        "list": "sam_exclusions",
        "record_id": "SAM-EX-98123",
        "added": "2024-08-03"
      }
    ]
  }
}

The idempotency_key combines entity_master_id, list name, and effective date. It is stable across retries: if the Cloudflare Queue delivers the same event twice due to a transient delivery failure, the subscriber can detect the duplicate by caching received keys. The 24-hour cache window is sufficient — the same entity will not generate a second entity_list_addition event for the same list within one day unless the record is removed and re-added, which would produce a new key.

The cross_agency_context.lists array reflects the entity's complete current list footprint at the moment the event fires, not just the triggering record. In the example above, the OFAC SDN addition is the new event, but the subscriber immediately learns the entity was already on the BIS Entity List since November 2025 and SAM exclusions since August 2024. That context changes the severity assessment considerably: this is not a first-time designation but a third enforcement list appearance.

Severity scoring

The min_severity filter uses a 0–10 scale assigned per list type. The scoring reflects the regulatory urgency and typical compliance response requirement for each list, not a subjective priority ranking:

cisa_kev at severity 3 is unusual in this table — it is a CVE-level signal rather than an entity-level enforcement action. Its value is in vendor monitoring: a compliance team watching a software vendor can receive a low-priority alert when CISA adds a known vulnerability from that vendor's products to the KEV catalog, without that alert appearing alongside OFAC sanctions and SAM debarments. Setting min_severity: 5 suppresses CISA KEV events entirely while preserving FDIC, CFPB, and everything above.

Bulk portfolio monitoring

The subscription model scales directly to portfolio-level monitoring via POST /v1/subscriptions/bulk. A compliance team managing 200 active supplier relationships resolves each supplier to an entity_master_id at onboarding time (either manually via the entity lookup API or via the batch resolution endpoint) and submits the full portfolio in a single bulk create request:

POST /v1/subscriptions/bulk
{
  "subscriptions": [
    {
      "entity_master_id": "em_7f3a9c",
      "min_severity": 6,
      "delivery": { "channel": "webhook", "webhook_url": "...", "webhook_secret": "..." }
    },
    {
      "entity_master_id": "em_2b1d4e",
      "min_severity": 6,
      "delivery": { "channel": "webhook", "webhook_url": "...", "webhook_secret": "..." }
    }
    // ... up to 500 per request on Standard tier
  ],
  "shared_delivery": {
    // optional: apply one delivery config to all if all route to the same endpoint
    "channel": "webhook",
    "webhook_url": "https://compliance.example.com/regulatory-alerts",
    "webhook_secret": "whsec_...",
    "include_entity_context": true
  }
}

The bulk endpoint returns a subscription_ids array and a failed array for any entity IDs that could not be resolved. Failed entries are returned with a reason code rather than silently dropped.

The operational value of this pattern is clearest under mass-designation scenarios. When Russia's SWIFT-cutoff sanctions package landed on OFAC in March 2022, the hub would have fired 847 entity-specific events within 6 minutes of the XML publication: 10-minute polling interval, with the ingest and fan-out pipeline adding approximately 4 minutes of processing overhead. A compliance team with 200 pre-registered suppliers would have received only the events relevant to their portfolio — not 847 raw events for every newly designated entity across the entire SDN list, but a filtered set covering only their counterparties. The entity subscription is a portfolio-monitoring primitive: it scales to the counterparty set the team actually manages, not to the full list.

RSS feed channel

Not every compliance workflow is built around webhook infrastructure. For teams that prefer polling over push delivery, each entity subscription exposes a stable Atom feed at:

GET /v1/subscriptions/{subscription_id}/feed.xml
Authorization: Bearer <api_key>

# Returns RFC 4287 Atom feed
# Last 100 EntityChangeEvents for this subscription
# Supports conditional GET: ETag and Last-Modified headers
# Response includes ETag for subsequent If-None-Match requests

The Atom feed carries the same EntityChangeEvent content as the webhook channel, serialized into <entry> elements with the event_id as the Atom id field and effective_date as updated. The cross_agency_context block appears as a JSON-encoded <content type="application/json"> element.

ETag support makes the RSS channel bandwidth-efficient even at high polling frequency: a conditional GET that returns HTTP 304 Not Modified costs only a D1 read against the subscription's last-modified timestamp, not a full event enumeration. Teams running compliance dashboards that refresh every 5 minutes will see 304 responses on the vast majority of requests.

Subscription quotas

Subscription limits are enforced per API key at the subscription-creation endpoint. Pattern subscriptions are counted separately from entity-pinned subscriptions because they carry higher per-ingest-cycle compute cost:

Enterprise subscribers receive dedicated queue priority, which means their delivery events are processed before Standard tier events when the Cloudflare Queue consumer is under load from a large ingest batch. In practice this matters during the minutes immediately following a large OFAC designation package, when fan-out volume spikes across all subscribers simultaneously. Enterprise events typically land within 90 seconds of ingest completion even during peak load; Standard tier events may queue for 3–5 minutes under the same conditions.

Enterprise accounts also unlock the bulk resolve API: a dedicated endpoint for resolving large entity sets to entity_master_id in batch, with confidence scores and match explanations returned for each input. This is the recommended path for compliance teams onboarding large vendor or customer portfolios — resolve once at onboarding, subscribe by entity_master_id, and let the entity-pinned subscription model handle continuous monitoring from that point.

For the webhook delivery system that routes these subscription events — queue semantics, HMAC signing, and retry backoff: Federal Regulatory Data Hub change alerts: near-real-time OFAC sanctions, SAM debarments, and enforcement action webhooks →

For the entity bridge that makes cross-list fan-out possible — how 30+ lists are joined under a single entity_master_id: Cross-agency regulatory entity graph: joining OFAC, SAM, SEC, EPA, and 25+ federal lists under a single entity identifier →

For the risk scoring layer built on top of subscription data — how severity scores and cross-agency context combine into a counterparty risk signal: Compliance screening across 30+ federal enforcement lists: how the risk score works →

For how OFAC SDN data is ingested and kept fresh — conditional GET polling, entity normalization, and the record hash index: OFAC SDN integration in the Federal Regulatory Data Hub: conditional GET, entity normalization, and sub-second screening →