Writing · topic · 20 articles
Campaign finance, lobbying, foreign agents, dark money, redistricting, and election-integrity analysis.
Since 1981 American universities have disclosed 62 billion dollars in foreign gifts and contracts under Section 117 of the Higher Education Act — 117,152 transactions at 528 institutions. A reading of the federal ledger: who received it, which countries and governments sent it, how it concentrates at the top, and what the disclosure regime does and does not reveal.
Ownership and consolidation · Research and education · Money in politics · Transparency and open data
Three public datasets describe how organizations try to shape government and what they receive: FEC campaign finance (who gives), lobbying disclosures (who lobbies), and USAspending (who wins contracts). Joined on organization + parent name — there is no shared identifier — with the honest correlation-not-causation caveat.
Money in politics · Government operations · Transparency and open data · Engineering and infrastructure
The IRS publishes Form 990 filings for political organizations — 527 committees (direct political spending) and 501(c)(4) social welfare organizations (the dark money vehicle). The data covers revenue, expenditures, officer compensation, and political activities for 65,000+ organizations. Here is what the data contains, how to access it via ProPublica Nonprofit Explorer and the IRS bulk XML, and what it reveals about the shadow infrastructure of US political spending.
Federal data · Transparency and open data · Money in politics
The hardest and most valuable move in federal data analysis is following one company across four disconnected systems — the political money it gives, the lobbying it pays for, the contracts it wins, and the fraud cases brought against it. None of them share a key, so the work is entity resolution: normalizing names, mapping subsidiaries to parents, and living with fuzzy matches.
Engineering and infrastructure · Money in politics · Government operations
The Lobbying Disclosure Act forces every paid lobbyist to file the client, the issues, the agencies contacted, and the money — a quarterly public ledger of the influence industry. This guide walks the LD-2 and LD-203 filings, the standardized issue-area codes, the Senate and House disclosure systems, and how the data joins to the campaign-finance and foreign-agent records.
Before a single dollar of federal campaign money can be traced, the spender has to be named — and the FEC committee registry is where every candidate committee, party committee, traditional PAC, and Super PAC is identified by a unique C-prefixed ID. This guide covers FECA and the registration threshold, the committee taxonomy, how Citizens United and SpeechNow created the Super PAC, and how the committee ID joins to the itemized money.
The FEC publishes bulk data on every contribution and expenditure in federal elections — candidates, PACs, super PACs, and party committees. Here is how to download the full dataset, trace money from donor to expenditure, and identify the shell-company layer that obscures dark money flows.
The Foreign Agents Registration Act database maintained by the DOJ National Security Division is the federal government authoritative record of foreign influence operations in the United States — covering every individual and firm registered as a foreign agent, the foreign governments and entities that retained them, and the lobbying activities, media campaigns, and political contacts conducted on their behalf.
The FEC independent expenditure database covers every Super PAC and outside group that spent money to influence federal elections — over $4 billion in disclosed outside spending in the 2020 election cycle, plus dark money flowing through nonprofit organizations not required to disclose donors.
The Federal Election Commission Matters Under Review database tracks every campaign finance complaint and enforcement action — from contribution limit violations and disclosure failures to foreign national contributions and coordinated expenditure violations.
FARA (Foreign Agents Registration Act, 22 U.S.C. §§ 611-621, 1938) requires agents of foreign governments and political parties to register with DOJ's National Security Division and file semi-annual disclosure statements. ~500-600 active registrations at any time. Registration: Form RA-1 (within 10 days) → Form NSD-3 (semi-annual supplement) disclosing principal identity, activities, compensation, disbursements, political contacts. LDA exemption (22 U.S.C. § 613(h)): agents registering under the Lobbying Disclosure Act whose principal is not a foreign government or political party may use LDA instead -- DOJ IG 2016 report criticized this gap. Mueller-era surge 2018-2022: Manafort convicted, Flynn retroactively registered (Turkey/Gülen), Barrack acquitted (UAE), Podesta Group and Mercury LLC retroactively registered. Saudi Arabia post-Khashoggi: $14M+ annually, $450M+ since 2016; firms retained: Squire Patton Boggs, Akin Gump, BGR Group. Chinese state media: CGTN and Xinhua registered as foreign agents 2019. Criminal penalty: 22 U.S.C. § 618 felony, up to 5 years + fines. Electronic Reading Room: justice.gov/nsd-fara; eFARA bulk CSV at efile.fara.gov/bulk/. OpenSecrets and POGO maintain secondary databases. Here is registration mechanics, LDA exemption gap, Mueller-era cases, Saudi Arabia and China enforcement, eFARA bulk data structure, and a Python analysis of FARA disbursements by country.
The FEC administers FECA (1971/1974) for federal elections only. Committee types: PCC, party committees, PAC ($5k/election limit), Super PAC (post-Citizens United, unlimited), SSF, Leadership PAC. 2024 federal spending ~$14B. Individual to candidate limit $3,300/election. FEC bulk data: cm.zip (committees), indiv.zip (individual contributions >$200 with employer/occupation), pas2.zip (PAC-to-candidate), oppexp.zip (disbursements). OpenFEC API at api.open.fec.gov/v1/. 501(c)(4) dark money: no donor disclosure required. Here is all eight bulk files, Super PAC mechanics, MURs, and a Python occupation partisan lean analysis.
Public Law 94-171 mandates the Census Bureau to deliver block-level population data to states for legislative redistricting by April 1 of the year following the decennial census — the foundational dataset for every congressional and state legislative district. Here is the five data tables (P1–P5, H1), the geographic hierarchy to census block, the one-person-one-vote case law (Reynolds v. Sims, Wesberry v. Sanders), the 2020 apportionment results (Texas +2, New York missed a seat by 89 people), differential privacy and the TopDown Algorithm controversy, the 63-combination race/ethnicity schema, Census API variable naming (P2_006N syntax), VRA Section 2 and the Gingles three-part test, and a Python Census API tract-level racial composition analysis.
The Lobbying Disclosure Act requires quarterly filings with the Senate SOPR — covering lobbyist identities, issue codes, specific bills lobbied, and dollar amounts for every registered lobbying engagement. Here is the LDA API, the relationship to FARA and LD-203 contribution reports, and how to connect lobbying spending to legislative outcomes.
Federal data · Money in politics · Transparency and open data
The DOJ buries the FARA bulk download inside an Oracle APEX URL that looks broken. Behind it: daily CSV exports of every DC firm registered to lobby for a foreign government — who they represent, what they're paid, and what activities they conduct. Here is how to use it.
How we detect coordinated amplification campaigns across 58M daily posts: MinHash LSH (128 hash functions, 16 bands, Jaccard threshold 0.80) for content similarity, Redis sorted-set burst detection (≥5 accounts within 15 minutes, inverse-sqrt account age weighting), seven account-feature logistic regression, network amplification ring detection via cycle enumeration, cross-platform timing joins, and a 0–100 coordination score with 70/90 thresholds for human review and auto-flagging.
Machine learning and OSINT · Engineering and infrastructure · Money in politics
How the election intelligence pipeline resolves FEC committee identity across 1.3M records: the 10-code committee type taxonomy (H/S/P/X/Y/N/Q/O/I/U), a JointFundraisingCommittee dataclass with JFCAllocation and resolve_jfc_participants() from Form 99, normalize_entity_name() with iterative legal-suffix stripping, a four-pass resolution table (exact ID 63.4% → exact name 82.1% → alias 91.7% → TF-IDF char 3-gram 95.5% cumulative recall), and LLC chain disambiguation via FinCEN/EDGAR/SOS cross-reference.
Anomaly detection across 47 races in 23 states: Benford's law with magnitude-range validity checks, XGBoost turnout model (20 features, SHAP attribution, MAD-based z-scores, 3.1pp MAE), ARIMA(2,1,2) reporting-curve detection, DBSCAN campaign finance clustering (near-identical amounts + 3-day burst), and full triage workflow (12 flags → 9 explained, 2 false positives, 1 persistent).
The statistical methods behind AI Analytics' election anomaly detection — first-digit analysis, last-digit uniformity testing, turnout z-scores, and why these signals require cross-validation with social and media data before generating an alert.
Money in politics · Machine learning and OSINT · Censorship and information control
How the election intelligence pipeline ingests AP Election API feeds, state authority data (JSON/CSV/HTML scraping), social media signals, and media coverage in real time: Kafka election.precinct_results topic (50 partitions by state FIPS), PrecinctResult protobuf schema, state scraper StateScraperConfig, ElectionSentimentConsumer, narrative divergence scoring, FIPS normalization edge cases (Connecticut planning regions, Alaska districts), and p50/p99 latency targets for all four streams.
Money in politics · Engineering and infrastructure · Machine learning and OSINT
All topics: the writing index.