see the Glossary in the comment following this post.
Design Considerations: Presidential Campaign Data Warehouse
Purpose of this document. This is not an architecture. It is a checklist of constraints that any architecture has to satisfy. It exists so that (a) whoever builds this can be asked the right questions early, and (b) a proposal that ignores these constraints can be identified as incomplete before money is spent on it.
Audience. Technical leadership and non-specialist decision-makers. A companion glossary defines every term used here.
Contents
- Why a generic data warehouse design fails in this context — the five structural differences from a commercial warehouse: fixed deadline, statutory boundaries, nation-state threat model, explosive-then-zero headcount, licensed rather than owned data.
- Legal and regulatory considerations — entity separation as an architectural boundary; state-by-state voter file restrictions; consent as a data model; post-campaign records retention.
- Data acquisition and licensing — existing party data infrastructure; the three incompatible ingestion patterns; refresh cadence over the cycle; primary vs. general.
- Identity resolution and modeling — the person-ID layer, model governance, experimentation, and activation (reverse ETL).
- Architecture and operations — immutable raw data, the compute cost curve, Election Day as a single point of failure, non-SQL consumers, not rebuilding what exists.
- Security — documented nation-state targeting, misconfiguration as the largest known incident, insider access, field minimization, surge-scale provisioning, minimum controls checklist.
- Lifecycle and wind-down — archive, legal hold, disposition of licensed data, and platform-agnostic infrastructure.
- Questions to put to the domain expert — 15 questions across legal/structural, data, analytics, operations, and build-vs-buy. Intended as the agenda for that conversation.
- Warning signs in a proposal — the checklist for identifying an under-considered proposal before money is spent.
- What this document is not — scope boundary and next step.
Sources
1. Why a generic data warehouse design fails in this context
A presidential campaign data warehouse differs from a commercial one in five structural ways. Each of these changes the design, not just the operations plan.
| Property |
Commercial warehouse |
Presidential campaign warehouse |
| Deadline |
Roadmap slips are absorbed |
Election Day does not move. A feature that ships November 4 has zero value. |
| Legal boundaries |
Internal policy, contractual |
Federal statute. Certain groups are legally prohibited from seeing each other's data, and violations are enforcement matters. |
| Threat model |
Financially motivated criminals |
Nation-state intelligence services, plus criminals, plus insiders leaking to press. |
| Org size |
Stable headcount |
Grows from dozens to thousands, then dissolves entirely within weeks of Election Day. |
| Data ownership |
You own your data |
Most of the core dataset is licensed, under 50+ separate agreements with different use restrictions. |
The last two are the ones most often missed. A design that assumes stable staffing and owned data will fail on both counts.
2. Legal and regulatory considerations
2.1 Entity separation is a hard architectural boundary, not a permissions setting.
A presidential operation involves multiple legal entities: the principal campaign committee, one or more joint fundraising committees, the national party committee, state parties, and — separately and at arm's length — super PACs and 501(c)(4) organizations. FEC regulations define when communication between these entities becomes a "coordinated communication," which is treated as an in-kind contribution and is generally illegal for a super PAC. FEC rules contemplate a formal firewall: it must prohibit the flow of information between the employees or consultants serving one side and those serving the other, and it must be described in a written policy statement distributed to all affected employees, consultants, and clients.
- Implication: the warehouse may need to be multiple warehouses, or a single platform with enforced tenancy that a compliance lawyer will sign off on.
- Implication: audit logs are not just a security control. They may become legal evidence that the firewall held.
- Get General Counsel into the design conversation before the schema is written, not after.
2.2 Voter file use is restricted state by state.
State-level laws governing the use, storage, and transmission of voter files vary significantly and can impose strict limits on third-party uploads, data matching, and cross-platform activation. Some states prohibit commercial use; some prohibit re-disclosure; some require signed use agreements naming specific individuals.
- Implication: the warehouse needs to track provenance and license terms per row or per source, and be able to answer "can this record legally be sent to this ad platform?"
- A design that merges all 50 states into one undifferentiated table and forgets where each row came from cannot answer that question.
2.3 Contact consent is a data problem, not just a compliance checkbox.
Political campaigns are not exempt from the Telephone Consumer Protection Act (TCPA), a position the Supreme Court affirmed in Barr v. American Association of Political Consultants. Violations carry penalties of $500 per call or text, trebled to $1,500 for willful violations — attractive territory for class-action plaintiffs. Presidential campaigns have been named in TCPA class actions.
- Implication: consent provenance (who consented, when, through what form, to what) must be stored, versioned, and queryable at the individual level, and it must survive the trip through the warehouse into the texting platform.
2.4 Records retention obligations outlive the campaign.
FEC recordkeeping requirements apply after the committee winds down. The warehouse design must include a defensible archive, not just deletion.
3. Data acquisition and licensing
3.1 You are not the only data operation in your ecosystem.
National party infrastructure already exists on both sides. On the Republican side, Data Trust, founded in 2011, describes itself as the core infrastructure provider for conservative campaigns and says it supported more than 5,000 Republican campaigns in the 2024 cycle. On the Democratic side, the Democratic Data Exchange (DDx), founded in 2019, operates a real-time blind exchange of voter contact data between aligned organizations, parties, committees, and campaigns. Separately, Catalist and TargetSmart function as major voter-file managers for progressive organizations, while state parties maintain and license their own files.
- Implication: interoperability with party infrastructure is a requirement, not a nice-to-have. Design decisions that make it hard to exchange data with these systems have a political cost, not just a technical one.
- Implication: someone has to negotiate and track data-sharing agreements. That is a role, and it has a technical counterpart.
3.2 Source data arrives in incompatible shapes and cadences.
Roughly three ingestion patterns coexist:
- REST APIs — CRMs, fundraising, events, comms tools. Rate-limited, paginated, inconsistently documented.
- Bulk file drops — SFTP or object storage. Voter files, vendor matchbacks, early-vote files. Delivered as full snapshots, not deltas, and large.
- Warehouse-native shares — some vendors publish directly to warehouse marketplaces; L2, for example, makes its national databases available through the Snowflake Data Marketplace.
Any proposal that assumes a single ingestion pattern is underspecified.
3.3 Refresh cadence changes over the cycle.
Weekly refreshes are adequate in the spring. In the final three weeks, early-vote and absentee-return files drive daily — sometimes hourly — resource reallocation. The system has to support both without a rebuild.
3.4 Primary and general election are different systems in practice.
The primary is a sequential, state-by-state contest with different rules per state. The general is a national contest concentrated in a handful of battlegrounds, with party coordination that did not exist during the primary. Data models built exclusively for one phase tend to break at the transition.
4. Identity resolution and modeling
4.1 Identity resolution is the hardest problem in the build.
A donor record, an email subscriber, a canvass contact, a text opt-in, and a voter-file row all describe the same person and share no common key. Entity resolution — also called record matching or identity resolution — is the technique that identifies and matches records across multiple sources to produce a single unified record, and it is the engine underneath any master data management program.
- Implication: a persistent person-ID layer with match confidence scores, not a one-time deduplication script.
- Implication: matching decisions must be reproducible and reversible. Bad merges are worse than missed merges, because they corrupt downstream targeting and fundraising compliance.
- Vendors sell matching as a service; evaluate build-vs-buy explicitly here.
4.2 Models drive resource allocation, so model governance is a real requirement.
Turnout propensity and support scores determine where field staff, money, and the candidate's time go. That makes model versioning, reproducibility, backtesting, and score lineage operational necessities, not data-science hygiene.
4.3 The campaign will run experiments.
Randomized controlled trials on messaging and contact methods are standard practice. The warehouse has to support treatment/control assignment, holdout groups, and outcome measurement — which means it must record who was assigned to what, not just who was contacted.
4.4 Activation is half the value.
Scores and universes are useless in the warehouse. They have to get back into the CRM, the texting platform, and the ad platforms. This is reverse ETL, and it is routinely under-scoped because it looks like an afterthought on an architecture diagram.
5. Architecture and operations
5.1 Preserve raw data immutably.
Voter files get re-cut; vendors reissue corrected files; models get retrained. If the raw landing layer is overwritten, past universes cannot be reproduced and disputes cannot be settled.
5.2 Plan for the compute cost curve.
Query volume and model training spike enormously in the final six weeks. Budget and quota design should anticipate this. A platform whose pricing model punishes bursty usage is a poor fit.
5.3 Election Day is a single point of catastrophic failure.
There is no "we'll fix it tomorrow." Disaster recovery, failover, and on-call rotation for the final 72 hours need explicit design, including a documented degraded-mode plan: what still works if the warehouse is down.
5.4 Most consumers are not SQL users.
State directors, field organizers, and finance staff need self-service reporting. The BI layer and the set of curated, trustworthy tables are as important as the pipelines.
5.5 Do not rebuild what the ecosystem already provides.
Parsons, a Python package maintained by The Movement Cooperative, exists specifically as a universal adapter for this ecosystem and normalizes data regardless of source. It ships connectors for VAN, ActBlue, Mobilize, Action Network, TargetSmart, Catalist, PDI, Hustle, Twilio, Redshift, S3 and dozens of others. Managed platforms exist too. Writing 40 bespoke API integrations is a common, avoidable, and expensive mistake.
6. Security
6.1 The threat model is nation-state, and this is documented, not speculative.
In 2020, Google's Threat Analysis Group reported that a China-linked APT group targeted Biden campaign staff and an Iran-linked APT group targeted Trump campaign staff with phishing emails. In 2024, an Iran-backed group used the compromised email account of a former senior advisor to phish a high-ranking official in a presidential campaign, one of dozens of individuals targeted across both campaigns. The FBI, CISA and ODNI issued a joint statement in August 2024 attributing a cyberattack on a presidential campaign to Iranian actors.
- Implication: phishing-resistant MFA and SSO for every account, including vendor accounts, from day one.
- Implication: assume credential compromise will be attempted successfully. Design for blast-radius limitation.
6.2 The most damaging known incident in this space was a misconfiguration, not an attack.
In 2017, a firm working for the RNC left 1.1 terabytes of unsecured personal data on a publicly accessible cloud server, exposing details on approximately 198 million registered voters — names, dates of birth, home addresses, phone numbers, registration details, and modeled ethnicity and religion.
- Implication: object storage configuration, public-access blocks, and continuous posture monitoring are first-class requirements. "We'll put the files in a bucket" is the exact failure mode.
6.3 Insider access is a real risk category.
In 2015 a Sanders campaign staffer accessed the Clinton campaign's voter file, which led to the DNC revoking the campaign's access and a public dispute. The system must be able to prove who queried what.
6.4 Minimize sensitive fields at ingest.
State voter registration records commonly include full or partial Social Security numbers, driver's license numbers, and dates of birth, depending on the state. Fields the campaign has no operational use for should be dropped at the boundary rather than stored and protected.
6.5 Access provisioning at surge scale.
Staff count grows by an order of magnitude and turns over constantly, including large volunteer and fellow populations. Manual access management will fail. Role-based access tied to an identity provider, with automated deprovisioning, is a design requirement rather than an operational refinement.
6.6 Controls checklist (minimum).
Encryption in transit and at rest; centralized secrets management for dozens of vendor API credentials; role-based access control plus row-level security by state and program; column masking on PII; full query audit logging with retention; vendor data processing agreements; documented incident response; independent security review before the general election.
7. Lifecycle and wind-down
- The committee dissolves. The obligations do not. Plan the archive, the legal hold, and the disposition of licensed data (much of which must be destroyed or returned per license terms) as part of the initial design.
- A stated design goal in this space is platform-agnostic infrastructure — a warehouse that lets an organization switch vendors between cycles without losing its history. That goal conflicts with how campaign technology is usually procured, and the conflict should be named explicitly at the start.
- Documentation is the deliverable that determines whether the next cycle starts from zero.
8. Questions to put to the domain expert
Legal / structural
- What legal entities will touch this data, and where does counsel want the hard boundaries?
- Which party data infrastructure are we expected to interoperate with, and what does that require of us?
- Who owns the relationships and contracts with the voter-file vendors, and what do those licenses permit?
Data
4. Which source systems are in scope for cycle one, and which are deliberately deferred?
5. Are we buying identity resolution or building it?
6. What is the required data freshness in October versus in April?
7. What is the plan for consent provenance across SMS, email, and phone?
Analytics
8. Who are the actual consumers, and what is their SQL proficiency?
9. What models will exist, who builds them, and how are scores versioned and shipped back out?
10. What experimentation infrastructure is expected?
Operations
11. What is the on-call model for the final two weeks?
12. What is the degraded-mode plan if the warehouse is unavailable on Election Day?
13. What is the staffing plan — how many data engineers, and when do they start?
Build vs buy
14. What do we buy off the shelf (managed platform, matching, connectors) versus build?
15. What is the wind-down and archive plan?
9. Warning signs in a proposal
A proposal is likely under-considered if it does any of the following:
- Treats legal entity separation as a permissions problem rather than an architectural one.
- Contains no mention of identity resolution, or treats it as a deduplication step.
- Ingests voter file data without tracking source, license, and permitted use.
- Assumes a single ingestion pattern (all API, or all file-based).
- Has no reverse ETL / activation path back into operational tools.
- Has no consent data model.
- Presents a security section consisting only of "encryption and access controls," with no phishing-resistant MFA, no audit logging, no cloud posture management, and no reference to the nation-state threat model.
- Has no answer for Election Day availability or degraded operation.
- Proposes building all source integrations from scratch.
- Has no wind-down, archive, or data-destruction plan.
- Was designed without General Counsel in the room.
10. What this document is not
It is not an architecture, a vendor selection, a budget, or a staffing plan. It is the constraint set those things have to satisfy. The next step is a conversation with someone who has run a data operation on a national campaign before, using section 8 as the agenda.
Sources
- FEC, Coordinated communications — https://www.fec.gov/help-candidates-and-committees/candidate-taking-receipts/coordinated-communications/
- Mac Murray & Shuster, TCPA Compliance for Political Calls — https://mslawgroup.com/tcpa-compliance-for-political-calls/
- Wiley, TCPA Suits Against Political Campaigns on the Rise — https://www.wiley.law/newsletter-TCPA-Suit-Against-Political-Campaigns-Rise-Trump-Campaign-Facing-Separate-Class-Action-Suits
- UpGuard, The RNC Files: Inside the Largest US Voter Data Leak — https://www.upguard.com/breaches/the-rnc-files
- Dark Reading, Are 2024 US Political Campaigns Prepared for Coming Cyber Threats? — https://www.darkreading.com/remote-workforce/2024-us-political-campaigns-prepared-cyber-threats
- TechCrunch, Google: China- and Iran-backed hackers targeted Trump and Biden campaigns — https://techcrunch.com/2020/06/04/google-china-iran-trump-biden/
- Overture Global, The Secret Power of Political Data Trusts — https://www.overtureglobal.io/story/the-secret-power-of-political-data-trusts
- NASS, Public Voter Registration Information and Security — https://www.nass.org/sites/default/files/surveys/2020-10/NASS-briefing-FAQ-info-security-2020_0.pdf
- Data Trust — https://thedatatrust.com/
- Parsons (The Movement Cooperative) — https://www.parsonsproject.org/
- Senzing, What Is Entity Resolution? — https://senzing.com/what-is-entity-resolution/
- Sutton Smart, Legacy Data Archiving — https://suttonsmart.com/political-consulting/legacy-data-archiving-building-a-database-for-a-career-not-just-a-cycle/
see the Glossary in the comment following this post.
Design Considerations: Presidential Campaign Data Warehouse
Purpose of this document. This is not an architecture. It is a checklist of constraints that any architecture has to satisfy. It exists so that (a) whoever builds this can be asked the right questions early, and (b) a proposal that ignores these constraints can be identified as incomplete before money is spent on it.
Audience. Technical leadership and non-specialist decision-makers. A companion glossary defines every term used here.
Contents
Sources
1. Why a generic data warehouse design fails in this context
A presidential campaign data warehouse differs from a commercial one in five structural ways. Each of these changes the design, not just the operations plan.
The last two are the ones most often missed. A design that assumes stable staffing and owned data will fail on both counts.
2. Legal and regulatory considerations
2.1 Entity separation is a hard architectural boundary, not a permissions setting.
A presidential operation involves multiple legal entities: the principal campaign committee, one or more joint fundraising committees, the national party committee, state parties, and — separately and at arm's length — super PACs and 501(c)(4) organizations. FEC regulations define when communication between these entities becomes a "coordinated communication," which is treated as an in-kind contribution and is generally illegal for a super PAC. FEC rules contemplate a formal firewall: it must prohibit the flow of information between the employees or consultants serving one side and those serving the other, and it must be described in a written policy statement distributed to all affected employees, consultants, and clients.
2.2 Voter file use is restricted state by state.
State-level laws governing the use, storage, and transmission of voter files vary significantly and can impose strict limits on third-party uploads, data matching, and cross-platform activation. Some states prohibit commercial use; some prohibit re-disclosure; some require signed use agreements naming specific individuals.
2.3 Contact consent is a data problem, not just a compliance checkbox.
Political campaigns are not exempt from the Telephone Consumer Protection Act (TCPA), a position the Supreme Court affirmed in Barr v. American Association of Political Consultants. Violations carry penalties of $500 per call or text, trebled to $1,500 for willful violations — attractive territory for class-action plaintiffs. Presidential campaigns have been named in TCPA class actions.
2.4 Records retention obligations outlive the campaign.
FEC recordkeeping requirements apply after the committee winds down. The warehouse design must include a defensible archive, not just deletion.
3. Data acquisition and licensing
3.1 You are not the only data operation in your ecosystem.
National party infrastructure already exists on both sides. On the Republican side, Data Trust, founded in 2011, describes itself as the core infrastructure provider for conservative campaigns and says it supported more than 5,000 Republican campaigns in the 2024 cycle. On the Democratic side, the Democratic Data Exchange (DDx), founded in 2019, operates a real-time blind exchange of voter contact data between aligned organizations, parties, committees, and campaigns. Separately, Catalist and TargetSmart function as major voter-file managers for progressive organizations, while state parties maintain and license their own files.
3.2 Source data arrives in incompatible shapes and cadences.
Roughly three ingestion patterns coexist:
Any proposal that assumes a single ingestion pattern is underspecified.
3.3 Refresh cadence changes over the cycle.
Weekly refreshes are adequate in the spring. In the final three weeks, early-vote and absentee-return files drive daily — sometimes hourly — resource reallocation. The system has to support both without a rebuild.
3.4 Primary and general election are different systems in practice.
The primary is a sequential, state-by-state contest with different rules per state. The general is a national contest concentrated in a handful of battlegrounds, with party coordination that did not exist during the primary. Data models built exclusively for one phase tend to break at the transition.
4. Identity resolution and modeling
4.1 Identity resolution is the hardest problem in the build.
A donor record, an email subscriber, a canvass contact, a text opt-in, and a voter-file row all describe the same person and share no common key. Entity resolution — also called record matching or identity resolution — is the technique that identifies and matches records across multiple sources to produce a single unified record, and it is the engine underneath any master data management program.
4.2 Models drive resource allocation, so model governance is a real requirement.
Turnout propensity and support scores determine where field staff, money, and the candidate's time go. That makes model versioning, reproducibility, backtesting, and score lineage operational necessities, not data-science hygiene.
4.3 The campaign will run experiments.
Randomized controlled trials on messaging and contact methods are standard practice. The warehouse has to support treatment/control assignment, holdout groups, and outcome measurement — which means it must record who was assigned to what, not just who was contacted.
4.4 Activation is half the value.
Scores and universes are useless in the warehouse. They have to get back into the CRM, the texting platform, and the ad platforms. This is reverse ETL, and it is routinely under-scoped because it looks like an afterthought on an architecture diagram.
5. Architecture and operations
5.1 Preserve raw data immutably.
Voter files get re-cut; vendors reissue corrected files; models get retrained. If the raw landing layer is overwritten, past universes cannot be reproduced and disputes cannot be settled.
5.2 Plan for the compute cost curve.
Query volume and model training spike enormously in the final six weeks. Budget and quota design should anticipate this. A platform whose pricing model punishes bursty usage is a poor fit.
5.3 Election Day is a single point of catastrophic failure.
There is no "we'll fix it tomorrow." Disaster recovery, failover, and on-call rotation for the final 72 hours need explicit design, including a documented degraded-mode plan: what still works if the warehouse is down.
5.4 Most consumers are not SQL users.
State directors, field organizers, and finance staff need self-service reporting. The BI layer and the set of curated, trustworthy tables are as important as the pipelines.
5.5 Do not rebuild what the ecosystem already provides.
Parsons, a Python package maintained by The Movement Cooperative, exists specifically as a universal adapter for this ecosystem and normalizes data regardless of source. It ships connectors for VAN, ActBlue, Mobilize, Action Network, TargetSmart, Catalist, PDI, Hustle, Twilio, Redshift, S3 and dozens of others. Managed platforms exist too. Writing 40 bespoke API integrations is a common, avoidable, and expensive mistake.
6. Security
6.1 The threat model is nation-state, and this is documented, not speculative.
In 2020, Google's Threat Analysis Group reported that a China-linked APT group targeted Biden campaign staff and an Iran-linked APT group targeted Trump campaign staff with phishing emails. In 2024, an Iran-backed group used the compromised email account of a former senior advisor to phish a high-ranking official in a presidential campaign, one of dozens of individuals targeted across both campaigns. The FBI, CISA and ODNI issued a joint statement in August 2024 attributing a cyberattack on a presidential campaign to Iranian actors.
6.2 The most damaging known incident in this space was a misconfiguration, not an attack.
In 2017, a firm working for the RNC left 1.1 terabytes of unsecured personal data on a publicly accessible cloud server, exposing details on approximately 198 million registered voters — names, dates of birth, home addresses, phone numbers, registration details, and modeled ethnicity and religion.
6.3 Insider access is a real risk category.
In 2015 a Sanders campaign staffer accessed the Clinton campaign's voter file, which led to the DNC revoking the campaign's access and a public dispute. The system must be able to prove who queried what.
6.4 Minimize sensitive fields at ingest.
State voter registration records commonly include full or partial Social Security numbers, driver's license numbers, and dates of birth, depending on the state. Fields the campaign has no operational use for should be dropped at the boundary rather than stored and protected.
6.5 Access provisioning at surge scale.
Staff count grows by an order of magnitude and turns over constantly, including large volunteer and fellow populations. Manual access management will fail. Role-based access tied to an identity provider, with automated deprovisioning, is a design requirement rather than an operational refinement.
6.6 Controls checklist (minimum).
Encryption in transit and at rest; centralized secrets management for dozens of vendor API credentials; role-based access control plus row-level security by state and program; column masking on PII; full query audit logging with retention; vendor data processing agreements; documented incident response; independent security review before the general election.
7. Lifecycle and wind-down
8. Questions to put to the domain expert
Legal / structural
Data
4. Which source systems are in scope for cycle one, and which are deliberately deferred?
5. Are we buying identity resolution or building it?
6. What is the required data freshness in October versus in April?
7. What is the plan for consent provenance across SMS, email, and phone?
Analytics
8. Who are the actual consumers, and what is their SQL proficiency?
9. What models will exist, who builds them, and how are scores versioned and shipped back out?
10. What experimentation infrastructure is expected?
Operations
11. What is the on-call model for the final two weeks?
12. What is the degraded-mode plan if the warehouse is unavailable on Election Day?
13. What is the staffing plan — how many data engineers, and when do they start?
Build vs buy
14. What do we buy off the shelf (managed platform, matching, connectors) versus build?
15. What is the wind-down and archive plan?
9. Warning signs in a proposal
A proposal is likely under-considered if it does any of the following:
10. What this document is not
It is not an architecture, a vendor selection, a budget, or a staffing plan. It is the constraint set those things have to satisfy. The next step is a conversation with someone who has run a data operation on a national campaign before, using section 8 as the agenda.
Sources