Data hygiene

How data hygiene works

Data hygiene is a set of routines that run on every record a team holds. Each routine targets a different way a record goes wrong, and IBM's data quality dimensions give a useful map of those failure types 1.

• Validity. Values match the format a field expects, such as a bare domain in a website field.
• Uniqueness. Each person and each company appears once.
• Accuracy. Values match the real world, such as the employer a contact works for today.
• Completeness. Fields that routing and outreach read are filled.
• Timeliness. Records are refreshed before they go stale.

A hygiene program turns those dimensions into four routines.

**Entry checks.** Forms, imports and API writes reject or fix bad values before they land. A problem caught here has not reached any downstream system yet.

**Standardization.** Fields such as country, company name, phone and website follow one format, so filters, joins and reports treat equal values as equal. RichAPI's normalize_company strips legal suffixes and punctuation from a company name, and clean_domain reduces a messy URL to a root domain 3.

**Deduplication.** Duplicate records are found and merged, which is the job of deduplication. CRMs rely on clean keys for this. HubSpot deduplicates contacts by email address and companies by company domain name 2, so the quality of those two fields decides how well its automatic checks work.

**Verification and refresh.** Email addresses are checked before sends, and fields that change, such as title and employer, are refreshed on a schedule. Data decay explains how records age, and data freshness covers how to measure that age.

The routines depend on each other. Standardization makes deduplication possible, and deduplication stops enrichment from paying twice for the same person.

What to measure in a data hygiene audit

An audit gives the program a baseline. Each check below produces a number that can be tracked from month to month, so the team can see whether the routines are working.

**Duplicate rate.** Count records that share a normalized email or a root domain. Run company names through a normalizer first, since ACME TECHNOLOGIES, INC. and Acme Technologies are one account to a person and two strings to a database 3.

**Format failures.** Count values that break the field's rule, such as websites stored with paths or phone numbers in mixed formats. IBM's validity dimension measures how much data matches the required format for business rules 1.

**Fill rate on required fields.** Measure the share of records with email, title, company and country present. Low fill rate on a field that routing reads sends leads to the wrong queue.

**Bounce history.** Track the share of sends that hard bounce, split by source and by import. A rising bounce rate can point to one list or one source that skipped verification.

**Age of key fields.** Record when title, employer and email were last checked. A field with no timestamp cannot be refreshed on purpose.

**Ownership.** Name who fixes each failure type and which system wins when two disagree. IBM notes that when several sources report on the same metric, a primary source of truth should be designated 1.

Run the audit before a large import or a new campaign, and again after the fixes, so the change is measured on the same checks.

Example of data hygiene before an outbound import

A RevOps team is about to load a purchased event list and a year-old CRM segment into a new outbound program. Before anything reaches the sequencer, it runs a hygiene pass in a fixed order, so each step works on cleaner input than the last.

Step 1   Split cells that hold several values (normalize_list)
Step 2   Strip legal suffixes from company names (normalize_company)
Step 3   Reduce website values to root domains (clean_domain)
Step 4   Merge rows that share an email or a root domain
Step 5   Re-find missing work emails (email_finder)
Step 6   Verify every email that will be sent
Step 7   Stamp last_checked on each field that changed

Merging before normalizing would miss rows where one copy says ACME TECHNOLOGIES, INC. and the other says Acme Technologies 3. Enriching before merging would pay twice for the same person.

The cleaning steps cost 0.5 credits per call each and bill on any 2xx response, including an empty result 3. The email_finder step costs 5 credits per call, and a call where no source has the address settles at zero credits 4. The team verifies only the rows it plans to send, so the verification bill tracks the campaign size.

After the import, two routines keep the database from sliding back. New records pass the same normalization at entry, so the next import starts clean. A scheduled job re-checks emails and titles that have not been verified within the team's refresh window, and writes the date of each check next to the field.

The team also keeps the raw values from the event list in separate fields. If a later review finds a wrong merge, the original spelling and URL are still there to compare, and the merge can be reversed without guessing.

After the pass, the duplicate check, the routing rule and the bounce report all read the same values.

Data hygiene vs data enrichment

Hygiene keeps the values a team already holds correct and consistent. Enrichment adds values the team does not have yet. A sensible order runs hygiene first, so enrichment lands on one clean record per person.