Discovery Agent

Scan a source, profile columns, detect PII, infer semantics, and propose mappings for BA review.

Purpose

The Discovery Agent collapses source onboarding from months to days. It connects to a new source system, automatically ingests technical metadata, profiles every column, classifies PII, infers the business meaning of fields using the enterprise glossary, detects relationships, and drafts a star-schema mapping aligned to the Algonomy Data Agent semantic layer — all reviewed and approved by a Business Analyst before anything is published.

Solves
Manual SME interviews, tribal knowledge, duplicate dimensions, undocumented PII, slow onboarding.
Delivers
Ranked tables, semantic dictionary, source-to-target mapping, lineage, PII flags, BA-signed report.
Handoff
Approved data product moves to the Data Quality Agent for rule generation and monitoring.
Choose your scenario
The same agent — three different starting conditions. Pick the one that matches your source situation.
UC1Active scenario · deep dive· New source · No documentation
Systems in scope · greenfield source only
Source (undocumented)
modelled
Retail POS · Egypt (Oracle 11g)
Newly acquired estate. No data dictionary, Arabic column comments, no SME available.
Destination (governed model)
target
Algonomy Data Agent · Sales Transaction / Inventory Availability data products
Existing governed target model on Databricks Unity Catalog. Agent aligns the new source to it.
Reference (glossary)
read-only
Enterprise Business Glossary · Purview + Unity tags
Used to match cryptic column names (str_cd, itm_nbr) to business terms with evidence.
1Select Source
2Scan & Profile
3Understand Semantics
4Map Tables
5BA & QA Review

Step 1 · Onboard an undocumented source (UC1)

No data dictionary, no SME on the line. Give the agent enough context to start a cold scan.

Zero-doc mode. Fields below describe the unknown source. Documentation status defaults to None — the agent will infer everything from metadata + sampled values.
→ Agent will cold-scan every table, infer semantics from evidence, and draft a first-time star schema.