BCBS 239 Compliance with AI

BCBS 239 requires banks to aggregate risk data accurately, with full data lineage, and deliver reports on time. Most banks treat this as a burden: enormous manual effort on data lineage tracing, cross-system reconciliation, and narrative report generation.

The manual effort problem

A single regulatory submission requires tracing data from source systems (trading, lending, treasury) through transformations to final reports. Reconciliation across siloed risk systems reveals discrepancies that must be investigated. Report narratives must explain risk positions with data provenance.

The Lakeflow data engineering stack

• Lakeflow Connect ingests data from risk systems via managed connectors
• Spark Declarative Pipelines handle reconciliation ETL across systems (materialized views for golden source data, streaming tables for real-time feeds)
• Lakeflow Jobs orchestrate scheduled reporting workflows (DAGs with conditional logic, event triggers)

Data flows into Delta Lake as the golden source for risk data.

Agentic compliance on Databricks

Three agents work together:

1. Data Lineage Agent: Uses Unity Catalog lineage APIs to trace every data point from source to report. Unity Catalog provides table and column-level lineage natively, this is built into the platform, not a custom build.
2. Reconciliation Agent: Uses ai_query for cross-system consistency checks against Delta Lake tables, prioritizing discrepancies by materiality.
3. Report Generation Agent: Uses ai_summarize and ai_gen to draft regulatory reports with data provenance and source citations.

Agent Bricks Multi-Agent Supervisor coordinates the pipeline, ensuring all data quality checks pass before report generation begins.

Unity Catalog as the BCBS 239 backbone

Unity Catalog is the foundational governance layer for BCBS 239 Principle 2 (data architecture and IT infrastructure). It governs data, models, functions, pipelines, vector search indexes, and serving endpoints. Data lineage is native, not bolted on.

All models are accessed via AI Gateway on Databricks Model Serving. AI Functions (ai_query, ai_summarize, ai_gen) handle the data analysis and narrative generation directly in SQL or PySpark.

From burden to advantage

Manual reconciliation effort drops by 90%. Real-time data lineage replaces periodic lineage audits. Regulatory submissions are faster and more accurate. And the same data quality improvements that satisfy regulators also improve risk management and strategic decision-making.