A Governance-Centric Lakehouse Architecture For Ai-Ready Public Sector Decision Intelligence Using Medallion Data Engineering Patterns

Abstract

Artificial Intelligence (AI) is being used in public sector organizations to improve policymaking, service delivery, and evidence-based decision-making. Broadly, however, separate data ecosystems, poor data quality, a lack of interoperability, and inadequate governance processes remain obstacles to creating reliable, AI-enabled data analytics environments. In this paper, the researcher proposes a roadmap for the data engineering lifecycle, including governance principles, through a Governance-Centric Lakehouse Architecture for the AI-Ready Public Sector Decision Intelligence. The suggested framework builds upon bronze, silver, and gold data layers for a sequentially enhanced level of data quality, standardization, traceability, analysis, and governance aspects of data stewardship, security, privacy, compliance, and accountability for data use in AI. Additionally, the architecture includes metadata management, data lineage, and access control restrictions based on policies, thereby enabling transparent and explainable AI applications in public-sector settings. The framework integrates contemporary lakehouse features with governance needs, creating a scalable data environment that facilitates the development and utilization of trusted data assets. The framework's combination of modern lakehouse features and governance requirements enables the building and leveraging of trusted data assets to support predictive analytics, decision intelligence, and strategic planning. This study highlights a scalable and interoperable blueprint for governments seeking to rapidly digitize their operations without compromising regulatory requirements, operational efficiency, or ethical AI use in a data-informed government sphere.