Skip to main content

Technology Stack

Understanding the technology stack is a prerequisite for configuring and implementing the platform correctly. Each technology listed here plays a specific role in the data flow, from ingesting policy data through to delivering results in Power BI. The Component Inventory in the Solution Architecture Guide maps each of these technologies to its corresponding architectural component and functional role.

The WhatIF Data Platform is built on the following technologies:

  • Microsoft Fabric Lakehouse, the primary storage layer for all Delta tables, including policy sets, XML payloads, rating outputs, and summary views.
  • Fabric Lakehouse SQL Analytics Endpoint, exposes Lakehouse Delta tables as queryable SQL views, used by the semantic model and for ad-hoc analysis.
  • Microsoft Fabric PySpark Notebooks, the execution environment for all data transformation, XML build, XML parse, and API processing logic.
  • Delta Tables, the storage format for all structured data in OneLake, supporting ACID transactions, schema evolution, and incremental merge patterns.
  • Spark SQL, used within notebooks for data transformation, view creation, and analytical query logic.
  • YAML-driven configuration, controls the behavior of XML build and parse flows at runtime, eliminating the need for code changes when mapping rules evolve.
  • SQL Server metadata and control table reads, the Scenario DB and related control tables are read by notebooks to drive scenario processing and report configuration.
  • XML build and XML parse logic, the two primary data transformation patterns used to construct Rating Engine request payloads and parse responses.
  • External Rating API integration, the Rating Engine is invoked via distributed Spark HTTP calls, with one request per policy per scenario.
  • Analytical SQL views for reporting, a Kimball-style reporting layer built on top of the Lakehouse SQL Analytics Endpoint, consumed by the semantic model.
  • Power BI Semantic Model in Import mode, the shared semantic model (import_sm_whatif) that serves all report artifacts.
  • Live-connected Power BI report artifacts, report files connected to the shared semantic model rather than directly to the data source.
  • XMLA/TOM and Fabric REST for semantic model maintenance, used by the schema-drift automation notebook to add or remove columns from the semantic model without manual intervention.
  • Fabric CI/CD Deployment Pipeline, governs the promotion of Fabric artifacts across workspaces.