Architecture Walkthrough
This section provides a detailed technical description of each of the 9 numbered steps in the What-If Solution Architecture. Each step is described in terms of its inputs, processing logic, outputs, key technology choices, and design considerations.
The user interacts with the What-If UI to define and select specific What-If parameters - such as rating factors, coverage options, or policy endorsements. Each user-defined configuration is persisted as a scenario record in the Scenario DB (SQL), creating an audit-ready trail of every what-if request. The UI enforces input validation and presents pre-configured scenario templates to guide analysts through consistent parameter entry.
Design Considerations:
- Scenario templates should be versioned to ensure reproducibility of historical analyses.
- UI should enforce mandatory field validation before persisting to Scenario DB.
- Consider a scenario cloning feature to accelerate creation of variant scenarios.
A Spark Notebook retrieves the target policy population from the Data Platform (Delta Tables on OneLake). These policy set will be used for scenario selection in Step1, it filters and assembles the precise set of policies that will be re-rated. This What-If Policy Set represents the exposure universe for the analysis - typically a cross-section of in-force policies chosen by line of business, effective date range, territory, or custom segment criteria. The output is a structured Delta table persisted in OneLake.
Each policy record in the What-If Policy Set is transformed into a Rating Request XML conforming to the Rating Engine's schema. A dedicated Notebook applies field mappings, code lookups, and business logic to produce a complete, schema-valid XML document per policy. The resulting Rating Request XMLs captures the precise input payload that would have been submitted in a production rating call.
Design Considerations:
- XML schema validation should be applied to every generated document before persisting.
- Maintain a schema (Mapping Configuration) field in each XML to handle Rating Engine format.
Scenario metadata and the Rating Request XML set are combined into a unified Spark distributed payload. Leveraging Apache Spark's parallel execution model, this step partitions the work across cluster nodes, enabling thousands of policy XMLs to be prepared concurrently. The payload structure includes the scenario identifier, batch identifiers, and each serialised XML document, providing the complete context for a distributed Rating Engine invocation.
The What-If Server orchestrates the distribution of the Scenario + Rating Payload to the external Rating Engine. Each partition of the distributed payload is submitted as an independent HTTP call to the Rating Engine endpoint. Spark workers execute these calls in parallel, maximising throughput against the engine's available concurrency. The Rating Engine applies the current rate tables, rules, and factors to each policy XML, computing premium for each what-if scenario.
Design Considerations:
- Implement exponential back-off and retry logic for transient Rating Engine failures.
- Respect the Rating Engine's concurrency limits - use Spark's maxConnections configuration.
Raw rating responses - typically XML payloads - are captured from the Rating Engine for every submitted policy. Each response contains the computed premium elements, sub-limit breakdowns, applied rating factors, and any engine-level warnings or error codes. Responses are persisted to OneLake in their raw form before any transformation, preserving auditability and enabling re-processing without re-invoking the engine.
Design Considerations:
- Store raw responses in their original format (XML) before parsing - this enables re-processing without re-invoking the engine.
- Log every call attempt, response time, and HTTP status code for SLA monitoring.
- Record response timestamp, engine version, and batch identifier alongside each response for full auditability.
A parsing Notebook reads the raw Rating Engine responses and applies a standardisation transform. Premium elements are extracted into a columnar schema, rating factors are decoded against reference tables, and monetary amounts are normalised to a consistent currency and precision. The Rating Output table in OneLake is the primary analytical artifact - it joins scenario identifiers, policy keys, and re-rated premium components into a single queryable fact set.
Design Considerations:
- Apply schema evolution handling for Rating Engine output changes - use MERGE INTO Delta pattern.
- Derive calculated columns (e.g., premium delta vs baseline) at this stage to avoid repeated computation downstream.
The Rating Output is aggregated into a consolidated Summary View that presents what-if impact metrics at multiple levels of granularity: portfolio total, line of business, territory, segment, and individual scenario dimensions. Key measures include written premium delta (absolute and percentage), loss ratio sensitivity, rate change indices, and scenario-vs-baseline variance. The Summary View is the primary artefact consumed by the Power BI layer in subsequent steps.
Design Considerations:
- Implemented Record Modified column for incremental refresh support.
Power BI dashboards are refreshed with the latest Summary View data via a scheduled dataset refresh. Interactive reports allow actuaries, underwriters, and pricing managers to slice what-if results by scenario, territory, product, and time period. Expose individual policy re-ratings, and comparison views allow side-by-side scenario benchmarking. Reports are governed by row-level security aligned to the user's business unit and access tier.
Design Considerations:
- Use Incremental Refresh in Power BI to avoid full dataset reloads on each what-if run.
- Apply Row-Level Security (RLS) based on business unit or geography to restrict data visibility.
- Pin Summary KPI tiles to the default landing dashboard for executive-level quick read.