Data Engineer - R01571442

Data Engineer

Job requirements

• Key Responsibilities Logic Discovery and Requirements Analysis
• Work with pricing analysts to identify custom definitions, calculations, transformations, dependencies, assumptions, edge cases, and ambiguities currently implemented in the Gold layer.
• Reverse-engineer analyst-written SQL, spreadsheet logic, and ad hoc calculations to define intended business behaviour.
• Translate legacy logic into clear technical requirements and migration plans.
• Migration to the Silver Layer
• Re-implement approved pricing definitions as governed, reusable, documented dbt models in the Silver layer.
• Follow data modelling, naming, testing, version control, and code review standards.
• Build maintainable, scalable, auditable models independent of undocumented analyst knowledge.
• Apply dbt tests for uniqueness, not-null, accepted values, relationships, and business rules.
• Parity Validation and Quality Assurance
• Compare migrated Silver-layer results with existing Gold-layer outputs and resolve differences.
• Validate results across representative periods, segments, boundary conditions, and known exceptions.
• Confirm migrations only after parity review and pricing analyst approval.
• Analyst Collaboration and Sign-Off
• Partner with pricing analysts to clarify calculations, confirm intended behaviour, and align on expected results.
• Lead walkthroughs and reviews; document analyst approval before retiring legacy definitions.
• Communicate risks, open questions, dependencies, and decisions to technical and business stakeholders.
• Documentation and Lineage
• Document business meaning, calculation rules, assumptions, exceptions, source inputs, ownership, lineage, tests, and downstream consumers in dbt and Confluence.
• Ensure documentation enables future engineers and analysts to maintain logic without relying on the original analyst.
• Gold-Layer Cleanup and Cutover
• Decommission ad hoc Gold-layer logic after the Silver replacement is validated, approved, adopted, and downstream dependencies are updated.
• Verify retired logic is no longer used and operational documentation reflects the new source of truth.
• Required Qualifications
• 5–8 years of data engineering experience on analytical data platforms.
• Strong SQL skills: joins, window functions, CTEs, aggregations, conditional logic, and optimisation.
• Hands-on dbt experience across models, tests, documentation, and lineage.
• Strong Snowflake experience, including modelling, performance tuning, and warehouse concepts.
• Experience refactoring, migrating, or modernising complex analytical logic.
• Ability to untangle ad hoc SQL and spreadsheet calculations and validate outputs.
• Strong understanding of data quality, reconciliation, testing, and release controls.
• Skilled at working with non-engineering stakeholders to clarify requirements and validate outcomes.
• Excellent communication and documentation habits.
• Core Skills and Tools
• SQL; Python a plus
• Snowflake
• dbt, dbt Cloud
• Airflow
• Confluence, Jira, Slack
• Data modelling, quality validation, reconciliation, migration planning, stakeholder collaboration, and technical documentation.