Data Engineers - Public Sector

Summary

Our public sector client is seeking senior consultants with experience and expertise to migrate data from legacy applications to new Dynamics 365 solution and new data warehouse/lakehouse platform.

Responsibilities

• Data Cleansing and Validation: cleanse and validate the extracted data to ensure its accuracy, completeness, and consistency. Help with identifying and resolving data quality issues, performing deduplication, and applying business rules to ensure the integrity of the migrated data;
• Data Mapping and Transformation: map the data elements from the legacy application to the corresponding entities and fields in Dynamics 365 CE, F&O and Data Lakehouse. Handle necessary data transformations, ensuring compatibility and consistency between the legacy data and the target system;
• Ongoing Support: provide post-migration support, analyze and address data-related issues or questions; help optimize data management processes in the new environment;
• Data Migration Strategy and Execution: review the present migration strategy that outlines the overall approach, sequence, and timeline for migrating the data from the legacy application to Dynamics 365 using a delta-load approach; execute the migration plan efficiently, managing data transfers and ensuring minimal disruption to ongoing operations;
• Data Lakehouse: help expand the current data lakehouse implementation and help build a gold layer for reporting; conduct data analysis and profiling; develop data transformation and cleansing pipelines; implement data integration and harmonization;
• Documentation: document the entire data migration process, including data mapping rules, transformation logic, migration scripts, and any specific configurations;
• Data Testing and Quality Assurance: conduct thorough testing to verify the accuracy and integrity of the migrated data; define test cases, perform data reconciliation, and address any issues or discrepancies that arise during the testing phase; develop KPIs to report on the progress, completeness and quality of the data migration effort;
• Data Extraction: help extract the required data from the legacy application, develop and implement extraction processes to retrieve data from various sources, such as databases, files, APIs, or other relevant legacy systems;
• Data Analysis: analyze the existing data in the legacy applications, understand its structure, quality, and relationships and help in designing an appropriate migration strategy;

Skills & Qualifications

Must Haves:

• Proficiency in scripting languages like Python, and experience with Azure-specific scripting using PowerShell or Azure CLI.
• Knowledge of integration technologies commonly used with Dynamics, such as DataVerse / Common Data Service (CDS), Data Entities, and APIs.
• Proficiency in Azure Data Factory, Azure Synapse workspaces, PolyBase, including knowledge of pipeline creation, data flows, integration runtimes, triggers, and monitoring.
• Familiarity with version control systems (e.g., Azure Repos) and collaboration tools (e.g., Azure DevOps) for managing code, tracking changes, and collaborating with team members.
• Expertise in data transformation techniques, such as data cleansing, aggregation, enrichment, and normalization using Azure cloud technologies
• Experience in integrating various data sources and systems, both on-premises and in the cloud, using Azure
• Experience with continuous integration/continuous deployment (CI/CD) processes around DevOps, data workflows, Synapse workspaces.
• Strong SQL skills and experience working with Azure SQL Databases and Dataverse; good understanding of Azure storage concepts and technologies.
• Understanding of data quality management practices, including data profiling, data validation, and error handling
• Understanding of data governance principles, data privacy regulations and experience working with high-sensitivity data, and knowledge of best practices for data security and compliance in Azure.
• Expert in data manipulation languages (T-SQL, PL/SQL), data definition languages, physical database design, data modelling, query performance analysis & tuning
• Experience with SSIS, SSRS, PowerBI
• Familiarity with data warehousing concepts and experience working with Azure Synapse Analytics or similar platforms for building and managing large-scale data warehousing/lakehouse solutions.
• Experience with SQL Server Management Studio, Azure data management tools, XRM toolbox, data modeling tools (preferably, ERWIN).

Nice To Haves

• Experience with continuous integration/continuous deployment (CI/CD) processes around DevOps, data workflows, Synapse workspaces
• Knowledge of integration technologies commonly used with Dynamics, such as DataVerse / Common Data Service (CDS), Data Entities, and APIs.
• Azure cloud certifications (e.g. Azure fundamentals, Azure Data Engineer associate, Azure Database Administrator associate)
• Proficiency in Azure Data Factory, Azure Synapse workspaces, PolyBase, including knowledge of pipeline creation, data flows, integration runtimes, triggers, and monitoring.
• Strong SQL skills and experience working with Azure SQL Databases and Dataverse; good understanding of Azure storage concepts and technologies.
• Expert proficiency with data manipulation languages (T-SQL, PL/SQL), data definition languages, physical database design, data modeling, query performance analysis & tuning
• Experience with PowerApps platform, Power Automate
• Proficiency in scripting languages like Python, experience with Azure-specific scripting using PowerShell or Azure CLI
• Familiarity with data warehousing concepts and experience working with Azure Synapse Analytics or similar platforms for building and managing large-scale data warehousing/lakehouse solutions.

About Raise

Raise is an established government, IT and engineering hiring firm with over 60 years’ experience connecting talented candidates with meaningful work. When you apply, you’ll get more than just a chance at a great job—you’ll become part of a vast network of employers that are always changing.

We strive to build teams that reflect the diversity of the community we work in and encourage applications from traditionally underrepresented groups such as women, visible minorities, Indigenous peoples, people identifying as LGBTQ2SI, veterans, and people with disabilities.

#GOV

#MON