Lead Data Engineer

Accountabilities:: You will lead the technical direction of data migration and modernization initiatives, ensuring the successful transition of legacy systems to scalable cloud-based infrastructure. Your responsibilities will include: Define and own the technical approach for migrating legacy data warehouse environments and pipelines to an Azure and Databricks lakehouse architecture. Establish standardized migration frameworks, technical patterns, and best practices to guide data engineering teams. Design and develop reusable data migration pipelines using Azure Data Factory and Azure Databricks. Define data reconciliation strategies and validation processes to ensure accuracy and consistency between source and target systems. Establish data quality controls, testing standards, and quality gates throughout the migration lifecycle. Provide technical leadership in resolving complex pipeline challenges, performance bottlenecks, and data engineering issues. Optimize data processing workflows to improve scalability, reliability, and overall system performance. Mentor data engineers, provide technical guidance, and review engineering deliverables to maintain high standards of quality. Collaborate with Product, Quality Assurance, and Platform teams to coordinate technical activities and ensure successful project delivery. Promote consistent engineering practices and scalable architectural solutions across migration workstreams. Communicate technical decisions, progress, risks, and recommendations clearly to internal stakeholders and customers. Contribute to additional technical and operational responsibilities as required. Requirements : The ideal candidate is an experienced data engineering leader with a strong background in cloud modernization, large-scale data migrations, and distributed data processing. You should bring: A Bachelor's degree in a relevant field. Proven experience leading data engineering teams or technical workstreams within large-scale migration or modernization programs. Deep expertise in Databricks and/or Apache Spark, with hands-on experience designing and implementing data processing solutions. Strong experience with Azure Data Factory and cloud-based data pipeline development. Advanced SQL and Python programming skills. Demonstrated experience migrating legacy ETL processes and data warehouse workloads to cloud environments. Strong understanding of lakehouse architecture, medallion architecture, and Delta Lake. Experience designing reusable data pipelines, data validation frameworks, and reconciliation processes. Ability to troubleshoot complex data engineering problems and optimize pipeline performance. Proven technical leadership skills, including mentoring engineers, reviewing code, and establishing engineering best practices. Excellent communication and collaboration skills, with the ability to explain technical concepts and align stakeholders around technical direction. Experience working effectively in remote, collaborative Agile environments. A proactive, adaptable, and solution-oriented mindset. Additional qualifications that would be advantageous include: Experience working in federal, government, or other regulated data environments. Familiarity with government data modernization initiatives and associated compliance requirements. Additional Employment Requirements: Must be legally authorized to work in the United States. Depending on federal contract requirements, certain positions may require U.S. citizenship. Ability to obtain and maintain a federal background investigation and/or security clearance, if required. Benefits : Annual salary range of $110,000–$145,000. Fully remote work arrangement. Comprehensive employee benefits package. Technology and education stipend. 401(k) retirement plan with company matching. Flexible paid time off (PTO). Paid holidays. Opportunities for professional development and technical growth. Collaborative environment within a small, impact-driven organization. Opportunity to contribute to meaningful technology initiatives supporting the public sector.