Mid-Level Data Engineer, Databricks

Accountabilities: Design, develop, test, and maintain data pipelines using Databricks, Python, PySpark, and SQL to ingest, transform, and organize data from source systems into layered, reusable tables. Build and maintain data products that support analytics, reporting, and automation across multiple business areas. Validate datasets against source systems, investigate discrepancies, troubleshoot data quality issues, and implement corrections before data reaches downstream reporting and analytics processes. Apply established ETL/ELT practices, data modeling principles, and quality controls to ensure data accuracy, consistency, and reliability. Document data pipelines, table definitions, transformation logic, and business rules to support maintainability, collaboration, and knowledge sharing. Take ownership of assigned engineering tasks from initial development through delivery and production support, communicating progress and raising risks or blockers promptly. Collaborate with senior engineers, data scientists, government stakeholders, and other team members to clarify requirements and deliver solutions aligned with business needs. Adapt to changing project priorities and develop proficiency with additional tools and technologies as new requirements emerge. Contribute to continuous improvements in data engineering workflows, pipeline reliability, and the usability of shared datasets. Requirements: Bachelor's degree in computer science, data science, information systems, engineering, or a related discipline, or equivalent professional experience. At least 5 years of experience in data engineering, data analysis, or a related technical field. Hands-on experience building and supporting data pipelines in Databricks, including notebooks, PySpark or Spark SQL, and Delta tables. Proficiency in Python and SQL for data transformation, querying, validation, and troubleshooting. Solid understanding of ETL/ELT processes, data modeling, data integration, and data quality practices. Ability to investigate data discrepancies, identify root causes, and implement effective solutions. Strong documentation, communication, and problem-solving skills, with the ability to explain technical issues and collaborate with both technical and non-technical stakeholders. Demonstrated ownership of assigned work, effective time management, and the ability to adapt to shifting priorities while keeping stakeholders informed. U.S. citizenship is required. Must be eligible to obtain and maintain federal suitability, a government-issued Personal Identity Verification (PIV) credential, and all required system access. Nice to Have: Experience integrating data through APIs. Familiarity with Databricks Unity Catalog, Databricks Jobs or Workflows, and CI/CD deployment practices. Experience working with Google Cloud Platform (GCP). Previous experience supporting federal government clients or projects. Benefits: Salary: Annual compensation range of $100,000–$140,000 USD. Work arrangement: Fully remote position within the United States. Health coverage: Comprehensive medical, dental, and vision insurance. Insurance protection: Short-term disability and life insurance coverage. Retirement savings: 401(k) plan with matching employer contributions. Paid time off: Paid leave, including 11 holidays and a day off for your birthday. Professional development: Opportunities and resources to support ongoing learning, technical growth, and career advancement. Fitness benefits: Fitness membership support. Remote-work support: Home office allocations to help support your remote working environment. Performance incentives: Performance bonuses and profit-sharing opportunities. Employee referral program: Referral bonuses for eligible employee referrals. Career development and support: A collaborative environment within a growing organization, with an emphasis on employee development, curiosity, authenticity, and helping team members reach their professional goals. Inclusive workplace: Commitment to equal employment opportunities and fair consideration for qualified applicants.