Mid Data Developer ( Azure + Databricks )

Accountabilities:: Design, develop, maintain, and optimize robust ETL/ELT data pipelines, applying best practices for low latency, resilience, data quality, observability, and reliable data processing. Develop scalable data processing workflows using Apache Spark, PySpark, and Databricks, applying efficient partitioning strategies and cost optimization techniques for large-scale workloads. Structure, organize, and manage data using SQL Server and Delta Lake, contributing to reliable and scalable storage architectures for modern data platforms. Apply DevOps practices across data engineering workflows, including Git-based version control, CI/CD automation, deployment routines, and continuous monitoring of production processes. Investigate production incidents, identify root causes, implement corrective actions, and contribute to the continuous improvement of data pipelines and production routines. Contribute to data platform modernization initiatives, supporting the development of scalable, product-oriented architectures and helping evolve data engineering practices. Requirements: Proven experience in data engineering, with hands-on experience developing and maintaining data pipelines and working with modern data processing technologies. Experience working in production environments, including monitoring, troubleshooting, incident investigation, and continuous improvement of operational data workflows. Experience handling large volumes of data and developing scalable processing solutions capable of meeting performance and reliability requirements. Experience participating in data migration, modernization, or data lake development projects, with an understanding of modern data platform architectures. Strong experience with cloud-based data environments, particularly Azure, and practical knowledge of Databricks for distributed data processing. Proficiency with Apache Spark and PySpark, including experience with data transformation, processing optimization, partitioning, and large-scale workloads. Experience with SQL and SQL Server, as well as knowledge of Delta Lake and data lake/data warehouse environments. Familiarity with Git and CI/CD practices, with the ability to apply DevOps principles to data engineering and deployment workflows. Strong analytical and problem-solving skills, particularly the ability to investigate incidents, perform root cause analysis, and implement sustainable solutions. Experience with AI projects and pipeline automation is considered an advantage, as is knowledge of data-oriented architectures and data governance best practices. Familiarity with Databricks MLflow for model management is a plus, particularly for professionals interested in the intersection of data engineering, AI, and machine learning. Benefits: Health insurance and dental insurance. Food and meal allowance. Childcare assistance. Extended parental leave. Access to Wellhub (Gympass) and TotalPass, with partnerships supporting fitness, health, and wellbeing. Profit Sharing and Results (PLR). Life insurance. Continuous learning opportunities through a dedicated learning platform. Discounts through partner programs. Free online platform focused on physical health, mental health, and overall wellbeing. Pregnancy and responsible parenthood courses. Partnerships with online learning platforms. Language learning platform. Additional benefits and programs designed to support professional development, wellbeing, and inclusion.