Engenheiro(a) de Dados Sênior – Azure Databricks

Accountabilities: Design, develop, maintain, and optimize data pipelines using Azure Databricks, PySpark, Azure Data Factory (ADF), Python, and Azure Data Lake. Build and support ETL/ELT processes to integrate enterprise data and meet analytical and business requirements. Perform data ingestion, transformation, validation, and reconciliation across multiple source systems. Conduct data profiling and quality assessments to identify inconsistencies, resolve issues, and ensure data accuracy and reliability. Develop, write, and optimize complex SQL queries to improve data processing performance. Perform performance tuning and troubleshoot large-scale data workloads to ensure efficiency, scalability, and reliability. Provide Level 2 production support for incidents involving Azure Data Factory pipelines, Databricks jobs, and data processing workflows. Monitor and maintain production pipelines, ensuring availability, operational stability, and efficient resource utilization. Support deployment and release processes across development, testing, and production environments. Manage competing priorities across multiple projects, working independently while meeting deadlines and maintaining high delivery standards. Requirements More than eight years of professional experience in data engineering, including the development and maintenance of ETL processes. Strong hands-on experience with Azure Databricks and Azure Data Factory (ADF). Extensive knowledge of ETL/ELT development, data transformation, and enterprise data integration. Advanced SQL skills, including complex query development, optimization, and performance tuning. Practical experience with Python and PySpark for data processing and engineering. Experience working with Azure Data Lake and integrating data from multiple enterprise systems. Proven ability to perform data profiling, validation, reconciliation, and quality assurance. Experience monitoring, maintaining, and troubleshooting data pipelines in production environments. Advanced English proficiency, both written and spoken, to collaborate effectively in a global working environment. Strong analytical and investigative skills, with a structured, solution-oriented approach to complex technical challenges. Ability to work autonomously, organize activities, manage competing demands, and prioritize effectively. Excellent verbal and written communication skills, with the ability to collaborate across multidisciplinary teams. Commitment to code quality, performance, reliability, and timely delivery. Nice to have: Experience implementing CI/CD pipelines and DevOps practices. Familiarity with Git, Azure DevOps, or similar version control and deployment tools. Knowledge of cloud monitoring, logging, and observability solutions. Benefits Fully remote work arrangement. CLT employment contract in Brazil. Opportunity to contribute to large-scale enterprise data engineering initiatives in a global environment. Salary, healthcare coverage, paid time off, and additional benefits are not specified in the provided job description.