Engenheiro de Dados Senior

Accountabilities: Develop, maintain, and optimize large-scale data pipelines in Databricks, primarily using SQL and also PySpark. Apply generative AI, LLMs, and agents to automate data collection, extraction, structuring, and validation, including from unstructured sources. Integrate new internal and external data sources while ensuring traceability and cost efficiency. Build and maintain APIs that make data available to platform products. Ensure data quality and integrity through automated tests, validation processes, and monitoring, addressing issues at their root cause. Apply privacy and data protection best practices in compliance with Brazil’s LGPD. Optimize processing performance and costs across Databricks and AWS environments. Contribute to architecture decisions and define data engineering standards and best practices. Use AI-powered development tools, including Claude Code, throughout the development lifecycle, helping the team adopt these tools with quality and critical judgment. Share knowledge through code reviews, pair programming, and technical collaboration. Work closely with Product, Data Science, Platform, and business teams to translate requirements into effective solutions and communicate results clearly. Document data pipelines, business rules, and data architecture. Requirements: Degree in Computer Science, Data Engineering, Information Systems, or a related field, or equivalent professional experience. Professional experience as a Data Engineer or in a similar role. Advanced SQL skills and solid knowledge of data modeling. Hands-on experience with Databricks. Practical experience using AI in data-related workflows, such as LLMs or agents for information collection, extraction, classification, or validation. Daily experience with AI-powered development tools such as Claude Code, Cursor, GitHub Copilot, or similar solutions. Knowledge of Python and PySpark. Familiarity with AWS services such as S3, Lambda, and EKS. Experience with Git and automated testing. Ability to work independently, communicate results clearly, and collaborate effectively. Experience working in agile and collaborative environments. Nice to have: Experience with Claude Code. Knowledge of Delta Live Tables (DLT) and/or streaming data pipelines. Experience with Unity Catalog. Expertise in optimizing Databricks pipeline performance and processing costs. Experience building data APIs. Knowledge of CI/CD practices and code review standards. Experience with web scraping and large-scale public data collection. Experience building AI agents or integrating with LLM APIs. Knowledge of LGPD requirements as applied to data. English proficiency for reading technical documentation. Benefits: Meal and food allowances. Wellhub or TotalPass access. Home-office allowance. Health insurance. Optional dental insurance. Childcare assistance for children up to 6 years old. Extended maternity, paternity, and adoption leave. Life insurance. Birthday day off, available on the birthday or during the birthday month. Family Day: an additional day off for parents between May and August. Mental Health Break: one consecutive week off in December. Career development and growth opportunities. Collaborative, inclusive, and entrepreneurial work environment. Opportunity to contribute to innovative technology and products with broader market impact.