Director, Data Engineer (Python, Pyspark, MLOPs, LLMOPs, SQL, Databricks)

About this role: As a Director of Data Engineering in the Tech Market Economics team, you will serve as a core engineering leader for a rapidly growing capability, operating with the agility of a startup but with the backing and scale of Gartner. We are building Gartner's next generation of analytical platforms, transforming how we generate insights by moving from artisanal research to a continuous, AI-powered insight engine. This is a strategic leadership role designed for a builder who wants to leave a lasting architectural footprint. You will not be inheriting a legacy maintenance project; you will be architecting the destination state for our data, analytics, and MLOps platforms. Partnering with cross-functional leaders, you will help evolve our operating model, establishing the engineering systems, telemetry, and automated evaluation frameworks necessary to scale complex AI workflows across the enterprise. If you are looking to step up, shape a modern engineering culture, and build combinatorial AI models that will influence the high-tech industry, this is your platform. What you will do: Architect the destination state for high-volume data pipelines that support both model training and complex, multi-prompt inference workflows at enterprise scale Design robust evaluation frameworks, guardrails, and automated quality gates to manage consistency, reliability, and variance in non-deterministic AI outputs Establish MLOps/LLMOps practices and system telemetry to monitor pipeline health, detect data and model drift, and ensure continuous performance in production Partner with Data Science, Product, and Research leaders to define technical roadmaps, moving prototype models into scalable, production-grade capabilities Architect solutions for data isolation, metadata management, and data classification across internal, external, and unstructured data sources Establish engineering standards, reusable code design principles, and CI/CD governance frameworks that accelerate engineering velocity while reducing technical debt Lead the modernization of production data platforms, identifying architectural bottlenecks, improving system resilience, and expanding end-to-end automation What you will need: Experience: 9+ years of progressive experience in data engineering, advanced analytics, MLOps, and complex distributed system architecture. Education: Bachelor’s degree required; Master’s degree preferred in Computer Science, Data Science, Software Engineering, or a related quantitative discipline. Pipeline Architecture: Expertise in designing and optimizing robust enterprise pipelines, data workflows, and modern orchestration frameworks for complex AI/ML systems. AI/MLOps & Deployment: Proven track record of deploying, monitoring, and scaling AI models in production, including experience managing output consistency and implementing automated regression/evaluation suites. Cloud & Infrastructure: Hands-on experience architecting scalable data systems within major cloud environments and modern enterprise data platforms. Programming & Tooling: Strong proficiency in Python, with deep experience using numerical processing, orchestration, and automated testing libraries to develop scalable enterprise solutions. Quality & Telemetry: Proven ability to build evaluation tooling, system monitoring dashboards, and alerting frameworks to catch performance degradation in production. Who you are A strategic technical builder who drives enterprise value by creating scalable systems, reusable assets, and durable architectures rather than just executing isolated tasks. An exceptional problem solver capable of taking loosely defined requirements or prototype models and translating them into actionable engineering architecture. Data-driven and analytically curious, with a passion for enterprise technology markets and the role AI plays in creating combinatorial insights. Thrives in a fast-paced, high-growth environment, balancing multiple priorities while effectively managing risk, technical debt, and architectural trade-offs. An effective communicator and technical leader capable of translating complex architectural concepts, system behaviors, and MLOps methodologies to diverse business, research, and technical audiences. What you will get: Competitive salary, generous paid time off policy, charity match program, Group Medical Insurance, Parental Leave, Employee Assistance Program (EAP) and more! Collaborative, team-oriented culture that embraces diversity Professional development and unlimited growth opportunities #LI-PM3 Who are we? At Gartner, Inc. (NYSE:IT), we guide the leaders who shape the world. Our mission relies on expert analysis and bold ideas to deliver actionable, objective business and technology insights, helping enterprise leaders and their teams succeed with their mission-critical priorities. Since our founding in 1979, we’ve grown to 20,000 associates globally who support over 13,000 client enterprises in ~90 countries and territories. We do important, interesting and substantive work that matters. That’s why we hire associates with the intellectual curiosity, energy and drive to want to make a difference. The bar is unapologetically high. So is the impact you can have here. What makes Gartner a great place to work? Our vast, virtually untapped market potential offers limitless opportunities – opportunities that may not even exist right now – for you to grow professionally and flourish personally. How far you go is driven by your passion and performance. We hire remarkable people who collaborate and win as a team. Together, our singular, unifying goal is to deliver results for our clients. Our teams are inclusive and composed of individuals from different geographies, cultures, religions, ethnicities, races, genders, sexual orientations, abilities and generations. We invest in great leaders who bring out the best in you and the company, enabling us to multiply our impact and results. This is why, year after year, we are recognized worldwide as a great place to work. Gartner is the world authority on AI At Gartner, you’ll join a company at the very center of the AI revolution. Gartner has proactive, objective guidance throughout clients’ AI journeys. We set the standard for how organizations leverage artificial intelligence to drive meaningful impact. You’ll have access to unmatched resources, expertise, and technology, and play a key role in helping Gartner and our clients innovate and grow as we leverage AI to transform business and technology landscapes. It’s an exciting time to be at Gartner, with limitless opportunities to make a real impact, grow your skills, and build a lasting, meaningful career in a field that’s reshaping the way we operate. If you’re passionate about AI and want to be part of a team that’s guiding the leaders who shape the world, Gartner is the place for you. What do we offer? Gartner offers world-class benefits, highly competitive compensation and disproportionate rewards for top performers. In our hybrid work environment, we provide the flexibility and support for you to thrive — working virtually when it's productive to do so and getting together with colleagues in a vibrant community that is purposeful, engaging and inspiring. Ready to grow your career with Gartner? Join us. The policy of Gartner is to provide equal employment opportunities to all applicants and employees without regard to race, color, creed, religion, sex, sexual orientation, gender identity, marital status, citizenship status, age, national origin, ancestry, disability, veteran status, or any other legally protected status and to seek to advance the principles of equal employment opportunity. Gartner is committed to being an Equal Opportunity Employer and offers opportunities to all job seekers, including job seekers with disabilities. If you are a qualified individual with a disability or a disabled veteran, you may request a reasonable accommodation if you are unable or limited in your ability to use or access the Company’s career webpage as a result of your disability. You may request reasonable accommodations by calling Human Resources at +1 (203) 964-0096 or by sending an email to ApplicantAccommodations@gartner.com . Job Requisition ID:114333 By submitting your information and application, you confirm that you have read and agree to the country or regional recruitment notice linked below applicable to your place of residence. Gartner Applicant Privacy Link: https://jobs.gartner.com/applicant-privacy-policy For efficient navigation through the application, please only use the back button within the application, not the back arrow within your browser.