Senior Data Pipeline Engineer

Accountabilities: You will be responsible for building and maintaining reliable data infrastructure, ensuring the accuracy and availability of critical information, and supporting continuous improvements across the data engineering environment. Your main responsibilities will include: Architect, develop, and maintain ETL/ELT pipelines and C#/.NET pipeline services using technologies such as SSIS, Apache Airflow, AWS Lambda, data enhancement services, and message-processing solutions. Support the evolution of the cloud data platform, including Apache Iceberg ingestion and transformation pipelines, DynamoDB integrations, and query performance optimization using Trino/Starburst. Monitor end-to-end data quality, investigate production pipeline incidents, identify root causes, document findings, and implement corrective actions through to successful data reprocessing. Lead the onboarding of new external and regulatory data sources, coordinating requirements and ensuring seamless integration across ingestion and transformation layers. Contribute to data schema design, architecture decisions, and technical improvements across the data platform. Maintain standard operating procedures, technical documentation, and engineering best practices. Mentor Associate-level engineers, review their work, and support their technical development. Participate in release management, deployment activities, and processes across development, QA, and production environments. Partner with the Data Operations Manager on technical architecture, platform strategy, and roadmap decisions. Provide escalation-level support for business-critical data pipeline incidents, including after-hours assistance when required. Requirements : The ideal candidate is an experienced data engineering professional with strong technical ownership, advanced database expertise, and the ability to operate independently within a distributed team. The following qualifications and skills are expected: 5+ years of experience in database engineering, data engineering, or ETL development, with demonstrated ownership of production data pipelines. Expert-level proficiency in Transact-SQL, including stored procedures, views, and database indexing. Hands-on development experience with C#/.NET. Proven experience designing and implementing ETL processes using SQL Server Integration Services (SSIS). Familiarity with orchestration platforms such as Apache Airflow is preferred. Experience working with AWS cloud services and open table formats such as Apache Iceberg and Parquet. Familiarity with AWS Lambda, DynamoDB, and Trino/Starburst is an advantage. Strong data quality management, troubleshooting, and root-cause analysis capabilities, particularly in production environments. Understanding of source control, CI/CD practices, and release and deployment workflows across Dev, QA, and Production environments. Ability to mentor other engineers, conduct technical reviews, and communicate complex technical concepts effectively. Excellent written and verbal communication skills. Strong business judgment, independent prioritization, and the ability to manage competing responsibilities. Ability to collaborate effectively within a remote and distributed team. Willingness to take on additional responsibilities and special projects as required. Demonstrated sensitivity and respect for racial, gender, sexual orientation, and cultural differences. Commitment to inclusive values, curiosity, accessibility, transparency, and a collaborative approach to work. Must be legally authorized to work in the United States without current or future employer sponsorship. Benefits : Annual salary ranging from $100,000 to $130,000 for the New York City area, with adjustments based on location, skills, and experience. Fully remote working arrangement. 35-hour workweek, Monday through Friday. Comprehensive health insurance, including medical, dental, and vision coverage. Employer retirement contributions, with an additional matching option. Employer-paid life insurance and Accidental Death and Dismemberment (AD&D) coverage. Short-term and long-term disability insurance. Generous paid leave options, including paid time off, compassionate leave, volunteer leave, holidays, and parental leave. Flexible Spending Accounts (FSA) and supplemental insurance options. Pre-tax commuter and transit benefits. Summer working hours. Eligibility for the Public Service Loan Forgiveness (PSLF) program. Opportunities to participate in annual, weeklong all-staff summits, with additional in-person meetings depending on leadership responsibilities. Travel is expected for employees based outside the New York City area.