Senior Lead Software Engineer

Be an integral part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products. As a Senior Lead of Software Engineering – Payment Data Platform Infrastructure at JPMorgan Chase within the Commercial and Investment Banking – Data Analytics Payments Team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. Promote significant business impact through your capabilities and contributions, and apply deep technical expertise and problem-solving methodologies to tackle a diverse array of challenges that span multiple technologies and applications. Job responsibilities Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems Produces architecture and design artifacts for complex applications while being accountable for ensuring design constraints are met by software code development Deploy and operate ClickHouse clusters across environments (on-prem and/or cloud), including upgrades, scaling, backups, and disaster recovery. Designs and implements scalable, secure, and cost-effective AWS infrastructure solutions. Develops and maintains infrastructure as code (IaC) using tools such as Terraform, CloudFormation, or AWS CDK. Monitors and optimizes the performance, availability, and scalability of AWS infrastructure. Implements and manages security best practices, including identity and access management (IAM), encryption, and network security. Automates infrastructure provisioning, configuration, and deployment processes. Identifies opportunities for cost optimization and implement strategies to reduce AWS spending. Develops and maintain scripts and tools to enhance operational efficiency. Stays up-to-date with the latest AWS services, features, and best practices. Required qualifications, capabilities, and skills Formal training or certification on software engineering concepts and 8+ years applied experience Hands-on experience deploying and supporting ClickHouse in production (multi-node clusters). Strong knowledge of ClickHouse internals and performance levers: MergeTree family, parts/merges, partitions, primary key vs ordering key, compression, TTLs, sampling, distributed query execution. Experience with ClickHouse Keeper / ZooKeeper coordination and cluster topology design. Experience with Linux systems engineering and troubleshooting (CPU, memory, disk, networking). Expertise in AWS cloud platform (S3, Lambda, EKS, DynamoDB, EventBridge, Kafka, Opensearch), big data technologies (Apache Spark, Glue, Redshift), and data lake/lakehouse architectures (Iceberg). Experience with containerization/orchestration and automation (Kubernetes and Terraform) Stays up-to-date with the latest AWS services, features, and best practices. Contributes to software engineering communities of practice and events that explore new and emerging technologies Experience leading adoption of agentic AI-enabled engineering practices (using enterprise-authorized tools within the work environment) across teams, including defining operating expectations (human-in-the-loop validation, quality gates), measuring outcomes, and ensuring secure handling of sensitive inputs/outputs.