Software Developer (Content Integration & Data Pipelines)

Accountabilities: Build scalable web scrapers and crawlers: Develop and maintain production-ready scraping and crawling solutions that collect online content responsibly, following standards such as robots.txt, rate limiting, and applicable terms of use. Develop content integration services: Implement parsing, extraction, and normalization logic using TypeScript and JavaScript to transform content from diverse sources into consistent, usable data. Design event-driven pipelines: Build and maintain Kafka-based services that process, enrich, and route content in near real time, supporting reliable and scalable data flows. Integrate external content sources: Connect third-party content providers, RSS and Atom feeds, and external APIs to expand and maintain content coverage. Develop backend services: Design and implement Java-based RESTful services that support content processing, integration, and delivery across the platform. Improve data quality and reliability: Establish and apply best practices for error handling, source management, validation, data normalization, and fault-tolerant pipeline operations. Monitor and optimize pipeline performance: Implement observability and monitoring practices, investigate processing issues, and improve system performance, reliability, and operating costs. Contribute to technical architecture: Participate in system design discussions, evaluate scalability trade-offs, and help define the technical direction of content acquisition and data processing systems. Support search and indexing capabilities: Contribute to content indexing and retrieval workflows, collaborating with search technologies such as Solr to ensure content is structured and accessible. Collaborate across the development team: Work with technical colleagues and other stakeholders to deliver maintainable solutions aligned with product and business needs. Mentor and share knowledge: Support junior developers, contribute to code reviews and engineering best practices, and promote continuous learning within the team. Requirements: Education and experience: A degree in Computer Science or a related field, or equivalent practical experience, with at least three years of professional software development experience. TypeScript and JavaScript proficiency: Strong programming skills in TypeScript or JavaScript, particularly for content integration, parsing, and data processing. Backend development experience: Solid experience building backend services with Java or a comparable programming language. Production web scraping expertise: Hands-on experience developing and operating web scrapers and crawlers in production environments, with an understanding of robots.txt, rate limiting, responsible crawling practices, and applicable terms of use. Event-driven architecture: Practical experience with event-driven systems and message-streaming technologies, particularly Kafka. Content processing skills: Knowledge of HTML parsing, content extraction, data transformation, and normalization techniques. Data pipeline engineering: Experience designing or maintaining high-volume, fault-tolerant data pipelines that process and deliver information reliably. Database and data modeling knowledge: Familiarity with SQL and data modeling principles. Development tools and environments: Experience working in Linux environments and using Git and modern software development workflows. Familiarity with Maven is also expected. Search technology knowledge: Experience with Solr, including indexing, schema design, and query tuning, is an asset. Relevant industry experience: Experience building content ingestion pipelines for news, media, or similar data-intensive applications is strongly preferred. System design capabilities: Understanding of distributed systems, scalability trade-offs, reliability, and performance optimization. Ownership and initiative: Ability to work independently, take responsibility for projects, and proactively identify opportunities for technical improvement. Analytical problem-solving: Strong troubleshooting skills and the ability to investigate complex technical issues and develop practical solutions. Communication and teamwork: Ability to communicate effectively with technical and non-technical stakeholders, collaborate across teams, and share knowledge. Interest in innovation: Enthusiasm for data engineering, automation, emerging technologies, and AI-powered capabilities. Benefits: Competitive compensation: Expected annual salary range of CAD $75,000–$110,000, depending on experience, skills, and internal equity. Remote work: Fully remote working environment. Health coverage: Health, dental, and vision benefits. Retirement savings: RRSP matching. Financial protection: Life insurance. Employee support: Access to an Employee Assistance Program (EAP). Paid time off: Paid vacation, personal days, and sick days. Holiday break: A week off between Christmas and New Year's. Meeting-free Fridays: No internal meetings scheduled on Fridays. Career development: Opportunities for professional growth and career progression. Collaborative culture: A supportive working environment with tools and practices that encourage teamwork, knowledge sharing, and continuous improvement. Inclusive workplace: A commitment to an inclusive environment where people with diverse backgrounds and perspectives are encouraged to apply. Reasonable accommodations can be arranged for candidates who require support during the recruitment process.