Staff Software Engineer, Observability Platform

About the role

Join Wealthsimple's mission to make financial services accessible and intuitive for everyone. As a Staff Software Engineer on the Observability Platform team, you will be instrumental in building the foundational systems that empower all engineers and AI agents to understand production environments. You will shape the technical direction and ensure the highest quality in our observability solutions.

Key facts

What you'll do

• Architect and implement systems for handling wide-event data models, high-cardinality storage, and high-throughput ingestion and query pipelines.
• Develop and promote telemetry libraries and SDKs to simplify instrumentation for internal engineering teams.
• Establish and enforce instrumentation standards, including naming, tagging, sampling, and adherence to OpenTelemetry specifications.
• Create fast query foundations and access patterns that enable AI agents to assist in production incident investigation and change verification.
• Utilize AI tools to accelerate prototyping, understand complex systems, and enhance team workflows.
• Investigate and make significant, well-reasoned changes to unfamiliar codebases.
• Pilot new observability approaches with select teams, measure their effectiveness, and scale successful solutions.
• Mentor other engineers, review complex architectural designs, and lead technical initiatives across teams.

Requirements

• At least 8 years of software engineering experience, with a focus on building platforms.
• Hands-on experience writing production code in multiple programming languages, such as Kotlin and Ruby.
• Demonstrated experience building telemetry platforms, ingestion pipelines, and developer-facing SDKs.
• Deep understanding of structured event data, streaming and columnar data stores, cardinality, sampling, and context propagation.
• Proficiency with OpenTelemetry or similar standards, and experience designing SLIs and SLOs.
• Proven ability to quickly understand, refactor, and impact complex and ambiguous codebases.
• Daily use of AI development tools, such as Claude Code, to enhance productivity and quality.
• Strong communication and influencing skills to collaborate effectively with cross-functional teams.

Nice to have

• Experience in the fintech industry or in regulated environments.
• Practical experience with ClickHouse or other columnar storage solutions.
• Familiarity with Kubernetes and progressive delivery tools like Argo Rollouts.
• Experience building observability solutions tailored for LLM and agent-based workloads.

Skills & tools

• Kotlin
• Ruby
• OpenTelemetry
• ClickHouse
• Kubernetes
• Argo Rollouts
• Claude Code

Practical notes

• Wealthsimple offers comprehensive health benefits, life insurance, and a group savings plan with employer matching.
• Employees receive 20 vacation days, 4 wellness days, and unlimited sick and mental health days annually.
• The company provides a 90 days away program, allowing employees to work outside of Canada for up to 90 days per year.
• Various employee resource groups are available, including Rainbow, Women of WS, and Black at WS.
• Applications are encouraged from all individuals, and accommodations will be provided throughout the hiring process upon request.
• AI tools may be used to support parts of the hiring process, but all final decisions are made by humans.