Staff Software Engineer

About the role

Grafana Labs is seeking a Staff Software Engineer to join the Adaptive Telemetry group within the Databases department. This role focuses on optimizing how telemetry data is stored and managed within Grafana Cloud, ensuring that customers receive the most valuable insights while controlling costs. You will contribute to the technical vision and delivery of solutions that enhance the efficiency and relevance of metrics, logs, traces, and profiles.

Key facts

What you'll do

• Define technical strategy and roadmaps, influencing product and engineering decisions.
• Lead the full lifecycle of major, cross-functional projects from planning to operation.
• Oversee the architecture, reliability, performance, and cost efficiency of critical systems.
• Establish reliability targets (SLOs/SLIs) and manage incident response, including post-mortems and automation.
• Improve observability, automation, and operational readiness through telemetry, alerting, and capacity planning.
• Coordinate with Product, Design, and other teams to align priorities and resolve blockers.
• Mentor engineers, conduct design reviews, and elevate engineering standards.
• Represent engineering strategy to both technical and non-technical stakeholders.

Requirements

• Demonstrated experience delivering and operating large-scale distributed systems, showing technical leadership.
• Strong understanding of system design principles, including latency, consistency, availability, scaling, and cost tradeoffs.
• Practical experience with cloud-native architectures (microservices, containers, Kubernetes, Infrastructure as Code) and operational best practices.
• Ability to define SLOs/SLIs, perform capacity planning, tune performance, and manage reliability initiatives.
• Excellent coding and design skills; proficiency in Go is preferred, but experience with Python, C, C++, or Rust is acceptable.
• Comfort and practical experience with AI-assisted development tools and integrating them into team workflows.
• Familiarity with streaming/messaging systems like Kafka and observability tools such as Prometheus/Grafana.
• Proven ability to influence cross-functional stakeholders and drive outcomes in a remote environment.
• Clear written and verbal communication skills for both engineers and non-technical audiences.

Skills & tools

• Go
• Python
• C
• C++
• Rust
• Kubernetes
• Prometheus
• Grafana
• Kafka
• Mimir
• Loki
• Tempo
• Pyroscope
• AI coding assistants (OpenAI, Anthropic, Google models)

Practical notes

This is a fully remote position for candidates located in Canadian time zones. Compensation includes equity and potential bonuses, with specific benefits detailed on the Grafana Labs careers page. Grafana Labs supports AI-assisted development with company-funded usage budgets and access to advanced AI models. The company offers 30 days of annual leave, with 3 days reserved for company-wide shutdown days. An in-person onboarding experience is provided.