🏢
Grafana
Staff AI Engineer | Canada | Remote
Job Description
ABOUT THE ROLE
Grafana Labs is a 100% remote company with a global collaborative culture, a passion for meaningful work, and a commitment to innovation. We're scaling fast and staying true to our open-source legacy. Our team thrives in an environment where transparency, autonomy, and trust fuel everything we do.
We're looking for a Staff AI Engineer to own the AI agent infrastructure and automation platform that powers our GTM teams. This is a high-autonomy role where you'll identify the highest-leverage problems, design solutions, and ship them.
WHAT YOU'LL DO
As a Staff AI Engineer, you'll be responsible for building multi-agent architectures, LLM integrations, and backend services that connect AI models to internal and third-party data platforms. You'll ship production systems that teams depend on daily and define the technical direction for the automation platform.
Your key responsibilities will include:
Agentic Tool Development
- Owning end-to-end development of multi-agent AI systems, from architecture and implementation through testing, deployment, and ongoing operation
- Building modular, composable agentic systems using orchestration frameworks (LangChain, CrewAI, Anthropic MCP, or similar) that operate 24/7 across teams
- Developing reusable agentic skills that agents invoke across interfaces (Slack, dashboards, internal apps, CLIs)
- Implementing observability and feedback loops including logging, performance metrics, prompt iteration, model evaluation, and cost management
- Establishing governance and compliance standards for AI workflows including access controls, audit trails, PII handling, and human-in-the-loop escalation paths
Systems Integration & Backend Services
- Building MCP servers, APIs, CLIs, and microservices connecting AI models to business systems (BigQuery, Slack, CRMs, email, calendars, analytics tools)
- Architecting data flows for retrieval-augmented generation (RAG), connecting LLMs to internal knowledge bases, customer data, and real-time business context
- Building serverless or containerized services (GCP Cloud Functions, Cloud Run) that scale with usage and integrate with Grafana's cloud infrastructure
Automation & Workflow Manufacturing
- Partnering with RevOps, Demand Generation, Regional Marketing, and SDR teams to scope high-impact automation problems, identify bottlenecks, and build solutions with measurable business outcomes
- Designing and deploying workflows using orchestration tools (n8n, Workato, or custom platforms) with CI/CD, testing, and production reliability standards
- Building systems designed for self-service with documentation, playbooks, and enablement materials that let partner teams operate independently
WHAT YOU'LL NEED
To be successful in this role, you'll need:
- 8+ years of software engineering experience with depth in backend development, systems integration, or data/analytics engineering
- 2+ years hands-on experience applying LLMs/AI to production workflows, not just prototypes
- Strong proficiency in Python and JavaScript/Node.js with Git-based workflows, code review practices, and testing discipline
- Hands-on experience with LLM frameworks and patterns including prompt engineering, RAG, function calling/tool use, structured output parsing, and evaluation
- Experience building and operating