A few months ago I was on a call with an engineering director who uses CompassHQ. She had a board meeting in two hours and needed to answer a straightforward question: “Why did cycle time spike in January, and should we be worried?”
The data was all there. DORA dashboard had the numbers. The SPACE survey from three weeks earlier had relevant comments. The release timeline showed a major infrastructure migration overlapping with two feature deadlines. But connecting those dots meant opening three different views, mentally correlating timelines, and then writing it up in a way that a non-technical board member would understand. All of that for one question.
That conversation is what finally pushed Compass AI from “we should build this” to “we’re building this now.”
What It Actually Is
Compass AI is a chat interface built directly into your CompassHQ dashboard. You type a question in plain English, and it answers using your actual data. Not hallucinated numbers. Not industry averages. Your organization’s real metrics, pulled from the same data your dashboards display.
Ask it “How are we doing on DORA metrics?” and it’ll pull your deployment frequency, lead time, change failure rate, and recovery time. It knows your delivery profiles, so it compares against your targets, not someone else’s benchmarks. If your team ships on a two-week release cycle and that’s intentional, Compass AI won’t penalize you for not deploying daily.
Ask it “Which services need attention?” and it’ll look across change failure rates, deployment gaps, and open incidents to surface the ones that are drifting.
The thing I care most about: every number it shows you comes with a source. You can see exactly which tool call produced which data point. No black box answers.
Why This Matters More Than Another Dashboard Tab
Dashboards are great for monitoring. You check them regularly, you spot trends, you drill down when something looks off. But dashboards don’t explain things. They show you what happened but not why.
The question engineering leaders actually ask isn’t “what’s our deployment frequency?” They can read the chart. The question is “why did it drop, and is the cause something we should fix or something we expected?” Answering that requires cross-referencing multiple data sources, understanding context, and knowing what the team was trying to accomplish.
That’s the gap Compass AI fills. It sits on top of all three dimensions CompassHQ tracks — delivery metrics, developer experience, and strategic alignment — and can correlate across them in seconds.
A few examples of things you can ask:
- “Show me deployment frequency trends for the past quarter” — and it’ll generate a chart inline, right in the conversation
- “What’s our change failure rate and how does it compare to last month?”
- “Which teams have the highest review cycle time?”
- “Summarize how Team Platform is doing across all metrics”
How It Works Under the Hood
For the technically curious: Compass AI is powered by Claude and uses the same MCP (Model Context Protocol) tools that power our external API. When you ask a question, the AI figures out which tools to call, executes them against your organization’s data, and synthesizes the results into a response.
This means it has exactly the same access as your dashboard — no more, no less. The tools are scoped to your organization and your role. An Admin sees everything. A Contributor sees what Contributors see. There’s no way for the AI to access data outside your org because the authorization is enforced server-side, not in the prompt.
Responses stream in real-time so you’re not staring at a spinner. And when a chart would help explain the data, Compass AI generates one inline — line charts, bar charts, area charts — whatever fits the question.
What It Won’t Do
I want to be direct about limitations because I think AI product launches tend to oversell.
Compass AI won’t make decisions for you. It won’t tell you to fire someone or reorganize your teams. It surfaces data, explains trends, and highlights risks. You decide what to do about it.
It won’t fabricate data. If there’s no information available for a question, it’ll say so. We built hard rules into the system prompt: every factual claim must cite a tool call. If the tool returned nothing, the AI tells you it returned nothing. This was a non-negotiable design decision. An AI assistant that makes up plausible-sounding metrics is worse than no AI assistant at all.
It also won’t pretend to be something it’s not. Ask it to ignore its instructions or act like a different system and it’ll politely refuse. Security isn’t a feature you bolt on later.
Context-Aware
One detail I’m particularly happy with: Compass AI knows what screen you’re looking at when you open it. If you’re on the DORA dashboard and you ask “why is this number low?”, it understands that “this number” refers to whatever metric you’re viewing. Small thing, but it makes the interaction feel natural instead of forcing you to spell out the full context every time.
Who This Is For
If you’re an engineering leader who spends time pulling data from multiple sources to answer questions from stakeholders, this is for you. If you’re a manager who wants a quick health check on your teams without clicking through five pages, this is for you. If you’re prepping for a board meeting or a quarterly review and need a summary of how engineering is doing, this is definitely for you.
The director from that call I mentioned? She was one of our beta testers. Last week she told me she used Compass AI to prep for her board meeting in fifteen minutes instead of the usual two hours. That’s the kind of thing that makes building this worth it.
Available Now
Compass AI is live for all CompassHQ users today. You’ll see it in the navigation as “Compass AI.” Click it, type a question, and see what comes back.
We’re early with this. The tool set will grow — predictive health scoring, automated weekly digests, deeper SPACE survey analysis — but the foundation is solid and the core experience is ready. I’d rather ship something focused and useful than wait for a feature list that sounds impressive in a press release.
If you run into something unexpected or have ideas for what Compass AI should be able to answer, reach out at hello@compass-hq.app. We’re listening.
Compass AI is included in Pro and Enterprise plans at no additional cost. Pro plans include 100 requests per user per hour and 5,000 per organization per day. Enterprise plans have unlimited usage.