Tactical Excellence, Strategic Truth: Who Checks Whom

One Call, Two People, Many Meters

Picture a call center where agents make outbound calls to customers. Custom software dials and routes the calls, a phone company provides the lines, and a manager runs the floor. The product owner, who is accountable for all of it, has one goal this quarter: reduce dropped calls.

One afternoon a customer picks up, and an agent starts walking them through an order. Four minutes in, the customer's voice cuts out. The agent says hello twice, hears nothing, and the line goes dead before the order is complete.

Only the customer knows why. They may have set the phone down, stopped listening, walked away, or driven into a tunnel. The agent and the software see the same thing in every case, silence and then a dead line, and the agent types "customer hung up" in the notes and moves on. I call that a dropped call, and it is the only kind I mean: a conversation that started but did not complete a transaction.

The call left tracks in many places. The software logged four minutes and an open order, and depending on how it is configured, it counts that as a dropped call or a completed one. The phone company's records show a call that was answered and ended normally, since from its side nothing went wrong.

The manager's floor report shows agents busy and utilization high, so by that measure the afternoon went well. Days later a satisfaction survey comes back with one line in the comment box: "my call dropped." It is the only record written from the customer's side, and it cannot say which call it was about.

Now the product owner asks whether dropped calls are getting better. Engineering shows a chart where drops are down. QA has a count of scenarios that failed again after release. The manager brings the floor report, customer experience brings the satisfaction score, and analytics has a monthly trend line that nobody in the room quite trusts yet. Forty minutes later the group has debated five numbers and decided nothing.

When I find myself in that meeting, the tooling is rarely the problem. Each dashboard is probably accurate for the system it records. The trouble is that every group is answering every question, and nobody has agreed which question belongs to whom.

Three Questions, Three Owners

The cleanest fix is a division of labor, and each job serves the same goal: fewer dropped calls. Developers improve the system. QA checks that the improvement is real. Analytics measures whether it mattered. It helps me to write the questions down, because they sound similar and are not.

Development owns diagnosis and action. That means finding the problem, tracing the root cause, and designing the fix, along with building enough operational visibility to see what the system is doing. QA owns the claim that the fix holds. Analytics owns the longer view, which is whether the thing everyone agreed was worth fixing got better in a way a customer would notice.

Development and QA share a horizon, which is why they often get blurred together. One person believing a fix works and another person proving it works are separate acts.

Tactical Reporting, Strategic Analysis

Tactical thinking is about the present: what is broken right now, where agents are getting stuck, what changed after the last deployment. Development and QA work here.

Tactical work lives in the tools engineers already use: operational dashboards, log queries, error monitoring, and the QA test suites. A good tactical dashboard is messy, specific, and changes as the problems change.

Strategic thinking asks a slower question, and it belongs to analytics. Is quality improving? Are drops declining over a quarter, not an afternoon?

No single source can answer that, because each holds one slice of what happened. The answer requires the agent's notes, the software log, the phone company's records, and the customer's own words stitched together over time. That work lives in the analytics platform and the executive dashboards built on it, and those dashboards should be stable and boring, because a trend line only means something if the definition under it stays put.

The stitching forces a decision the tactical teams never had to make: what counts as a dropped call? I gave the definition at the start, a conversation that started but did not complete a transaction. It covers a customer whose signal died, a customer who set the phone down, and a customer who decided mid-order they were done, whatever the cause.

Each source records those differently. The software may log one as a completed call, the phone company's records show none of them as a failure, and the agent's notes say "customer hung up" for all three. Applying the definition the same way across every source is analytics' job.

The mistake runs in both directions. A monthly stitched trend cannot tell you why one particular call dropped, so using it to troubleshoot an incident goes nowhere. A count from one source can shift just because that team changed what it records as a drop, so using it for executive reporting produces a number that moves for the wrong reasons.

Why the Fixer Shouldn't Also Be the Judge

The group that makes a change shouldn't be the only group that decides whether the change worked.

I'm a developer, and I know how it feels to ship a fix. The error disappears from the logs, the deployment goes green, and the part of my brain that wants the work to be finished quietly agrees that it is. Every team wants its own work to succeed, and that shapes what it looks for.

So the same fix gets three different questions. Developers say, "We fixed it." QA asks whether they can prove it. Analytics asks, weeks later, whether the trend improved.

A team ships a change so the agent software saves the order when a call drops and offers a callback to finish it. The developers verify what they can see: the lost-order errors no longer appear, the code behaves as intended, the deployment succeeded. From where they sit, the problem is solved.

QA tests that conclusion against the conditions that actually produce a dropped call, such as a customer's signal vanishing mid-order, a customer setting the phone down and walking away, or a customer who simply stops responding. They might find that one of those still loses the order, that the requirement was understood slightly differently than it was written, or that the fix changed behavior somewhere else. Any of those gaps can exist in a strong team's work. QA sees how the fix behaves in the conditions of a real call center, a view the developers' own checks may not include.

I wrote about the handoff side of this in QA Was Always My Last Battle Front, which is about how often I made QA reconstruct what I meant before they could even begin validating it. The separation of roles only works if each side gets what it needs from the other.

The Trend Is Someone Else's Question

Suppose the dropped-call fix passes QA. The tests are green, stakeholders are pleased, and the release is called a success. Analytics is the group that comes back weeks later with the sources stitched together and asks whether the monthly trend moved.

A fix can work exactly as intended and still leave the trend roughly where it was, because the calls it addressed were a small share of the whole. It can also go the other way, where a modest fix has an outsized effect on the trend. Neither result is visible from inside the release, and neither contradicts the tactical dashboards, which were answering a different question. Without that longer view, an organization can't tell a fix that shipped from a fix that mattered, and it will count the first as the second.

This is also where the temptation to let engineering self-report gets expensive. I explored a version of it in The Engineering Zoo, where the dashboard improved steadily while the thing the dashboard was supposed to represent stayed exactly where it was.

Analytics and development read the same data differently because they ask from different horizons. The analytic contract is one way I've found to make each side's assumptions visible before a report breaks.

Whom the Product Owner Asks

The product owner asked whether dropped calls are getting better and was handed five numbers, each answering a different question.

Only one of the five answers the product owner's question, and it is the slowest. The other four answer earlier questions that can be settled this week, while the trend may take months of stitched data before it means anything. When someone brings a number to a meeting, I start by asking which question it answers.

Without that agreement, dueling dashboards turn the meeting into an audit of the numbers instead of a decision about the work. Disagreements change character too. If QA says the fix failed and analytics says the trend improved, both statements can be true, because they cover different horizons.

The Loop, and Where It Can Break

The goal is fewer dropped calls, and when the three roles work they form a feedback loop that serves it. Developers change how the agent software handles a dropped call, and QA validates the change under real conditions. Analytics measures the trend, and leadership uses that to decide whether the goal is being met and where to invest next. That decision points developers at the next improvement, and then it starts again.

Each handoff can fail. If developers skip QA's validation, bad behavior reaches customers while the dashboard looks fine. If analytics lacks a stable definition of a dropped call, leadership gets a trend nobody trusts, and the next fix goes to whoever presents the most convincing chart.

The third handoff is the one most likely to fail: leadership acting on what analytics reports. The other three groups can each do excellent work and the loop still stalls if nobody with authority is willing to be redirected by what the measurement says, a tension I looked at from another angle in Accountability and Authority.

Each group wants its own work to succeed, so the design does not rely on anyone being unbiased and has a second group check each stage. That is a more realistic foundation than hoping everyone is objective. The real question for any team, mine included, is who is willing to say when a fix the team is proud of leaves dropped calls right where they were.

Explore More

• QA Was Always My Last Battle Front — the handoff problem between developers and independent validation
• The Engineering Zoo: When the Dashboard Gets Better but the Zoo Does Not — what happens when measurement and the work it describes drift apart
• Stop Digging Through Logs. Start Designing for Learning. — making the analytics and development relationship explicit
• Accountability and Authority: Walking the Tightrope — responsibility without the power to act on it