The Questions Your Bot Could Not Answer Are the Useful Part

TL;DR

• The gap queue is the most valuable thing an employee assistant produces and almost nobody reads it.
• Most entries are not missing content. They are a question phrased in words your documents do not use, which is a headings problem rather than a writing problem.
• Three categories: a genuine documentation gap, a correct refusal needing no action, and a vocabulary mismatch. Only the first needs a document.
• Do not write a document for a question asked once. Cluster first, then write one document per cluster.
• The person reading the queue has to be somebody who can write the answer. Give it to an analyst and it becomes a report.
• The gap rate should fall and never reach zero. A queue with nothing in it means people have stopped asking.

Three Hundred Questions and Nobody Looked

A company ran an HR chatbot for five months and considered it a success. Usage was steady, the answers were good, and the HR team said their inbound had dropped noticeably.

Then somebody opened the log of questions it had not been able to answer. Three hundred and twelve entries. Reading them took forty minutes and produced a list of nine topics, four of which the team had been answering by hand the entire time without noticing they were the same four.

The most useful output of the whole deployment had been accumulating for five months in a screen nobody had a reason to open. One of the nine was asked 41 times: what happens to unused leave at the end of the year. The policy existed. It was in a document called Annual Leave and Absence Management, under a heading that said Carry-Over Provisions, and nobody searching in plain language had found it.

Recommended Tools

Best tools for AI HR Tools

B BambooHR HR management system L Lattice Performance management D Deel Global payroll & EOR

View all recommended tools

That is the shape of this. The queue is not a defect list. It is a list of what your people actually need to know, written in their own words, which is a thing companies normally pay to find out.

What Is Actually In The Queue

Four things get logged, and products differ on which they distinguish. Establish this before you read anything, or the categories will blur.

Questions it could not answer at all. No source matched, so it said it did not know. The cleanest signal and the smallest category.

Questions it answered badly. It found something and the something was wrong, incomplete or from the wrong document. Harder to detect, because it looks like a success unless somebody rated it or asked again.

Questions it refused on purpose. Out of scope, sensitive, or routed to a human. These are the system working and they need no action, but they will sit in the same list and inflate it.

Questions asked again immediately. The strongest signal in the whole dataset and the one most often unlogged. Somebody rephrasing within thirty seconds is telling you the first answer failed, whatever the system recorded.

Ask your vendor which of the four it records and whether you can filter them apart. A queue that mixes deliberate refusals with real gaps is a queue nobody will trust after the second reading.

Most Gaps Are Vocabulary, Not Content

This is the finding that changes what you do with the queue, and it is counterintuitive enough that most teams get it wrong on the first pass.

When a question comes back unanswered, the instinct is that the policy is missing. Usually it is not. The policy exists, in a document whose title and headings use the organisation's internal language, and the question used the words a person actually says.

The carry-over example above is the canonical shape. Nobody asks about carry-over provisions. They ask what happens to the leave they did not take. Those are the same question in two vocabularies, and the retrieval failed on the gap between them.

So the first action on any apparent gap is to search your own corpus by hand for the answer. Two minutes. If you find it, you do not have a content gap, you have a findability problem, and the fix is a heading, a synonym line or a short plain-language summary at the top of the existing document rather than a new document.

Three habits that close most of these.

Write headings as questions people ask. "What happens to leave I have not taken" beats "Carry-Over Provisions" and costs nothing.

Add the vernacular terms into the document text. If people say sick note and the policy says fit note, the document should contain both words somewhere. This is not keyword stuffing, it is writing in the reader's language.

Put a one-line plain answer at the top of each section. Retrieval tends to find and quote the first direct statement, so leading with it improves the answer as well as the finding.

When the Queue Is Not Worth Reading

In the first week

The first week is dominated by people testing it. Questions about the weather, questions designed to make it say something silly, and questions about topics far outside scope. Reading that week as a needs analysis will send you writing documents nobody wanted. Start at week two.

When the corpus is deliberately narrow

If you launched with the handbook only and everybody knows it, the queue will be full of questions about things you consciously excluded. That is not a gap, it is your scope working. Read it to decide whether the scope was right, not to write documents.

When nobody can act on it

A queue read by somebody who cannot commission or write a document produces a list and no outcome. If the only available reader is an analyst, the honest answer is to wait until the right person has thirty minutes, because an unactioned report read monthly trains everybody that the exercise is decorative.

The edge case that forces attention

The same question appearing more than about ten times in a month. At that volume it is not a gap, it is a thing your people need and cannot get, and somebody is answering it by hand repeatedly without the pattern reaching anybody who could fix it.

Reading It Properly, in Forty Minutes

The mechanics matter because the obvious approach, sorting by frequency and working down, produces the wrong priorities.

Export the whole period, not a top-ten view. A frequency sort hides the clusters, because fifteen differently-worded versions of one question each appear once.

Strip the deliberate refusals first. They are noise for this purpose. If you cannot filter them, they are usually recognisable in a glance.

Group by topic, by hand, before counting anything. This is the step that does the work. Fifteen phrasings of the leave question become one cluster of fifteen, which changes its rank completely.

Then count the clusters and sort those. Now the order reflects demand rather than wording.

For each of the top clusters, search the corpus by hand. Content gap or vocabulary gap. Two minutes each, and the answer determines whether you write something new or edit something existing.

Write down the clusters you are not acting on, and why. This is the part that makes the next reading faster, because half the long tail recurs and you will otherwise re-investigate it every month.

Forty minutes for five months of data in the example above. It is not an analytics project and it does not need a dashboard.

The One-Document-Per-Cluster Rule

The failure mode after a good first reading is writing too much, and it is worth guarding against explicitly.

One document per cluster, not per question. A cluster of fifteen phrasings about untaken leave needs one clear section, not fifteen entries in an FAQ. The assistant will match all fifteen phrasings to one well-written passage.

Resist writing for anything asked once. A single question is not evidence of a need. Park it, and if it returns next month it has earned a document. Because every new document is something somebody has to maintain, a corpus grown from the long tail becomes unmaintainable faster than it becomes useful.

Prefer editing an existing document to adding a new one. Two documents answering the same question slightly differently is the worst outcome available, and it is what happens when a new FAQ is written beside an existing policy nobody thought to check.

And put the new content where the answer already lives. If the leave question belongs in the leave policy, it goes in the leave policy. A separate Frequently Asked Questions document becomes a second source of truth within a quarter.

Who Should Read It

Whoever can write the answer. That is the whole requirement and it rules out most of the obvious candidates.

Not an analyst. They will produce a good summary of a problem they cannot fix, which adds a handoff and loses the detail that makes the clusters obvious.

Not the platform owner. If IT owns the assistant, IT will read the queue as a retrieval performance report rather than as a content specification, which is a reasonable reading and the less useful one.

Ideally whoever owns the handbook. They can tell a vocabulary gap from a content gap in seconds, because they know what the documents say, and they can make the edit themselves without commissioning anything.

With one person from the HR inbox, for the first two readings. They have been answering these questions by hand and will recognise the clusters immediately. In the opening example, this is the person who would have said four of the nine within a minute.

So the practical arrangement is a standing thirty minutes, monthly, in one named person's calendar, with the inbox person joining for the first two. Not a meeting, not a report, and not a dashboard.

What to Measure, and What Not To

Three figures are worth tracking and one commonly-reported one is misleading.

The gap rate. Unanswered questions as a proportion of all questions, with deliberate refusals excluded. This should fall over the first two quarters and then flatten. A figure that keeps falling towards zero is usually a sign that usage is falling rather than that coverage is improving.

The size of the top cluster. More useful than the total, because it tells you whether there is a single thing worth fixing this month. When the largest cluster drops below about five, you have reached the long tail and the monthly reading can become quarterly.

Repeat questions from the same person. Somebody asking the same thing three times in a fortnight has not got an answer they can use, whatever the system recorded. This is the closest thing to a satisfaction measure the queue offers.

And the figure to distrust: a bare deflection or resolution percentage. It counts answers given, not answers that worked, and it will look excellent while the carry-over question fails 41 times. If somebody asks for one number, give the gap rate instead and say why.

The Privacy Side, Briefly

The queue is a list of questions employees asked, which makes it employee data, and that has consequences for how you handle it even though its purpose is benign.

Three rules that keep this straightforward. Read it in aggregate by default, since clusters are the useful unit and individual identity adds nothing to the task. Restrict access to the named reader rather than making it a shared dashboard. And never build the report that lists questions by person, which is the most useful-sounding artefact in this area and the one guaranteed to be experienced as monitoring.

The exception is the repeat-question measure, which needs a stable identifier to compute. Compute it as a count rather than a list, so the output is that four people asked something three or more times rather than who they were.

What obligations apply to holding and processing this differs by jurisdiction and by circumstance, so settle that with local advice rather than from an article, and settle it before anybody requests a dashboard rather than after.

The Options

Matram

Disclosure: Matram is owned by the same people who publish HROpsLab. It answers from the sources you give it, with strict mode so it declines rather than inventing when the answer is not there, which is what makes a gap queue meaningful in the first place. A product that always produces something confident has no gaps to log. Pricing is flat per month with unlimited seats at $29, $69 and $199, and it supports 95 or more languages.

Its limits: no free tier, it answers rather than actioning tickets, and it is not an HR or ITSM workflow platform. For this subject the question to put to it and to every alternative is whether the queue separates deliberate refusals from genuine misses, and whether you can export the period rather than only view a top ten.

SiteGPT

Best for a document-trained assistant where the published surface is the main one.

Read on 5 October 2026: $468 and $948 per year, billed yearly, with monthly options at $59 and $129. The lower monthly equivalents frequently quoted for this product require annual billing and should never appear without that qualifier.

Where it struggles for this subject: a public-facing deployment produces a queue mixing customers and staff, which needs separating before it means anything.

Freshservice

Best where the assistant sits in a service platform, because the ticket history gives you a second source of demand data to compare the queue against.

Read on 5 October 2026: $19, $49 and $99 per agent per month, plus the Freddy AI add-on at $29 per agent per month. It publishes no language count.

Where it struggles: per-agent pricing prices the answerers, which is the wrong direction for a tool whose purpose is fewer questions reaching them.

Zendesk

Best for larger teams already standardised on it, with mature reporting around the conversation data.

Read on 5 October 2026: $55 per agent per month plus $50 per agent per month for Copilot. It geo-redirects, so confirm your currency on the page.

Where it struggles: the reporting is built around agent performance rather than content gaps, so the queue reading described here is something you assemble rather than something it hands you.

Leena AI and Moveworks

Both built for enterprise employee support, where gap analysis is a first-class feature rather than a log you export.

Both publish no price, so comparison requires a sales process. For a company whose requirement is forty minutes of reading a month, that evaluation is out of proportion.

The Comparison

The first column is the one to insist on, because a queue that counts your deliberate refusals as failures produces a gap rate that cannot improve and a report nobody believes after the second month.

The Decision Table

Detecting the Answers That Were Wrong

The hardest category to work with is the one that does not appear in the queue at all: questions the assistant answered confidently and incorrectly. Those log as successes, and finding them needs a different method.

Four signals, in descending order of how easy they are to get.

The immediate rephrase. Somebody asking a near-identical question within a minute. If your product timestamps questions per session, this is computable and it is the single best proxy for a failed answer. Count sessions containing two similar questions inside sixty seconds.

The thumbs-down, if it exists and anybody uses it. Rating widgets get very low engagement, typically a few per cent, so treat the absence of negative ratings as no information rather than as good news. The ones you do get are worth reading individually, since somebody bothered.

The handoff that arrives with a complaint. When a conversation escalates to a person, the first thing the employee says is often about the answer they just received. Those messages are a quality signal sitting in a channel nobody mines for it.

And a sample you check by hand. Twenty answers a month, read against the source documents. Tedious, unglamorous, and the only method that finds the confidently-wrong answer nobody reported. Twenty is enough to tell a 5 per cent error rate from a 30 per cent one, which is the distinction that matters.

Then act on what the sample tells you about the corpus rather than about the product. A wrong answer traced back to its source is almost always a document that contradicts another document, or one carrying a figure that changed. Those are content defects the assistant surfaced rather than created, and they were producing wrong answers from human colleagues too.

The Nine Topics, and What They Usually Are

Because the clusters recur across companies with surprising consistency, it is worth naming the ones that come up most, so a first reading has something to check itself against rather than starting cold.

Leave arithmetic. Not the policy, the arithmetic. What happens to untaken days, how a part year is pro-rated, whether a bank holiday falling in a booked week counts. The policy almost always exists and almost never answers the arithmetic.

The practical mechanics of a process. Where the form is, who approves it, how long it takes. Documents describe what the process is and skip where it lives, because the author knew where it lived.

Anything involving a date. When the pay date is, when the review cycle opens, when the benefits window closes. These change annually and the document frequently carries last year's dates, so the answer is confidently wrong rather than missing.

The edge of an entitlement. Whether something applies in a specific circumstance. These are often genuinely unanswerable from a document and the right outcome is a clean handoff, not a new paragraph.

Questions about somebody else. What a manager can see, whether a conversation was recorded, who was told. Mostly correct refusals and worth checking that the refusal is graceful, since a blunt one on a question like this reads badly and the person asking is usually anxious rather than idle.

Benefits detail. Not whether a benefit exists but how to use it, which is usually in a provider's document nobody added to the corpus. Adding those documents is often the single largest coverage improvement available, and it is blocked less by effort than by nobody owning the relationship with the provider's materials.

Things that changed recently. A policy updated three weeks ago and an assistant still quoting the old version, which is its own subject and worth treating separately because the failure is a lag rather than a gap.

Anything a contractor asks. Questions about entitlements that do not apply to their arrangement, producing a confident answer from the wrong document.

And the one nobody predicts, which differs per company and is the most valuable entry in the list, because it is the thing specific to how your organisation works that nobody thought to write down. It is usually a local convention that everybody who has been there two years knows and no document states: which system is authoritative for something, who actually signs a particular thing off, or a step in a process that exists only because of how the company grew.

So check your clusters against those nine. Seven or eight will usually match, and the one that does not is the one worth the most attention.

Comparing the Queue Against the Inbox

So the queue tells you what people asked the assistant, which is not the same as what they needed, and the gap between those two is worth one afternoon.

Pull a month of the HR inbox alongside a month of the queue. Group both by topic using the same categories.

Look for topics in the inbox that never reach the queue. This is the valuable direction. A question people email but never ask the assistant means either they do not believe it can answer, or the question needs a person and they know it. The first is a communications problem and the second is correct behaviour, and you can usually tell which by the topic.

Look for topics in the queue that never reach the inbox. Questions people will ask a machine and not a colleague. In practice this cluster is small, consistent and revealing: it tends to contain the things people are slightly embarrassed to ask, which is a genuine argument for having the assistant at all.

Then check whether the inbox volume actually fell. Not whether it feels lower. Count a month before and a month after, same month of the year if you can, because seasonality in HR volume is substantial and a January comparison against August proves nothing.

And do this once rather than monthly. It is a calibration exercise, not a metric, and its purpose is to tell you whether the queue is a fair picture of demand or only of the demand people route to a machine.

What to Fix First

And when the clusters are ranked, the order to work in is not simply top to bottom, because the cheapest fixes are often not the largest clusters.

First, anything where the answer exists and is buried. A heading change and a plain-language first line. Minutes per item, and it converts a failure into a success without writing anything new.

Second, anything with a wrong date in it. These produce confidently incorrect answers, which are worse than no answer, and the fix is a find-and-replace rather than a drafting job.

Third, the largest genuine content gap. Now the writing starts, and it is one document for the whole cluster.

Fourth, the refusals that are ungraceful. A correct refusal delivered badly reads as the assistant being useless, and rewording the refusal to name a person costs one line of configuration.

And last, the long tail, which mostly means recording that you have decided not to act on it. That record is what stops the next reading re-investigating the same twenty entries, and it should name the reason rather than just the decision, because "needs a person" and "asked once" lead to different actions if the entry returns.

Working in that order means the first two sessions produce visible improvement without anybody writing a document, which matters more than it sounds when the exercise needs to survive a change of priorities.

What Getting This Wrong Costs

The visible cost is nothing, which is the problem. An unread queue produces no error, no complaint and no incident, so it sits there indefinitely.

The second cost is the handwork that never stops. In the opening example four topics were being answered manually for five months by people who had no way to see they were the same four. Because each individual answer takes three minutes and nobody counts them, the aggregate is invisible, and the assistant gets credit for reducing inbound while the residual inbound is entirely addressable and nobody knows it.

The third is the one that decides whether the deployment survives its first review. Somebody will eventually ask what the assistant is worth, and the honest answer requires knowing what it could not do and what you did about it. A team that can say the gap rate fell over two quarters, here are the nine topics we wrote, has a defensible case. A team with a usage chart has a number that proves people clicked.

The reframing question: what did your people ask last month that they could not get an answer to, and who read the list?

When You're Ready to Make It Routine

The honest position is that this is thirty minutes a month that nobody owns, and it stays unowned because the queue lives in a product administered by whoever installed it rather than by whoever writes the content.

The trigger worth acting on is the first month of real usage ending. Not a problem, not a complaint. Week five, with four weeks of non-test data available.

The sequence costs nothing. Ask the vendor which entry types the queue records and whether you can export a period rather than view a top ten. Export week two onwards, strip the deliberate refusals, and group by topic by hand before counting anything. Search your own corpus for each top cluster to tell a vocabulary gap from a content gap. Then write one document per cluster, park everything asked once, and record what you parked.

Put the next reading in a calendar before you close the file, owned by whoever owns the handbook, with the person from the HR inbox joining the first two. Of everything in this article, inviting that second person is the part that will surprise you, because they have been holding the answer in their head for months.

One broader point worth separating from the mechanics, because it changes how you justify the time. An employee assistant is usually bought to reduce inbound questions, and that is a reasonable thing to buy. But the queue it produces is a different and arguably more valuable product: a continuous, unprompted, honestly-worded record of what your own people do not know, collected without a survey, without a focus group and without anybody having to admit in a meeting that they cannot find the leave policy.

Companies spend real money trying to obtain that. Engagement surveys ask people to report what they need and get back what they think they should say. An intranet search log comes close and lacks the phrasing. The queue has both the question and the words, and it refreshes every week.

So the case for the thirty minutes is not maintenance. It is that you have accidentally bought a research instrument and are using a fraction of it. The nine topics in the opening example were not a defect list; they were a specification for what the people team should have written two years earlier, and it took forty minutes to obtain once somebody opened the file.

Frequently Asked Questions

HROpsLab takes no vendor money and publishes no paid placements, which is why the owner's own product is named above with its three limitations rather than its three best features.