What an HR Chatbot Exposes, and How to Narrow It

TL;DR

• The risk in a document-trained assistant is almost never the model. It is the corpus somebody pointed it at on the first afternoon.
• Strict mode, where the assistant answers only from your documents, is a hallucination control. It is not an access control, and the two get confused constantly.
• Four exposure routes matter: an over-broad corpus, inference across two harmless documents, attempts to make it reveal its own instructions, and the question logs.
• The test that settles it in ten minutes is asking it something it should not know, from an ordinary employee account.
• Ask every vendor where the text is processed, whether it trains on your content, which model provider sits behind it, and what the retention is. Get it in writing.
• What any of this obliges you to do differs by jurisdiction and by circumstance, so take local advice on the obligations and treat this as the operational half.

Forty Documents and One Folder Nobody Checked

A company switched on an HR chatbot and pointed it at the shared drive folder where the people team kept its documents. Forty-one files: the handbook, the policies, the benefits summaries, the onboarding pack.

It worked immediately and well. Three weeks later somebody asked it, idly, what the salary bands were for their job family. It told them, accurately, because file 37 was a spreadsheet export from a compensation review that had been saved in that folder in March and forgotten.

Nothing had malfunctioned. The assistant had done exactly what it was built to do, which is answer questions from the documents it was given, and somebody had given it a document nobody had read in eight months. The configuration was correct, the model behaved properly, and the exposure was a filing decision made by a person who had no idea it would ever be queryable.

Recommended Tools

Best tools for AI HR Tools

B BambooHR HR management system L Lattice Performance management D Deel Global payroll & EOR

View all recommended tools

That is the shape of this problem. It is a corpus curation problem wearing a security costume, and almost every control that matters sits on your side rather than the vendor's.

The Four Exposure Routes

Worth separating, because they need different controls and only one of them is about the vendor.

An over-broad corpus. The folder contained something it should not have. This is the overwhelming majority of real incidents in this category and it is entirely within your control.

Inference across documents. Two files that are each unremarkable combine into something that is not. A team list and a salary band document are both fine; together they answer a question neither was meant to. No access rule catches this, because no individual document is sensitive.

Attempts to make it reveal its own configuration. Somebody asks it to repeat its instructions, or to list everything it knows, or phrases a request as a system message. A well-built assistant refuses. Test yours rather than assuming.

The question logs. What people asked is itself employee data, and it is the route nobody thinks about until somebody asks for a report. That has enough in it to be its own subject and is treated separately.

Strict Mode Is Not Access Control

This is the distinction worth taking away, because the two features are routinely sold and bought as if they were one.

Strict mode means the assistant answers only from the documents you supplied, and says it does not know when the answer is not there. It exists to stop the model inventing something plausible. It is a very good feature and most companies should have it on.

It has nothing whatever to do with who may see what. If the compensation spreadsheet is in the corpus, strict mode will quote it accurately and confidently to anybody who asks, because quoting your documents is precisely what it was told to do. Turning strict mode on makes the wrong answer less likely and does nothing about the wrong audience.

So you need both, and they come from different places. Strict mode comes from the product. Audience separation comes from how you structure the corpus and which assistant a given person is talking to, which is a configuration decision rather than a toggle.

The practical consequence: a vendor demonstrating strict mode has not answered your access question, and you should ask it separately.

When This Is Not Worth a Project

When the corpus is genuinely public

If everything the assistant can read is already on your intranet with no restrictions, available to every employee, then the exposure is close to zero and a curation review is enough. Many first deployments are exactly this and are fine.

When it is a pilot with ten people

A two-week pilot with a named group and a handful of documents does not need a governance process. It needs somebody to look at the document list once. Build the process when you widen the audience, not before.

When you have no sensitive documents in scope

Worth checking rather than asserting. The test is whether anybody could name a file in the folder that should not be read by everyone, and the honest answer at most companies is that nobody has looked.

The edge case that forces the full treatment

A second audience. The moment managers and employees both use it, or contractors are added, you have an access problem rather than a curation problem, and the controls are different.

Five Questions People Ask First

"Can it leak our data to the model provider?" That depends entirely on the arrangement, which is why it is a question for the vendor in writing rather than an assumption. Ask which model provider sits behind the product, whether your content is used for training, whether it is retained by that provider and for how long. A vendor that answers precisely is telling you something useful; a vague answer here is the one to push on hardest.

"Is strict mode enough?" No, and the section above is why. It is a correctness control. Treat it as mandatory and insufficient.

"Should we classify our documents first?" Usually not, and this is where companies lose six months. A full classification exercise is a large project with a poor completion rate. The cheaper move is to curate the specific list the assistant reads, which is typically between 20 and 60 documents, and review that list rather than your entire estate.

"What about people asking it things to see what happens?" Some will, in the first week, and it is better treated as free testing than as misuse. Expect a handful of people to try to make it say something silly. Watch what they try, because they will find the gaps faster than any review.

"Who should sign this off?" Whoever signs off access to the underlying documents today. If the handbook needs no approval and the compensation folder does, the assistant inherits that distinction, and the useful framing in a security conversation is that you are not creating new access so much as creating a new route to existing access.

Curating the Corpus Instead of the Estate

The single highest-value piece of work here, and it is an afternoon rather than a programme.

List the documents individually, not the folder. A folder pointer means the corpus changes whenever anybody saves anything, which is how file 37 arrived. An explicit list changes only when somebody decides it should. This one change prevents the most common incident outright.

Put a name against each document. Who owns it, so there is somebody to ask when it is wrong or out of date.

Mark each one with its intended audience. Everybody, managers, or a specific group. You are not classifying for secrecy; you are recording who the document was written for, which is a question its author can answer in seconds.

Review the list quarterly, and on any change of scope. Fifteen minutes. The review that matters is not reading the documents, it is checking that nothing new has appeared and that nothing on the list has become something else.

And search the corpus for the obvious terms before you launch. Salary, grievance, disciplinary, redundancy, termination, bank, passport, national insurance, date of birth. Not as a classification exercise, as a smoke test. Because the thing that goes wrong is a document nobody remembered rather than a document nobody understood, a keyword sweep across 40 files finds it in two minutes.

The Ten-Minute Test

Nothing in this article is worth as much as doing this once, from an ordinary account, before anybody else uses it.

Ask it something it should not know. Pick the most sensitive thing plausibly in scope. Salary bands, somebody's performance notes, a disciplinary process about a named person. Phrase it naturally rather than cleverly, because that is how a real question arrives.

Ask it to list what it has read. A reasonable product will summarise its sources. An unreasonable one will enumerate file names, which tells you the corpus is visible even where the content is not.

Ask it to repeat its own instructions. Then ask it to ignore them. Then ask it the sensitive question again, framed as though you were an administrator. None of this is sophisticated and all of it is what somebody will try in week one.

Ask it something that requires combining two documents. The inference route. Something like which team has the highest band, which no single file answers.

Ask it a question with no answer in the corpus. You are checking that it says it does not know rather than producing something confident and wrong, which is the strict mode behaviour you were promised.

Write down what happened in each of the five. That page is your security assessment, it took ten minutes, and it is considerably more useful than a vendor questionnaire.

What to Ask the Vendor, in Writing

Six questions. Written answers, because the willingness to put them in writing is part of the information.

The first row is the one most often skipped and is frequently the most consequential, because a product that calls a third-party model is sending your handbook text to a company you have not assessed. That is not automatically wrong, and it is something to know rather than discover.

What obligations any of this creates for you differs by jurisdiction and by the commitments you have already given your own customers, so settle that with local advice rather than from a vendor's summary or from this article.

The Options

Matram

Disclosure: Matram is owned by the same people who publish HROpsLab. It is a document-trained assistant that answers employee questions, with strict mode so it answers only from the sources you give it, a human handoff, and 95 or more languages. Pricing is flat per month rather than per seat: Basic at $29, Standard at $69 and Pro at $199, with unlimited seats on each.

Its limits, stated plainly because the masthead depends on it: there is no free tier, it answers questions rather than actioning tickets, and it is not an HR or ITSM workflow platform. For the subject of this article that last point cuts both ways, since a product that cannot action anything has a smaller blast radius than one that can.

SiteGPT

Best for teams that want a document-trained assistant on a published site and are comfortable with annual billing.

Read on 5 October 2026: Starter is $468 per year and Growth is $948 per year, both billed yearly, with monthly options at $59 and $129. Anybody quoting a lower monthly figure for this product is quoting the annual-billing equivalent without saying so.

Where it struggles for this subject: the questions above still have to be asked, and the corpus discipline is yours regardless of product.

Crisp

Best where you want a support suite rather than an answering assistant, and where the team already uses it for customer conversations.

Read on 5 October 2026: Pro at $45, Unlimited at $95 and Enterprise at $295 per month, priced per workspace rather than per seat, which is unusual and favourable for a larger team. It publishes no language count.

Where it struggles: it is a conversation platform first, so the document-answering behaviour and its controls are less central than in a purpose-built assistant.

Freshservice

Best where the HR assistant should sit inside the same platform as IT service management.

Read on 5 October 2026: $19, $49 and $99 per agent per month, with the Freddy AI add-on at $29 per agent per month. It publishes no language count.

Where it struggles for this subject: per-agent pricing means the cost scales with the people answering rather than the people asking, which is the wrong direction for a deflection tool.

Zendesk

Best for larger teams already standardised on it.

Read on 5 October 2026: $55 per agent per month for Suite Team plus $50 per agent per month for the Copilot add-on. It geo-redirects, so confirm your own currency on the page.

Where it struggles: the same per-agent direction, and the agent-focused model means the employee-facing assistant is one capability among many rather than the product.

Leena AI and Moveworks

Both aimed squarely at enterprise employee support and both publish no price at all, so a comparison requires a sales process before you have a number.

Where they struggle for a mid-market buyer: the evaluation itself is a project, and the governance questions above are the same whichever you pick.

The Comparison

The fourth column is the one that interacts with this subject in a way people miss. A per-seat product prices the answerers; an employee assistant exists to reduce the number of questions reaching answerers. Those two incentives point in opposite directions over a three-year term.

The Decision Table

Widening From Pilot to Everybody

The pilot is almost always safe and the widening is where risk arrives, so it is worth naming what changes rather than treating scale as a volume question.

Four things change at once, and only the first is obvious.

The audience stops being self-selected. A pilot group is usually people who volunteered, which means they are curious in a constructive way. A whole company includes people who will test the edges, people who will screenshot a bad answer, and people who will ask it something personal because it feels private. All three are ordinary behaviour and none of them appeared in your pilot.

The question distribution changes shape. Pilot questions cluster on the topics the pilot group cares about. Opening it up produces a long tail nobody anticipated, and the tail is where the corpus gaps and the awkward answers live.

The corpus gets pressure to grow. Within about two weeks somebody will ask for their department's documents to be added, reasonably, and that request is the moment the curated list becomes a folder again if nobody is guarding it. Decide in advance who approves an addition.

And the stakes of a wrong answer rise. A bad answer to a pilot participant is feedback. The same answer to 400 people is an incident, and the difference is entirely about audience size rather than about the answer.

So the widening checklist is short. Re-run the ten-minute test, because the corpus has probably changed since you last did it. Confirm the audience separation if you now have more than one population. Name the person who approves corpus additions. Check the audit trail retention is long enough to cover the fortnight after launch. And tell people what it can and cannot do before they find out by asking, which is a communications job rather than a security one and is covered separately.

The Inference Problem, Which No Rule Catches

Worth its own treatment, because it is the one exposure route that survives a perfectly curated corpus and a correctly configured product.

Two documents, each entirely appropriate for everybody. A team directory listing who reports to whom. A pay structure document giving the band for each job title. Neither is sensitive, both are routinely published, and most companies would approve both without hesitation.

Together they answer a question neither was written to answer, which is roughly what a named individual earns. Nobody uploaded salary data. No access rule was broken. The assistant simply did the thing it is good at, which is reading across sources to construct an answer.

So the control cannot be document-level, because the problem is not in any document. Three things help, none of them a setting.

Think in pairs when you approve the list. For each document, ask what it would answer when combined with the two or three others most likely to be in scope. That is a thirty-second question per file and it is the only way to see this before an employee does.

Keep the directory and the structural documents apart. In practice the highest-risk combination in HR is almost always an organisational list plus a quantitative framework. Separating those two into different audiences removes most of the risk at the cost of very little utility.

Test the combination explicitly. One of the five questions in the ten-minute test exists for this, and it is the one most often left out because it requires knowing your own corpus well enough to construct the question.

And accept that you will not catch all of it. A sufficiently determined person with twenty questions can assemble things from material you were right to publish, which is true of your intranet as well and has been for years. The assistant changes the effort required rather than the possibility, and the honest aim is to remove the accidental discoveries rather than to defeat a deliberate effort.

When It Answers Something It Should Not

So decide the response now, while nothing has happened, because the decisions made in the twenty minutes after somebody reports this are the ones that determine whether the assistant survives.

Five steps, in this order, and the order matters more than the speed.

Remove the document from the corpus, not the assistant from service. This is the step people invert. Suspending the whole assistant is a bigger action that feels safer and is usually wrong: it removes a working capability to address a single file. Pull the file, re-index, confirm the question now returns nothing.

Confirm it is actually gone by asking again. Re-indexing is not instant on every product, and a document removed from a source list can still sit in an index for a period. Ask the original question, from the same kind of account, and keep asking until the answer changes. Until it does, the exposure is open regardless of what the configuration says.

Establish who asked and how many times. This is what the audit trail is for, and it is the difference between one curious person and a circulating screenshot. The answer changes what you tell people and whether you tell them at all.

Work out how the document arrived. Almost always a folder pointer, a bulk upload, or a source added by somebody who did not know the scope. The fix for the cause is different from the fix for the instance, and only one of them prevents a recurrence.

Then decide on disclosure, with advice. Whether you have to tell anybody, and whom, depends on what the document contained and on obligations that differ by jurisdiction and by circumstance. That is a question for local advice on the day, and the useful preparation is knowing who to call rather than deciding the answer in advance.

Write those five lines down and put them wherever your other incident notes live. It is a quarter of a page and the reason to have it is that the alternative is improvising under pressure, which reliably produces the suspension that nobody reverses.

The Corpus Review, as a Standing Fifteen Minutes

Because nothing breaks when the corpus review is skipped, it will be skipped unless it has a date and a name against it. This is the same shape as every other invisible maintenance task, and it needs the same treatment.

Put it in a calendar, quarterly, owned by one person. Not a team. Fifteen minutes.

Diff the list rather than reading the documents. The review is not a content review. You are checking three things: has anything appeared that nobody decided to add, has anything on the list changed its nature since it was approved, and does every entry still have a living owner.

Re-run the keyword sweep. Salary, grievance, disciplinary, redundancy, termination, bank, passport. Two minutes across a list of 40 files, and it catches the one thing a diff will not, which is a document that was always on the list and has since had a sensitive section added to it.

Check the owners are still employed. An orphaned document is one nobody will correct when it is wrong, and the assistant will keep quoting it with total confidence for years.

Record the date you did it and what changed. One line. After four quarters that log is the only evidence that the control exists, which is what somebody will ask for.

And treat any change of scope as an unscheduled review. A new audience, a new department's documents, a new country, a migration of the shared drive. Each of those is a moment when the corpus changes shape rather than size, and the quarterly rhythm is not frequent enough to catch it.

What the Audit Trail Has to Carry

And the audit trail is the control you will care about most on the one day it matters, which is why it is worth checking what yours actually records before you need it.

Four things, and products vary considerably on the last two.

The question, verbatim. Without it you cannot reconstruct what was asked, and paraphrases are useless when the precise wording is what triggered the answer.

The answer given, and the sources it cited. The sources are the part that identifies the offending document immediately rather than after an hour of searching the corpus by hand.

Who asked, or a stable identifier for them. This is in tension with the privacy question and both considerations are real. The workable position for most companies is that the identity exists in the log, access to it is restricted to one or two named people, and it is used for incidents rather than for reporting.

When, to the minute, and retained long enough to be useful. A trail that rolls off after seven days is not a trail. Ask for the retention figure and compare it against how long it realistically takes somebody to report something awkward, which is usually weeks rather than days.

Then test that you can actually retrieve it. Ask the vendor to show you the log for a question you asked five minutes earlier, in the product, during the evaluation. A product that logs everything into a place only its own support team can query has given you a feature you cannot use, and discovering that during an incident is the worst available moment.

What Getting This Wrong Costs

The visible cost is the incident, and in most companies it is handled quietly: somebody notices, the document comes out of the corpus, and the assistant stops knowing.

The second cost is the shutdown. An exposure in week three of a deployment does not usually produce a better corpus; it produces a suspended assistant and a committee. Because the people who paused it are not the people who would curate the document list, the pause tends to outlast the problem, and the company ends up with a capability it has paid for and switched off. That is the most common bad outcome in this area, and it is prevented by ten minutes of testing before launch rather than by any amount of review afterwards.

The third is the trust cost with employees, which runs the other way from what people expect. The risk is not that staff fear the assistant knows too much. It is that one person discovers it answers a question it should not, tells colleagues, and the story that circulates is that HR has put everything into a bot. Nobody files a complaint about that and everybody hears it.

The reframing question: if an ordinary employee asked your assistant the most sensitive question plausibly in scope, right now, what would it say, and has anybody checked?

When You're Ready to Widen the Audience

The honest position is that the first deployment is usually safe by accident, because the corpus is small and somebody chose it carefully. Risk arrives with scale, and specifically with the second audience.

The trigger worth acting on is a request to add a group. Managers, contractors, a second country, a department with its own documents. Each one converts a curation question into an access question, and the controls for those are different.

The sequence costs an afternoon and no budget. Replace any folder pointer with an explicit document list, each with an owner and an intended audience. Run the keyword sweep across that list. Do the ten-minute test from an ordinary account and write down the five answers. Send the vendor the six written questions. Then settle the obligations side with local advice, because that is the part that cannot be worked out from first principles.

Do that and widening the audience becomes a decision about which documents a new group may read, which is an ordinary question with an ordinary answer. Skip it and the first thing that goes wrong will be a file somebody saved in March.

One last framing, because it is the thing that makes this conversation go well with a security team rather than badly. An employee assistant does not create access. Every document in that corpus was already readable by somebody, and in most cases by everybody, sitting on a drive that has been searchable for years. What the assistant changes is the effort required to find it: a question in plain language replaces knowing which folder to open and what the file was called.

That is a real change and it is a narrower one than it sounds. It means the right comparison is not against a locked filing cabinet but against your existing intranet search, and the honest version of the risk is that an assistant surfaces what search had effectively buried. Buried is not a control, it was never a control, and a company that discovers an exposed document through its own assistant in week one has had a cheap and useful warning about a file that was available to anybody patient enough to look.

So the work above is worth doing properly and it is not a reason to delay. The companies that get this wrong are not the ones that moved quickly; they are the ones that pointed it at a folder, never tested it, and had no plan for the morning somebody reported something.

Frequently Asked Questions

HROpsLab takes no vendor money and publishes no paid placements, which is why the owner's own product is named above with its three limitations rather than its three best features.