Can an AI Assistant Use Your Company's Procedures?
Bart Porcincula · Published
A company assistant needs direct instructions, enough relevant source material, and reliable records of important details. Separate experiments on one small model show why supplying documents alone does not establish reliability.
Your business may already have the information an assistant needs: a pricing sheet, a returns policy, supplier notes, and instructions written by the person who knows how everything works.
Giving those documents to an AI is a plausible starting point. The harder question is whether it uses the right information, follows the rules, and retains the details needed for the next step.
I tested parts of that problem on a small local model using synthetic notes and work logs. The results suggest three things worth checking before building a company assistant.
Write the instruction directly
Consider the difference between “our importer rejects commas in amounts” and “write every amount without commas.” A person may understand both as the same instruction. A model may not.
In my test, the model followed explicitly stated rules on every probe. When the same rules appeared only as explanations, it followed them about a quarter of the time. A preliminary step asking it to list the rules did not reliably close the gap.
This was a controlled test of six rules, not a guarantee of perfect compliance with any written policy. It does suggest a useful editing task: turn implied expectations into direct instructions, then test the assistant on cases where following each rule matters.
Check questions that need two documents
There are different ways to supply company knowledge. One is to give the model the whole small collection. Another is to search for relevant passages and supply only those.
I compared those approaches on folders of 10 to 120 synthetic project notes. Supplying the entire folder was more accurate at every size tested. Searching first was faster and worked well for questions answered by one note, but the tested search setups struggled on questions that needed two notes together; at 10 notes, two setups answered some correctly.
That does not mean searching documents is a bad approach. It means that finding a relevant passage and finding all the passages needed for an answer are different requirements.
For your trial, include questions such as “does this customer's exception change the standard policy?” alongside straightforward lookups. Make sure the assistant sees both the general rule and the exception.
Keep important details in a reliable record
I also tested what happens when a model summarises its own growing work history. With the original history available, it recovered 88% of the planted details. After a free-form summary, it recovered 3%. A structured summary improved that to 20%, and a copy-exactly template to 33%.
These figures come from generated work logs and one model. They do not describe every assistant's memory. They do show why a readable summary should be checked for the exact details the next task needs.
For a business trial, keep approved prices, dates, identifiers, and policy versions in explicit records. Verify that the assistant actually reads those records when needed; simply providing a file-reading tool was insufficient in my test.
Start with one procedure and a set of questions a staff member can judge. Expand the document collection after the assistant handles that small job well, including exceptions and missing information.
What a useful first assistant might do
Imagine a small wholesaler whose staff need to answer questions about delivery arrangements. The standard policy is in one document, a particular customer's agreed exception is in another, and a later note changes the cut-off date. This is a hypothetical example of a question that requires several pieces of information.
A useful assistant would prepare an internal answer, identify the documents supporting it, and show which policy version it used. If it finds conflicting instructions, it would leave the question for the responsible employee. A fluent answer based only on the standard policy could be wrong for that customer.
Start with an internal assistant for one procedure, where someone familiar with the work can judge the proposed answers. That gives you a concrete collection of questions and a person who can resolve disagreements about what the business actually does.
Sort out the documents before adding the assistant
An implementer needs to know which documents are authoritative. If an old pricing sheet and a revised one sit in the same folder without clear dates, the system has an information-management problem before it has an AI problem.
For the first procedure, choose an owner, identify the current source, and mark documents that have been superseded. Put important exceptions in a form staff can recognise and maintain. Decide who can approve an update and what should happen to answers prepared from an earlier version.
This work also exposes gaps. If two experienced staff members give different answers to the same question, asking the model to choose will not establish company policy. Resolve the rule with the people responsible, then turn it into a direct instruction and an example the assistant can be tested against.
Ask what the assistant actually receives
You may hear the term “retrieval”: software searches the document collection and passes selected passages to the model. That can reduce how much text the model needs to read, but a relevant passage may still be an incomplete basis for the answer.
Ask the implementer to show the passages supplied for a difficult question. In the wholesaler example, did the assistant see the customer's exception and the later change as well as the standard rule? This makes a failure easier to diagnose than looking only at the final sentence.
My whole-folder versus retrieval study used a small synthetic collection. It does not establish the best approach for a large company archive. Its useful lesson for a trial is to test questions that need information from several places, rather than approving the system after a few successful single-document lookups.
Give the implementer a question pack
Build a small question pack with the employee who owns the procedure. Include routine questions, exceptions, questions needing two documents, outdated instructions, and questions with no answer in the approved collection. Write the expected answer and supporting sources before the final demonstration.
A first implementation brief could require:
- Answers limited to one approved procedure and its named source collection.
- Visible source references and document versions for proposed answers.
- A clear unresolved response when information is missing or conflicting.
- Tests of ordinary cases, exceptions, and direct instructions about output format.
- A demonstrated update process, including a check that an old answer does not persist after the policy changes.
- An operating note identifying who maintains the sources and who handles unresolved questions.
These are proposed acceptance conditions, not capabilities established by supplying the model with a folder. Have the technical person demonstrate each one with your question pack.
Keep conversation history separate from business records
A chat can help an employee work through a question, but it is a poor place to leave the only copy of an approved price or customer exception. A later conversation may not contain that detail, and a summary may omit the exact identifier needed to find it again.
In the summary experiment, even structured summaries lost many planted details. For a proposed company workflow, ask for important facts to live in explicit records with an owner and version, and require the assistant to consult them when needed.
The first useful result may be a dependable way to find and explain one procedure, with a person handling unresolved cases. Expand from there once staff can show that it helps. The trial guide can help you turn that limited scope into something you can accept or reject on evidence.
Frequently asked questions
Can an AI assistant answer questions using our company procedures?
It can use relevant procedures supplied to it, but providing documents alone does not establish reliability. Test answers against your own questions and check that the assistant identifies the relevant source.
What if the procedure does not answer an employee’s question?
The assistant should make the gap visible and direct the person to an appropriate human or source, rather than presenting an unsupported answer as policy.
How should we test an assistant before staff rely on it?
Use representative questions, including ambiguous cases and missing information. Have someone who knows the procedures check the answers, sources, and handling of uncertainty before wider use.
Research behind this article
These are separate controlled experiments, not a deployed company knowledge system.
Cite this guide
Bart Porcincula. "Can an AI Assistant Use Your Company's Procedures?." Published 2026-11-08. https://porsync.com/ai-for-business/company-procedures