Skip to content

How-to guides

Chatbot Conversation Analytics That a Small Business Can Use

Seven chatbot metrics a small business can act on weekly: unanswered questions, lead quality, handover completion and real outcomes, not message volume.

The Pivra Team5 min read

The chatbot analytics a small business can use are the ones that lead to a decision each week. Track seven things: conversations with a clear intent, answered versus unresolved questions, qualified leads, lead completion, human-handover completion, the top missing knowledge topics and downstream outcomes. Treat message volume as capacity, not value.

A scenario#

An electrician's office manager opens the chatbot dashboard on Monday morning. There are 140 conversations from the past week. Reading them all would take most of the morning, and nothing would change as a result.

Instead she spends 30 minutes on three questions: what did people ask that the bot could not answer, which enquiries were missing a suburb, and did every request for a person reach someone? She finds that six visitors asked about EV charger installation, which the website never mentions. That is a content fix, a training fix and possibly a sales opportunity, found in half an hour.

The seven metrics#

  1. Conversations with a clear customer intent. Exclude widget opens with no message and test chats. This is your real demand.
  2. Answered versus unresolved questions. Count the times the bot said it did not have the information, or a visitor gave a thumbs-down.
  3. Qualified leads, using your written definition: valid contact details, in your area, a service you offer, a stated need.
  4. Lead-completion rate: of the conversations where someone started giving details, how many finished?
  5. Human-handover rate and completion: how often visitors asked for a person, and how many were actually picked up.
  6. Top missing knowledge topics: the three or four subjects behind most unresolved questions.
  7. Downstream outcomes, where available: booked, won or lost.

Unresolved questions are a content queue#

Treat every unresolved question as a work item:

  • A repeated question about a service you offer means the website page and the chatbot source both need a clear answer.
  • A repeated question about a service you do not offer means the exclusion should be clearer. Do not train the bot to sell around it.
  • A question that needs judgement (a diagnosis, a complaint, a custom price) should go to a person. Make sure the handover path is obvious.

Fix the visible website page first, then retrain the chatbot source, so customers and the bot see the same information.

Segment before you conclude#

Averages hide useful patterns. Where your tools allow, look at:

  • Time: after-hours visitors often ask different questions, such as urgency and callbacks, than daytime visitors.
  • Landing page: visitors from a paid campaign page may need a faster route to giving their details.
  • Device: mobile visitors abandon long sequences sooner, so keep questions short.

Treat sentiment with caution#

Automated sentiment scores can misread short, local or sarcastic language. "Yeah nah, that's fine" is not negative. Use sentiment, and thumbs-up or thumbs-down ratings, as a flag for review, not as a verdict on the customer's experience.

Protect privacy during review#

Conversations can contain information the business never asked for, such as health details, addresses or family circumstances. When reviewing:

  • limit who can read transcripts;
  • avoid exporting full conversations for routine reports;
  • use counts and themes for recurring summaries;
  • delete exported files you no longer need.

For collection design, see privacy-first AI chatbot lead capture.

What Pivra shows you#

On Starter and above, the Pivra analytics dashboard includes (analytics):

  • conversations this month and a daily trend over 7, 30 or 90 days;
  • total leads captured;
  • lead conversion rate: leads captured ÷ conversations over the last 30 days;
  • positive and negative visitor ratings, useful for spotting knowledge gaps;
  • knowledge sources by type and status, including any that failed to train.

Note that the analytics count includes sessions where a visitor opened the widget without typing; the Conversations tab shows only sessions with at least one exchange.

Beyond the dashboard:

  • Conversations lets you read transcripts and has a Handoff tab listing requests for a person, with how long each has waited and whether it has been claimed (human handover).
  • Save as refinement: when the chatbot gives a reply you want it to give every time, save it from Conversations as a draft refinement, check it and turn it on (refinements).
  • Exports: there is no export from the Analytics page itself. You can export conversations as CSV from the Conversations tab and leads as CSV from the Leads page (lead capture).
  • Keeping knowledge current: retrain sources after you fix a page, or set website sources to retrain automatically (keep your chatbot up to date; improving responses).

Pivra does not track what happens after the enquiry. For booked and won outcomes, match exported leads to your job system; the approach in how to attribute chatbot leads without overclaiming helps avoid double-counting.

The 30-minute weekly ritual#

Every week, produce three concrete changes:

  1. Fix one missing answer: update the website page and retrain the source.
  2. Remove one unnecessary question: if a field is rarely used by the team, stop asking for it.
  3. Follow up one broken handover: find a request for a person that was not picked up and find out why.

Write the three changes in a simple log with the date. After a month, the log is a better report than any dashboard screenshot.

Checklist#

  • Seven metrics defined, with "qualified" written down.
  • Unresolved questions reviewed weekly.
  • Ratings treated as review flags.
  • Transcript access limited.
  • Three changes logged each week.

Next step: make every analytics review produce three concrete changes, and write them down.

Topics

  • Chatbot conversation analytics metrics
  • Chatbot analytics
  • Conversation review
  • Lead quality
  • Small business AI
TP

Author

The Pivra Team

Pivra

The Pivra team writes from product, onboarding, and customer support work with Australian tradies, local services, and small business operators.

AI chat setupLead captureAustralian small business workflows
View all →