New Feature Announcement: LLM BI Report
We are excited to announce the release of a new LLM (Large Language Model) report, which offers detailed visibility into your users' interactions with LLM chat agents, such as ChatGPT. You can find this new report under BI Reports > LLM.
This report is designed to help you monitor and understand how your team utilizes AI/LLM tools by allowing you to:
View key details, including the direction of the interaction (user's input/prompt vs. AI agent's response), chat title, the specific LLM model used, the full message content, and any attached files uploaded by the user.
Access the full context of any conversation by right-clicking on a row and selecting the View conversation option.
Download attachments using the Download attachments option.
Enabling LLM Monitoring:
To begin tracking, enable LLM monitoring within your monitoring profile(s) and specify the LLM applications you wish to track.
Current Limitations:
Only ChatGPT Web is currently supported for tracking.
The system tracks only uploaded files at this time; file downloads are not captured.
Some data points, such as audio input or prompts embedded within images, are not captured.