What is OCR and How it Works?
Teramind’s Optical Character Recognition or OCR feature allows you to discover onscreen Personally Identifiable Information (PII), Protected Health Information (PHI), Private Financial Information (PFI), and other sensitive information using keywords, patterns, and regular expressions.
Teramind’s proprietary OCR engine continuously captures, indexes, and analyses a user’s desktop using machine learning and sophisticated pattern recognition algorithms.
With OCR, you can quickly search for textual information displayed on the screen; even inside images or videos for forensic investigation purposes. You can also build powerful activity rules and automatically get notified when a piece of information is displayed on the user’s screen.
The OCR works with multi-monitor setups and virtual desktops, including the ones from the Terminal Servers/RDP sessions.
Some use cases of Teramind’s OCR features are:
- Alert when a user sees a full credit card number on the screen violating the PCI DSS compliance requirements.
- Find out how often a user saw certain sensitive records and recall desktop videos & screenshots of those occurrences for investigation purposes.
- Create rules for applications that are not easily parseable because the text is inside images or videos.
- Notify an admin when certain text appears on the screen.
- Prevent embargoed data exposure such as M&A announcement, stop insider trading, etc.
- Monitor the activity of third-party vendors, contractors, and other external users on a Terminal Server without needing to install an Agent.
Teramind automatically analyzes any content displayed on the user’s screen in the background and creates an index of text it detects. This allows you to conduct a high-speed search of any onscreen text using the OCR search feature.
You can access the OCR search screen from the MONITORING > OCR menu:
- You can use Full Text, Regular Expressions, Wildcards, or Contains (phrases) for your search terms.
- From the search results, you can click the Camera icon to view a screen snapshot. The snapshot will show the user’s desktop with the areas highlighted in semi-transparent Yellow color where the OCR text was detected:
- You can also use the Movie Camera icon to view a session recording of the exact date/time when the text appeared on the screen.
You can use OCR-based Activity rules to detect text on screen(s). You can optionally limit the detection to some specific applications, specify detection thresholds (e.g., how many times the text was detected in a day) and risk levels:
What is Session Mining? What are OCR Mining Node and OCR Database Node?
You will hear these terms when working with OCR in Teramind. Basically, session mining means capturing the screens from the user desktop and storing them in the server for later processed by the OCR engine.
The OCR engine uses two nodes/servers. The Mining Node runs the OCR engine which analyzes and identifies texts from the screen recordings. The Database Node (Elasticsearch) facilities the storing and searching of the text. You will need to set up both of these nodes for the OCR to work properly in your on-premise or private cloud deployments (see below). For Cloud deployments, the nodes will be set up and managed by us.
You can see the nodes displayed under the Nodes section on the Settings > Server Management screen:
OCR Server Requirements
If you are using the on-premise or private cloud deployment, you need to set up at least one OCR Database Node and one Mining Node for the OCR features to work. Additional nodes may be required for larger deployments of 200 or more users.
Generally speaking, the more powerful a server is, the better performance you will get. See the OCR Performance and Mining Delays section below for more information.
The deployment guides below have the detailed specifications for OCR node sizes, CPU and RAM requirements for each deployment scenario.
- On-Premise Deployment Guide > OCR Server Requirements
- AWS Deployment Guide > OCR Server Requirements
- Azure Deployment Guide > OCR Server Requirements
How to Set Up OCR for Your Deployment
If it is a Cloud trial, send an email to firstname.lastname@example.org.
You will need to set up the OCR nodes in your deployment. Follow the instructions here to set up OCR for your on-premise deployment.
You will need to set up the OCR nodes in your deployment. Follow the instructions here to set up OCR for your AWS deployment.
You will need to set up the OCR nodes in your deployment. Follow the instructions here to set up OCR for your Azure deployment.
Enabling OCR on the Monitoring Settings
Some of the Screen monitoring settings control how the OCR feature will work. For example, you can specify the OCR language. Other settings such as update screen on events only, no. of frames per second, etc. will affect how the OCR behaves.
For example, if you record only during rule violations, the OCR will not be able to capture texts for the entire duration of the user sessions. Or, for example, if you have configured the Screen settings to auto-delete recordings after certain days; the OCR search will be able to find previous texts it captured but will not be able to show the screen snapshot or the video recording of the time.
Asynchronized Screen Upload and OCR
If the user is using a Hidden Agent, ASYNC SCREEN UPLOAD will force Teramind to use a queue for screen recordings instead of uploading them in real-time. It is suitable for a slower network or a busy OCR server. However, you might experience some delay between the user activity and the recording appearing on the dashboard when ASYNC is enabled.
OCR Performance and Mining Delays
The OCR, due to how it works, is not real-time. So, sometimes the OCR search will not find information if the OCR engine has not processed the screen recording yet. Or, an OCR-based rule might trigger slightly later than when an action actually took place. This is why you will notice that the OCR only supports the Notify action.
Several factors might affect the performance of the OCR processing. The number of screens, no. of users, user’s activity level, the OCR node configurations, network bandwidth, etc. Usually adding more CPU/RAM and disk space can increase the performance of the OCR engine.
You can view how your OCR servers are performing from the Settings > System Health screen. Under the Session mining stats you will see the OCR processing status: