Analyst Metrics

Managing Analysts – A Critical Challenge for the Mature ArcSight SOCs

For maturing ArcSight Security & Operations Centers (SOCs), the primary focus gradually shifts from log source onboarding and event correlation to improving the efficiency and effectiveness of the analysts. Are your analysts responding to events within specified time thresholds? Are any events slipping through the cracks? Are events handled correctly and following the appropriate Standard Operating Procedures (SOPs)? These are just a few challenges a mature ArcSight SOC faces in both in its day-to-day operations and when justifying its existence to management. The most advanced threat intelligence and event correlation is only valuable if the frontline analysts are reacting appropriately and consistently to these events.

Measuring ArcSight Analyst Metrics – The Basis of Efficient and Effective Analysts

Most management challenges can be solved by studying the metrics. SEMplicity’s ArcSight Analyst Metrics package is a large body of content, including dashboards, reports and FlexConnectors that are designed to give you unparalleled visibility into your analyst activities. We track a variety of statistics over time, both across all analysts and broken-down by analyst, shift, event, stage and SOPs. We can measure:

  • Response Time: Which analysts have outlying average response times? Which events are taking in inordinate amount of time to handle? Which events suggest automation or removal from the main channel?
  • Resolution: Are your analysts moving events from stage to stage in the same way? (i.e., Queued, False Positive, Escalated, Remediated, Review, Case Opened?) Are different analysts resolving similar events differently?
  • Standard Operating Procedures: Are similar events being handled with the same SOPs? Are some analysts overusing (or under-using) some SOPs? Are some analysts using the wrong SOPs?
  • Annotations: What is the average number of events each analyst includes in each distinct annotation? Are analysts grouping annotations appropriately? Are there opportunities for tuning event aggregation?
  • Marked as Similar: How many events are marked as similar by each analyst? Which analysts are overusing or under-using this feature?
  • Channel Statistics: How many events are going to the main channel? How many are slipping through the cracks? How many events are going to the review channel, and who is handling them there?

This package addresses these, along with many other, important ArcSight SOC management metrics. Designed based on the Micro Focus Security Information and Operations Consulting (SIOC) best practices, our ArcSight Analyst Metrics package has been implemented at some of the most advanced ArcSight SOCs in the world. This powerful package will give you full visibility into how your analysts respond to ArcSight events. With our ArcSight Analyst Metrics package you no longer need to export analyst data to a spreadsheet and manipulate it; all of your analyst metrics are available for viewing and analysis from with ArcSight!

Component Benefits
Dashboard: Metrics by Analyst A snapshot of your Analyst event annotation times and counts over the previous 30 days. Contains drill-downs to focus on a specific analyst and their response times and counts by event name, stage and SOP. Also contains comparative metrics between analysts, identifying outlying response times and counts.
Dashboard: Metrics by Stage A snapshot of your analyst event handling activities, grouped by resolution, stage over the previous 30 days. Displays times and counts, with drill downs to focus on activities for specific analysts, events and SOPs. Stacked bar charts provide a quick visual display of events handled inconsistently between analysts.
Dashboard: Metrics by Event Shows the 99 most common events handled, with response times and counts. Contains drill-downs to focus on specific event handling activities by analyst, stage and SOP. Stacked bar charts indicate events handled inconsistently between analysts.
Dashboard: Metrics by SOP Shows standard operating procedures implemented and broken down by event name to show inconsistent SOP application. Contains drill-downs to focus on analyst activities for a specific event and SOP.
Dashboard: Marked as Similar and Unique Annotations Displays events counts for annotations made by the Marked-as-Similar (MaS) feature. Also indicates average events per unique annotation by analysts. Drill-downs to investigate specific MaS annotation patterns and events grouped into a single annotation.
Dashboard: Channel Statistics Displays statistics over time on the events handled within your main and review channels. Includes data on the average events in the channel, alerts for spikes or dips in main channel volume, events expiring from the channel without being handled, and similar statistics for the review channel.
Report: Metrics by Analyst, Event, Stage, SOP A comprehensive summary report run and archived every 30 days detailing the metrics indicated in the above dashboards. Managers may use these for monthly analysis and yearly reviews of analyst performance.
Report: Marked as Similar and Unique Annotations A summary report run and archived every 30 days detailing the usage of the MaS feature by analyst, and also the average events included in each annotation, sorted by analyst.
Rules: Service Level Anomalies Using various rules to trigger reports based on client-specified anomalies in event response times, event annotation counts, and other analyst event handling activities, as well as event spikes or dips on the main and review channels.

Requirements:

  • A two-hour ArcSight SOC review, with the ArcSight SOC manager
  • Admin credentials to the ArcSight Manager
  • Remote access, if the implementation is performed remotely