Skip to content

Algorithms Used Within Beacon Huntress

Beacon Huntress can be configured to search for beacons using the algorithms below. Each Machine Learning (ML) algorithm has its own unique parameters and is used to search for slow or fast beacons based on time intervals.

Advanced Search for Beacons

Below are the current algorithms that can be used when searching for beacons.

Note

Algorithms can be set by using the cluster_type option in config.conf. The default algorithm is Quick Cluster Search with Slow Beacon parameters used for dbscan_by_variance().

Beacon Searches

Cluster Search is also known as DBSCAN, which stands for Density-Based Spatial Clustering of Applications with Noise. DBSCAN can identify clusters of different sizes within large data samples that contain noise and outliers. DBSCAN primarily uses two parameters: Minimum Points and EPS (Epsilon). The Minimum Points parameter represents the minimum number of data points (the threshold) that must be clustered together for a region to be considered dense. EPS is the maximum distance between two data points for them to be considered part of the same cluster.

  • DBSCAN clustering is a good choice for searching either Fast Beacons or Slow Beacons. However, this is the slowest running algorithm.


image source

Parameters

  • Minimum Delta Time (int)
    The minimum time interval between connection requests (in minutes) for your search.

  • Time Spans (list)
    Spans of time that you wish to search, in list format.
    Example: Will search within two time spans, 0-5 mins and 5-10 mins:
    [[0, 5], [5, 10]]

  • Minimum Cluster Points (int)
    The minimum number of cluster points/connections needed to identify a potential beacon.

  • Likelihood Percentage (int)
    The likelihood percentage used as a threshold by the Machine Learning algorithm in order to flag a potential beacon.

Below are two examples for finding fast and slow beacons using DBSCAN clustering.

Searching for beacons with 70% likelihood, time spans (0-5 mins, 2-15 mins, 15-35 mins, 30-60 mins), at least 10 connections, and minimum delta time of 1 minute.

Minimum Delta Time = 1
Time Spans = [[0, 5], [2, 15], [15, 35], [30, 60]]
Minimum Cluster Points = 10
Likelihood Percentage = 70

Searching for beacons with 70% likelihood, time spans (0-5 mins, 2-15 mins, 15-35 mins, 30-60 mins), at least 10 connections, and minimum delta time of 20 minutes.

Minimum Delta Time = 20
Time Spans = [[0, 5], [2, 15], [15, 35], [30, 60]]
Minimum Cluster Points = 10
Likelihood Percentage = 70

Hierarchical Search uses agglomerative clustering, which is a hierarchical clustering technique used to group objects based on similarity. Each item is treated as a singleton cluster, and clusters that are sufficiently similar are merged together into a larger cluster, working from the bottom up. This process continues until all of the clusters are placed into a single large cluster (see image below).

  • Agglomerative clustering works well when searching for Fast Beacons.


image source

Parameters

  • Maximum Variance Percentage (int)
    Variance threshold for any potential beacons.

  • Beacon Callback Count (int)
    Minimum number of delta records to search.

  • Clustering Factor Percentage (int)
    The likelihood percentage for a cluster.

  • Process Lines (list)
    Line amounts to process at a time, in list format.

  • Minimum Callback Time (ms) (int)
    Minimum delta time to search by, in milliseconds.

Below are two examples for finding fast and slow beacons using agglomerative clustering.

Note

Hierarchical Search is NOT a recommended algorithm for searching for Slow Beacons.

Fast Beacon Search

Searching for beacons with 70% likelihood, 12% max variance, at least 10 connections, and delta time of 60 seconds.

Maximum Variance Percentage = 12
Beacon Callback Count =  10
Clustering Factor Percentage = 70
Process Lines = 1
Minimum Callback Time (ms) = 60000

Slow Beacon Search

Searching for beacons with 70% likelihood, 12% max variance, at least 10 connections, and delta time of 15 minutes.

Maximum Variance Percentage = 12
Beacon Callback Count =  10
Clustering Factor Percentage = 70
Process Lines = 1
Minimum Callback Time (ms) = 900000

Quick Cluster Search uses the same principles as Detailed Cluster Search but some records will be filtered out before the scan if they surpass the user-set variance percentage. If the variance is above the configured threshold, it is excluded from the scan. This feature provides all of the benefits of a DBSCAN without the performance overhead.

  • DBSCAN by Variance clustering is a good choice for searching either Fast Beacons or Slow Beacons.
  • This provides a performance increase over Detailed Cluster Search because some connections will be pre-filtered by the variance setting.

Parameters

  • Average Delta Time (int)
    Average delta time to include in the search using your delta column. Less than or equal (<=).

  • Connection Count (int)
    Total connection count for filtering. Greater than or equal (>=).

  • Time Span Average (int)
    The percentage to increase and decrease from the connections total delta span.
    Example: 15 will decrease 15% from the minimum and maximum delta span.
    delta min = 5
    delta max = 10
    span min = 4.25 (5 - (5 _ 15%))
    span max = 11.5 (10 + (10 _ 15%))

  • Variance Percentage (int)
    Total variance percentage for filtering. Greater than or equal (>=).
    Default = 4

  • Minimum Likelihood Percentage (int)
    Likelihood value (threshold) used to identify a potential beacon.

Below are two examples for finding fast and slow beacons using DBSCAN by variance clustering (Quick Cluster).

Fast Beacon Search

Searching for beacons with 70% likelihood, at least 10 connections, 15% time span average, minimum variance of 15% and minimum delta time of 5 minutes.

Average Delta Time = 5
Connection Count = 10
Time Span Average = 15
Variance Percentage = 15
Minimum Likelihood Percentage = 70

Slow Beacon Search

Searching for beacons with 70% likelihood, at least 10 connections, 15% time span average, minimum variance of 15% and minimum delta time of 20 minutes.
Note that this will also search for fast beacons, as avg_delta <= value encompasses both fast and slow.

Average Delta Time = 20
Connection Count = 10
Time Span Average = 15
Variance Percentage = 15
Minimum Likelihood Percentage = 70