Skip to main content

Cut Trisul storage per day by tuning flows, edges and counter groups

FieldValue
App Note IDAN-009
AudienceTrisul admins, NOC and network engineers, ISP operators
Applies toThe Hub database (flows, counters, edges); flow and edge settings are made on the Probe

Summary​

By default Trisul stores every flow and every metric at full resolution, which uses a lot of disk. You first measure how much the Hub database grows per day on the DB Status page and see which parts of a slice use the most space. You then reduce the growth with three settings: VolumeCutoffBytes for the flow database, the Edges options in the Probe configuration file, and the storage settings of each counter group. Less storage per day lets you keep more days on the same disk.

The problem​

A default Trisul deployment keeps every metric at its native resolution and stores every flow, whether it moved 10 bytes or 10 gigabytes. This design supports incident response, post-breach audits and compliance, where a small flow can be the evidence you need. The cost is disk space.

The Hub keeps 96 days of data by default: 32 days each in the Operational, Reference and Archive parts of the database. If your disk fills before that window, or you need more days on the same hardware, you have to reduce how much Trisul writes per day. You need to know which parts of the database grow fastest before you change anything, and you need to understand what each setting gives up.

How it works​

The Hub writes its database in slices. A slice is the unit of storage in Trisul, and a day typically has two slices. To get the storage for one day, add the sizes of the slices that carry that date. The SlicePolicy in the Hub configuration file sets a daily slice window and 32 slices each for Operational, Reference and Archive. As slices age they move from Operational to Reference to Archive.

Each slice holds several storage subsections. Three of them respond to tuning:

  • Flow (session) database. This is typically the largest part. A flow record holds the protocol, source and destination IP and port, bytes, packets and timestamps. Each stored flow uses a fixed amount of storage regardless of how much data it moved. The VolumeCutoffBytes setting of the session group stores only flows whose volume is equal to or greater than the cutoff. The volume counts both directions and includes the Ethernet and IP headers. The default is 0, which stores all flows.
  • Edges. When Edges are enabled, the Probe generates an Edge stream that records relationships between keys, for example between an application and a host. Flow edges are added when each flow ends. Edges are used mainly for security investigations and can use a significant amount of disk.
  • Counter groups. Each counter group stores the keys it tracks. The High Water and Low Water marks limit how many keys are kept, Topper Traffic Only stores only the keys that reach a topper list, and the topper count sets how many toppers are kept.

The retention window itself (how many slices are kept) is a separate setting. This note reduces what each slice holds; it does not change the slice counts.

Reference: Tuning database usage, Configure Retention Policy, Trisul Edges

Prerequisites​

  • Trisul is installed and has written several full-day slices, so the DB Status page has data to compare.
  • You can log in to the web interface as admin.
  • For the Edges change: shell access to the Probe node and root privileges to edit trisulProbeConfig.xml.
  • A decision on your use case. If you use Trisul for security, incident response or compliance, read the trade-off in Part B before you set a flow cutoff.

Procedure​

Work through the parts in order. Part A gives you the baseline and tells you which of Parts B, C and D will save the most space.

Part A – Measure disk usage per day​

navigation

👉 Log in as admin and go to Context: default → Admin Tasks → DB Status

  1. Under Total counts of objects, note the counts for Sessions (flows), Resources, FTS (Full Text Search) and Alerts.
  2. Scroll to Database slices. Note the Total and Average sizes of the Hub database and the sizes of the Oper, Ref and Archive parts.
  3. On the Operational tab, find the slices that carry the same date and add their SIZE values. The sum is the storage for that day. For example, two slices of 4.07 GB and 4.11 GB on the same date give 8.18 GB for that day.
  4. Repeat step 3 for a few recent days to get a typical value per day.
  5. If the Probe stores packets, check Raw packet storage slices for the PCAP usage on each Probe.

To see what fills a slice, analyze it:

  1. In the TOOLS column of a recent full-day slice, click the menu icon.
  2. Under Storage Analyzer, click View. Click Download instead if you want a file to share.

The DB Analyzer page opens. It shows the slice details, a donut chart of the largest subsections by size, and a DB SIZE table with the columns NAME, SIZE, TOPI, BLKI, TRKI, CUTOFF, HIWATER, BUCKETSIZE and GUID. Sort the table by SIZE.

Check: Note which rows are largest. Session rows (for example TCP Sessions) point to Part B, Edge Graph points to Part C, and counter group rows point to Part D. Keep the per-day value from step 4 as your baseline.

Part B – Set Volume Cutoff Bytes for the flow database​

The flow database is typically the largest component. A cutoff of 1 MB keeps every flow that moved 1 MB or more and drops the smaller ones. The Trisul docs estimate that this reduces flow storage by roughly 80%, because about 80% of flows carry less than 10% of total traffic. Adjust the value for your network.

Decide first whether you can give up small flows:

With a 1 MB cutoff
You keepEvery flow that transferred 1 MB or more, so you still have full visibility of large flows
You gainA much smaller Trisul database, and more days of data on the same storage
You loseFlows under 1 MB. During incident response you cannot find them, and malware may use only small flows

Trisul recommends a 1 MB cutoff when your main use cases are visibility and network monitoring. Keep the default of 0 (store all flows) if you rely on Trisul for security investigations, incident response or compliance.

navigation

👉 Log in as admin and go to Context: default → profile0 → Session Groups

  1. Open the session group you want to tune for editing.
  2. Fill in the field:
Field NameDescription
VolumeCutoffBytesEnter 1000000 for 1 MB. The field also accepts a number with a K, M or G suffix, in decimal units (1M = 1,000,000 bytes). Only flows whose volume, in both directions including Ethernet and IP headers, is equal to or greater than this value are stored. Default: 0 (store all flows).
  1. Save the session group.

The Probe reads this setting only when it starts. Restart the Probe:

navigation

👉 Go to Context: default → Admin Tasks → Start/Stop Tasks

  1. Restart the Probe.

Reference: Flow Handling Settings

Part C – Disable Edges​

If the DB Analyzer shows a large Edge Graph row and you do not use Edge graph analytics, turn Edges off on the Probe.

  1. Log in to the Probe node.
  2. Open the Probe configuration file as root. For the default context it is:
/usr/local/etc/trisul-probe/domain0/probe0/context0/trisulProbeConfig.xml
  1. In the <Edges> section, set Enable to false to turn off the Edge feature. To keep Edges but stop generating edges for every flow, leave Enable as it is and set EnableFlowEdges to false instead.
<Edges>
	<Enable>false</Enable>
	<EnableFlowEdges>false</EnableFlowEdges>
</Edges>
  1. Save the file.
  2. Restart the Probe from Context: default → Admin Tasks → Start/Stop Tasks. The Probe reads the Edges options when it starts.

Reference: Trisul Probe Configuration: Edges

Part D – Tune counter groups​

Tune the counter groups that the DB Analyzer shows as large.

navigation

👉 Log in as admin and go to Context: default → profile0 → Counter Groups

  1. Click the name of the counter group you want to tune.
  2. Fill in the fields:
Field NameDescription
High Water MarkThe maximum number of keys tracked in this counter group before it is pruned down to the low water mark. A lower value stores fewer keys.
Low Water MarkThe number of keys the counter group is pruned down to when the high water mark is crossed. Keep it below the High Water Mark.
  1. To store only the keys that reach a topper list, select Topper Traffic Only.
  2. Click Update.

To control how many toppers are stored:

  1. In the counter group list, click the action button for the counter group and select Toppers.
  2. Set a lower top count for the meters you want to reduce.
  3. Save the topper policy.

Counter group changes take effect only after a restart:

navigation

👉 Go to Context: default → Admin Tasks → Start/Stop Tasks

  1. Restart Trisul.

Reference: Counter Group Settings

Verify the result​

  1. Wait until the Hub has written full-day slices after your changes.
  2. Go to Context: default → Admin Tasks → DB Status and add up the slice sizes for a new day, as in Part A step 3. Compare the result with your baseline.
  3. Open Storage Analyzer → View for a new slice. For the session rows, the CUTOFF column shows the cutoff in effect. For counter groups, the HIWATER column shows the high water mark in effect. Compare the SIZE of the Edge Graph, session and counter group rows with the baseline slice.

Troubleshooting​

Symptom or messageCauseFix
The CUTOFF column still shows the old valueThe Probe reads VolumeCutoffBytes only when it startsRestart the Probe from Admin Tasks → Start/Stop Tasks.
Edge Graph storage does not dropThe Probe reads the Edges options only when it starts, or the change was made in another context's fileCheck that you edited the trisulProbeConfig.xml of the right context, then restart the Probe.
Counter group changes have no effectCounter group changes apply only after a restartRestart Trisul.
You can't Add/Enable more than 1024 counter groupsMore than 1,024 counter groups would be activeDisable counter groups you do not need.
User-defined counter groups report zero usageA built-in counter group they depend on was disabledEnable the built-in counter group again.
Invalid values for track hi/lo watermark in the Probe logA session group has a track high water below 10, or a low water equal to or above the high waterSet the high water to 10 or more and the low water below it, then restart the Probe.

Limitations​

  • Flows below the volume cutoff are not stored. You cannot find them later during incident response or a compliance audit.
  • Disabling Edges removes the Edge graph data that Edge analytics uses for investigations.
  • Every change in this note needs a restart of the Probe or of Trisul before it takes effect.
  • This note reduces storage per slice. To keep more or fewer days, change the slice counts as described in Configure Retention Policy.
In this page: