Skip to main content

Netflow Configuration File

How to edit

On the Trisul-Probe type sudo /usr/local/share/trisul-probe/cfgedit then select “Netflow” to edit. See Plugin Configuration

Location

In a file calledPI-7CA09636-02D4-45E7-AA00-BE0D49B94E26.xmlin/usr/local/etc/trisul-probe/domainX/probeX/contextXYou can edit the config file directly by typing the full file name as shown below

# vi /usr/local/etc/trisul-probe/domain0/probe0/context0/PI-7CA09636-02D4-45E7-AA00-BE0D49B94E26.xml 

Parameters

These are the parameters that control Netflow processing.

Case insensitive

Boolean parameters are case-insensitive. True, true,TRUEmean the same thing

ParametersDefaultsDescription
AppModeApplication mode. Set this to ipdr to enable IPDR mode of Netflow processing. Leave it to blank for default processing
MeterHostsTRUEWhether or not traffic stats for each host is metered.
MeterSubnetsTRUEWhether or not traffic stats for each subnet is metered.
MeterApplicationsTRUEWhether or not traffic stats for each application is metered. In this context application refers to the UDP/TCP port or to special network level apps like ICMP, IPSEC, etc. In case of TCP/UDP ports, the default behavior is to only meter the lower numbered port. This works in most cases, but if you want to count both ports, set the MeterBothPorts parameter.
MeterNetflowSourcesTRUEIf set, Trisul tracks traffic volume from each exporting router. The meters tracked are
Netflow export rate in bytes
Netflow export rate in records
Total traffic volume on router in bytes
MeterNetflowInterfacesTRUEIf set, Trisul tracks traffic into and out of network interfaces.
MergeMultipleSourcesTRUEIf set, duplicate flows from different routers will be merged.
MatchBiDirectionalFlowsTRUENetflow records are unidirectional. If set, Trisul will merge two unidirectional flows into one bidirectional flow.
If you set this option to false, Trisul will retain the uni-directional flow as-is.
UseRouterTimestampsTRUEIf set, Trisul will use the timestamp from the Netflow records.
If not set, Trisul will use the timestamp at the server running Trisul. This is the default and recommended option unless you have routers which synchronize their timestamps automatically.
MergeHTTPFALSEFor HTTP flows, ignore the Source Ephemeral Port and set it to 65535. NOTE When the MergeX option is used flows will not be terminate but timed out. Use the MergeX options to reduce the number of unique flows
MergeHTTPSFALSEFor HTTPS flows, ignore the Source Ephemeral Port and set it to 65535.
MergeDNSFALSEFor DNS flows, ignore the Source Ephemeral Port. Set to FALSE, every DNS transaction will be a separate flow
IgnoreESPFALSEIgnore IPSEC ESP flows. These are tunneled interfaces containing no flow information within them.
MeterBothPortsFALSEWhen set to FALSE : The default behaviour of Trisul is to only meter the low numbered port. The assumption is that low numbered ports represent servers. This is usually accurate for traditional server applications that run below port 1024.
When set to TRUE : Meters both ports for applications above port 1024. Ports below 1024 (such as HTTP) are still counted in the normal way.
Set this to TRUE if you have P2P / VoIP Traffic you wish to track.
MeterHomeNetworkTRUEClassifies traffic relative to your Home Network.
**INCOMING **Destination IP is in your home network but Source IP is not
OUTGOING Source IP is in your home network but Destination IP is not
INTERNAL Both the Source IP and Destination IP are in your home network
**TRANSIT **Both the Source IP and Destination IP are not in your home network
MeterBothPortsFALSEWhen set to FALSE : The default behaviour of Trisul is to only meter the low numbered port. The assumption is that low numbered ports represent servers. This is usually accurate for traditional server applications that run below port 1024.
When set to TRUE : Meters both ports for applications above port 1024. Ports below 1024 (such as HTTP) are still counted in the normal way.
Set this to TRUE if you have P2P / VoIP Traffic you wish to track.
MeterTCPConnectionsTRUEMeter connection count for the flow TCP end point IPs. This allows you to get reports for hosts with maximum connections, etc
MeterAppConnectionsTRUEMeter connection count for source and destination ports for TCP based application.
FilterInterfacesIncludeA comma separated list of interfaces.
When set, Trisul will accept and process flows that exit or enter these interfaces. This can be in routerip_ifindex format or Trisul key format
An example
If you only want to accept flows that use the WAN link (if Index = 108) on router (122.166.4.242) and GigE link (ifIndex = 120) also on router (122.166.4.242)
You should specify the interfaces as 122.166.2.242_108,122.166.242.100_120
FilterInterfacesExcludeA comma separated list of interfaces.
When set, Trisul will only process flows that DO NOT use these interfaces.
The format is same as the parameter FilterInterfacesInclude
Note: that you cannot use the FilterInterfacesInclude and Exclude parameters at the same time. This rule applies to all such parameters
FilterFlowASNIncludeOnly process netflow records containing one of the ASN in this comma separated list
FilterFlowASNExcludeOnly process netflow records which DO Not contain any ASN in this CSV list. USECASE Upstream ISP filtering out its downstream ISP ASNs for monitoring and logging purposes.
FilterHostsIncludeOnly process netflow records containing IP address in this CSV list (IPv4 only)
FilterHostsExcludeOnly process netflow records not containing ANY IP in this CSV list (IPv4 only)
Feature used in ISP environments
FilterAppsIncludeOnly process these ports. Use comma separated port numbers
CheckPacketsForCorruptionFALSETurns on heuristics to check each netflow record for corruption. When set to TRUE, Trisul validates the timestamps, packets, bytes, etc. We have seen unpatched routers transmit corrupted records, which can distort the metrics and charts.
LogTemplateMessagesTRUEIf enabled , Trisul will log template updates to the probe log files. Disable if you feel there are too many of these messages.
IgnoreV9EgressFromDevicesSometimes we find that customers erroneously enable both Netflow v9 INGRESS and EGRESS on all ports. This will result in double counting. The solution is to only configure INGRESS+EGRESS on a single port such as the uplink ports. Oftentimes, we are unable to get the Netflow change request done. This option allows you to adjust for this. This option contains a comma separated list of IP addresses of devices from which Trisul will ignore EGRESS netflow V9 records. The following example ignores Netflow v9 EGRESS from two devices
<IgnoreV9EgressFromDevices>172.16.13.31,172.16.13.36
Egress flow records are those with NF9/IPFIX direction field with value=1
IgnoreAllEgressFALSEIgnore all flow records generated by egress processing. Egress flow records are those Netflow v9 and IPFIX records with direction = 1 (egress)
IgnoreOutCountsSet this option to true to ignore out packets and out bytes sent incorrectly by some devices. Some firewalls send the same metrics in IN and OUT counts by misinterpreting the template as post out counts. Use this if you see the same values of IN/OUT on all flows
EnableShimTunnelFALSEIf Enabled, Trisul will decapsulate the special SHIM tunnel used to forward Netflow packets from a remote network to the Trisul probe. Normal Netflow will also be processed correctly. For more information see on Github netflow-shim-tunnel
HomeASNumbersEnter a comma separated list of Home AS and Downstream AS numbers. Used in ISP deployments these are the AS numbers of the customer running Trisul. If Home AS numbers are present they are used instead of Home Network Prefixes to determine per-AS Upload (egress) and Download (ingress) traffic metrics per AS and also to split upstream vs downstream. For large and transit ISPs with changing downstream topologies see the parameter HomeASNumbersFile
AddEdgesTRUEAdds ASN to PREFIX edges. If enables, it allows you to select a AS and see which PREFIXES are active in that AS. Here active means prefixes which had some traffic in the selected interval
MeterLinkASFALSEFlow-Link-ASN is a special Crosskey Counter Group which tracks network interface > AS flows. Setting this option to TRUE, enables that counter group. You can then visualize these flows using the Sankey App
UsePostNATAddressesTRUEThis is Neflow Template option supported on some devices such Palo Alto. Set this to TRUE to use Post-NAT IP address which represent the actual end user in our organization rather than the WAN address.
TemplateDumpIntervalSeconds600How frequently must the template database be dumped by the probes. These are shown on the Admin > Show Template DB page. Set this to 0 to prevent template databases being dumped
MeterZeroFlowsFALSEUse case Large ISP compliance logging. When compliance only asks for connection logs without collecting byte and packet counts, enabel this option
AllowVariableLengthStringsFALSEIPFIX variable length strings in templates are used by some network vendors to export labels and tags using IPFIX template field type 65535
TagFlowsWithUserIDFALSESome vendors will export flows with a User-ID string which identifies the pre-NAT user of the flow, this appends a tag each flow.
MeterUserIDFALSEIf User-ID string is present, meter bytes, flows, connections for it
MeterAppIDFALSEIf App-ID string is present, meter bytes, flows, connections for it
AddL2AccountingBytes0Use this option if you want to closely match Netflow with SNMP metrics for accounting and billing purposes. Netflow being a Layer3 (IP) technology it does not account for link layer bytes and also any physical layer bytes such as Ethernet Frame Check sequence. A typical setting is to set this parameter to 18 to add 14 bytes of ethernet and 4 bytes of ethernet FCS to each packet. This option is typically only useful for customers who seek very close match with SNMP metrics for billing purposes
ReloadConfigCheckSecondsSet this to 60 to automatically check for changes to this configuration file every 60 seconds. If the modified timestamp of this configuration file changes, the configuration is automatically reloaded. The default behaviour is for Trisul Probe to require a restart for a Netflow configuration file change.
TagFlowsWithDirectionFALSETag each flow with a direction relative to the Home Networks (see user guide Home network section based on the AS Number information.
Every flow is tagged with one of the following
[dir]internet
when one of source and dest AS is within home network and the other is external
[tag]internal
when both source and dest AS are within the home network
[tag]transit
when both source and dest AS are outside the home network
Feature used in ISP environment
HomeASNumbersFileUsed with ISP licensed customers. Trisul automatically creates an online feed of currently observed downstream AS numbers based on BGP routing information from outside vantage points. Set this to the value downstream_origin_as.txt and configure the Geo plugin for automatic updates. For simpler applications see the parameter HomeASNumbers
Feature used in ISP environments
Use64BitHostByteOrderFALSESome netflow implentations incorrectly send 64 bit counters in template fields as two 32 bit fields in host byte order. If you are seeing bizzarely large numbers for bandwidth, bytes, and packets – try setting this to TRUE
MeterTosAsDSCPTRUEMeter TOS byte as 6-bit DSCP + 2-bit ECN. This is then mapped to DSCP values in Trisul UI such as for example
08” → “CS1/AF11-13”
“0A” → “CS1/AF11-13”
SmartNATRemovalTRUESet this to TRUE in firewall environments. When set to TRUE Trisul analyzers Netflow records and always selects the correct IP addresses to reflect the internal IPs rather than the public NAT IP
Home Prefixes FileA Trisul routes database or a CSV file containing all prefixes to be treated as home or downstream prefixes. Typically this file is named 0.0.0.0_routes.sql3 setup during configuraiton for ISP deployment
Feature used in ISP environments
AutoloadPreviousTemplatesFALSEWhen set to true Trisul will reload the most recently known netflow template database and immediately start processing instead of waiting for templates. Use this option in networks where template intervals are long
PrintRecordsUsed in DDoS report mode to print raw netflow records automatically, Do not set this option manually

Sampling rates

Flow sampling is a technique used by very busy devices to reduceCPUload. If you confiygure a sampling rate of128, the device will only process every 128 packets. InIPFIXtechnologies the sampling rate is typically sent in anoptions template. This section provides a manual way to configure sampling rates per router in the event you are not able to setup the options template. This is not a replacement for configuring sampling on the router or enabling the options template.

Add a separate line for each router as shown below

<SamplingRates>
<Rate router="180.179.9.2" rate="128" />
</SamplingRates>

Default sampling rate

To create a default sampling rate for all routers which do not have an explicit sampling rate set, use the IP address 0.0.0.0. For example to configure a sampling rate of 1000 for all routers. Use

<SamplingRates>
<Rate router="180.180.9.2" rate="128" />
<Rate router="0.0.0.0" rate="1000" />
</SamplingRates>

Advanced

Static Templates

Typically you dont need to use this section. We have used this section in the past to address various firmware bugs in some devices which did not export Netflow templates correctly. Templates fields specified in this section will override those received on the network.