TCP Reassembly

FRONT-END SCRIPT

Interface into the TCP Reassembly Engine using LUA. Your LUA scripts can

1. control what flows need reassembly
2. ability turn on or off mid way through the flow
3. co-operative algorithms allow multiple LUA scripts
4. handle reassembled chunks of data and do whatever you want

Structure

TCP Reassembly skeleton script

Table reassembly_handler

You need to supply code for one or more of the following functions.

filter	function( engine, timestamp, flowkey )	called when a new flow starts. Examine the flow tuples and determine if you are interested in reassembly.
onpayload	function(engine, timestamp, flowkey, direction, seekpos, buffer)	Called for every chunk of newly reassembled raw bytes
onnewflow	function(engine, timestamp, flowkey)	Called when a new flow is established
onterminateflow	function(engine, timestamp, flowkey)	Called when a flow is terminated
onattribute	function(engine, timestamp, flowkey, attr_name, attr_value)	Trisul’s native processors found some flow attribute. You get a chance to handle it. Examples are : HTTP Content Type, TLS Certificates, HTTP Hosts, etc

LUA functions reference

Your script needs to supply one or more of these functions listed below. Note that there will be other scripts too written by you or others executing in the same context. So do not assume any stateful behaviour between separate invocations of your script. It is however guaranteed that a given flow will always have all events on it called on the same context of your script.

Function filters

Purpose

Hint to the Trisul Engine whether to enable or disable reassembly on a per-flow basis.

When called

When a new flow is established.

Parameters

engine	An Engine object	use this object to add metrics, resources, or alerts into the Trisul framework
timestamp	number	Timestamp seconds when the first packet in the flow was seen
flowkey	A FlowID object	use this to determine IPs and Ports involved in the flow

Return value

Return true if you want files to be extracted for this flow
Return false if you are not interested in this flow.

Voting considerations

If you have multiple scripts S1, S2, .. SN each voting differently on filter(), the following rule is enforced.

1. ALL scripts have to vote NO to flush by returning false.
2. Even if one script Sx returns YES or does not implement filter(), the flow is reassembled.

Example

The following example only reassembles TCP from IP 192.168.1.8

filter = function(engine, timestamp, flow )

    if flow:ipa_readable() == "192.168.1.8" then
      return true
    else
      return false
    end

  end

Function onnewflow

Purpose

Whan a new flow is first seen. You can do housekeeping, setup some datastructures or files here.

When called

When a new flow is first seen. This method is not called when LUA plugins have all agreed using the filter function that reassembly be disabled for this flow.

Parameters

engine	An Engine object	use this object to add metrics, resources, or alerts into the Trisul framework
timestamp	number	Timestamp seconds when the first packet in the flow was seen
flowkey	A FlowID object	use this to determine IPs and Ports involved in the flow

Return value

Ignored

Example

See trisul-scripts on Github

Function onterminateflow

Purpose

Called when a flow is terminated.

When called

When a TCP flow is terminated either by the normal TCP procedures or by timeout.

Parameters

engine	An Engine object	use this object to add metrics, resources, or alerts into the Trisul framework
timestamp	number	Timestamp seconds. seconds since epoch Jan 1 1970
flowkey	A FlowID object	use this to determine IPs and Ports involved in the flow

Return value

Ignored

Example

Function onpayload

Purpose

If you want streaming access to raw reassembled TCP payload.

When called

Called when a new chunk of reassembled data is available on a TCP connection.

Stream read position

We allow multiple user written LUA scripts as well the Trisul’s native reassembly handlers to plug into the same reassembly engine. These are the rules that indicate when the read position is moved.

1. the stream read position (in each direction) is moved ahead
   1. Trisul native built in analysis gets first priority. For example until Trisul native function “TLS Certificate Analysis” has had a chance to read the entire TLS Certificate exchange, the read position will not be moved.
   2. If Trisul native analysis is not interested in any given stream, then after calling onpayload of each LUA script, the stream read position is moved.

Co-operative scripting note

Due to the co-operative execution of multiple LUA scripts as well as Trisul’s built in processing. You may get multiple calls of the same payload. You can use the seekpos parameter to detect that. See the example at the end of how to save payloads to a file.

Parameters

engine	An Engine object	use this object to add metrics, resources, or alerts into the Trisul framework
timestamp	number	Timestamp seconds when the first packet in the flow was seen
flowkey	A FlowID object	use this to determine IPs and Ports involved in the flow
direction	number	- 0 = IN payload in server → client direction - 1 = OUT payload in client → server direction (same as the first SYN)
seekpos	number	Seek position byte position from the beginning of the stream
buffer	A Buffer object	represents the reassembled bytes

Return value

Ignored

Usage

onpayload  = function(engine, time, flow, dir, seekpos, buff )
    ..
    buff:writetofile("myfile_in",seekpos)
  end,

Function onattribute

Purpose

When Trisul finds some flow attribute you can get informed. This way you can leverage Trisul’s native attribute extraction logic without re-inventing the wheel.

When called

When a new attribute about a flow is discovered by Trisul.

Leverage Trisul’s discovery
We encourage you to use the onattribute function if you can find your object of interest here. You can save CPU as well as work!

Parameters

engine	An Engineobject	use this object to add metrics, resources, or alerts into the Trisul framework
timestamp	number	Timestamp seconds when the first packet in the flow was seen
flowkey	A FlowID object	use this to determine IPs and Ports involved in the flow
attribute_type	string	The type of attribute. See note below
attribute_value	string	The attribute value

Supported attributes

Currently these are the attribute types that Trisul can report

TLS:Record	A complete TLS Handshake record (all messages except Application Data)
TLS:O	Organization found in the certificate
TLS:CA:ROOT	Root Certificate Authority
TLS:CA:INTER	Intermediate Certificate Authority
TLS:CIPHER	Cipher Suite negotiated. This is a hex string – see IANA TLS Ciphers for meaning
^D	A flow ended
""	A flow started
HTTP-Header	The entire HTTP Header
Content-Type	Content Type
User-Agent	The HTTP User Agent
Host	HTTP Host
URI	HTTP URI

Return value

Ignored

Usage

Typically you would update a counter, attach a flow counter, or generate some resource metadata or alerts.

onattribute  = function(engine, time, flow, attr_type, attr_value )

      if attr_type == "Content-Type" then

        -- some processing
        engine:add_flow_counter(..)

    end

  end,