Script selector cheat sheet
This page helps you select the correct script type for a specific task you are trying to accomplish.
How to go about Trisul scripting
- Using the table below you can get the script type most appropriate for your task
- Download the appropriate skeleton and start coding
| What I want to do | What script type should use | Remarks and examples |
|---|---|---|
| I want to count something at the packet level | simple_counter | First thing is to decide what protocol you want to attach to. Specify a protocol_guid from the list of known GUIDS Then write code in the onpacket() method. Use methods in the engine object passed to you to add metrics into TrisulExamples: pktlen.lua |
| I want to see the first X bytes of every TCP flow | reassembly_handler | Write code in the onpayload(..) method, the first bytes will arrive with seekpos==0 you can process that and reject all other calls with seekpos>0. Alternately you can use a Lua Table to keep track of active flows. |
| I want to listen to all URLs and then do something.. | resource_monitor | Plug into the HTTP Resources stream identified by the GUID {4EF9DEB9-4332-4867-A667-6A30C5900E9E} Then fill out onresource(). You will be passed a resource object. You can get the URL from the resource object using resource:uri()Example: dyndns-alert.lua ua-parser.lua |
| I want to listen to all Domains | resource_monitor | Same as the above question for HTTP URLs. By just plugging into the DNS Resource stream identified by a different GUID {D1E27FF0-6D66-4E57-BB91-99F76BB2143E} you access the DNS stream. The uri() and label() methods give you the DNS question name and the entire record |
| Whats the deal with the GUIDs | for all | Every entity type in Trisul is identified by a GUID. One of the first things you need is to know where to find the Entity-Type to GUID mapping. It is easy, you can start with the Well known GUID page See the section On GUIDS |
| I want to process IP flows | sg_monitor | In the sg_monitor script there are many methods you can write depending on what you want to do. Use onflush() to get access to the flow records just before they are flushed to the Hub node. All methods get a Flow object you can navigate that object to get the statistics for the flow.Examples: passive-dns.lua simple sg3.lua |
| I want to prevent some flows from being stored | sg_monitor | Write code in the flushfilter() method to examine the attributes in the Flow object that is passed to you. Then return false to prevent a flow from being stored in the Hubs |
| I want to enrich a flow with some tags | engine:tag_flow | Every LUA function in Trisul is passed an Engine object for you to interact with the Trisul pipelines. Say you are inspecting some traffic from any script and want to mark a particular flow with some label-tag to be queried later. Use engine:tag_flow() |
| I want to listen to all IDS alerts and take some action | alert_monitor | Plugin into the IDS Alert group. Then write code in onnewalert() or onflush() . You will have access to an alert object Since it is plain LUA you can do whatever you want with the alert. Send an email, write a log file, generate metrics, .. |
| I want to prevent some alert types from being stored | alert_monitor | Write code in the flushfilter method . Inspect the alert object fields like alert:sigid() and decide if you want it stored in the Hubs. Return false to prevent storage.Example: no-alienvault.lua |
| I want to process SSL certificates | fts_monitor | Attach to the FTS group SSL Certificates. FTS is full text search feature of Trisul. Some example of FTS Documents generated by Trisul are HTTP Headers, SSL Certificates, DNS records. The onnewfts() method contains a FTSDocument object The fts:text() returns the entire certificate chain in a canonical format. You can then use LUA methods to pick out fields you want using regex and text manipulation. This is the approach of Trisul use text manip instead of very fine grained protocol decodes.Examples : detect_sha1.lua roca.lua |
| I want to write a dissector for a TCP based protocol | reassembly_handler | Write code in the onpayload method of reassembly_handler. Here you can do port independent protocol detection and also dissect the protocol. Write the protocol in plain LUA but we reccommend using the excellent BITMAUL library to write your dissector. |
| I want to write a dissector for protocol running on IP / UDP/ Ethernet | protocol_handler | These dont require reassembly hence you can process on a packet-by-packet basis. Write code in the “parselayer()”protocol_handler.html#function_parselayer method to tell Trisul where the protocol begins and ends and what the next protocol on the stack is. You can also use the simple_counter script type |
| I want to add some information to Trisul | Engine object | For any script you write, Trisul will pass you an Engine object that lets you add information into the Trisul streaming pipeline. Metrics, tags, alerts.. |
| I want to create an SNMP poller | engine_monitor | The Engine Monitor is called when the flush window starts which is every minute. This is a good place to do “general purpose” scripting in a periodic manner without reference to any of Trisul streams. Here you would write code in onbeginflush to run snmpget.. get the metrics from the outside devices and update the Trisul metrics.Example: snmp.lua |
| I want to create new metrics | counter_group | Create a new counter group “My Group” and then create meters within it. Each counter group gets a new GUID |
| How can I update a metric | engine:update_counter | To update metrics for a counter group by calling engine:update_counter() |
| I want to “log” something | resource_group | In Trisul, logs are called resources. You create a new resource group and then add a log by adding a resource using the methods in the Engine object. |
| I want to write an actual log file, not the resource | resource_group | Create a new resource_group and also resource_monitor. Then in the resource_monitor just write to a file in plain LUA. The issue here is Trisul is multi threaded workers, so you may not be able to write a single log file without some kind of synchronization. You can write to multiple files from the onflush methods to a file that uses the engine:id() as part of the file name. Then you can use some 3rd party system like Graylog, Logstash, syslog-ng to merge these worker log files into a single file. |
| I want to control HTTP file extraction | fileextraction | You can minutely control what gets saved using flow based filters or content based filtering. See fileextract |
| I want to send HTTP extracted files to YARA | out of scope | Trisul’s work stops when the files are dumped into a particular directory. You need to write your own tooling to process those files outside the Trisul pipelines |
| I dont want to store packets from 192.168.2/x during working hours | packet_storage | Use the filter method which contains a flow object. Look at the IP address and Timestamps to decide if you want to store packets on this flow or not.Example: skip_youtube.lua |
| I dont want to store packets based on content | packet_storage | Write code in the filter_payload() where you can examine a part of the payload string to check if you want to store all packets in that flow. Only the first payload bytes is given to you. So your “signature” has to be right at the beginning |
| I want to use my own input framework | input_filter | The input filter script should be used to return the next packet using step or the next alert using step_alert . Then Trisul has to be run slightly differently to use this input filter instead of the normal PCAP file or AFPACKET network interface.Example: suricata_unixsocket.lua pure_lua_pcap.lua |
| I want to run some external task – like send an email, write a database record etc | just use normal LUA | In general you can write any LUA code to perform any type of task in your scripts. In reality, you are working with a real time streaming analytics system so you have time budgets. Rough rule. For Frontend script types, do not perform any significant I/O. For Backend script types you have a total of about 30 seconds to complete all your tasks in every 1 minute interval |
| I want to run some long running external task | T.async framework | These tasks are scheduled on a separate thread and they call back on the main thread when they complete. So most I/O should be safe to perform in the backend category only |
| I want to do some really long running tasks for 1000s of elements | you cant !! | Trisul is a streaming analytics real time system. The default window is 1 minute. So ALL of the processing needs to be done within the global time budget of 1 minute. If you exceed that time budget, Trisul will simply drop everything to stay in sync with the next window that has started. If you have a few long running tasks, you can use the T.async framework. But if you have thousands of such tasks, then you have to break it up. Save the inputs from Trisul somewhere on disk and process them separately using another process. |
| I want to check against malware indicators | sg_monitor , resource_monitor , etc | It all depends on the exact type of indicator. For IP addresses link up to cg_monitor on Host Counter Group, for DNS Resources, URL, SSL plugin use those types. Load the intel file into plain LUA table and lookup and alert using Engine:add_alert |
| I want to load 1-Million indicators too big for LUA tables | use a LevelDB backend | Compile the indicators into a LevelDB database and load that in the onload function of your script. You can use the open source tris_leveldb LUA library for that.Example: tris_leveldb.lua Umbrella-Top1Million.lua |
| I want to support the FIX protocol.. | reassembly_handler | TCP based protcols plugin to reassembly_handler:onpayload and then use the BITMAUL library to get started. Example: BITMAUL examples for working code |
| Happy coding |