Update Analytics Performance by nasbench · Pull Request #3866 · splunk/security_content
This PR enhances the performance of a couple of detections as part of a continued effort that I was tracking from the issue #3809. Details are below.
Note All tests were executed on Endor
First batch was addressed in #3850
Updated Analytics [11]
Common Ransomware Notes- From 164 seconds to 33.44 seconds - Switch to using a subsearch since the lookup is small enough. Added a warning in the description to inform the user about this too.File with Samsam Extension- From 160 seconds to 11 seconds - Moved the extension search to thewhereand kept the extraction for better output.Detect Rare Executables- From 300~ seconds to 100~ seconds - Reduces usage ofvaluesand switched them tolatestso that we can reduce MV calcs which are expensive.3CX Supply Chain Attack Network Indicators- Added a where clause withDNS.query=*to ensure the existence of the field, since it is used as a filter in the lookup. This is a best effort and will only reduce search time by some time. The same thing was applied to the followingDetect hosts connecting to dynamic domain providers,Detect Remote Access Software Usage DNSDetect Remote Access Software Usage URL- Added a where clause withWeb.url_domain=*to ensure the existence of the field, since it is used as a filter in the lookup.Detect Remote Access Software Usage Traffic- Added a where clause withAll_Traffic.app=*to ensure the existence of the field, since it is used as a filter in the lookup.Common Ransomware Extensions- To enhance this we added filter for known extensions to reduce the search space. By excluding stuff like.exe...etc from file creation we already get rid of a lot. This can ofc be further enhanced by adding more to the list.Detect Remote Access Software Usage File- In order to also enhance perf, we allow only the extensions that are referenced by the lookup itself. Because this is using an file creation event, this will reduce the search space by a lot.Windows DotNet Binary in Non Standard Path- From 852 seconds to 7 seconds - Using a subsearch to enhance performance with the accepted limitation just because the lookup is small.
nasbench
marked this pull request as ready for review
| | `drop_dm_object_name("All_Traffic")` | ||
| | `security_content_ctime(firstTime)` | ||
| | `security_content_ctime(lastTime)` | ||
| | lookup remote_access_software remote_appid AS app OUTPUT isutility, description as signature, comment_reference as desc, category |
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we move these lookup by the where clause like other detection updates? eg : Windows DotNet Binary in Non Standard Path
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can't just because the app field is inconsistent in that CIM it can contain both apps categories and binaries, but more importantly the lookup is big with 600 entries, which will results in a big OR clause that results in the same performance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters