L-SM-TH

Logging(Host/Network) / Security Monitoring / Threat Hunting

Table of Contents

https://www.youtube.com/watch?v=YGJaj6_3dGA

https://adsecurity.org/wp-content/uploads/2017/04/2017-BSidesCharm-DetectingtheElusive-ActiveDirectoryThreatHunting-Final.pdf

https://fortinetweb.s3.amazonaws.com/fortiguard/research/Learn_How_to_Build_Your_Own_Utility_to_Monitor_Malicious_Behaviors_of_Malware_on%20macOS_KaiLu.pdf https://www.blackhat.com/us-18/arsenal.html#learn-how-to-build-your-own-utility-to-monitor-malicious-behaviors-of-malware-on-macos https://jpcertcc.github.io/ToolAnalysisResultSheet/ https://techcommunity.microsoft.com/t5/Azure-Sentinel/Identifying-Threat-Hunting-opportunities-in-your-data/ba-p/915721

https://www.peerlyst.com/posts/security-monitoring-and-attack-detection-with-elasticsearch-logstash-and-kibana-martin-boller https://github.com/littl3field/Audix https://digital-forensics.sans.org/blog/2019/02/09/investigating-wmi-attacks https://github.com/hunters-forge/API-To-Event https://www.peerlyst.com/posts/threat-hunting-basics-getting-manual-soc-prime

https://www.youtube.com/watch?v=iweEI60PWeY

  • Container Forensics: What to Do When Your Cluster is a Cluster - Maya Kaczorowski & Ann Wallace(CloudNativeConEU19)

    • When responding to an incident in your containers, you don’t necessarily have the same tools at your disposal that you do with VMs - and so your incident investigation process and forensics are different. In a best case scenario, you have access to application logs, orchestrator logs, node snapshots, and more. In this talk, we’ll go over where to get information about what’s happening in your cluster, including logs and open source tools you can install, and how to tie this information together to get a better idea of what’s happening in your infrastructure. Armed with this info, we’ll review the common mitigation options such as to alert, isolate, pause, restart, or kill a container. For common types of container attacks, we'll discuss what options are best and why. Lastly, we’ll talk about restoring services after an incident, and the best steps to take to prevent the next one.

  • Get Cozy with OpenBSM Auditing...the good, the bad, & the ugly - Patrick Wardle

  • Getting Cozy With OpenBSM Auditing On MacOS - Patrick Wardle

    • With the demise of dtrace on macOS, and Apple’s push to rid the kernel of 3rd-party kexts, another option is needed to perform effective auditing on macOS. Lucky for us, OpenBSM fits the bill. Though quite powerful, this auditing mechanism is rather poorly documented and suffered from a variety of kernel vulnerabilities. In this talk, we’ll begin with an introductory overview of OpenBSM’s goals, capabilities, and components before going ‘behind-the-scenes’ to take a closer look at it’s kernel-mode implementation. Armed with this understanding, we’ll then detail exactly how to build powerful user-mode macOS monitoring utilities such as file, process, and networking monitors based on the OpenBSM framework and APIs. Next we’ll don our hacker hats and discuss a handful of kernel bugs discovered during a previous audit of the audit subsystem (yes, quite meta): a subtle off-by-one read error, a blotched patch that turned the off-by-one into a kernel info leak, and finally an exploitable heap overflow. Though now patched, the discussion of these bugs provides an interesting ‘case-study’ of finding and exploiting several types of bugs that lurked within the macOS kernel for many years https://github.com/maus-/slack-auditor

  • When Macs Come Under ATT&CK - Richie Cyrus(Derbycon2018)

    • Macs are becoming commonplace in corporate environments as a alternative to Windows systems. Developers, security teams, and executives alike favor the ease of use and full administrative control Macs provide. However, their systems are often joined to an active directory domain and ripe for attackers to leverage for initial access and lateral movement. Mac malware is evolving as Mac computers continue to grow in popularity. As a result, there is a need for proactive detection of attacks targeting MacOS systems in a enterprise environment. Despite advancements in MacOS security tooling for a single user/endpoint, little is known and discussed regarding detection at a enterprise level. This talk will discuss common tactics, techniques and procedures used by attackers on MacOS systems, as well as methods to detect adversary activity. We will take a look at known malware, mapping the techniques utilized to the MITRE ATT&CK framework. Attendees will leave equipped to begin hunting for evil lurking within their MacOS fleet. https://blog.stealthbits.com/windows-file-activity-monitoring/ https://github.com/salesforce/bro-sysmon

  • Detecting Kerberoasting activity using Azure Security Center - Moti Bani

  • Practical PowerShell Security: Enable Auditing and Logging with DSC - Ashley McGlone

  • Detecting Offensive PowerShell Attack Tools - adsecurity.org https://github.com/djhohnstein/EventLogParser https://blog.redteam.pl/2019/08/threat-hunting-dns-firewall.html?m=1

  • Windows 10, version 1809 basic level Windows diagnostic events and fields

  • How to Detect Overpass-The-Hash Attacks - Jeff Warren

  • Implementing Sysmon and Applocker - BHIS

    • In almost every BHIS webcast we talk about how important application whitelisting and Sysmon are to a healthy security infrastructure. And yet, we have not done a single webcast on these two topics. Let's fix that. In this webcast we cover how to implement Sysmon and Applocker. We cover overall strategies for implementation and how to deploy them via Group Policy. We walk through a basic sample of malware and show how both of these technologies react to it. Finally, we cover a couple of different "bypass" techniques for each. Everything in security has weaknesses, and these two technologies are no exception.

  • The Role of Evidence Intention - Chris Sanders

  • $SignaturesAreDead = “Long Live RESILIENT Signatures” wide ascii nocase - Matthew Dunwoody, Daniel Bohannon(BruCON 0x0A)

    • Signatures are dead, or so we're told. It's true that many items that are shared as Indicators of Compromise (file names/paths/sizes/hashes and network IPs/domains) are no longer effective. These rigid indicators break at the first attempt at evasion. Creating resilient detections that stand up to evasion attempts by dedicated attackers and researchers is challenging, but is possible with the right tools, visibility and methodical (read iterative) approach. As part of FireEye's Advanced Practices Team, we are tasked with creating resilient, high-fidelity detections that run across hundreds of environments and millions of endpoints. In this talk we will share insights on our processes and approaches to detection development, including practical examples derived from real-world attacks. https://github.com/miriamxyra/EventList

  • Different Approaches to Linux Monitoring - Kelly Shortridge

  • Detecting the Elusive Active Directory Threat Hunting - Sean Metcalf(BSidesCharm2017)

    • Attacks are rarely detected even after months of activity. What are defenders missing and how could an attack by detected? This talk covers effective methods to detect attacker activity using the features built into Windows and how to optimize a detection strategy. The primary focus is on what knobs can be turned and what buttons can be pushed to better detect attacks. One of the latest tools in the offensive toolkit is ""Kerberoast"" which involves cracking service account passwords offline without admin rights. This attack technique is covered at length including the latest methods to extract and crack the passwords. Furthermore, this talk describes a new detection method the presenter developed. The attacker's playbook evolves quickly, defenders need to stay up to speed on the latest attack methods and ways to detect them. This presentation will help you better understand what events really matter and how to better leverage Windows features to track, limit, and detect attacks.
    • Slides

  • What’s in a name? TTPs in Info Sec - Robby Winchester

https://blog.kolide.com/monitoring-macos-hosts-with-osquery-ba5dcc83122d?gi=e42e60717e0 https://blog.trailofbits.com/2017/11/09/how-are-teams-currently-using-osquery/ https://blog.trailofbits.com/2017/12/21/osquery-pain-points/ https://blog.trailofbits.com/2018/04/10/what-do-you-wish-osquery-could-do/ https://github.com/davehull/Kansa

https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/appendix-l--events-to-monitor https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/ https://docs.microsoft.com/en-us/windows/security/threat-protection/auditing/advanced-security-audit-policy-settings https://www.microsoft.com/en-us/download/confirmation.aspx?id=52630 https://www.microsoft.com/en-us/download/details.aspx?id=50034

https://www.youtube.com/watch?v=YwR7m3Qt2ao&feature=youtu.be

https://posts.specterops.io/threat-hunting-with-jupyter-notebooks-part-1-your-first-notebook-9a99a781fde7

OSQuery * https://github.com/facebook/osquery/tree/master/packs * https://osquery.readthedocs.io/en/stable/

  • Mordor
    • The Mordor project provides pre-recorded security events generated by simulated adversarial techniques in the form of JavaScript Object Notation (JSON) files for easy consumption. The pre-recorded data is categorized by platforms, adversary groups, tactics and techniques defined by the Mitre ATT&CK Framework. The pre-recorded data represents not only specific known malicious events but additional context/events that occur around it. This is done on purpose so that you can test creative correlations across diverse data sources, enhancing your detection strategy and potentially reducing the number of false positives in your own environment.

ThreatHunting * https://github.com/ThreatHuntingProject/ThreatHunting * https://sqrrl.com/media/Framework-for-Threat-Hunting-Whitepaper.pdf * https://www.threathunting.net/files/huntpedia.pdf * https://www.sans.org/reading-room/whitepapers/threats/paper/37172

https://posts.specterops.io/threat-hunting-with-jupyter-notebooks-part-4-sql-join-via-apache-sparksql-6630928c931e

https://blog.redteam.pl/2019/04/dns-based-threat-hunting-and-doh.html https://www.peerlyst.com/posts/security-monitoring-and-attack-detection-with-elasticsearch-logstash-and-kibana-martin-boller

https://www.youtube.com/watch?v=SzbABydoz0k

  • https://github.com/Patrowl/PatrowlManager https://medium.com/@cryps1s/detecting-windows-endpoint-compromise-with-sacls-cd748e10950

https://www.blackhat.com/docs/eu-17/materials/eu-17-Atkinson-A-Process-Is-No-One-Hunting-For-Token-Manipulation-wp.pdf

https://github.com/kolide/fleet

https://github.com/deviantony/docker-elk https://techcommunity.microsoft.com/t5/Microsoft-Defender-ATP/Hunting-for-reconnaissance-activities-using-LDAP-search-filters/ba-p/824726

https://github.com/github/vulcanizer

http://penconsultants.com/blog/crown-jewels-monitoring-vs-mitigating/

https://github.com/Yelp/elastalert

  • Mordor
    • The Mordor project provides pre-recorded security events generated by simulated adversarial techniques in the form of JavaScript Object Notation (JSON) files for easy consumption. The pre-recorded data is categorized by platforms, adversary groups, tactics and techniques defined by the Mitre ATT&CK Framework. The pre-recorded data represents not only specific known malicious events but additional context/events that occur around it. This is done on purpose so that you can test creative correlations across diverse data sources, enhancing your detection strategy and potentially reducing the number of false positives in your own environment.

Network Security Monitoring/Logging/Threat Hunting

  • History
  • 101
  • Educational
  • Courses
  • General
  • Articles/Presentations/Talks/Writeups
    • IDS/IPS
    • Logging
    • Traffic Analysis
      • Network Profiling Using Flow
        • This report provides a step-by-step guide for profiling—discovering public-facing assets on a network—using network flow (netflow) data. Netflow data can be used for forensic purposes, for finding malicious activity, and for determining appropriate prioritization settings. The goal of this report is to create a profile to see a potential attacker’s view of an external network. Readers will learn how to choose a data set, find the top assets and services with the most traffic on the network, and profile several services. A cas e study provides an example of the profiling process. The underlying concepts of using netflow data are presented so that readers can apply the approach to other cases. A reader using this repor t to profile a network can expect to end with a list of public-facing assets and the ports on which each is communicating and may also learn other pertinent information, such as external IP addresses, to which the asset is connecting. This report also provides ideas for using, maintaining, and reporting on findings. The appendices include an example profile and scripts for running the commands in the report. The scripts are a summary only and cannot replace reading and understanding this report.
      • Making the Most of OSSEC
      • Using SiLK for Network Traffic Analysis
      • Current State of Virtualizing Network Monitoring
      • The Quieter You Become, the More You’re Able to (H)ELK - Nate Guagenti, Roberto Rodriquez - BSides Colombus Ohio 2018
        • Enabling the correct endpoint logging and centralizing the collection of different data sources has finally become a basic security standard. This allows organizations to not just increase the level of visibility, but to enhance their threat detection. Solutions such as an (Elastic) ELK stack have largely been adopted by small and large organizations for data ingestion, storage and visualization. Although, it might seem that collecting a massive amount of data is all analysts need to do their jobs, there are several challenges for them when faced with large, unstructured and often incomplete/disparate data sets. In addition to the sisyphean task of detecting and responding to adversaries there may be pitfalls with organizational funding, support, and or approval (Government). Although “everyone” is collecting logs and despite the many challenges, we will show you how to make sense of these logs in an efficient and consistent way. Specifically when it comes to Windows Event logs (ie: Sysmon, PowerShell, etc) and the ability to map fields to other logs such as Bro NSM or some other network monitoring/prevention device. This will include different Windows Event log data normalization techniques across the 1,000+ unique Event IDs and its 3,000+ unique fields. Also, proven data normalization techniques such as hashing fields/values for logs such as PowerShell, Scheduled Tasks, Command Line, and more. These implementations will show how it allows an analyst to efficiently “pivot” from an endpoint log to a NSM log or a device configuration change log. However, we will also show how an analyst can make an informed decision without degrading/hindering their investigation as well as to enhance their decision. Whether this is preventing an analyst from excluding keywords that a malicious actor may include as an “evasion” technique or adding additional analysis techniques (ie: graphing).
  • Breach Detection/Response
    • Articles/Blogposts/Presentations/Talks/Writeups
    • Tools
      • Infection Monkey
        • The Infection Monkey is an open source security tool for testing a data center's resiliency to perimeter breaches and internal server infection. The Monkey uses various methods to self propagate across a data center and reports success to a centralized Command and Control(C&C) server.
      • 411
        • Configure Searches to periodically run against a variety of data sources. You can define a custom pipeline of Filters to manipulate any generated Alerts and forward them to multiple Targets.
      • Pattern
        • Pattern is a web mining module for Python. It has tools for: Data Mining: web services (Google,; Twitter, Wikipedia), web crawler, HTML DOM parser; Natural Language Processing: part-of-speech taggers, n-gram search, sentiment analysis, WordNet; Machine Learning: vector space model, clustering, classification (KNN, SVM, Perceptron); Network Analysis: graph centrality and visualization.
  • Building a ___(/Lab/)
  • Infrastructure Monitoring
  • General Tools
    • General
      • Security Onion
        • Security Onion is a Linux distro for intrusion detection, network security monitoring, and log management. It's based on Ubuntu and contains Snort, Suricata, Bro, OSSEC, Sguil, Squert, Snorby, ELSA, Xplico, NetworkMiner, and many other security tools. The easy-to-use Setup wizard allows you to build an army of distributed sensors for your enterprise in minutes!
    • Data Tranformation
      • Pip3line, the Swiss army knife of byte manipulation
        • Pip3line is a raw bytes manipulation utility, able to apply well known and less well known transformations from anywhere to anywhere (almost).
      • dnstwist
        • Domain name permutation engine for detecting typo squatting, phishing and corporate espionage
    • DNS
      • DNSChef
        • DNSChef is a highly configurable DNS proxy for Penetration Testers and Malware Analysts. A DNS proxy (aka "Fake DNS") is a tool used for application network traffic analysis among other uses. For example, a DNS proxy can be used to fake requests for "badguy.com" to point to a local machine for termination or interception instead of a real host somewhere on the Internet.
      • Passive DNS
        • A tool to collect DNS records passively to aid Incident handling, Network Security Monitoring (NSM) and general digital forensics. * PassiveDNS sniffs traffic from an interface or reads a pcap-file and outputs the DNS-server answers to a log file. PassiveDNS can cache/aggregate duplicate DNS answers in-memory, limiting the amount of data in the logfile without losing the essense in the DNS answer.
    • HTTP Traffic
      • Captipper
        • CapTipper is a python tool to analyze, explore and revive HTTP malicious traffic. CapTipper sets up a web server that acts exactly as the server in the PCAP file, and contains internal tools, with a powerful interactive console, for analysis and inspection of the hosts, objects and conversations found.
    • PCAPs/Packet Capture
      • CapLoader
        • CapLoader is a Windows tool designed to handle large amounts of captured network traffic. CapLoader performs indexing of PCAP/PcapNG files and visualizes their contents as a list of TCP and UDP flows. Users can select the flows of interest and quickly filter out those packets from the loaded PCAP files. Sending the selected flows/packets to a packet analyzer tool like Wireshark or NetworkMiner is then just a mouse click away.
      • Netdude
        • The Network Dump data Displayer and Editor is a framework for inspection, analysis and manipulation of tcpdump trace files. It addresses the need for a toolset that allows easy inspection, modification, and creation of pcap/tcpdump trace files. Netdude builds on any popular UNIX-like OS, such as Linux, the BSDs, or OSX.
      • Stenographer
        • Stenographer is a full-packet-capture utility for buffering packets to disk for intrusion detection and incident response purposes. It provides a high-performance implementation of NIC-to-disk packet writing, handles deleting those files as disk fills up, and provides methods for reading back specific sets of packets quickly and easily.
      • PCAPDB
        • PcapDB is a distributed, search-optimized open source packet capture system. It was designed to replace expensive, commercial appliances with off-the-shelf hardware and a free, easy to manage software system. Captured packets are reorganized during capture by flow (an indefinite length sequence of packets with the same src/dst ips/ports and transport proto), indexed by flow, and searched (again) by flow. The indexes for the captured packets are relatively tiny (typically less than 1% the size of the captured data).
      • Network Miner
        • NetworkMiner is a Network Forensic Analysis Tool (NFAT) for Windows (but also works in Linux / Mac OS X / FreeBSD). NetworkMiner can be used as a passive network sniffer/packet capturing tool in order to detect operating systems, sessions, hostnames, open ports etc. without putting any traffic on the network. NetworkMiner can also parse PCAP files for off-line analysis and to regenerate/reassemble transmitted files and certificates from PCAP files.
      • SilLK
        • Silk
          • The SiLK analysis suite is a collection of command-line tools for processing SiLK Flow records created by the SiLK packing system. These tools read binary files containing SiLK Flow records and partition, sort, and count these records. The most important analysis tool is rwfilter, an application for querying the central data repository for SiLK Flow records that satisfy a set of filtering options. The tools are intended to be combined in various ways to perform an analysis task. A typical analysis uses UNIX pipes and intermediate data files to share data between invocations of the tools.
        • Administering/Installing SiLK
        • SiLK Tool Tips
        • SiLK Reference Guide
        • SiLK Toolsuite Quick Reference Guide
        • flowbat
          • Awesome flow tool, SiLK backend
    • ShellCode Analysis
      • Shellcode Analysis Pipeline
        • I recently required an automated way of analyzing shellcode and verifying if it is detected by Libemu, Snort, Suricata, Bro, etc. Shellcode had to come from public sources like Shell-Storm, Exploit-DB and Metasploit. I needed an automated way of sourcing shellcode from these projects and pass it on to the analysis engines in a pipeline-like mechanism. This posts documents the method I used to complete this task and the overall progress of the project.
  • Logging
  • Threat Hunting
  • Traffic Analysis
    • Behavioral Analysis using DNS, Network Traffic and Logs, Josh Pyorre (@joshpyorre)
      • Multiple methods exist for detecting malicious activity in a network, including intrusion detection, anti-virus, and log analysis. However, the majority of these use signatures, looking for already known events and they typically require some level of human intervention and maintenance. Using behavioral analysis methods, it may be possible to observe and create a baseline of average behavior on a network, enabling intelligent notification of anomalous activity. This talk will demonstrate methods of performing this activity in different environments. Attendees will learn new methods which they can apply to further monitor and secure their networks
    • DNS
    • SMB
    • TLS
      • TLS client fingerprinting with Bro
      • JA3 - A method for profiling SSL/TLS Clients
        • JA3 is a method for creating SSL/TLS client fingerprints that are easy to produce and can be easily shared for threat intelligence.
        • Talk/Presentation
          • In this talk we will show the benefits of SSL fingerprinting, JA3’s capabilities, and how best to utilize it in your detection and response operations. We will show how to utilize JA3 to find and detect SSL malware on your network. Imagine detecting every Meterpreter shell, regardless of C2 and without the need for SSL interception. We will also announce JA3S, JA3 for SSL server fingerprinting. Imagine detecting every Metasploit Multi Handler or [REDACTED] C2s on AWS. Then we’ll tie it all together, making you armed to the teeth for detecting all things SSL.
    • Tools
      • Frameworks
      • General
        • DNSpop
          • Tools to find popular trends by analysis of DNS data. For more information, see my blog post on the most popular subdomains on the internet. Hit the results directory to get straight to the data.
        • Yeti
          • Yeti is a platform meant to organize observables, indicators of compromise, TTPs, and knowledge on threats in a single, unified repository. Yeti will also automatically enrich observables (e.g. resolve domains, geolocate IPs) so that you don't have to. Yeti provides an interface for humans (shiny Bootstrap-based UI) and one for machines (web API) so that your other tools can talk nicely to it.
        • Malcom - Malware Communication Analyzer
          • Malcom is a tool designed to analyze a system's network communication using graphical representations of network traffic, and cross-reference them with known malware sources. This comes handy when analyzing how certain malware species try to communicate with the outside world.
        • BeaconBits
          • Beacon Bits is comprised of analytical scripts combined with a custom database that evaluate flow traffic for statistical uniformity over a given period of time. The tool relies on some of the most common characteristics of infected host persisting in connection attempts to establish a connection, either to a remote host or set of host over a TCP network connection. Useful to also identify automation, host behavior that is not driven by humans.
  • IDS/IPS Tools
  • IDS/IPS Monitoring Tools
    • Snorby
    • Snorby - Github
      • Snorby is a ruby on rails web application for network security monitoring that interfaces with current popular intrusion detection systems (Snort, Suricata and Sagan). The basic fundamental concepts behind Snorby are simplicity, organization and power. The project goal is to create a free, open source and highly competitive application for network monitoring for both private and enterprise use.
    • Squil
      • Sguil (pronounced sgweel) is built by network security analysts for network security analysts. Sguil's main component is an intuitive GUI that provides access to realtime events, session data, and raw packet captures. Sguil facilitates the practice of Network Security Monitoring and event driven analysis. The Sguil client is written in tcl/tk and can be run on any operating system that supports tcl/tk (including Linux, * BSD, Solaris, MacOS, and Win32).
      • Squil FAQ
    • Squert
      • Squert is a web application that is used to query and view event data stored in a Sguil database (typically IDS alert data). Squert is a visual tool that attempts to provide additional context to events through the use of metadata, time series representations and weighted and logically grouped result sets. The hope is that these views will prompt questions that otherwise may not have been asked.
      • Slide Deck on Squert
      • Install/setup/etc - Github
  • ELK Stack