威胁猎杀实战（三）：基于Wazuh, Snort/Suricata和Elastic Stack的SOC
整合HIDS、NIDS和Elastic Stack，在此基础上实现SOC
The Elastic Stack delivers security analytics capabilities that are widely used for threat detection, visibility, and incident response. The speed and scale at which Elasticsearch can index and search security-related information enable security analysts to work more efficiently, while Kibana dashboards provide wide visibility and enable interactive threat hunting. And the machine learning engine can automate the analysis of complex datasets, making it possible to spot intruders that otherwise would’ve gone unnoticed.
Popular Intrusion Detection Systems (IDS), such as Wazuh or Snort/Suricata, use a signature-basedapproach to threat detection. That is, they compare patterns found in files, logs, and network traffic against a database of patterns known to be associated with malicious activity, alerting when a match is found. They provide useful rulesets to analyze and correlate data, usually generating thousands or millions of alerts per day in a production environment.
Casting a wide net can ensure that all potential security events are caught, but it also adds the work of sifting through thousands (or millions) of alerts every day. Elastic machine learning features help reduce the noise by automatically identifying unusual behaviors. This is a clear use case where anomaly-based and signature-based technologies complement each other, making threat detection easier and investigations more efficient.
目录：
1.部署 Snort/Suricata
Ubuntu sudo add-apt-repository ppa:oisf/suricata-stable sudo apt-get update sudo apt-get install suricata RHEL/CentOS yum install epel-release yum install suricata 参考： Suricata https://github.com/tianyulab/dalton/blob/master/dalton-agent/Dockerfiles/Dockerfile_suricata https://suricata.readthedocs.io/en/suricata-4.0.5/install.html Snort https://github.com/tianyulab/SnortCP/blob/master/Scripts/Snort_Wireshark.sh
2.配置Suricata Eve JSON Output
# 配置举例： vi /etc/suricata/suricata.yaml outputs: - eve-log: enabled: yes filetype: regular #regular|syslog|unix_dgram|unix_stream|redis filename: eve.json types: - alert: metadata: yes tagged-packets: yes xff: enabled: yes mode: extra-data - http: extended: yes - dns: query: yes # enable logging of DNS queries answer: yes # enable logging of DNS answers - tls: extended: yes # enable this for extended logging information - files: force-magic: no # force logging magic on all logged files - smtp: extended: yes # enable this for extended logging information - ssh - flow 参考： https://suricata.readthedocs.io/en/suricata-4.0.5/configuration/suricata-yaml.html#eve-extensible-event-format
3.部署 Wazuh Stack
Wazuh stack包含3个组件： 1.Wazuh server: 包含Wazuh manager，API 和 Filebeat（Filebeat仅在分布式架构下使用） 2.Elastic Stack: 包含Elasticsearch，Logstash，Kibana 和 Wazuh Kibana app，读取，解析，索引和存储Wazuh服务器生成的警报数据。 3.Wazuh agent # 分布式架构：在不同主机上运行Wazuh服务器和Elastic Stack集群（一个或多个服务器） https://documentation.wazuh.com/current/_images/installing_wazuh2.png # 单主机架构：在同一系统上运行Wazuh服务器和Elastic Stack https://documentation.wazuh.com/current/_images/installing_wazuh_singlehost2.png # 本文采用分布式架构，分别在Ubuntu 16.04上部署Wazuh server，CentOS 7.x上部署Elastic Stack Ubuntu 16.04 1.部署Wazuh server # 1.添加Wazuh Repositories ## 1.1）安装依赖软件包 apt-get update apt-get -y install curl apt-transport-https lsb-release # if [ ! -f /usr/bin/python ]; then ln -s /usr/bin/python3 /usr/bin/python; fi # 可选 curl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | apt-key add - echo "deb https://packages.wazuh.com/3.x/apt/ stable main" | tee -a /etc/apt/sources.list.d/wazuh.list # 2.安装wazuh-manager apt-get update apt-get -y install wazuh-manager systemctl status wazuh-manager systemctl enable wazuh-manager # 3.安装Wazuh API ## 3.1）安装依赖软件包，NodeJS >= 4.6.1，Python >= 2.7 curl -sL https://deb.nodesource.com/setup_8.x | bash - apt-get -y install nodejs apt-get -y install wazuh-api systemctl status wazuh-api systemctl enable wazuh-api # 4.配置Wazuh Kibana app 参考： https://documentation.wazuh.com/current/user-manual/kibana-app/connect-kibana-app.html # 5.安装Filebeat（分布式架构） curl -s https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add - echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee /etc/apt/sources.list.d/elastic-6.x.list apt-get update apt-get -y install filebeat=6.4.2 curl -so /etc/filebeat/filebeat.yml https://raw.githubusercontent.com/wazuh/wazuh/3.6/extensions/filebeat/filebeat.yml 修改/etc/filebeat/filebeat.yml中ELASTIC_SERVER_IP的值为Elastic Stack服务器IP systemctl daemon-reload systemctl enable filebeat.service systemctl start filebeat.service # 6.验证 systemctl status wazuh-manager systemctl status wazuh-api systemctl status filebeat.service # 7.一键部署脚本 https://github.com/tianyulab/Threat_Hunting_with_ELK/tree/master/HIDS_NIDS_ELK/Deploy_Wazuh_server.sh 2.部署Elastic Stack CentOS 7.x # 1.安装依赖软件包，JRE curl -Lo jre-8-linux-x64.rpm --header "Cookie: oraclelicense=accept-securebackup-cookie" "https://download.oracle.com/otn-pub/java/jdk/8u191-b12/2787e4a523244c269598db4e85c51e0c/jre-8u191-linux-x64.rpm" rpm -qlp jre-8-linux-x64.rpm > /dev/null 2>&1 && echo "Java package downloaded successfully" || echo "Java package did not download successfully" yum -y install jre-8-linux-x64.rpm rm -f jre-8-linux-x64.rpm # 2.安装elasticsearch、logstash、kibana rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch cat > /etc/yum.repos.d/elastic.repo << EOF [elasticsearch-6.x] name=Elasticsearch repository for 6.x packages baseurl=https://artifacts.elastic.co/packages/6.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md EOF yum install elasticsearch-6.4.2 systemctl daemon-reload systemctl enable elasticsearch.service systemctl start elasticsearch.service curl "localhost:9200/?pretty" # Load the Wazuh template for Elasticsearch: curl https://raw.githubusercontent.com/wazuh/wazuh/3.6/extensions/elasticsearch/wazuh-elastic6-template-alerts.json | curl -XPUT 'http://localhost:9200/_template/wazuh' -H 'Content-Type: application/json' -d @- yum install logstash-6.4.2 # Download the Wazuh configuration file for Logstash: curl -so /etc/logstash/conf.d/01-wazuh.conf https://raw.githubusercontent.com/wazuh/wazuh/3.6/extensions/logstash/01-wazuh-remote.conf systemctl daemon-reload systemctl enable logstash.service systemctl start logstash.service yum install kibana-6.4.2 export NODE_OPTIONS="--max-old-space-size=3072" sudo -u kibana /usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.6.1_6.4.2.zip # 参考：https://github.com/wazuh/wazuh-kibana-app#installation # /etc/kibana/kibana.yml # 可选 # server.host: "0.0.0.0" systemctl daemon-reload systemctl enable kibana.service systemctl start kibana.service # 3.验证 curl "localhost:9200/?pretty" systemctl status logstash.service systemctl status kibana.service # 4.一键部署脚本 https://github.com/tianyulab/Threat_Hunting_with_ELK/tree/master/HIDS_NIDS_ELK/Deploy_Elastic_Stack.sh
4.在Suricata服务器上安装Wazuh Agent
Ubuntu 16.04 # 1.部署Wazuh Agent apt-get -y install curl apt-transport-https lsb-release curl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | apt-key add - echo "deb https://packages.wazuh.com/3.x/apt/ stable main" | tee /etc/apt/sources.list.d/wazuh.list apt-get update apt-get -y install wazuh-agent # 2.注册Wazuh Agent # Wazuh Manager 上执行： openssl req -x509 -batch -nodes -days 365 -newkey rsa:2048 -keyout /var/ossec/etc/sslmanager.key -out /var/ossec/etc/sslmanager.cert /var/ossec/bin/ossec-authd -i # Wazuh Agent 上执行： sed -i "s/MANAGER_IP/8.8.8.8/" /var/ossec/etc/ossec.conf /var/ossec/bin/agent-auth -m 8.8.8.8 systemctl restart wazuh-agent # 参考： https://documentation.wazuh.com/current/user-manual/registering/index.html # 3.验证 systemctl status wazuh-agent # 4.一键部署脚本 https://github.com/tianyulab/Threat_Hunting_with_ELK/tree/master/HIDS_NIDS_ELK/Deploy_Wazuh_agent.sh # 注：此脚本为交互模式
5.在Wazuh Manager服务器上配置Wazuh rules处理Suricata日志
sed -i 's/id="86600" level="0"/id="86600" level="4"/g' /var/ossec/ruleset/rules/0475-suricata_rules.xml sed -i 's/id="86602" level="0"/id="86602" level="4"/g' /var/ossec/ruleset/rules/0475-suricata_rules.xml sed -i 's/id="86603" level="0"/id="86603" level="4"/g' /var/ossec/ruleset/rules/0475-suricata_rules.xml sed -i 's/id="86604" level="0"/id="86604" level="4"/g' /var/ossec/ruleset/rules/0475-suricata_rules.xml systemctl restart wazuh-manager.service # /var/ossec/bin/ossec-control restart
6.在Suricata服务器上配置Wazuh Agent读取Suricata的eve.json文件
vi /var/ossec/etc/ossec.conf # 在<ossec_config> tag里添加如下内容 # Modify ossec.conf - read localfile suricata EVE json log <localfile> <log_format>syslog</log_format> <location>/var/log/suricata/eve.json</location> </localfile> # 验证 systemctl restart wazuh-agent systemctl status wazuh-agent
7.在Elastic Stack上配置wazuh logstash filter
# 1.wazuh logstash filter 配置
在Elastic Stack服务器上执行，
vi /etc/logstash/conf.d/01-wazuh.conf
# 新增以下内容
filter {
if [data][src_ip] {
mutate{
add_field => [ "[data][srcip]","%{[data][src_ip]}"]
remove_field => [ "[data][src_ip]" ]
}
}
if [data][dest_ip] {
mutate{
add_field => [ "[data][dstip]","%{[data][dest_ip]}"]
remove_field => [ "[data][dest_ip]" ]
}
}
if [data][dest_port] {
mutate{
add_field => [ "[data][dstport]","%{[data][dest_port]}"]
remove_field => [ "[data][dest_port]" ]
}
}
if [data][src_port] {
mutate{
add_field => [ "[data][srcport]","%{[data][src_port]}"]
remove_field => [ "[data][src_port]" ]
}
}
}
# 配置验证
/usr/share/logstash/bin/logstash -f 01-wazuh.conf --config.test_and_exit
# 重启Logstash服务
systemctl restart logstash.service
8.Wazuh + Snort/Suricata 联动（active response）
# 1.Snort # 修改snort输出为alert_fast： vi /etc/snort/snort.conf output alert_fast: snort.log 128M systemctl restart snort # 配置Wazuh agent vi /var/ossec/etc/ossec.conf <localfile> <log_format>snort-full</log_format> <location>/var/log/snort/snort.log</location> </localfile> systemctl restart wazuh-agent # 配置Wazuh Manager vi /var/ossec/etc/ossec.conf # 添加如下内容： <!-- Active response --> <global> <white_list>127.0.0.1</white_list> <white_list>^localhost.localdomain$</white_list> <white_list>8.8.8.8</white_list> </global> <command> <name>disable-account</name> <executable>disable-account.sh</executable> <expect>user</expect> <timeout_allowed>yes</timeout_allowed> </command> <command> <name>restart-ossec</name> <executable>restart-ossec.sh</executable> <expect></expect> </command> <command> <name>firewall-drop</name> <executable>default-firewall-drop.sh</executable> <expect>srcip</expect> <timeout_allowed>yes</timeout_allowed> </command> <command> <name>host-deny</name> <executable>host-deny.sh</executable> <expect>srcip</expect> <timeout_allowed>yes</timeout_allowed> </command> <command> <name>route-null</name> <executable>route-null.sh</executable> <expect>srcip</expect> <timeout_allowed>yes</timeout_allowed> </command> <command> <name>win_route-null</name> <executable>route-null.cmd</executable> <expect>srcip</expect> <timeout_allowed>yes</timeout_allowed> </command> <command> <name>win_route-null-2012</name> <executable>route-null-2012.cmd</executable> <expect>srcip</expect> <timeout_allowed>yes</timeout_allowed> </command> <command> <name>netsh</name> <executable>netsh.cmd</executable> <expect>srcip</expect> <timeout_allowed>yes</timeout_allowed> </command> <command> <name>netsh-win-2016</name> <executable>netsh-win-2016.cmd</executable> <expect>srcip</expect> <timeout_allowed>yes</timeout_allowed> </command> <!-- Active Response Config --> <active-response> <!-- This response is going to execute the host-deny - command for every event that fires a rule with - level (severity) >= 6. - The IP is going to be blocked for 600 seconds. --> <command>host-deny</command> <location>local</location> <level>12</level> <timeout>600</timeout> </active-response> <active-response> <!-- Firewall Drop response. Block the IP for - 600 seconds on the firewall (iptables, - ipfilter, etc). --> <command>firewall-drop</command> <location>local</location> <level>12</level> <timeout>600</timeout> </active-response> <!-- Snort active response 配置 --> <active-response> <command>firewall-drop</command> <location>local</location> <rules_id>20101</rules_id> <timeout>600</timeout> </active-response> <active-response> <command>host-deny</command> <location>local</location> <rules_id>20101</rules_id> <timeout>600</timeout> </active-response> # 重启服务， systemctl restart wazuh-manager.service 参考： https://groups.google.com/forum/#!msg/wazuh/8cu1hZ9PHCM/RiPK41gWAgAJ https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/localfile.html?highlight=localfile#log-format /var/ossec/ruleset/decoders/0285-snort_decoders.xml /var/ossec/ruleset/rules/0240-ids_rules.xml https://github.com/wazuh/wazuh-ruleset/blob/master/decoders/0285-snort_decoders.xml # 2.Suricata 省略 参考： https://github.com/wazuh/wazuh/issues/202 https://documentation.wazuh.com/current/user-manual/ruleset/custom.html https://documentation.wazuh.com/current/user-manual/ruleset/json-decoder.html 调试/测试工具： /var/ossec/bin/ossec-logtest
效果图：
9.未完待续
1.机器学习实例 2.Wazuh和Snort/Suricata事件关联 3.下期我们会讲一讲Wazuh