How to configure log collection from Cisco FirePower to Splunk
In this article we are going to describe the process of connecting FirePower Threat Defense with Splunk in case of using Firepower Management Center.
First of all, we are about to share some notes about preparation to this task. When the necessity of log collection from Firepower appeared, guys who did it before said that it was a really difficult task. There was an add-on which was written in Perl and during the configuration process you received too many errors and had no idea how to manage it. But when we started reviewing possible methods, we found new opportunities to provide this.
So let’s review possible methods of sending logs from FTD to Splunk. There are two variants: through syslog and through estreamer.
If we are talking about syslog, so first of all it’s not very reliable way to send logs. Even Splunk doesn’t advise you to use it, if there is another way in place. On the other hand we should manually create all necessary alerts via Firepower Management Center. And as we read on forums, if we use syslog there, less dashboards will be riched by default.
The other way is sending logs via eStreamer. When we firstly heard about this methods, there were many problems with perl modules, and other technology which was used by the Splunk estreamer Add-on. Let’s have a closer look at eStreamer.
The FireSIGHT System Event Streamer (eStreamer) uses a message-oriented protocol to stream events and host profile information to the client application. Your client can request event and host profile data from a Defense Center, and intrusion event data only from a managed device. The client application initiates the data stream by submitting request messages, which specify the data to be sent, and then controls the message flow from the Defense Center or managed device after streaming begins. More details you can find here. The other feature of using this method is that communication between devices is encrypted over SSL.
That’s why there are many recommendations to use estreamer protocol for log collection instead of syslog. So we are about to share our experience of configuration log collection based on estreamer protocol.
First of all we found the Splunk Add-on for eStreamer. There are two versions:
- Cisco eStreamer for Splunk, which uses perl modules and its support SourceFire system version 5.2+ and Splunk 6.
- eStreamer Splunk solution which is available for Firepower customers running FMC version 6.x – Splunk eStreamer eNcore add-on and app.
As we deal with Firepower and FMC version 6.+. We start working with second add-on which was written in Python.
There is one important thing here that FTD and FMC should be in one network as Splunk with eStreamer add-on. Here could appear one interesting thing, if you have installed Splunk in the cloud (such as AWS, Azure, Google Cloud) and have an office, which is located in a business center where your local network is hosted behind the NAT with one white IP address for many companies. Your Splunk wouldn’t have the possibility to communicate with your device directly.
There are two possible solution methods here:
- The first one is to organise Destination NAT, or port forwarding on core router for your FTD appliance. But we haven’t tested it, so maybe it won’t work.
- The other one is to create Splunk Heavy Forwarder in your corporate network where was the possibility for add-on to access FTD and FMC devices directly.
As we understand the version of Splunk eStreamer eNcore add-on and app (the new one) is developed for second scenario. Because the add-on installs on the heavy forwarder and provides only log collection, the other part is eStreamer eNcore App which provides log transformation, data model log mapping to CIM, and consist of many dashboards for monitoring.
So let’s go to some configuration. On the picture below we describe how the possible infrastructure looks like.
Before the start, we should have configured Splunk Heavy Forwarder. In our case, we have installed it on Ubuntu server, because eStreamer eNcore add-on works only on Linux systems.
We should check if all required packets and libs are installed. They are:
- python 2.7
If it isn’t in place you can simply install it using your packet manager.
After that, we are going to download eNcore add-on http://apps.splunk.com/app/3662 and install it on HF.
You must have a valid PKCS12 file for your Splunk server. Once you have the PKCS12 file you must rename the file to “client.pkcs12” and place it on the Splunk server here:
This will require some form of SSH, SCP or console access to the server.
Let’s describe the process of creation the key for eStreamer on FMC. For that go to your FMC and navigate System->Integration -> eStreamer check out what type of events you want to log and save.
Next visit Splunk web face. Before making any further steps be certain that all necessary inputs are enabled. Go to the Settings -> Data inputs:
- In File & directories check $SPLUNK_HOME /etc/apps/TA-eStreamer/data is enabled.
- After that go to the Script and check that all script inputs from TA–eStreamer app are enabled.
- Add your FMC IP address.
- Check out the box near Process PKCS12 file and type your password from the previous step.
- Define what data should be collected.
- Click save.
“Encountered the following error while trying to update: Error while posting to url=/servicesNS/nobody/TA-eStreamer/encore/configure/main”
go to Troubleshooting Example 1 and provide all described steps.
The eNcore eStreamer app has to be installed on the Search Head. You can complete that through web interface and App management by installing from Splunkbase directly or from file on your local computer.
Make sure that you have created indexer named as estreamer.
https://underdefense.com/msi-packages/https://underdefense.com/msi-packages/The app provides you with many useful dashboards so you can easily use them for your needs. It’s possible that you could have some problems with data model mapping to CIM. If it’s necessary, you can look out to Example 2 where we provide the solution for how to fix that.
In this article, we try to clarify the process of connecting Cisco Firepower Threat Defense with Splunk for log analysis and event correlation with events from other devices in the infrastructure. We describe different methods of log collection, define the pros and cons of them and provide the instructions how to do that using eNcore eStreamer Add-on and App for Splunk. Moreover, we describe troubleshooting of probable errors which could occur during the configuration process.
Read more handy dandy articles from UD team on:
Make sure your certificate file is named client.pkcs12
Check if the addon scripts are enabled on your Splunk HF:
Go to the:
Open the configure.sh script
Here could be the problem with basepath at #SPLUNK_HOME variable don’t work.
You should replace the raw:
basepath = SPLUNK_HOME/etc/apps/TA-ESTREamer …
with next raw:
basepath = /opt/splunk/etc/apps/TA-ESTREamer …
like in the picture below
- exec &>configuration.log
- exec >>configuration.log 2>&1
Rerun the setup process – manually run this script
To fix the problem with data model mapping, if you use eNcore app version 3.0.0:
Download it from Splunkbase, unzip the file and navigate the default folder.
Check the tags.conf and eventtypes.conf.
To provide that – replace “-” with “_”
And finally your files have looks like on the picture below.
To check the result navigate Data models, find Network Traffic data model and open Pivot. But previously edit the constraints in Network Traffic data model
especially search macros that are used in constraint.
For that go to Advanced Search-> Search macros
Find the cim_Network_Traffic_indexes macros edit it and add “OR index=estreamer” to the search statement.
Open The Networh Traffic datamodel in the Pivot add to filter sourcetype=cisco*. And you should receive any result. Or can visualise smth, like in the picture below.