Notes from November.2016 meeting

Thanks to everyone that came out to The Learning Academy for today’s meeting.  We had a great meeting with lots of discussion.

  • I brought up the idea of getting together at the beginning of December for a happy hour.  A couple people seemed interested.  I will send out an email to everyone in a few weeks to see who is interested.
  • Adam Tice with Splunk gave us an overview of what is new in Splunk 6.5
  • I showed building a rsyslog server and getting the logs in to Splunk
    • create sheet available below
  • The grouped talk about building your own dashboard by taking code from other application
    • Using other applications searches to build the dashboard of just want you want
    • Combining different dashboards in to a single dashboard
  • The group also talked to the students and each other about opportunities in security and specially in Splunk in the workforce


### Build your own rsyslog and send all the data to Splunk ###

In this test/demo environment I used three CentOS 7 machines on VMware Workstation.  This sheet is focusing on the rsyslog server machine and assumes that you have the client setup to forward syslog to the server and that the Splunk indexer is setup and set to receive Splunk data from a universal forwarder.


Splunk Indexer =
RSyslog Remote =
RSyslog Server =


## On the rsyslog server ##

# Make sure that we have the latest updates installed #
yum update -y

# Install the rsyslog and the documents for rsyslog
yum install rsyslog rsyslog-doc

# Edit the rsyslog config file
vi /etc/rsyslog.conf

# remove the comment (#) for the following lines:
$ModLoad imudp
$UDPServerRun 514

$ModLoad imtcp
$InputTCPServerRun 514

# Add the firewall rules to accept the data from other machines / devices
firewall-cmd –zone=public –permanent –add-port=514/tcp
firewall-cmd –zone=public –permanent –add-port=514/udp
firewall-cmd –reload

# Restart the rsyslog service
systemctl restart rsyslog

# Check to make sure that the system is responding to the ports
netstat -antup | grep 514

# You should see something like the following:



# Now watch the log file (/var/log/messages) to see if you are getting syslog data from the remote machine
tail -f /var/log/messages

## On the rsyslog remote ##

# Setup / Verify that it is set up to send syslog to the rsyslog server
vi /etc/rsyslog.conf

# Typically it will look like the following:
*.* @remote-host:514

# Then we can write a message to the message file to be sent
logger -t Splunk402 Splunk Rules!

## On the rsyslog server ##

# You should see a message pop up like:
Nov 16 12:17:33 remotemachine01 Splunk402: Splunk Rules!

# Now we want to install the Splunk Universal Forwarder.  Go to and select .rpm version that is the same as the OS for you.  If you are not logged in to the site, you will need to login or sign up for an account.  Click on the “Download via Command Line (wget)” and get the wget code.
wget -O splunkforwarder-6.5.0-59c8927def0f-linux-2.6-x86_64.rpm ‘’

# Install Splunk with YUM
yum –nogpgcheck localinstall splunkforwarder-6.5.0-59c8927def0f-linux-2.6-x86_64.rpm

# Start Splunk accepting the license
/opt/splunkforwarder/bin/splunk start –accept-license

# Enable Splunk to start at boot
/opt/splunkforwarder/bin/splunk enable boot-start

# If you are not using a Deployment Server to set the output.conf file, edit the file to send the data to Splunk
vi /opt/splunkforwarder/etc/system/local/outputs.conf



# Have Splunk grab the data from the /var/log folder
vi /opt/splunkforwarder/etc/system/local/inputs.conf


# Depending on your needs, you may need to add which index to send to or other factors.  Please consult the docs for inputs.conf.

# Now we need to restart Splunk to have all of our settings go in to effect.
/opt/splunkforwarder/bin/splunk restart

## You should now see the logs in Splunk.  Happy Splunking! ##