The Elasticsearch, Kibana, Logstash (ELK) stack has become very popular recently for cheap and easy centralized logging. The developer of Logstash, Jordan Sissel, was recently hired by Elasticsearch which has led to some great things for the future of Logstash, my favorite of which is that Elasticsearch now provides package feeds for Logstash. This makes getting everything set up a lot easier!

I'm writing this guide as I set ELK up to capture Event Logs from some Windows Server 2008 boxes, and a Sophos UTM (Astaro) firewall. To capture the Event Logs, I'm using nxlog-ce to serialize the Event Logs to JSON and send them to Logstash.

I'm setting ELK up on an older Dell Precision T3400 workstation running a fresh install of Ubuntu Server 14.04 LTS.

Personal note: For whatever reason, Ubuntu still doesn't ship with htop. That's always the first thing that I install: sudo aptitiude install htop

Let's get started!

First, I'm going to add the repositories for Elasticsearch and Logstash. From the Elasticsearch site:
Download and install the Public Signing Key:

wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -  

NOTE: You probably won't see the password prompt for sudo, so just type your password and hit enter when wget looks done.

Edit /etc/apt/sources.list and add the Elasticsearch and Logstash repositories to the end of the file:

deb http://packages.elasticsearch.org/elasticsearch/1.1/debian stable main  
deb http://packages.elasticsearch.org/logstash/1.4/debian stable main  

Don't forget to run a quick sudo aptitude update after adding these new repositories!

Now to actually install some things!

To get started, I'm going to install Logstash, who's current version at the time I'm writing this is 1.4.1.

sudo aptitude install logstash  

Logstash (and Elasticsearch) requires Java, so this will take some time depending on your internet connection

Logstash requires specific versions of Elasticsearch (for the native Elasticsearch output), and for v1.4.1 that's ES 1.1.1. So I'll install that next:

sudo aptitude install elasticsearch=1.1.1  

As you might have noticed in the Aptitude output, Elasticsearch won't be started at boot, by default. To fix this, run their suggested command:

sudo update-rc.d elasticsearch defaults 95 10  

Next, I'm going to tell APT to not upgrade Logstash or Elasticsearch because there could be breaking changes between versions.

sudo aptitude hold elasticsearch logstash  

Now, a quick test to make sure that Elasticsearch was installed correctly and works. To do that, start Elasticsearch with sudo service elasticsearch start and run curl http://localhost:9200. Curl should output something like:

{
  "status" : 200,
  "name" : "Karthon the Quester",
  "version" : {
    "number" : "1.1.1",
    "build_hash" : "f1585f096d3f3985e73456debdc1a0745f512bbc",
    "build_timestamp" : "2014-04-16T14:27:12Z",
    "build_snapshot" : false,
    "lucene_version" : "4.7"
  },
  "tagline" : "You Know, for Search"
}

Last, but definitely not least, we need to install a web server to host Kibana. I'm going to be using nginx, so:

sudo aptitude install nginx  

Quick overview of important locations for files and what-not:

Elasticsearch:

• Binaries and stuff: /usr/share/elasticsearch
• Plugin manager: /usr/share/elasticsearch/bin/plugin
• Configuration: /etc/elasticsearch/elasticsearch.yml
• Data: /var/lib/elasticsearch/<cluster-name>

Logstash

• Binaries and stuff: /opt/logstash
• Configuration: /etc/logstash/conf.d
• Logs: /var/log/logstash

Kibana

Kibana is 'just' some html and javascript, so download the tarball and extract it to where you want to put it. I'm going to put it in /srv/www/kibana, so I had to make those directories and change their owner to www-data:

sudo mkdir -p /srv/www/kibana  
wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.0.tar.gz  
sudo tar xf kibana-3.1.0.tar.gz -C /srv/www/  
sudo chown -R www-data:www-data /srv/www/  

So, kibana will actually be extraced to /srv/www/kibana-3.1.0 and that's fine with me.

Configurations

Elasticsearch

ES has sane defaults that work well for Logstash, so I'm pretty much going to leave it alone. But, there are some plugins that I like to have installed:

Bigdesk:

In simple words bigdesk makes it very easy to see how your Elasticsearch cluster is doing. Just install it as an Elasticsearch plugin, download locally or run online from the web, then point it to the Elasticsearch node REST endpoint and have fun.

To install, run:

sudo /usr/share/elasticsearch/bin/plugin -install lukas-vlcek/bigdesk/2.4.0  

We want v2.4.0 for ES 1.1.1

Bigdesk will be accessible at http://your-es-host:9200/_plugin/bigdesk/

elasticsearch-head:

elasticsearch-head is a web front end for browsing and interacting with an Elastic Search cluster.

To install, run:

sudo /usr/share/elasticsearch/bin/plugin -install mobz/elasticsearch-head  

elasticsearch-head will be accessible at http://your-es-host:9200/_plugin/head/

Kibana

Kibana's defaults work great as well, so I won't be changing its configuration either. But I do have to tell nginx where to find and host Kibana.

/etc/nginx/sites-available/default

server {  
        listen 80 default_server;

        root /srv/www;
        index index.html index.htm;

        # Make site accessible from http://localhost/
        server_name localhost;

        location / {
                try_files $uri $uri/ =404;
        }
        location /kibana {
                alias /srv/www/kibana-3.1.0/;
                try_files $uri $uri/ =404;
        }
}

Don't forget to make a copy of the actual default config, you might want it again later!
Then, a quick sudo service nginx reload will (should) have it working!

Logstash

This is the biggie! When logstash is started using its initscript, it'll simply check /etc/logstash/conf.d for configuration files and load them in. I highly recommend reading through the documentation on Logstash's website, especially the Configuration Overview page. It's pretty good and gives a good overview of how Logstash works, and how to configure it. However, I'll cover my actual Logstash configuration in another post.

Once you've written your configuration, you can test it by running /opt/logstash/bin/logstash -t -f /etc/logstash/conf.d/. Then, when the config-test succeeds, just run sudo service logstash start to get going, and tail -f /var/log/logstash/logstash.log to make sure that everything is hunky-dory (there might not be any output if everything is OK). As you might have noticed when running the config-test, Logstash takes some time to start up, so be patient!

All that's left is to head on over to Kibana, choose the included "Logstash Dashboard" and look at all your pretty logs!