Wrangling Logs with Logstash and Elasticsearch at OSCON

by Nate Jones and David Castro

Here you will find all you need to get started playing with the following tools:

Some seriously awesome tools. Thanks to all the developers!

From scratch

If you are familiar with vagrant and/or want to have an identical environment to what was demonstrated during the presentation, building the VM from scratch is the way to go.

Install Vagrant and then:

$ git clone https://github.com/mediatemple/log_wrangler.git
$ cd log_wrangler
$ PROVISION=1 vagrant up

After bootup, the following ports are open:

Port Function Info
6999 Logstash netcat port Pipe logs in with:
cat [logfile] | nc localhost 6999
8080 Kibana http://localhost:8080/kibana/
9200 Elasticsearch http://localhost:9200/_plugin/head/
55672 RabbitMQ management console http://localhost:55672/#/queues/%2F/elasticsearch
Login: guest/guest

Pre-built VM

Download a pre-built VM to get up and running even faster.

After downloading, you will need to log into the VM (vagrant/vagrant) and run /sbin/ifconfig to discover the IP address. Then use that IP instead of localhost to access the VM.

If you do not already have a host-only adapter in VirtualBox, you must add one in File -> Settings -> Network (Mac: Preferences -> Network). Ensure the adapter has DHCP enabled.

This VM's firewall allows all access to ports 22, 80, 6999, 9200, and 55672, so if you configure the VM with bridged networking, other people will be able to access it.

Download Now »

Resources

Other stuff we put together ourselves.

OSCon 2012 Slides

RPM Package Repository

Vagrant CentOS 6.2 Basebox