Graylog2 on centos 6.2: easy visual log parsing

Here is a quick install for graylog2. This tool is a bit like sawmill or splunk but fully free ! You may discover some about your infrastructure today

Install Centos like usual, graphical interface is not needed

 Ensure you have correct DNS and reverse set-up

This is a fine way to ensure all name resolution will be done properly. In my case :

# host `hostname`
graylog.feedback.eu.org has address 172.16.86.34
graylog.feedback.eu.org has IPv6 address 2001:470:b6db:0:20c:29ff:fe54:ddea
# host 172.16.86.34
34.86.16.172.in-addr.arpa domain name pointer graylog.feedback.eu.org.
# host 2001:470:b6db:0:20c:29ff:fe54:ddea
a.e.d.d.4.5.e.f.f.f.9.2.c.0.2.0.0.0.0.0.b.d.6.b.0.7.4.0.1.0.0.2.ip6.arpa domain name pointer graylog.feedback.eu.org.

You may not have IPv6

 Add the secondary software repositories

We will need EPEL and passenger repositories. EPEL is a must for me as it provides nearly all that lack on main repository. Passenger is for the Apache ruby module. It will be needed for the web set-up

# rpm -ivh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpm
[...]
1:epel-release ########################################### [100%]
[root@graylog ~]# yum install http://passenger.stealthymonkeys.com/rhel/6/passenger-release.noarch.rpm
...

Install all needed package

#yum -y install lsb gcc make java-1.6.0-openjdk mongodb-server rubygems.noarch ruby-devel mod_passenger
[...]

Installation can be a bit long… but this way, all prerequisites are installed in one time

 Configure mongodb

Ok only 2 lines need to be changed :

# cat /etc/mongodb.conf
#bind_ip = 127.0.0.1
auth = true

 Start mongo DB and configure accesses

Mongo is a standard service to run :

# /etc/init.d/mongod start
Starting mongod: [ OK ]
# chkconfig mongod on

To give access, we need to create db and users :

# mongo
MongoDB shell version: 1.8.2
connecting to: test
> use admin
switched to db admin
> db.addUser('admin', 'xxx')
{
"user" : "admin",
"readOnly" : false,
"pwd" : "xxx"
}
> db.auth('admin', 'xxx')
1
> use graylog2
switched to db graylog2
> db.addUser('grayloguser', 'xxx')
{
"user" : "grayloguser",
"readOnly" : false,
"pwd" : "xxx"
}
> exit
bye

 Install elasticsearch

This new prerequisite does not exist yet as RPM

# wget https://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.18.6.zip
# unzip /home/fjacquet/elasticsearch-0.18.6.zip
Archive: /home/fjacquet/elasticsearch-0.18.6.zip
creating: elasticsearch-0.18.6/
[...]
inflating: elasticsearch-0.18.6/bin/plugin 

# /usr/local/elasticsearch-0.18.6/bin/elasticsearch

Will need to make a service wrapper …

Extract graylog2 server

Get the software

# wget https://github.com/downloads/Graylog2/graylog2-server/graylog2-server-0.9.6.tar.gz
# tar xzvf /tmp/graylog2-server-0.9.6.tar.gz
[...]
# cp graylog2.conf.example graylog2.conf
# mv graylog2-server-0.9.6/ /usr/local/

Trick the configuration location

# ln -s /usr/local/graylog2-server-0.9.6/graylog2.conf /etc/

Fix the root folder

# vim /usr/local/graylog2-server-0.9.6/bin/graylog2ctl
$NOHUP java -jar /usr/local/graylog2-server-0.9.6/graylog2-server.jar &

Starting graylog2-server … As it listen to syslog, you need to run as root ( port < 1024 … )

# /usr/local/graylog2-server-0.9.6/bin/graylog2ctl start
# /usr/bin/nohup: appending output to `nohup.out'

Web front installation

Get the software

# wget https://github.com/downloads/Graylog2/graylog2-web-interface/graylog2-web-interface-0.9.6.tar.gz
# tar xzvf /tmp/graylog2-web-interface-0.9.6.tar.gz
# mv graylog2-web-interface-0.9.6/ /usr/local/
# cd /usr/local/graylog2-web-interface-0.9.6

Install ruby modules installer

# gem install bundler
Successfully installed bundler-1.0.21
1 gem installed
Installing ri documentation for bundler-1.0.21...
Installing RDoc documentation for bundler-1.0.21...

# bundle install
Fetching source index for http://rubygems.org/
[...]
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.

Backup and edit configuration files

# cd config
# for i in *.yml; do j=$(echo $i.orig); cp $i $j; done
# vim *yml

yml file must be configured with same login informations for mongo. The rest is likely to stay untouched. New Relic may be or not installed

Start as apache site

# cd /etc/httpd/conf.d
# vim passenger.conf

LoadModule passenger_module modules/mod_passenger.so
<IfModule mod_passenger.c>
PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-3.0.11
PassengerRuby /usr/bin/ruby
PassengerTempDir /var/run/passenger
</IfModule>
<VirtualHost *:80>
    ServerAdmin root@localhost
    DocumentRoot /usr/local/graylog2-web-interface-0.9.6/public
    <Directory /usr/local/graylog2-web-interface-0.9.6/public>
        Allow from all
        Options -MultiViews
    </Directory>
    ErrorLog /var/log/httpd/error.log
    LogLevel warn
    CustomLog /var/log/httpd/access.log combined
</VirtualHost>

Start Apache and enjoy

# /etc/init.d/httpd restart
Stopping httpd: [FAILED]
Starting httpd: [OK]

About these ads