on graylog | graylog2 | elasticsearch | mongodb | centos

Setting up Graylog2 server on CentOS

The following description is for CentOS 5.5 and currently tle latest version of Graylog2 centralized logging platform. We will use tarball packages so that the setup wont require a priviledged user.

Here are the dependencies with the exact versions tested to be working.

MongoDB

Let's download and unpack the MongoDB package. We will use the 64 bit version for our CentOS 5.5.

$ wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.6.1.tgz
$ tar -xzf mongodb-linux-x86_64-2.6.1.tgz
$ mv mongodb-linux-x86_64-2.6.1 mongodb

I have renamed the directory to mongodb to make the paths simpler. The next step is to create a configuration file for MongoDB. As it does not come with an example config you can grab a great template from here.

$ cd mongodb && mkdir conf && cd conf
$ wget https://raw.githubusercontent.com/andrewgross/mongo.conf/master/mongodb.conf

Make the following changes in the configurations for basic setup.

# Set the path to your data directory.
# The default one wont be available if you
# are not a priveledged user.
dbpath = /path/to/data/directory

# Comment out the replication config.
# replSet = replica

# Set the server as a master.
master = true

# Set the path to log files.
logpath = /path/to/log/directory/rs.log

These are all preparations for database setup, you may go ahead and start the server now.

$ ./bin/mongod -f conf/mongodb.conf &

Create a database for Graylog2.

$ ./bin/mongo
> use graylog2

Note that we have not setup authentication, as this assumes a local development server and not considers any security.

ElasticSearch

Download and extract the package in the same directory as you did. This is just for convenience, so you can choose your own directory scheme.

$ wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.13.tar.gz
$ tar -xzf elasticsearch-0.90.13.tar.gz
$ mv elasticsearch-0.90.13 elasticsearch
$ cd elasticsearch

Let's set the elasticsearch configurations for Graylog2. The file is in the conf directory conf/elasticsearch.yml.

# Change the cluster name to have the same as in the
# Graylog2 configurations, which is done a bit
# later in the post.
cluster.name: graylog2

# Lock the memory to disable swapping.
bootstrap.mlockall: true

# Disable multicast discovery
discovery.zen.ping.multicast.enabled: false

# Set the list of hosts to look for unicast discovery
discovery.zen.ping.unicast.hosts: ["localhost"]

Start the server by simply running the script in bin/ directory.

$ ./bin/elasticsearch

Graylog2 Server

We have finally got to step where the actual logging server to be install. Download the archive from the mentioned URL and extract in the desired directory.

I did the same with the naming of the directory as above: renamed to a shorter name.

$ wget https://github.com/Graylog2/graylog2-server/releases/download/0.20.2/graylog2-server-0.20.2.tgz
$ tar -xzf graylog2-server-0.20.2.tgz
$ mv graylog2-server-0.20.2 graylog2

Create a seaparate directory for the configuration file.

$ cd graylog2 && mkdir conf && cd conf

As we consider that the user you use for the installation has no priviledged access, the directories in the configuration should be chosen correspondingly with write access.

Copy the example log file into conf/ directory and rename to graylog2.conf.

$ mv ../graylog2.conf.example ./graylog2.conf

The configurations will be as follows.

# Set the path to the auto generated node ID
node_id_file = /path/to/my/node/id/graylog2-server-node-id

# Generate a secret using `pwgen -s 96` command
# as described in the example log file.
password_secret = MyLongSecretHash

# Create a root password for the server.
# You can do it by running the command
# suggested by example config file
# echo -n yourpassword | shasum -a 256
root_password_sha2 = mypasswordhashhere

# Change the number of elasticsearch shards to one
elasticsearch_shards = 1

# Uncomment the cluster name. Note that it should be the
# same as on the ElasticSearch configuration
elasticsearch_cluster_name = graylog2

# Same with node name
elasticsearch_node_name = graylog2-server

# Uncomment the transport tcp port
elasticsearch_transport_tcp_port = 9350

# Disable multicast search and add unicast hosts
# for ElasticSearch.
elasticsearch_discovery_zen_ping_multicast_enabled = false
elasticsearch_discovery_zen_ping_unicast_hosts = localhost:9300

# Uncomment the MongoDB host configuration.
# No authentication will be needed, because of
# our MondoDB setup.
mongodb_host = localhost

Best approach is to run the server in debug mode and see if there are connection or other issues.

$ java -jar graylog2-server.jar -f conf/graylog2.conf -d

Finally, if everything is fine, change the log file path in the bin/graylog2ctl script

GRAYLOG2_CONF=${GRAYLOG2_CONF:=conf/graylog2.conf}

and start the service by the following command:

$ ./bin/graylog2ctl start

Graylog2 Web Interface

Final step to get everything up and running is to setup the web interface for the Graylog. This will allow to see all kind off statistics on the logs.

Let's download and extract the archive.

$ wget https://github.com/Graylog2/graylog2-web-interface/releases/download/0.20.2/graylog2-web-interface-0.20.2.tgz
$ tar -xzf graylog2-web-interface-0.20.2.tgz
$ mv graylog2-web-interface-0.20.2 graylog2-web-interface

This one comes with a config files that need only the Graylog2 server URIs and application secret

# Graylog2 server
graylog2-server.uris="http://localhost:12900/

# Secret
application.secret="mysecrethash"

Run the server on your desired port by the following command:

$ ./bin/graylog2-web-interface -Dhttp.port=8000 -Dhttp.address=0.0.0.0

According the command above the server should start running on port 8000 and listen to all the requests from the network.

When accessing the web interface (http://myhost:8000) it will require the password you set in the Graylog configurations for the root account. The user to login from admin interface is admin not root.