Daily Archives: 29.01.2015

Shipping your Nginx logs to Elasticsearch using Logstash

First of all create a new log format in /etc/nginx/nginx.conf and add/replace an access log directive: log_format logstash ‘$http_host ‘ ‘$remote_addr [$time_local] ‘ ‘”$request” $status $body_bytes_sent ‘ ‘”$http_referer” “$http_user_agent” ‘ ‘$request_time ‘ ‘$upstream_response_time’; access_log /var/log/nginx/access.log logstash; Install Elasticsearch like described here. Install Logstash like described here. Add the following pattern to logstash in /opt/logstash/paggerns/nginx: NGUSERNAME […]

Setting up Logstash on Debian/Ubuntu

Install Logstash: cd /tmp wget https://download.elasticsearch.org/logstash/logstash/logstash-1.4.2.tar.gz tar -xvpf logstash-1.4.2.tar.gz mv logstash-1.4.2 /opt/logstash Create an init script /etc/init.d/logstash (this one worked fine for me on Ubuntu 14.04): #! /bin/sh ### BEGIN INIT INFO # Provides: logstash # Required-Start: $network $remote_fs $named $elasticsearch # Required-Stop: $network $remote_fs $named $elasticsearch # Default-Start: 2 3 4 5 # Default-Stop: […]

Install Elasticsearch 1.4 on Debian/Ubuntu

Install Java: apt-get install openjdk-7-jre openjdk-7-jre-headless Add the repository and install Elasticsearch: wget -qO – https://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add – add-apt-repository "deb http://packages.elasticsearch.org/elasticsearch/1.4/debian stable main" apt-get update apt-get install elasticsearch Edit /etc/default/elasticsearch and change ES_HEAP_SIZE to limit memory usage depending on your machine. Start Elasticsearch: /etc/init.d/elasticsearch start Configure Elasticsearch to automatically start during bootup : […]