Shipping your Nginx logs to Elasticsearch using Logstash

First of all create a new log format in /etc/nginx/nginx.conf and add/replace an access log directive: log_format logstash ‘$http_host ‘ ‘$remote_addr [$time_local] ‘ ‘”$request” $status $body_bytes_sent ‘ ‘”$http_referer” “$http_user_agent” ‘ ‘$request_time ‘ ‘$upstream_response_time’; access_log /var/log/nginx/access.log logstash; Install Elasticsearch like described here. Install Logstash like described here. Add the following pattern to logstash in /opt/logstash/paggerns/nginx: NGUSERNAME […]

Install Elasticsearch 1.4 on Debian/Ubuntu

Install Java: apt-get install openjdk-7-jre openjdk-7-jre-headless Add the repository and install Elasticsearch: wget -qO – https://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add – add-apt-repository "deb http://packages.elasticsearch.org/elasticsearch/1.4/debian stable main" apt-get update apt-get install elasticsearch Edit /etc/default/elasticsearch and change ES_HEAP_SIZE to limit memory usage depending on your machine. Start Elasticsearch: /etc/init.d/elasticsearch start Configure Elasticsearch to automatically start during bootup : […]

Installing an Elasticsearch Cluster with 2 Nodes

Download the latest version of Elasticsearch from here and unpack it to /opt: cd /opt wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.4.tar.gz tar -xvpf elasticsearch-1.3.4.tar.gz ln -s elasticsearch-1.3.4 elasticsearch Given you want to create a cluster named my-ecs-cluster, having 2 nodes with IP address and (change it according to your setup, you might also want to use hostnames instead of IPs)   Then […]