Solr / Elasticsearch

This category contains everything around index searchers – SOLR, Elasticsearch, etc.

Install the latest Apache SOLR Build in a few minutes on Ubuntu/Debian

First install a Java runtime and needed utilities: apt-get update apt-get install openjdk-7-jre openjdk-7-jre-headless openjdk-7-jre-lib unzip lsof Then execute these as root: cd ~ wget tar xzf solr-5.3.1.tgz solr-5.3.1/bin/ –strip-components=2 chmod +x ./ solr-5.3.1.tgz Finally start Solr: service solr start You should now be able to access your server like this: http://localhost:8983/solr You […]

Shipping your Nginx logs to Elasticsearch using Logstash

First of all create a new log format in /etc/nginx/nginx.conf and add/replace an access log directive: log_format logstash ‘$http_host ‘ ‘$remote_addr [$time_local] ‘ ‘”$request” $status $body_bytes_sent ‘ ‘”$http_referer” “$http_user_agent” ‘ ‘$request_time ‘ ‘$upstream_response_time’; access_log /var/log/nginx/access.log logstash; Install Elasticsearch like described here. Install Logstash like described here. Add the following pattern to logstash in /opt/logstash/paggerns/nginx: NGUSERNAME […]

Setting up Logstash on Debian/Ubuntu

Install Logstash: cd /tmp wget tar -xvpf logstash-1.4.2.tar.gz mv logstash-1.4.2 /opt/logstash Create an init script /etc/init.d/logstash (this one worked fine for me on Ubuntu 14.04): #! /bin/sh ### BEGIN INIT INFO # Provides: logstash # Required-Start: $network $remote_fs $named $elasticsearch # Required-Stop: $network $remote_fs $named $elasticsearch # Default-Start: 2 3 4 5 # Default-Stop: […]

Install Elasticsearch 1.4 on Debian/Ubuntu

Install Java: apt-get install openjdk-7-jre openjdk-7-jre-headless Add the repository and install Elasticsearch: wget -qO – | sudo apt-key add – add-apt-repository "deb stable main" apt-get update apt-get install elasticsearch Edit /etc/default/elasticsearch and change ES_HEAP_SIZE to limit memory usage depending on your machine. Start Elasticsearch: /etc/init.d/elasticsearch start Configure Elasticsearch to automatically start during bootup : […]

Installing a 3 node Zookeeper Ensemble

First install the packages: apt-get install zookeeper zookeeper-bin zookeeperd As Zookeeper is launched using Java, you may want to update the Java options in /etc/default/zookeeper, for example: JAVA_OPTS="-Xms1024M -Xmx1024M -noverify -Duser.timezone=GMT+1" This would assign 1GB of RAM to the JVM for Zookeeper and set the timezone in Java to GMT+1. There are more options, this […]

Installing an Elasticsearch Cluster with 2 Nodes

Download the latest version of Elasticsearch from here and unpack it to /opt: cd /opt wget tar -xvpf elasticsearch-1.3.4.tar.gz ln -s elasticsearch-1.3.4 elasticsearch Given you want to create a cluster named my-ecs-cluster, having 2 nodes with IP address and (change it according to your setup, you might also want to use hostnames instead of IPs)   Then […]

Installing SOLR 4.9 with Tomcat 7 on Debian Wheezy

First of all, let's install Java: apt-get install openjdk-7-jre-headless   Installing Tomcat: apt-get install tomcat7 tomcat7-admin You should now be able to open http://<your-ip>:8080/ in a web browser.   Configuring Tomcat manager auth: The tomcat7-admin package comes with two web apps: Web Application Manager and Virtual Host Manager. For security reasons none of them are […]

Installing Solr 4.4 with Zookeeper and Tomcat 6 on Debian

Package installation: Install the needed tomcat6 packages first: apt-get update apt-get install tomcat6 tomcat6-admin tomcat6-common tomcat6-user  Pick a SOLR mirror here and download the latest stable release: cd /tmp wget tar –xvpf solr-4.4.0.tgz mv solr-4.4.0 /opt/solr Pick a ZOOKEEPER mirror here and download the latest stable release: cd /tmp wget tar -xvpf zookeeper-3.4.5.tar.gz mv zookeeper-3.4.5 /opt/zookeeper   Configure […]