Creating a Glusterfs volume replicated across 2 hosts

This worked perfectly on Ubuntu 14.04, but should work just the same on other Ubuntu or Debian-based distributions. 

First of all, install glusterfs on both hosts:

apt-get install glusterfs-server

I suggest, for simplicity to but the 2 hostnames in /etc/hosts on each server, for example: srv01 srv02

I'm going to use those hostnames in the commands below.

Let's connect the 2 hosts. Execute this on the first host for example:

gluster peer probe srv02

Now when executing

gluster peer status

You should see the following lines:

Number of Peers: 1
State: Peer in Cluster (Connected)

If your peer host is not marked "Connected", this may be or a network issue, a firewall problem or your glusterfs-server is not running.

In your firewall you need to open these tcp ports: 111/tcp, 111/udp, 24007/tcp, 24008/tcp, tcp ports 24009 – 24011 (24009 + number of bricks across all volumes, in our case 2), if you intend to mount your volume via nfs, you need to open also tcp ports 38465 – 38467 (38465 + number of gluster servers, in this case 2).

In this example I'm going to create a volume called "datavol" which resides in /datavol on both servers. In case that your volume will not be on your root partition, you can omit the "force" at the end. Let's create the volume (on one of the 2 hosts):

gluster volume create datavol replica 2 transport tcp srv01:/datavol srv02:/datavol force

When the creation was successful, just start the volume (on one of the 2 hosts):

gluster volume start datavol

With these 2 commands you can verify at any time the status of your volume (on both hosts the same):

gluster volume info datavol
gluster volume status datavol

Your volume should be marked with status "Started", the number of bricks should be 2.

Volume Name: datavol
Type: Replicate
Status: Started
Number of Bricks: 1 x 2 = 2

When adding additional servers that are using this volume, you are going to add bricks – which are the storage filesystem assigned to the volume. Bricks shall always be a multiple of the number of replica's. So to expand here, you'd add 2 additional bricks. You can have multiple bricks per server – here however, for simplicity, we have one server – one brick.

Ok, now that the volume is created and started, we should mount it. If, for example, you would like to mount it to /var/www, you can do it as follows:

mount.glusterfs srv01:/datavol /var/www

An entry in /etc/fstab would look like this:

srv01:/datavol /var/www glusterfs defaults,_netdev 0 0

You could mount your volume on both servers and create, modifiy, delete some files to verify that the replication works in both directions.