GlusterFS metadata split brain recovery
While investigating an error related to failed documents I came across following error in the GlusterFS healing daemon log file: [2015-11-08 23:22:38.700539] E [afr-self-heal-common.c:197:afr_sh_print_split_brain_log] 0-gfs-volume-vol1-replicate-0: Unable to self-heal contents of ” (possible split-brain). Please delete the file from all but…
GlusterFS internals
GlusterFS stores metadata info in extended attributes which is supported and enabled by default in the XFS file system we use for the bricks. This is different approach then some other distributed storage cluster systems like Ceph for example that…
HAProxy dynamically adjust server weight using external agent
Trying to utilize HAProxy-1.5/1.6 agent-check feature, see HAProxy documentation, I wrote this small script to check Tomcat system load and return back some values that HAP can use to dynamically adjust the server weight in the backend. This will run…
HAProxy OCSP stapling
The Online Certificate Status Protocol (OCSP) is an Internet protocol used for obtaining the revocation status of an X.509 digital certificate. It is used by https clients (browsers) to confirm that the certificate sent by the server they have connected…
Securing Logstash to Redis communication with Stunnel
Logstash is meant for private LAN usage since it doesn’t offer any kind of encryption support. If we need to ship sensitive data across WAN’s, like between Amazon VPC’s, we would like to have the communication channel secure. That’s where…
Centralized logs collection with Logstash, ElasticSearch and Kibana in Amazon AWS
Logstash is a tool for managing events and logs. It is very useful for collecting, parsing and storing logs for later use like for example searching. It comes with a web interface for searching through the logs. The picture bellow…
Setting up Encompass Maintenance Page with ELB, S3, Route53 and CloudFront
This is for the environments we have ELB (Elastic Load Balancer) instead of HAProxy. The idea is to host the maintenance page as static website in S3 bucket and then have a Failover DNS records in Route53 for the targeted…
Web site statistics with AWStats
Awstats (Apache Web Statistics) is powerful and highly customizable tool for collecting web site statistics. The purpose of this document is to show one way we are using it for data collection and presentation of heavily customized Apache logs from…
PostgreSQL Archiving Replication
In this mode PostgreSQL replicates the WAL archive logs. Configuring the master server First, create a new user in PostgreSQL, for replication purposes. We’ll use it to connect to the master instance from the slave and replicate data. To create…
PostgreSQL Streaming Replication
Streaming replication means the changes are synchronously applied from the master to the slave(s). First, create the replication user on the master: $ sudo -u postgres psql -c “CREATE USER replicator REPLICATION LOGIN ENCRYPTED PASSWORD ”;” The user created is…