{"id":52,"date":"2015-02-10T23:08:19","date_gmt":"2015-02-10T12:08:19","guid":{"rendered":"https:\/\/icicimov.com\/blog\/?p=52"},"modified":"2016-11-09T23:21:59","modified_gmt":"2016-11-09T12:21:59","slug":"postgresql-streaming-replication","status":"publish","type":"post","link":"https:\/\/icicimov.com\/blog\/?p=52","title":{"rendered":"PostgreSQL Streaming Replication"},"content":{"rendered":"<p>Streaming replication means the changes are synchronously applied from the master to the slave(s).<\/p>\n<p>First, create the replication user on the master:<\/p>\n<pre>\r\n$ sudo -u postgres psql -c \"CREATE USER replicator REPLICATION LOGIN ENCRYPTED PASSWORD '<replicator_password>';\"\r\n<\/replicator_password><\/pre>\n<p>The user created is called replicator. Make sure to create strong password for production use.<\/p>\n<p>Next, configure the master for streaming replication. Edit `\/etc\/postgresql\/9.1\/main\/postgresql.conf` file:<\/p>\n<pre>\r\n[...]\r\nlisten_address = '*'\r\nlog_line_prefix='%t [%p] %u@%d '\r\nwal_level = hot_standby\r\nmax_wal_senders = 3\r\ncheckpoint_segments = 8   \r\nwal_keep_segments = 16\r\n[...]\r\n<\/pre>\n<p>We&#8217;re configuring 8 x WAL segments here, each is 16MB. Consider increasing those values if we expect our database to have more than 128MB of changes in the time it will take to make a copy of it across the network to our slave, or in the time we expect our slave to be down for maintenance.<\/p>\n<p>Then edit the access control on the master to allow the connection from the slave in `\/etc\/postgresql\/9.1\/main\/pg_hba.conf` file:<\/p>\n<pre>\r\n[...]\r\nhostssl replication     replicator      <slave_ip>            md5\r\n[...]\r\n<\/slave_ip><\/pre>\n<p>Restart the server for the changes to take effect:<\/p>\n<pre>\r\n$ sudo service postgresql restart\r\n<\/pre>\n<p>Now on to the slave. In the slave&#8217;s `postgresql.conf` we add the following:<\/p>\n<pre>\r\n[...]\r\nlog_line_prefix='%t [%p] %u@%d '\r\nwal_level = hot_standby\r\nmax_wal_senders = 3\r\ncheckpoint_segments = 8   \r\nwal_keep_segments = 16\r\nhot_standby = on\r\n[...]\r\n<\/pre>\n<p>Then restart the slave. No changes are required in the slave&#8217;s `pg_hba.conf` specifically to support replication. We&#8217;ll still need to make whatever entries we need in order to connect to it from our application and run read-only queries, if we wish to have multiple db hosts read access. All writes should still go to the master only.<\/p>\n<p>Then on the slave:<\/p>\n<p>Stop PostgreSQL<\/p>\n<pre>\r\n$ sudo service postgresql start\r\n<\/pre>\n<p>Remove the database<\/p>\n<pre>\r\n$ sudo -u postgres rm -rf \/var\/lib\/postgresql\/9.1\/main\r\n<\/pre>\n<p>Start the base backup on the master to the slave as replicator user<\/p>\n<pre>\r\n$ sudo -u postgres pg_basebackup -h <master_ip> -D \/var\/lib\/postgresql\/9.1\/main -U replicator -v -P\r\n<\/master_ip><\/pre>\n<p>Create recovery.conf file `\/var\/lib\/postgresql\/9.1\/main\/recovery.conf`<\/p>\n<pre>\r\nstandby_mode = 'on'\r\nprimary_conninfo = 'host=<master_ip> port=5432 user=replicator password=<replicator_password> sslmode=require'\r\ntrigger_file = '\/tmp\/postgresql.trigger'\r\n<\/replicator_password><\/master_ip><\/pre>\n<p>Start PostgreSQL<\/p>\n<pre>\r\n$ sudo service postgresql start\r\n<\/pre>\n<p>After that we should see something like this in the slave log file:<\/p>\n<pre>\r\nuser@slave:~$ sudo tail -f \/var\/log\/postgresql\/postgresql-9.1-main.log\r\nLOG:  received smart shutdown request\r\nLOG:  autovacuum launcher shutting down\r\nLOG:  shutting down\r\nLOG:  database system is shut down\r\nLOG:  database system was interrupted; last known up at 2015-10-01 08:43:28 BST\r\nLOG:  entering standby mode\r\nLOG:  streaming replication successfully connected to primary\r\nLOG:  redo starts at 0\/66000020\r\nLOG:  consistent recovery state reached at 0\/66000688\r\nLOG:  database system is ready to accept read only connections\r\n<\/pre>\n<p>and find PostgreSQL WAL processes running on both servers. On the master:<\/p>\n<pre>\r\nuser@master:~$ ps ax -o pid,command | grep postgres | grep wal\r\n27075 postgres: wal writer process                                                                                               \r\n27934 postgres: wal sender process replicator <slave_ip>(19846) streaming 0\/6719D650 [slave_ip]\r\n<\/slave_ip><\/pre>\n<p>and on the slave:<\/p>\n<pre>\r\nuser@slave:~$ ps ax -o pid,command | grep postgres | grep wal\r\n26357 postgres: wal receiver process   streaming 0\/6719D650\r\n<\/pre>\n<p>We can notice the streaming numbers match. We can also check the xlog segments position to confirm they are same. On the master:<\/p>\n<pre>\r\ndatabase=# SELECT pg_current_xlog_location();\r\n pg_current_xlog_location\r\n--------------------------\r\n 0\/6713BEF8\r\n(1 row)\r\n<\/pre>\n<p>and on the slave:<\/p>\n<pre>\r\ndatabase=# select pg_last_xlog_replay_location();\r\n pg_last_xlog_replay_location\r\n------------------------------\r\n 0\/6713BEF8\r\n(1 row)\r\n \r\ndatabase=# select pg_last_xlog_receive_location();\r\n pg_last_xlog_receive_location\r\n-------------------------------\r\n 0\/6713BEF8\r\n(1 row)\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Streaming replication means the changes are synchronously applied from the master to the slave(s). First, create the replication user on the master: $ sudo -u postgres psql -c &#8220;CREATE USER replicator REPLICATION LOGIN ENCRYPTED PASSWORD &#8221;;&#8221; The user created is&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[],"class_list":["post-52","post","type-post","status-publish","format-standard","hentry","category-database"],"_links":{"self":[{"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/52","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=52"}],"version-history":[{"count":4,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/52\/revisions"}],"predecessor-version":[{"id":56,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/52\/revisions\/56"}],"wp:attachment":[{"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=52"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=52"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=52"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}