{"id":370,"date":"2014-03-13T13:18:17","date_gmt":"2014-03-13T02:18:17","guid":{"rendered":"https:\/\/icicimov.com\/blog\/?p=370"},"modified":"2017-01-09T23:20:25","modified_gmt":"2017-01-09T12:20:25","slug":"ipsec-vpn-tunnel-setup-between-two-amazon-vpcs-with-openswan-and-ec2-nat-instances","status":"publish","type":"post","link":"https:\/\/icicimov.com\/blog\/?p=370","title":{"rendered":"IPSEC VPN tunnel setup between two Amazon VPC’s with OpenSwan and EC2 NAT instances’"},"content":{"rendered":"

With services running in multiple VPC’s sooner or later a need will arise for secure clustering of instances across regions. This is especially important in case when such services do not have built in SSL\/TLS support or when the services are running on private only instances, ie instances that don’t have public ip’s and are meant to have private traffic only like databases, attached storage etc. There some low level tools we can use to support non SSL services, like stunnel, stud etc, but they all require public accessible instances. The VPN tunnel on other hand establishes a bridge between the private networks from different regions making the clustering much more easier as it makes it look like all instances are part of the same private network.<\/p>\n

Overview<\/h1>\n

To make internet access available for private subnets we need to create a NAT instance to route their traffic through. And to create the VPN tunnel between the VPC’s we will need one VPN instance. Instead having two separate instances in each VPC I have created single m1.small<\/code> instance that will serve both roles. Depending on the load we can scale this up if needed.<\/p>\n

On the VPC1 side we have public subnet of 10.1.1.0\/24<\/code> and private subnet 10.1.10.0\/24<\/code>. The NAT\/VPN instance has been created in the public subnet with ip of 10.1.1.254<\/code> and in the private subnet we have instance with ip of 10.1.10.41<\/code>. This private instance needs to get Internet access via the NAT instance (which is a VPN instance in the same time) for software update purposes but should also be able to talk to the private instances on the VPC2 side (and vice-versa).<\/p>\n

In the VPC2 VPC we launch NAT\/VPN instance with ip of 172.31.12.198<\/code> and private instance with ip of 172.31.200.200<\/code> created in the 172.31.200.0\/24<\/code> private subnet.<\/p>\n

Both NAT\/VPN instances should have EIP associated to their primary interface. For the VPC2 side that is 54.124.x.x<\/code> and 54.26.x.x<\/code> on the VPC1 side.<\/p>\n

Setup<\/h1>\n

NAT-ing<\/h2>\n

We need to set the NAT rules in that way that the VPN tunneled networks are not NAT’ed.<\/p>\n

On the VPC1 side:<\/p>\n

root@10.1.1.254:~# iptables -t nat -A POSTROUTING -o eth0 -s 10.1.0.0\/16 -d 172.31.0.0\/16 -j ACCEPT\nroot@10.1.1.254:~# iptables -t nat -A POSTROUTING -o eth0 -s 10.1.0.0\/16 -j MASQUERADE\n<\/code><\/pre>\n
On the VPC2 side:<\/p>\n
root@172.31.12.198:~# iptables -t nat -A POSTROUTING -o eth0 -s 172.31.0.0\/16 -d 10.1.0.0\/16 -j ACCEPT\nroot@172.31.12.198:~# iptables -t nat -A POSTROUTING -o eth0 -s 172.31.0.0\/16 -j MASQUERADE\n<\/code><\/pre>\n
Then for each VPC we need to:<\/p>\n
    \n
  • Set the NAT instance as default gateway in the VPC routing table for the private subnet(s)<\/li>\n
  • Modify the security group of the private instances that will be using this NAT\/VPN servers to allow ALL traffic from the NAT\/VPN security group (appropriate to the zone)<\/li>\n
  • Modify the security groups of the NAT\/VPN instances to allow ALL traffic from each other and the coresponding private instances security group<\/li>\n
  • Turn off the Source\/Destination check on the NAT instance<\/li>\n<\/ul>\n
    Configure the kernel<\/h2>\n
    Append (or modify if some of these already exist) the following to the end of the kernel config file.<\/p>\n
    net.ipv4.ip_forward=1\nnet.ipv4.conf.all.accept_redirects = 0\nnet.ipv4.conf.all.secure_redirects = 0\nnet.ipv4.conf.all.send_redirects = 0\nnet.ipv4.conf.default.accept_redirects = 0\nnet.ipv4.conf.default.secure_redirects = 0\nnet.ipv4.conf.default.send_redirects = 0\nnet.ipv4.conf.lo.accept_redirects = 0\nnet.ipv4.conf.lo.secure_redirects = 0\nnet.ipv4.conf.lo.send_redirects = 0\nnet.ipv4.conf.eth0.accept_redirects = 0\nnet.ipv4.conf.eth0.secure_redirects = 0\nnet.ipv4.conf.eth0.send_redirects = 0\n<\/code><\/pre>\n
    save the file and make the rules effective:<\/p>\n
    root@server:~# sysctl -p\n<\/code><\/pre>\n
    VPN tunnel<\/h2>\n
    We need the openswan<\/code> package installed on our NAT\/VPN instances. For Debian\/Ubuntu this is as simple as:<\/p>\n
    # aptitude install openswan\n<\/code><\/pre>\n
    On the VPC1 side we add to ipsec \/etc\/ipsec.conf<\/code> config file:<\/p>\n
    conn uk-to-au\n  authby=secret\n  auto=start\n  type=tunnel\n  left=10.1.1.254\n  leftid=54.26.x.x\n  leftsubnet=10.1.0.0\/16\n  right=54.124.x.x\n  rightsubnet=172.31.0.0\/16\n  ike=aes256-sha1;modp2048\n  phase2=esp\n  phase2alg=aes256-sha1;modp2048\n  forceencaps=yes\n<\/code><\/pre>\n
    On the VPC2 side we add to \/etc\/ipsec.conf<\/code> file:<\/p>\n
    conn au-to-uk\n  authby=secret\n  auto=start\n  type=tunnel\n  left=172.31.12.198\n  leftid=54.124.x.x  \n  leftsubnet=172.31.0.0\/16\n  right=54.26.x.x\n  rightsubnet=10.1.0.0\/16\n  ike=aes256-sha1;modp2048\n  phase2=esp\n  phase2alg=aes256-sha1;modp2048\n  forceencaps=yes\n<\/code><\/pre>\n
    And we add following two lines to ipsec security file \/etc\/ipsec.secrets<\/code> specifying the shared secret used for both directions:<\/p>\n
    54.26.x.x 54.124.x.x : PSK \"0xaec8d3991aaff8bc8e2e3f731a8f6882\"\n54.124.x.x 54.26.x.x : PSK \"0xaec8d3991aaff8bc8e2e3f731a8f6882\"\n<\/code><\/pre>\n
    on both sides. We can use the following command to generate the ipsec<\/code> secret(s):<\/p>\n
    $ ipsec ranbits --continuous 128\n0xaec8d3991aaff8bc8e2e3f731a8f6882\n<\/code><\/pre>\n
    The above given one is not the one used on the real servers of course.<\/p>\n
    At the end we restart the ipsec service on both sides:<\/p>\n
    # service ipsec restart\n<\/code><\/pre>\n
    After that the private instances from VPC2 and VPC1 VPC’s should be able to ping each other assuming the NAT part is already done.<\/p>\n
    Ping test from VPC2 VPC private instance to VPC1 VPC instance using its private ip only:<\/p>\n
    root@ip-172-31-200-200:~# ping -c 5 10.1.10.41\nPING 10.1.10.41 (10.1.10.41) 56(84) bytes of data.\n64 bytes from 10.1.10.41: icmp_req=1 ttl=62 time=340 ms\n64 bytes from 10.1.10.41: icmp_req=2 ttl=62 time=340 ms\n64 bytes from 10.1.10.41: icmp_req=3 ttl=62 time=340 ms\n64 bytes from 10.1.10.41: icmp_req=4 ttl=62 time=340 ms\n64 bytes from 10.1.10.41: icmp_req=5 ttl=62 time=340 ms\n\n--- 10.1.10.41 ping statistics ---\n5 packets transmitted, 5 received, 0% packet loss, time 4005ms\nrtt min\/avg\/max\/mdev = 340.253\/340.353\/340.585\/0.650 ms\n<\/code><\/pre>\n
    We listen for icmp traffic on the receiving side:<\/p>\n
    root@ip-10-1-10-41:~# tcpdump icmp\ntcpdump: verbose output suppressed, use -v or -vv for full protocol decode\nlistening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes\n13:20:12.420519 IP ip-172-31-200-200.eu-west-1.compute.internal > ip-10-1-10-41.eu-west-1.compute.internal: ICMP echo request, id 3631, seq 1, length 64\n13:20:12.420552 IP ip-10-1-10-41.eu-west-1.compute.internal > ip-172-31-200-200.eu-west-1.compute.internal: ICMP echo reply, id 3631, seq 1, length 64\n13:20:13.421785 IP ip-172-31-200-200.eu-west-1.compute.internal > ip-10-1-10-41.eu-west-1.compute.internal: ICMP echo request, id 3631, seq 2, length 64\n13:20:13.421827 IP ip-10-1-10-41.eu-west-1.compute.internal > ip-172-31-200-200.eu-west-1.compute.internal: ICMP echo reply, id 3631, seq 2, length 64\n13:20:14.423040 IP ip-172-31-200-200.eu-west-1.compute.internal > ip-10-1-10-41.eu-west-1.compute.internal: ICMP echo request, id 3631, seq 3, length 64\n13:20:14.423083 IP ip-10-1-10-41.eu-west-1.compute.internal > ip-172-31-200-200.eu-west-1.compute.internal: ICMP echo reply, id 3631, seq 3, length 64\n13:20:15.424580 IP ip-172-31-200-200.eu-west-1.compute.internal > ip-10-1-10-41.eu-west-1.compute.internal: ICMP echo request, id 3631, seq 4, length 64\n13:20:15.424612 IP ip-10-1-10-41.eu-west-1.compute.internal > ip-172-31-200-200.eu-west-1.compute.internal: ICMP echo reply, id 3631, seq 4, length 64\n13:20:16.426040 IP ip-172-31-200-200.eu-west-1.compute.internal > ip-10-1-10-41.eu-west-1.compute.internal: ICMP echo request, id 3631, seq 5, length 64\n13:20:16.426078 IP ip-10-1-10-41.eu-west-1.compute.internal > ip-172-31-200-200.eu-west-1.compute.internal: ICMP echo reply, id 3631, seq 5, length 64\n<\/code><\/pre>\n
    And we test the other way around:<\/p>\n
    root@ip-10-1-10-41:~# ping -c 5 172.31.200.200\nPING 172.31.200.200 (172.31.200.200) 56(84) bytes of data.\n64 bytes from 172.31.200.200: icmp_req=1 ttl=62 time=340 ms\n64 bytes from 172.31.200.200: icmp_req=2 ttl=62 time=340 ms\n64 bytes from 172.31.200.200: icmp_req=3 ttl=62 time=340 ms\n64 bytes from 172.31.200.200: icmp_req=4 ttl=62 time=340 ms\n64 bytes from 172.31.200.200: icmp_req=5 ttl=62 time=340 ms\n\n--- 172.31.200.200 ping statistics ---\n5 packets transmitted, 5 received, 0% packet loss, time 4002ms\nrtt min\/avg\/max\/mdev = 340.292\/340.459\/340.599\/0.114 ms\n<\/code><\/pre>\n
    The icmp traffic on the receiving side:<\/p>\n
    root@ip-172-31-200-200:~# tcpdump icmp\ntcpdump: verbose output suppressed, use -v or -vv for full protocol decode\nlistening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes\n13:24:03.959726 IP ip-10-1-10-41.ap-southeast-2.compute.internal > ip-172-31-200-200.ap-southeast-2.compute.internal: ICMP echo request, id 7820, seq 1, length 64\n13:24:03.959770 IP ip-172-31-200-200.ap-southeast-2.compute.internal > ip-10-1-10-41.ap-southeast-2.compute.internal: ICMP echo reply, id 7820, seq 1, length 64\n13:24:04.960371 IP ip-10-1-10-41.ap-southeast-2.compute.internal > ip-172-31-200-200.ap-southeast-2.compute.internal: ICMP echo request, id 7820, seq 2, length 64\n13:24:04.960419 IP ip-172-31-200-200.ap-southeast-2.compute.internal > ip-10-1-10-41.ap-southeast-2.compute.internal: ICMP echo reply, id 7820, seq 2, length 64\n13:24:05.960773 IP ip-10-1-10-41.ap-southeast-2.compute.internal > ip-172-31-200-200.ap-southeast-2.compute.internal: ICMP echo request, id 7820, seq 3, length 64\n13:24:05.960820 IP ip-172-31-200-200.ap-southeast-2.compute.internal > ip-10-1-10-41.ap-southeast-2.compute.internal: ICMP echo reply, id 7820, seq 3, length 64\n13:24:06.961421 IP ip-10-1-10-41.ap-southeast-2.compute.internal > ip-172-31-200-200.ap-southeast-2.compute.internal: ICMP echo request, id 7820, seq 4, length 64\n13:24:06.961468 IP ip-172-31-200-200.ap-southeast-2.compute.internal > ip-10-1-10-41.ap-southeast-2.compute.internal: ICMP echo reply, id 7820, seq 4, length 64\n13:24:07.961948 IP ip-10-1-10-41.ap-southeast-2.compute.internal > ip-172-31-200-200.ap-southeast-2.compute.internal: ICMP echo request, id 7820, seq 5, length 64\n13:24:07.961992 IP ip-172-31-200-200.ap-southeast-2.compute.internal > ip-10-1-10-41.ap-southeast-2.compute.internal: ICMP echo reply, id 7820, seq 5, length 64\n<\/code><\/pre>\n
    This confirms that both sides are now connected via VPN tunnel.<\/p>\n","protected":false},"excerpt":{"rendered":"
    With services running in multiple VPC’s sooner or later a need will arise for secure clustering of instances across regions. This is especially important in case when such services do not have built in SSL\/TLS support or when the services…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,12],"tags":[32,36],"class_list":["post-370","post","type-post","status-publish","format-standard","hentry","category-aws","category-devops","tag-aws","tag-vpn"],"_links":{"self":[{"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/370","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=370"}],"version-history":[{"count":1,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/370\/revisions"}],"predecessor-version":[{"id":371,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/370\/revisions\/371"}],"wp:attachment":[{"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=370"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=370"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/icicimov.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=370"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}