Yesterday I helped a friend to figure out why he couldn’t connect to his Amazon RDS database inside the Amazon VPC (Virtual Private Cloud). It was the second time someone asked me to help with the Amazon Web Services (AWS), and it was the first time I was actually helpful. Yey!
While I do use quite a few of the Amazon Web Services, I don’t have any experience with the Amazon RDS yet, as I’m managing my own MySQL instances. It was interesting to get my toes wet in the troubleshooting.
Here are a few things I’ve learned in the process.
Lesson #1: Amazon supports two different ways of accessing the RDS service. Make sure you know which one you are using and act accordingly.
If you run an Amazon RDS instance in the VPC, you’ll have to setup your networking and security access properly. This page – Connecting to a DB Instance Running the MySQL Database Engine – will only be useful once everything else is taken care of. It’s not your first and only manual to visit.
Lesson #2 (sort of obvious): Make sure that both your Network ACL and Security Groups allow all the necessary traffic in and out. Double-check the IP addresses in the rules. Make sure you are not using a proxy server, when looking up your external IP address on WhatIsMyIP.com or similar.
Lesson #3: Do not use ICMP traffic (ping and such) as a troubleshooting tool. It looks like Amazon RDS won’t be ping-able even if you allow it in your firewalls. Try with “telnet your-rds-end-point-server your-rds-end-point-port” (example: “telnet 1.2.3.4 3306” or with a real database client, like the command-line MySQL one.
Lesson #4: Make sure your routing is setup properly. Check that the subnet in which your RDS instance resides has the correct routing table attached to it, and that the routing table has the default gateway (0.0.0.0/0) route configured to either the Internet Gateway or to some sort of NAT. Chances are your subnet is only dealing with private IP range and has no way of sending traffic outside.
Lesson #5: When confused, disoriented, and stuck, assume it’s not Amazon’s fault. Keep calm and troubleshoot like any other remote connection issue. Double-check your assumptions.
There’s probably lesson 6 somewhere there, about contacting support or something along those lines. But in this particular case it didn’t get to that. Amazon AWS support is excellent though. I had to deal with those guys twice in the last two-something years, and they were awesome.