AWS VPC identify private and public subnet

Question

I have a VPC in AWS account and there are 5 subnets associated with that VPC. Subnets are of 2 types - Public and private. How to identify which subnet is public and which is private ? Each subnet has CIDR 10.249.?.? range.

Basically when I launch an EMR in that subnet with lists of ec2SubnetIds , it says ***The subnet configuration was invalid: Provided subnet list contains both public and private subnet. Only one type of subnet is allowed.

---

How to recify this error.

Itemize

you can check the subnets for NAT and IGW. IGW suggests public subnet, whereas NAT suggests private subnet. – Chery

@VarunChandak I think you mean "check the default routes of the subnets", rather than "check the subnets". – Susurrous

@Susurrous apologies for missing out on semantics – Chery

Answer 1

The question is how to identify public subnets vs. private subnets, and the answer lies in what it means in AWS for a subnet of a VPC to be 'public' vs. 'private'.

Public subnets have a default route to an Internet Gateway; private subnets do not.

So, to determine if a given subnet is public or private, you need to describe the route table that is associated with that subnet. That will tell you the routes and you can test for a 0.0.0.0/0 route with a gateway ID of igw-xxxxxxxxxxxxxxxxx (as opposed to local).

You can tell if a subnet is public in the AWS VPC Console by reviewing the subnet's route table, for example:

You can also do this as follows for a given subnet ID, using the awscli:

aws ec2 describe-route-tables \
    --filter Name=association.subnet-id,Values=subnet-0a123fc414ad5b999 \
    --query "RouteTables[].Routes[]"

The output will look like this:

[
    {
        "DestinationCidrBlock": "10.0.0.0/16",
        "GatewayId": "local",
        "Origin": "CreateRouteTable",
        "State": "active"
    },
    {
        "DestinationCidrBlock": "0.0.0.0/0",
        "GatewayId": "igw-0fca21fadaa22a1b2",
        "Origin": "CreateRoute",
        "State": "active"
    }
]

Here, you can see a destination route of 0.0.0.0/0 with a target that is an Internet Gateway (its GatewayId is igw-xxxxxxxxxxxxxxxxx). This confirms that you are looking at a public subnet.

Susurrous

how do you actually do this "test for a "0.0.0.0/0" route with a gateway ID of igw-xxxxxxxx (as opposed to "local")." part? – Elfland

@Elfland have added an example using the awscli. – Susurrous

Answer 2

The best solution is to specify only a single subnet, so that you don't incur cross-AZ data charges.

There's no definite way to identify public and private subnets without looking at their routing tables: a public subnet will route to an Internet Gateway, while a private subnet won't. If you're creating your clusters via some program then maybe that's a reasonable check, but I wouldn't go there.

The best alternative is to give the subnet a name when you create it: something like "Private B" for a private subnet in availability zone B (us-east-1b, us-west-1b, whatever). If you're launching your cluster via the console that name should be shown in the list of available subnets (I haven't manually launched an EMR cluster in years, so don't know for sure).

Alternatively, you can give your subnets arbitrary tags. This is probably most useful if you're accessing them programmatically.

Semiconductor

And after identifying private subnet,Since EMR does not allow mix of private and public subnet, I deleted public subnet from list and kept only private subnet then it says a warning No Amazon S3 endpoint found in subnet-***. S3 endpoints are recommended for EMR to access data in S3. Whats the resolution of this. NatGateway is associated with subnet. – Itemize

Answer 3

I found this to be rather challenging. I hope others find this useful. This approach uses aws cli and jq. As of mid 2021 I'm using current versions of both. To caveat, this is specific to only IPv4, and VPCs with only 1 internet gateway.

From a bash shell start off by identifying the VPC in which you are considering the subnets. In my use case I wanted the VPC hosting an EKS cluster.

VPC_ID=`aws eks describe-cluster \
  --name ${CLUSTER_NAME} \
  --query "cluster.resourcesVpcConfig.vpcId" \
  --output text`

Find the internet gateway for that VPC

IGW_ID=`aws ec2 describe-internet-gateways \
  --filters Name=attachment.vpc-id,Values=${VPC_ID} \
  --query "InternetGateways[].InternetGatewayId" \
  | jq -r '.[0]'`

Identify the public subnets

PUBLIC_SUBNETS=`aws ec2 describe-route-tables \
  --query  'RouteTables[*].Associations[].SubnetId' \
  --filters "Name=vpc-id,Values=${VPC_ID}" \
    "Name=route.gateway-id,Values=${IGW_ID}" \
  | jq . -c`

Show the public subnets

echo $PUBLIC_SUBNETS

List the private subnets by excluding public subnets

aws ec2 describe-subnets \
  --filter Name=vpc-id,Values=${VPC_ID} \
  --query 'Subnets[].SubnetId' \
  | jq -c '[ .[] | select( . as $i | '${PUBLIC_SUBNETS}' | index($i) | not) ]'

Quittor

Answer 4

I was reading through this user guide VPC with a single subnet and found this line helpful.

"A subnet that's associated with a route table that has a route to an internet gateway is known as a public subnet"

Jeaz