Can I use AWS LightSail with AWS CloudWatch?

Question

I've recently started testing out LightSail, but I would like to keep my logging centralized in CloudWatch, but cannot seem to find anything that would enable this. Interestingly LightSail instances do not appear in the EC2 Dashboard. I thought they were just EC2 instances beneath the surface.

Nathalia

Answer 1

I thought they were just EC2 instances beneath the surface.

Yes... but.

Conceptually speaking, you are the customer of Lightsail, and Lightsail is the customer of EC2.

It's as though there were an intermediary between you and AWS. The Lightsail resources are in EC2, but they're not in your EC2. They appear to be owned by an AWS account other than your AWS account, so you can't see them directly.

Parallels for this:

RDS is a "customer" of EC2/EBS. RDS instances are EC2 machines with EBS volumes. Where are they in the console? They aren't there. The underlying resources aren't owned by your account.

In EC2, EBS snapshots are stored in S3. Which bucket? Not one that you can see. EBS is a "customer" of S3. It has its own buckets.

S3 objects can be migrated to the Glacier storage class. Which Glacier vault? Again, not one that you can see. S3 is a "customer" of Glacier. It has its own vaults.

Every API Gateway endpoint is automatically front-ended by CloudFront. Which distribution? You get the idea... API Gateway is a "customer" of CloudFront.

I am not implying in any way that Lightsail is actually a separate entity from AWS in any meaningful sense... I don't know how it's actually organized... but operationally, that is how it works. You can't see these resources.

Buehler

Thank you so much for clarifying this. It actually makes sense now. – Nathalia

Answer 2

It's possible to get it working. The problem is that Lightsail instances are EC2 instances under the hood, but without access to all of the EC2 configuration. The CloudWatch agent documentation explains how to set up IAM roles for EC2 instances to assume, but Lightsail boxes only use a single role which can't be changed and can't be edited. As a result, you need to follow instructions for setting it up as an on-premise server.

The problem you will then hit is as David J Eddy saw in his answer:

2018-10-20T16:04:37Z E! WriteToCloudWatch failure, err:  AccessDenied: User: arn:aws:sts::891535117650:assumed-role/AmazonLightsailInstanceRole/i-0788a602f758b836f is not authorized to perform: cloudwatch:PutMetricData status code: 403, request id: b443ecc6-d481-11e8-a551-6d030b8667be

This is due to a bug in the CloudWatch agent which ignores the argument to use on-premise mode (-m onPremise) if it detects it is running on an EC2 instance. The trick is to edit the common-config.toml file to force using a local AWS CLI profile for authentication. You will need to add the following lines to that file (which can be found at /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml on Debian - the installation location is OS dependent):

[credentials]
shared_credential_profile = "AmazonCloudWatchAgent"

Restart the agent and it should start reporting metrics. I've put together a full tutorial here

Spathic

Answer 3

Running the CloudWatch Agent on Lightsail does NOT work at this time. When the agent attempts to communicate with CloudWatch it receives a 403 from the STS service. Selecting EC2 or OnPremise options during configuration wizards yields the same results.

2018-10-20T16:04:37Z E! WriteToCloudWatch failure, err:  AccessDenied: User: arn:aws:sts::891535117650:assumed-role/AmazonLightsailInstanceRole/i-0788a602f758b836f is not authorized to perform: cloudwatch:PutMetricData status code: 403, request id: b443ecc6-d481-11e8-a551-6d030b8667be

Just to make sure, I installed the CloudWatch Agent on my Ubuntu 18.04 desktop and started the agent without error.

Plus, if it did work, why would people pay for EC2 at a higher prices point? CloudWatch is a free value added service for using the full services.

Illegitimate