How can I script an alert for when my Amazon Web Service usage goes above a certain amount?

Question

We're using S3, SimpleDB and SQS on quite a complicated project.

I'd like to be able to automatically track their usage, to be sure we don't suddenly spend large amounts of money when we didn't intend to (perhaps because of a bug).

Is there a way of reading the usage figures of all Amazon Web Services and/or the current real time dollar cost of an account from a script?

Or any service or script which provides alerts based on that?

Alternate

Good question - I love EC2, but I've always found it annoyingly dificult to figure out up to the minute usage costs. – Papoose

You might have better luck asking this on the EC2 support forum; please post the answer back here if you get one. – Papoose

Thanks Gareth - I've posted the question here developer.amazonwebservices.com/connect/… – Alternate

Answer 1

Amazon just announced that you can now "set alarms for any metric that Amazon CloudWatch monitors" (CPU utilization, disk reads and writes, and network traffic, etc). Also, all instances now come with basic monitoring for free.

Tragus

Answer 2

We just released a Lab Management service that adds policies to AWS usage: time limits, max number of instance, max machine sizes etc. You may want to try that and see if it helps: http://LabSlice.com. As this is a startup, we would really value feedback about how to resolve issues such as yours (ie. email me if you think the app could be better modified to meet your requirements).

I don't believe there is any direct way to control AWS costs to the dollar. I doubt that Amazon provides an API to get in-depth metrics on usage, as it's obviously not going to be in their interest to help you reduce costs. I actually ran into two instances where surprise costs arose in a company (bank) due to mis-configured scripts, so I know that it can be a problem.

Collusion

Answer 3

I ran into the same issue with EC2 instances, but addressed it in a different way -- instead of monitoring the instances, I had them automatically kill themselves after a set amount of time. From your description, it sounds like this may not be practical in your environment, but I thought I would share just in case it helps. My AMI was Fedora-based, so I created the following bash script, registered it as a service, and had it run at startup:

#!/bin/bash
# chkconfig: 2345 68 20
# description: 50 Minute Kill
# Source Functions
. /etc/rc.d/init.d/functions

start()
{
    # Shut down 50 minutes after starting up
    at now + 50 minutes < /root/atshutdown
}

stop()
{
    # Remove all jobs from the at queue because I'm not using at for anything else
    for job in $(atq | awk '{print $1}')
    do
        atrm $job
    done
}

case "$1" in
    start)
        start && success || failure
        echo
        ;;
    stop)
        stop && success || failure
        echo
        ;;
    restart)
        stop && start && success || failure
        echo
        ;;
    status)
        echo $"`atq`"
        ;;
    *)
        echo $"Usage: $0 {start | stop | restart}"
        RETVAL=1
esac    

exit $RETVAL

You might consider doing something similar to suit your needs. If you do this, be especially careful that you stop the service before modifying your image so that the instance does not shutdown before you get a chance to re-bundle.

If you wanted, you could have the instances shutdown at a fixed time (after everyone leaves work?), or you could pass in a keep-alive length/shutdown time via the -d or -f parameters to ec2-run-instances and parse it out into the script.

Tragus

Thank you - and we are planning to make all our instances die out after a time. However, that still doesn't stop a bug making 10,000 instances and them being charged for an hour. And it doesn't stop other bugs in use of SQS or S3 causing massive charges. – Alternate