Shared memcache vs. dedicated memcache & quota calculation

Question

I have an app that stores ~20000 entries in the memcache. Each entry is a Serializable with one String and two integers. I set the expiration time to 6 hours.

So I was using the shared/free memcache. It only seemed to store ~5000 entries -> ~7mb. The oldest entry is always just some minutes old. Why is that?

Then I thought: let's switch to dedicated memcache. Then the cache runs fine, it stores all entries, oldest entry is 6 hours old, everything is as expected. Except for the quota. After just some hours it says that I already used 18 "Gbyte hours".

Well my total cache size is ~11mb. So I would guess that the cost would be ($0.12 / Gbyte / hr) -> $0.12*~0.01Gb*24hrs per day, which would be just ~$0.03.

What am I doing wrong? Is my calculation wrong? Do I misunderstand the meaning of "Gbyte hours"?

Gwenette

Answer 1

The dedicated memcache bills by the gigabyte. So anything less than 1GB is billed as 1GB, and you probably had it running for 18hrs. Yeah, kinda sucks.

Decedent

Answer 2

AppEngine dedicated memcache is priced per 1GB chunks, not based on what you use in your dedicated allocated 1GB of storage. See here https://developers.google.com/appengine/docs/adminconsole/memcache

Your dedicated memcache will cost you a flat $2.88 per day.

The "dedicated" gives you space but operations per seconds (10,000 ops per second).

Regarding your experience on the shared (and free) memcache, what you are seeing here is not what you should generally expect. You are unfortunately most likely on an AppEngine cluster where some other apps are abusing the shared memcache.

Namesake

got it! for the cluster thing: can I switch the cluster my instances are running on? Or do anything else about it? – Gwenette

Nope, you're at the whim of who else is using the shared memcache with you. And since it's free, you can bet that everyone else is trying to use it as much as they can too. There's been numerous reports that shared memcache kinda sucks. I expect maybe if you get a ton of traffic, it'll work better for you, and if you get little traffic, it'll constantly be purged, cuz that's just the way memcache is supposed to work. – Decedent

I switched my apps to dedicated memcache, so I can only tell you what we have done before to improve the stickiness of our memcache content: what I noticed is that doing a put followed by a get on the same key was making this key stickier. If you do a put, but do not get the data shortly after, you reduce your cache hit rate. – Namesake