accumulation of value in Graphite

Question

I send one message each time a user connect to my site.

With this format:

"user_login 1 13xxxxxxx"  (key value timestamp)

My problem is Graphite give me a graph with only a line with the value "1", each minute. I send a random number of messages each minutes (between 6 and 60), but I think Graphite limits to the first (or last?) message, to display a value.

How to do to have an sum of each message, each minute ?

Daryn

Answer 1

You could also try carbon aggregation service so that metrics are aggregated when they are collected instead of in the Graphite UI. See http://graphite.readthedocs.org/en/latest/config-carbon.html#aggregation-rules-conf for aggregation rule configuration. Note that you need to send the metrics to carbon aggregation service port that is running in different port than the normal carbon collector port.

Tanning

Answer 2

You might be interested in running statsd or statsite on your graphite server. That software will aggregate all statistics per a configurable time period, and write the result out to graphite.

Take a look at github.com/etsy/statsd (in node.js) and github.com/kiip/statsite (python clone). Etsy's statsd contains a few code examples on how to use it.

Bibb

This is what I ended up doing. Kind of annoying to have another service running, but it's a little service. – Gynecoid

And I actually used the C (more recent) version of statsite: github.com/armon/statsite . The only annoying thing was that I had to change the default "sink" file to drop the "statsite.counts." prefix that statsite prepends before whatever stats you send it by default. – Gynecoid

Answer 3

use sumSeries target=sumSeries(graphite.counterName) or summarize function if you want to sum over more than one minute

Unfix

Answer 4

Jari is correct, carbon-aggregator was written specifically for this.

Make sure you use a sum function in aggregation-rules.conf, such as this:

bpu.<cluster>.<account>.<metric>_sum (1) = sum bpu.<cluster>.<account>.<metric>_sum

Semimonthly

Answer 5

hitcount(seriesList, intervalString, alignToInterval=False)

http://code.hootsuite.com/accurate-counting-with-graphite-and-statsd/

http://obfuscurity.com/2012/04/Unhelpful-Graphite-Tip-1

http://graphite.readthedocs.org/en/latest/functions.html#graphite.render.functions.hitcount

Cauthen

Answer 6

This may also be how your data is aggregated per metric, it defaults to averages which may not be what you expects. See also Tracking metrics using StatsD (via etsy) and Graphite, graphite graph doesn't seem to be graphing all the data.

Grandma

Answer 7

Use the cumulative function on your series.

By default, when a graph is drawn, and the width of the graph in pixels is smaller than the number of datapoints to be graphed, Graphite averages the value at each pixel. The cumulative() function changes the consolidation function to sum from average.

Disconcerted

Answer 8

by default, graphite only stores the last value sent for any incoming metric in the minimum retention granularity (not the mean value). if you want aggregates, the common solution is to use statsd.

however, you can also use carbon-aggregator. just be sure you are sending your metrics to the aggregator line port, then update aggregation-rules.conf with a rule like this:

user_login (10) = sum user_login$

note: this assumes your minimum retention period is still 10 seconds (the default) you can confirm that by looking in storage-schemas.conf

for a longer explanation, see: Why use statsd when graphite's Carbon aggregator can do the same job?

Belshazzar