Calculate cumulative sum (cumsum) by group

Question

With data frame:

df <- data.frame(id = rep(1:3, each = 5)
                 , hour = rep(1:5, 3)
                 , value = sample(1:15))

I want to add a cumulative sum column that matches the id:

df
   id hour value csum
1   1    1     7    7
2   1    2     9   16
3   1    3    15   31
4   1    4    11   42
5   1    5    14   56
6   2    1    10   10
7   2    2     2   12
8   2    3     5   17
9   2    4     6   23
10  2    5     4   27
11  3    1     1    1
12  3    2    13   14
13  3    3     8   22
14  3    4     3   25
15  3    5    12   37

How can I do this efficiently? Thanks!

Gilmore

Answer 1

df$csum <- ave(df$value, df$id, FUN=cumsum)

ave is the "go-to" function if you want a by-group vector of equal length to an existing vector and it can be computed from those sub vectors alone. If you need by-group processing based on multiple "parallel" values, the base strategy is do.call(rbind, by(dfrm, grp, FUN)).

Ticklish

Error in unique.default(x, nmax = nmax) : unique() applies only to vectors – Gilmore

I keep forgetting ... need to name the FUN argument. – Ticklish

Note that you can add additional id variables if multiple columns define each unique row. e.g., df$csum <- ave(df$value, df$id1, df$id2, FUN=cumsum). – Catastrophe

@42- plyr was mothballed as of 2013 (six years ago already). You should be recommending dplyr/tidyverse/data.table – Broeker

@smci: Did you look at the date of the comment? Are you suggesting I go back through all my comments and update them? And that's not to mention the fact that I don't really like either plyr or dplyr, anyway. (And I did mention data.table.) So I decided to just delete the comment and put the useful stuff in the answer. – Ticklish

@42- No, only important old ones that are both out-of-date and likely to be used as a close target for other questions. Thanks for updating the answer. Still, I never see ave used, these days it's all dplyr or data.table. (Why do you dislike them?) and could you edit your answer to list the latter two first? – Broeker

I didn't say I disliked data.table. I said I had mentioned it in the now deleted comment. I voted up A5C1D2H2I1M1N2O1R2T1's solution which I will admit is very clear – Ticklish

instead of do.call(rbind,...) I use Reduce(rbind,...). What is the difference? – Crudden

Answer 2

To add to the alternatives, data.table's syntax is nice:

library(data.table)
DT <- data.table(df, key = "id")
DT[, csum := cumsum(value), by = key(DT)]

Or, more compactly:

library(data.table)
setDT(df)[, csum := cumsum(value), id][]

The above will:

• Convert the data.frame to a data.table by reference
• Calculate the cumulative sum of value grouped by id and assign it by reference
• Print (the last [] there) the result of the entire operation

"df" will now be a data.table with a "csum" column.

Welsh

Answer 3

Using dplyr::

require(dplyr)
df %>% group_by(id) %>% mutate(csum = cumsum(value))

Myranda

Hey, I tried your method. Somehow the grouping is not working properly. It does cumsum for all the data points without grouping. any suggestions? – Szymanski

sometimes starting a fresh r session helps in those cases. try my code on the sample data. – Myranda

Just an update, you might have a package that has loaded plyr. Explicitly referencing dplyr will fix it also: ``` df %>% group_by(id) %>% dplyr::mutate(csum = cumsum(value)) ``` – Illinois

my explicit reference to dplyr does not yield an accurate by-group result; using ave() did – Carlinecarling

Answer 4

Using library plyr.

library(plyr)
ddply(df,.(id),transform,csum=cumsum(value))

Scum

Answer 5

Using base R

df <- data.frame(id = rep(1:3, each = 5)
                 , hour = rep(1:5, 3)
                 , value = sample(1:15))

transform(df , csum = ave(value , id , FUN = cumsum))
#>    id hour value csum
#> 1   1    1     4    4
#> 2   1    2    12   16
#> 3   1    3    13   29
#> 4   1    4     6   35
#> 5   1    5     5   40
#> 6   2    1    15   15
#> 7   2    2     1   16
#> 8   2    3     2   18
#> 9   2    4     8   26
#> 10  2    5     9   35
#> 11  3    1    11   11
#> 12  3    2     7   18
#> 13  3    3    10   28
#> 14  3    4     3   31
#> 15  3    5    14   45

^{Created on 2022-06-05 by the reprex package (v2.0.1)}

Precursor