quantile cut by group in data.table

Question

I would like to do quantile cuts (cut into n bins with equal number of points) for each group

qcut = function(x, n) {
  quantiles = seq(0, 1, length.out = n+1)
  cutpoints = unname(quantile(x, quantiles, na.rm = TRUE))
  cut(x, cutpoints, include.lowest = TRUE)
}

library(data.table)
dt = data.table(A = 1:10, B = c(1,1,1,1,1,2,2,2,2,2))
dt[, bin := qcut(A, 3)]
dt[, bin2 := qcut(A, 3), by = B]

dt
A     B    bin        bin2
 1:  1 1  [1,4]    [6,7.33]
 2:  2 1  [1,4]    [6,7.33]
 3:  3 1  [1,4] (7.33,8.67]
 4:  4 1  [1,4]   (8.67,10]
 5:  5 1  (4,7]   (8.67,10]
 6:  6 2  (4,7]    [6,7.33]
 7:  7 2  (4,7]    [6,7.33]
 8:  8 2 (7,10] (7.33,8.67]
 9:  9 2 (7,10]   (8.67,10]
10: 10 2 (7,10]   (8.67,10]

Here the cut without grouping is correct -- data lie in the bin. But the result by group is wrong.

How can I fix that?

Kilocycle

dt[, qcut(A, 3), by = B] works though – Deitz

Answer 1

This is a bug in handling of factors. Please check if it is known (or fixed in the development version) and report it to the data.table bug tracker otherwise.

qcut = function(x, n) {
  quantiles = seq(0, 1, length.out = n+1)
  cutpoints = unname(quantile(x, quantiles, na.rm = TRUE))
  as.character(cut(x, cutpoints, include.lowest = TRUE))
}

dt[, bin2 := qcut(A, 3), by = B]
#     A B    bin        bin2
# 1:  1 1  [1,4]    [1,2.33]
# 2:  2 1  [1,4]    [1,2.33]
# 3:  3 1  [1,4] (2.33,3.67]
# 4:  4 1  [1,4]    (3.67,5]
# 5:  5 1  (4,7]    (3.67,5]
# 6:  6 2  (4,7]    [6,7.33]
# 7:  7 2  (4,7]    [6,7.33]
# 8:  8 2 (7,10] (7.33,8.67]
# 9:  9 2 (7,10]   (8.67,10]
#10: 10 2 (7,10]   (8.67,10]

Unshaped

without changing the function, dt[, bin2 := as.character(qcut(A, 3)), by=B] works too and if trying to convert it to a factor (dt[, bin2 := as.factor(as.character(qcut(A, 3))), by=B]) throws an error... – Deitz

Yeah, if you define factors per group, the final column (combining the groups) will just take attributes (like levels) from group 1, I think github.com/Rdatatable/data.table/issues/967 – Jaqitsch