Survival on binned data

Question

I have binned data I'm trying to perform a survival analysis on, example data below. n is a count of units at each group, time, failure indicator combination.

> df <- structure(list(group = structure(c(2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L), .Label = c("", "A", "B"), class = "factor"), t = c(0L, 1L, 2L, 3L, 1L, 2L, 3L, 0L, 1L, 2L, 3L, 1L, 2L, 3L), failure = c(0L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 1L, 1L, 1L), n = c(40000L, 30000L, 20000L, 10000L, 5L, 4L, 3L, 20000L, 15000L, 14000L, 11000L, 10L, 6L, 4L)), .Names = c("group", "t", "failure", "n"), row.names = c(NA, 14L), class = "data.frame")
> df
   group t failure     n
1      A 0       0 40000
2      A 1       0 30000
3      A 2       0 20000
4      A 3       0 10000
5      A 1       1     5
6      A 2       1     4
7      A 3       1     3
8      B 0       0 20000
9      B 1       0 15000
10     B 2       0 14000
11     B 3       0 11000
12     B 1       1    10
13     B 2       1     6
14     B 3       1     4

I know I can rep df by the n column so each row is one unit: (ref. How do I create a survival object in R?)

> library(survival)
> df2 <- df[rep(rownames(df),df$n),]
> sfit <- survfit(Surv(t,failure)~group, data = df2)

However, my actual data has about 10 million units. Is there a way to do survival with a count/frequency variable to avoid creating a 10 million row data frame?

Spherical

Answer 1

You'll want to use the weights parameter. You can compare the the two approaches to confirm that you have the same output.

With your data that you repeated:

sfit <- survfit(Surv(t,failure)~group, data = df2)
summary(sfit)
Call: survfit(formula = Surv(t, failure) ~ group, data = df2)

                group=A 
 time n.risk n.event survival  std.err lower 95% CI upper 95% CI
    1  60012       5    1.000 3.73e-05        1.000            1
    2  30007       4    1.000 7.63e-05        1.000            1
    3  10003       3    0.999 1.89e-04        0.999            1

                group=B 
 time n.risk n.event survival  std.err lower 95% CI upper 95% CI
    1  40020      10    1.000 0.000079        1.000            1
    2  25010       6    1.000 0.000126        0.999            1
    3  11004       4    0.999 0.000221        0.999            1

Now using weights:

weights <- df$n
sfit2 <- survfit(Surv(t,failure)~group, data = df, weights = weights)
summary(sfit2)
Call: survfit(formula = Surv(t, failure) ~ group, data = df, weights = weights)

                group=A 
 time n.risk n.event survival  std.err lower 95% CI upper 95% CI
    1  60012       5    1.000 3.73e-05        1.000            1
    2  30007       4    1.000 7.63e-05        1.000            1
    3  10003       3    0.999 1.89e-04        0.999            1

                group=B 
 time n.risk n.event survival  std.err lower 95% CI upper 95% CI
    1  40020      10    1.000 0.000079        1.000            1
    2  25010       6    1.000 0.000126        0.999            1
    3  11004       4    0.999 0.000221        0.999            1

Salford

Good answer to a good question, but there should be a warning that not all regression functions use the "weights" parameter in the same manner. These are "replicate" weights. – Sakai

I agree with 42-, I did not find the description of weights (under survfit.formula) helpful either: "The weights must be nonnegative and it is strongly recommended that they be strictly positive, since zero weights are ambiguous, compared to use of the subset argument." – Spherical