Shiny Reactivity Fails with data.tables

Question

As a follow up to Modifying a reactive value should trigger an observe chunk, I investigated the issue further and realised that the issue probably stems from my usage of data.tables. data.tables are modified by reference unlike data.frames which makes them more efficient to use but also apparently invisible to shiny's reactivity.

In the below example, pressing the commit button triggers the observe chunk in the data.frame case but not in the data.table case. A workaround could be to have a value linked to the changing of data.table which also helps trigger the reactivity.

with data.frame

shinyServer ( 

   function(input, output, session) {

      lReactiveValues = reactiveValues(a = data.frame(firstcol = runif(1)))

      voidaA = observeEvent(
         input$buttonCommit,
         {
            new = runif(1)
            cat(new,' one\t')
            lReactiveValues$a[letters[ncol(isolate(lReactiveValues$a))]] = new
         }
      )

      voidB = observe(priority = 50,{
         # input$buttonCommit
         cat(ncol(lReactiveValues$a))

         counter = runif(1)
         cat(counter,' two\t'); 

         if (counter > 0.5) {

            cat('\n')

            cat(ncol(lReactiveValues$a),' three\n')

         }
      }
      )

   }
)

with data.table

shinyServer ( 

   function(input, output, session) {

      lReactiveValues = reactiveValues(a = data.table(firstcol = runif(1)))
      # lReactiveValues = reactiveValues(a = data.frame(firstcol = runif(1)))

      voidaA = observeEvent(
         input$buttonCommit,
         {
            new = runif(1)
            cat(new,' one\t')
            setnames(
               lReactiveValues$a[, newcol := new],
               'newcol',
               letters[ncol(isolate(lReactiveValues$a))]
            )
            cat(ncol(lReactiveValues$a))
         }
      )

      voidB = observe(priority = 50,{
         # input$buttonCommit
         cat(ncol(lReactiveValues$a))

         counter = runif(1)
         cat(counter,' two\t'); 

         if (counter > 0.5) {

            cat('\n')

            cat(ncol(lReactiveValues$a),' three\n')

         }
      }
      )

   }
)

ui.r

dashboardPage(

   dashboardHeader(

      title = "Analytics"

   ),

   ## Sidebar content
   dashboardSidebar(
   menuItem("Analysis", tabName = "tabAnalysis", icon = icon("calculator"))
   ),

   ## Body content
   dashboardBody(
      tabItems(
         tabItem(
            tabName = "tabAnalysis",
            actionButton("buttonCommit", "Commit!")
         )
      )
      #, style='width: 100%; height: 100%;'

   )
)

Short summary of what the code does - Pressing button should print some text to the console which includes the string 'one'. The button should further trigger the observe chunk prompting the printing of some text containing the string 'two'. Depending on the if condition in the observe chunk, another set of text including 'three' might get printed. In the data.frame's server.r case this behaviour persists all the time the app runs. In the data.table's server.r case, this behaviour occurs for a few clicks of the button after which only the 'one' string is printed and the 'two' and 'three' aren't. This flip in behaviour, think, occurs after the if condition is found to be false for the first time.

Midwifery

you can try copy to force modify object's address in memory – Orlantha

If so, that's a good tip. But data tables are usually large datasets and I would like to avoid the copy if I can. – Midwifery

I've used a lot of data.table with shiny but never as reactive values. I was updating by reference and using reactive function and never had any problem because of that. I cannot figured out your example code to recommend you some workaround. – Orlantha

@Orlantha - good point. I've added a description to the question. – Midwifery

Answer 1

A data.table can be modified using [[ or $ and saved into the reactive expression. Such as:

dt <- reactiveVal(data.table(a=c(1,2),b=c(10,20))

Within a reactive expression the following will change value of b from 10 to 100 and trigger an observer

observeEvent(dt(), { print(dt()) })
observeEvent(input$mybutton, { 
  x <- dt()
  x[a==1][["b"]] <- 100
  dt(x)
})

Following using your example code and reactiveValues

server <- shinyServer ( 
  
  function(input, output, session) {
    
    lReactiveValues = reactiveValues(a = data.table(firstcol = runif(1)))
    # lReactiveValues = reactiveValues(a = data.frame(firstcol = runif(1)))
    
    voidaA = observeEvent(
      input$buttonCommit,
      {
        new = runif(1)
        cat(new,' one\t')
        
        lRV <- lReactiveValues$a
        lRV[["newcol"]] <- new
        lReactiveValues$a <- lRV
        
        cat(ncol(lReactiveValues$a))
      }
    )
    
    voidB = observe(priority = 50,{
      # input$buttonCommit
      cat(ncol(lReactiveValues$a))
      
      counter = runif(1)
      cat(counter,' two\t'); 
      
      if (counter > 0.5) {
        
        cat('\n')
        
        cat(ncol(lReactiveValues$a),' three\n')
        
      }
    }
    )
    
  }
)

ui <- dashboardPage(
  
  dashboardHeader(
    
    title = "Analytics"
    
  ),
  
  ## Sidebar content
  dashboardSidebar(
    menuItem("Analysis", tabName = "tabAnalysis", icon = icon("calculator"))
  ),
  
  ## Body content
  dashboardBody(
    tabItems(
      tabItem(
        tabName = "tabAnalysis",
        actionButton("buttonCommit", "Commit!")
      )
    )
    #, style='width: 100%; height: 100%;'
    
  )
)

shinyApp(ui,server)

Northwestward

Answer 2

A hacky solution if you are concerned about memory:

Add A DTControl value:

lReactiveValues = reactiveValues(a = data.table(firstcol = runif(1)), DTControl = 0)

Then pair that with the following after your reference change:

lReactiveValues$a[, newcol := new]
lReactiveValues$DTControl <- lReactiveValues$DTControl + 1

Then add this line in your observe function:

lReactiveValues$DTControl

Because your DTControl updates, the reactive function gets triggered. Takes a bit of diligence to make sure you are always pairing the two together, but it can override this reference issue without any memory load.

Regardless, this should be highlighted as an issue in shiny.

Tuinenga

Thanks, but I've already mentioned this work around in my question. – Midwifery

Answer 3

Note that there is a closed GitHub issue related to this (see here).

As mentioned in the issue, using dt$val <- x works where dt[, val := x] does not register as changed in shiny's reactivity.

A comparison would be:

library(shiny)

ui <- fluidPage(
  verbatimTextOutput("text_fails"),
  actionButton("go_fails", "This does not work!"),
  
  verbatimTextOutput("text_works"),
  actionButton("go_works", "This works")
)

server <- function(input, output, session) {
  
  state <- reactiveValues(
    dd_fails = data.table::data.table(x = 1),
    dd_works = data.table::data.table(x = 1)
  )
  output$text_fails <- renderPrint(print(state$dd_fails))
  output$text_works <- renderPrint(print(state$dd_works))
  
  observeEvent(input$go_fails, {
    # also fails:
    # dd_fails <- state$dd_fails
    # dd_fails[, x := rnorm(1)]
    # state$dd_fails <- dd_fails
    state$dd_fails[, x := rnorm(1)]
  })
  observeEvent(input$go_works, {
    # dd_works <- state$dd_works
    # dd_works$x <- rnorm(1)
    # state$dd_works <- dd_works
    # alternatively
    state$dd_works$x <- rnorm(1)
  })
}

shinyApp(ui, server)

Cucumber