Why would disposal of resources be delayed when using the "use" binding within an async computation expression?

Question

I've got an agent which I set up to do some database work in the background. The implementation looks something like this:

let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
    let rec loop = 
        async {
            let! data = inbox.Receive()
            use conn = new System.Data.SqlClient.SqlConnection("...")
            data |> List.map (fun e -> // Some transforms)
                 |> List.sortBy (fun (_,_,t,_,_) -> t)
                 |> List.iter (fun (a,b,c,d,e) ->
                    try
                       ... // Do the database work
                    with e -> Log.error "Yikes")
            return! loop
        }
    loop)

With this I discovered that if this was called several times in some amount of time I would start getting SqlConnection objects piling up and not being disposed, and eventually I would run out of connections in the connection pool (I don't have exact metrics on how many "several" is, but running an integration test suite twice in a row could always cause the connection pool to run dry).

If I change the use to a using then things are disposed properly and I don't have a problem:

let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
    let rec loop = 
        async {
            let! data = inbox.Receive()
            using (new System.Data.SqlClient.SqlConnection("...")) <| fun conn ->
              data |> List.map (fun e -> // Some transforms)
                   |> List.sortBy (fun (_,_,t,_,_) -> t)
                   |> List.iter (fun (a,b,c,d,e) ->
                      try
                         ... // Do the database work
                      with e -> Log.error "Yikes")
              return! loop
        }
    loop)

It seems that the Using method of the AsyncBuilder is not properly calling its finally function for some reason, but it's not clear why. Does this have something to do with how I've written my recursive async expression, or is this some obscure bug? And does this suggest that utilizing use within other computation expressions could produce the same sort of behavior?

Keffiyeh

Shouldn't that be let rec loop() and return! loop(), that is, a function not a value? – Kero

I've never been 100% sure if it has to be a function rather than a value. I've used both in the past and they seem to work the same, but it's possible there is a subtle difference which could be contributing to my issues. – Keffiyeh

To be clear, I've NOT tried changing the value to a function in this case...at least not yet. Give me about 10 minutes :) – Keffiyeh

Changing from a value to a function doesn't seem to make a difference in this case. Nice thought though. – Keffiyeh

Answer 1

This is actually the expected behavior - although not entirely obvious!

The use construct disposes of the resource when the execution of the asynchronous workflow leaves the current scope. This is the same as the behavior of use outside of asynchronous workflows. The problem is that recursive call (outside of async) or recursive call using return! (inside async) does not mean that you are leaving the scope. So in this case, the resource is disposed of only after the recursive call returns.

To test this, I'll use a helper that prints when disposed:

let tester () = 
  { new System.IDisposable with
      member x.Dispose() = printfn "bye" }

The following function terminates the recursion after 10 iterations. This means that it keeps allocating the resources and disposes of all of them only after the entire workflow completes:

let rec loop(n) = async { 
  if n < 10 then 
    use t = tester()
    do! Async.Sleep(1000)
    return! loop(n+1) }

If you run this, it will run for 10 seconds and then print 10 times "bye" - this is because the allocated resources are still in scope during the recursive calls.

In your sample, the using function delimits the scope more explicitly. However, you can do the same using nested asynchronous workflow. The following only has the resource in scope when calling the Sleep method and so it disposes of it before the recursive call:

let rec loop(n) = async { 
  if n < 10 then 
    do! async { 
      use t = tester()
      do! Async.Sleep(1000) }
    return! loop(n+1) }

Similarly, when you use for loop or other constructs that restrict the scope, the resource is disposed immediately:

let rec loop(n) = async { 
  for i in 0 .. 10 do
    use t = tester()
    do! Async.Sleep(1000) }

Deficit

Ah, ok, that actually makes sense now that you point it out directly. I now feel dumb for not being able to realize the recursive call occurs within the scope of the use binding. Thanks a bunch. – Keffiyeh

@Keffiyeh I'm glad my explanation made sense. It took me some time to understand what is actually going on - so it is definitely a tricky case! – Deficit