Just thought I'd share this in case anyone else has run into this.
I did something similar today and it took me a while to figure out why this was causing a problem at runtime.
This code:
Public Class foo
  Public bar As String = "blah"
End Class
Public Sub DoInline()
  Dim o As New foo
  Dim f As Func(Of String)
  With o
    f = Function() .bar
  End With
  Try
    Console.WriteLine(f.DynamicInvoke())
  Catch ex As Reflection.TargetInvocationException
    Console.WriteLine(ex.InnerException.ToString)
  End Try
End Sub
Throws a NullReferenceException. It seems as though the With is using the closure as its temp storage, and at the "End With", it sets the closure's variable to Nothing.
Here is that code in RedGate Reflector:
Public Shared Sub DoInline()
    Dim o As New foo
    Dim $VB$Closure_ClosureVariable_7A_6 As New _Closure$__1
    $VB$Closure_ClosureVariable_7A_6.$VB$Local_VB$t_ref$L0 = o
    Dim f As Func(Of String) = New Func(Of String)(AddressOf $VB$Closure_ClosureVariable_7A_6._Lambda$__1)
    $VB$Closure_ClosureVariable_7A_6.$VB$Local_VB$t_ref$L0 = Nothing 
    Try 
        Console.WriteLine(RuntimeHelpers.GetObjectValue(f.DynamicInvoke(New Object(0  - 1) {})))
    Catch exception1 As TargetInvocationException
        ProjectData.SetProjectError(exception1)
        Console.WriteLine(exception1.InnerException.ToString)
        ProjectData.ClearProjectError
    End Try
End Sub
Notice the
$VB$Closure_ClosureVariable_7A_6.$VB$Local_VB$t_ref$L0 = Nothing 
Only "question" I can really ask is; is this a bug or a strange design decision that for some reason I'm not seeing. I'm pretty much just going to avoid using "With" from now on.