Exception handling with Async Task functions

I am beginning to learn about Async / Task functions for running cancellable SQL queries in VB.NET.

I have the following code in a class library which runs two tasks and does not have any exception handling, as I would like to handle this in the application that calls the class library. I have the following code in the class library.

Public Async Function DirectoryList(ct As CancellationToken) As Task(Of List(Of Directory))
    ct.ThrowIfCancellationRequested()

    Dim ds As DataSet
    Dim dirs As List(Of Directory)
    Dim ctUnregister As CancellationTokenRegistration

    ds = Await Task.Run(Function()
                            Using newConnection = New SqlConnection(Me.InitializationData.ConnectionString)
                                Using sqlAdapter = New SqlDataAdapter("DirectoryList", newConnection)
                                    ctUnregister = ct.Register(Sub() sqlAdapter.SelectCommand.Cancel())

                                    With sqlAdapter
                                        .SelectCommand.CommandType = CommandType.StoredProcedure
                                        .SelectCommand.CommandTimeout = Me.InitializationData.CommandTimeout
                                    End With

                                    Dim newDataSet As DataSet = New DataSet()

                                    sqlAdapter.Fill(newDataSet)
                                    Return newDataSet
                                End Using
                            End Using

                            ' Remove the registration we set earlier so the cancellation token can be used normally.
                            ctUnregister.Dispose()
                        End Function, ct)

    dirs = Await Task.Run(Function()
                              Dim dirsResult As New List(Of Directory)

                              Dim tbl As DataTable = ds.Tables(0)

                              For Each row As DataRow In tbl.Select()
                                  ' process the data

                                  ct.ThrowIfCancellationRequested()
                              Next

                              Return dirsResult
                          End Function, ct)

    Return dirs
End Function

I then call it as follows:

Try
    dirs = Await databaseLibrary.DirectoryList(cts.Token)
    MsgBox("Query complete!")
Catch ex As System.Data.SqlClient.SqlException
    MsgBox("Cancelled (SQL)")
Catch ex2 As OperationCanceledException
    MsgBox("Cancelled")
Catch ex3 As Exception
    MsgBox("Cancelled")
End Try

Functionally it seems to work as expected - I can cancel the requests and exceptions are thrown as expected.

However I would like to handle the exceptions so I can gracefully cancel the tasks, but even when running in Debug mode within the IDE, the app still breaks and the exception (e.g. SqlException) is shown within the IDE. If I step on, eventually the Catch logic runs.

When the application is run outside of the IDE, the exception handling works as expected.

This seems different to the normal behaviour when running in the debugger- normally the exception handling runs but only breaks if the exception is unhandled.

Why is this behaviour different, presumably because of the Async functions?