Powershell script to capture the error response and headers of http request

I am getting 403 error randomly for an http request. To test this I am making a powershell script that hits http request and log the error response, headers and error code of all 4XX series errors. I am able to log both the error code, error response but not the headers.

Below is the script,

# Define the log file path
$logFilePath = "https_test_connection_log.txt"

# Function to log messages with a timestamp
function Log-Message {
    param (
        [string]$Message
    )
    $timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
    Add-Content -Path $logFilePath -Value "$timestamp - $Message"
}

# Send a GET request to the API
try {
    # Make the HTTP GET request
    $response = Invoke-WebRequest -Uri $apiUrl -Method Get -ErrorAction Stop

    # Log successful response details
    Log-Message "Request to $apiUrl successful."

    # Log response headers if they are available
    if ($response.Headers -ne $null) {
        Log-Message "Response Headers:"
        foreach ($header in $response.Headers.GetEnumerator()) {
            Log-Message "$($header.Key): $($header.Value)"
        }
    } else {
        Log-Message "No headers found in the response."
    }

    # Log the response body (converted to JSON if possible)
    $responseBody = $response.Content | ConvertTo-Json -Compress
    Log-Message "Response Body: $responseBody"
}
catch {
    # Log the error message
    $errorMessage = "Error: $($_.Exception.Message)"
    Log-Message $errorMessage

     # Log the response body (converted to JSON if possible)
    Log-Message "Error Response: $_.ErrorRecord.Exception"

    # Check if an error response is available
    $ErrorResponse = $_.ErrorRecord.Exception.Response

    # Log the error response body if available
    $ErrorContent = (New-Object System.IO.StreamReader($ErrorResponse.GetResponseStream())).ReadToEnd()
    Log-Message "Error Response Body: $ErrorContent"

    Log-Message "Error Response Headers:"
    foreach ($header in $ErrorResponse.Headers.GetEnumerator()) {
        Log-Message "$($header.Key): $($header.Value)"
    }
        
}

I tried using both invoke-webrequest and invoke-restmethod, I am getting empty headers.

Any suggestions are appreciated.