Unable to read .nettrace file collected by dotnet-trace

[cyang0718]

Description

Collected data using command
dotnet-trace collect --process-id **** --duration 00:00:00:30 --providers Microsoft-DotNETCore-SampleProfiler
But the result .nettrace file was not readable by either Visual Studio nor PerfView.
Visual Studio Log:

CreateEditorInstance
StreamJsonRpc.RemoteInvocationException: Not a understood file format: C:\Users\****\Downloads\w3wp.exe_20250919_140901.nettrace
   at StreamJsonRpc.JsonRpc.<InvokeCoreAsync>d__156`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.DiagnosticsHub.NetTraceDocumentConverter.<ConvertFromAsync>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.DiagnosticsHub.DocumentsService.<OpenAsConvertedAsync>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.DiagnosticsHub.DocumentsService.<OpenAsConvertedAsync>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.DiagnosticsHub.DocumentsService.<OpenAsConvertedAsync>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.DiagnosticsHub.VisualStudio.Shell.Views.NetTraceEditorFactory.<>c__DisplayClass6_0.<<CreateEditorInstance>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask.CompleteOnCurrentThread()
   at Microsoft.VisualStudio.Shell.VsTaskLibraryHelper.Run(JoinableTaskFactory joinableTaskFactory, String waitCaption, Func`3 asyncMethod, Nullable`1 delayToShowDialog)
   at Microsoft.DiagnosticsHub.VisualStudio.Shell.Views.NetTraceEditorFactory.<>c__DisplayClass6_0.<<CreateEditorInstance>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Shell.VsTaskLibraryHelper.<>c.<<FileAndForget>b__36_0>d.MoveNext()
RPC server exception:
FastSerialization.SerializationException: Not a understood file format: C:\Users\****\Downloads\w3wp.exe_20250919_140901.nettrace
      at FastSerialization.Deserializer.Initialize(IStreamReader reader, String streamName)
      at FastSerialization.Deserializer..ctor(IStreamReader reader, String streamName)
      at Microsoft.Diagnostics.Tracing.EventPipeEventSource.<>c__DisplayClass2_0.<.ctor>b__0()
      at Microsoft.Diagnostics.Tracing.EventPipeEventSource..ctor(PinnedStreamReader streamReader, String name, Boolean isStreaming)
      at Microsoft.Diagnostics.Tracing.EventPipeEventSource..ctor(String fileName)
      at Microsoft.Diagnostics.Tracing.Etlx.TraceLog.CreateFromEventPipeDataFile(String filePath, String etlxFilePath, TraceLogOptions options)
      at Microsoft.DiagnosticsHub.DataWarehouse.ServiceModule.Controllers.DataWarehouseTraceEventServiceController.ConvertAsync(Guid sessionId, ConvertDocumentPayload convertDocumentPayload, List`1 analyzerPayloads, CancellationToken cancellationToken)

PerfView Log:

Started with command line: PerfView.exe  w3wp.exe_20250919_140901.nettrace /ContinueOnError
PerfView Version: 3.1.24  BuildDate: Wed, 27 Aug 2025 11:35:13 GMT
PerfView Start Time 9/19/2025 2:20:41 PM
Started: View
Warning: PdbScope not found at **\Downloads\PerfViewExtensions\PdbScope.exe
Disabling the Image Size Menu Item.
Warning: ILSize not found at **\Downloads\PerfViewExtensions\ILSize.dll
Disabling the IL Size Menu Item.
Completed: View   (Elapsed Time: 0.098 sec)
Started: Opening w3wp.exe_20250919_140901.nettrace
Creating ETLX file **\AppData\Local\Temp\PerfView\w3wp.exe_20250919_140901_cdf8e802.etlx from **\Downloads\w3wp.exe_20250919_140901.nettrace
Error: Exception EventPipe conversion: System.FormatException: Invalid FastSerialization header
   at Microsoft.Diagnostics.Tracing.FastSerializationObjectParser.ValidateFastSerializationHeader()
   at Microsoft.Diagnostics.Tracing.EventPipeEventSource.ParseHeaders()
   at Microsoft.Diagnostics.Tracing.EventPipeEventSource..ctor(Stream stream, Boolean isStreaming)
   at Microsoft.Diagnostics.Tracing.Etlx.TraceLog.CreateFromEventPipeDataFile(String filePath, String etlxFilePath, TraceLogOptions options)
   at PerfView.EventPipePerfViewData.GetTraceLog(TextWriter log, Action`3 onLostEvents)
[Error: exception while opening EventPipe data.]
Completed: Opening w3wp.exe_20250919_140901.nettrace   (Elapsed Time: 0.123 sec)

Configuration

dotnet-trace Windows x64 version 9.0.12.21003 downloaded from https://aka.ms/dotnet-trace/win-x64
Perfview version is 3.1.19
Visual Studio 2022 Community version is 17.0.36407.19

Data collected on Windows Server 2022 x64 version 10.0.20348
Server is running IIS version 10.0
.net core runtime installed on server is 8.0.4

[cyang0718]

I noticed that FastSerialization in the stack-trace is expecting a header of 4 reserved bytes then followed by "!FastSerialization.1" in UTF8, so I'm including the first 32 bytes of my trace file here in hex:

0f 01 00 00 00 04 88 3b 0f 01 00 00 00 0c b0 4d 01 8d 98 01 01 00 00 00 0c bc 39 02 19 01 00 00

I don't know what this says but clearly not "!FastSerialization.1"

[mdh1418]

Hi @cyang0718, does this happen everytime you collect a trace? Did your dotnet-trace output look like

Q:\source\mdh1418\TestApp> dotnet-trace collect --process-id 49344 --duration 00:00:00:30 --providers Microsoft-DotNETCore-SampleProfiler

Provider Name                           Keywords            Level               Enabled By
Microsoft-DotNETCore-SampleProfiler     0x0000000000000000  Informational(4)    --providers

Process        : Q:\source\mdh1418\TestApi\bin\Debug\net8.0\win-x64\TestApi.exe
Output File    : Q:\source\mdh1418\TestApp\TestApi.exe_20250930_183218.nettrace
Trace Duration : 00:00:00:30

[00:00:00:30]   Recording trace 1.2078   (MB)
Press <Enter> or <Ctrl+C> to exit...
Stopping the trace. This may take several minutes depending on the application being traced.

Trace completed.

If those are the leading bytes of your .nettrace file, its likely that your trace is malformed/corrupted because typically nettrace files should start with

Are you able to reproduce the problem with a simple console app? Or does it only occur with w3wp.exe?

[cyang0718]

Hi @mdh1418 , Thank you for the reply.

does this happen everytime you collect a trace?

I ran the trace command multiple times on the server(which I temporarily do not have access to now until next week), none of the results were readable. These are the leading bytes from a different run:
00 0c fc 4c 0b 14 01 00 00 00 04 c4 08 11 01 00 00 00 0c b0 4d 01 cc b6 02 01 00 00 00 0c bc 39

Are you able to reproduce the problem with a simple console app? Or does it only occur with w3wp.exe?

I'm able to produce a readable trace with a console app running on my own workstation. However this was only possible if I run the console app directly via
dotnet consoleapp.dll
If I start the app using Visual Studio, even if I detach the VS debugger, I would still get the following error when running dotnet-trace on it:

Provider Name                           Keywords            Level               Enabled By
Microsoft-DotNETCore-SampleProfiler     0x0000000000000000  Informational(4)    --providers

[ERROR] System.TimeoutException: The operation has timed out.
   at System.IO.Pipes.NamedPipeClientStream.ConnectInternal(Int32 timeout, CancellationToken cancellationToken, Int32 startTime)
   at Microsoft.Diagnostics.NETCore.Client.IpcEndpointHelper.Connect(IpcEndpointConfig config, TimeSpan timeout)
   at Microsoft.Diagnostics.NETCore.Client.PidIpcEndpoint.Connect(TimeSpan timeout)
   at Microsoft.Diagnostics.NETCore.Client.IpcClient.SendMessageGetContinuation(IpcEndpoint endpoint, IpcMessage message)
   at Microsoft.Diagnostics.NETCore.Client.EventPipeSession.Start(IpcEndpoint endpoint, EventPipeSessionConfiguration config)
   at Microsoft.Diagnostics.NETCore.Client.DiagnosticsClient.StartEventPipeSession(EventPipeSessionConfiguration config)
   at Microsoft.Diagnostics.Tools.Trace.CollectCommandHandler.Collect(CancellationToken ct, CommandLineConfiguration cliConfig, Int32 processId, FileInfo output, UInt32 buffersize, String providers, String profile, TraceFileFormat format, TimeSpan duration, String clrevents, String clreventlevel, String name, String diagnosticPort, Boolean showchildio, Boolean resumeRuntime, String stoppingEventProviderName, String stoppingEventEventName, String stoppingEventPayloadFilter, Nullable`1 rundown, String dsrouter)

This is probably a whole other can of worm that I'm not really concerned with.

But I digress, so I think saying that it only occur with w3wp.exe is accurate.

[mdh1418]

So everytime you've collected a trace that was not readable by PerfView or Visual Studio it had an output like this?

Q:\source\mdh1418\TestApp> dotnet-trace collect --process-id 49344 --duration 00:00:00:30 --providers Microsoft-DotNETCore-SampleProfiler

Provider Name                           Keywords            Level               Enabled By
Microsoft-DotNETCore-SampleProfiler     0x0000000000000000  Informational(4)    --providers

Process        : Q:\source\mdh1418\TestApi\bin\Debug\net8.0\win-x64\TestApi.exe
Output File    : Q:\source\mdh1418\TestApp\TestApi.exe_20250930_183218.nettrace
Trace Duration : 00:00:00:30

[00:00:00:30]   Recording trace 1.2078   (MB)
Press <Enter> or <Ctrl+C> to exit...
Stopping the trace. This may take several minutes depending on the application being traced.

Trace completed.

It's not obvious to me what the culprit is that is corrupting your .nettrace files. I can only speculate, and it's not obvious to me that this is a problem with dotnet-trace itself.
I'm not at all familiar with w3wp.exe, but if you had some isolated repro that I could use to debug, that would be helpful.

When you open the .nettrace with a hex editor like in VS Code, do you see anything legible in the decoded text?

[cyang0718]

@mdh1418 I do not recall the exact output when I ran the collect command on the server but I'm certain I did not see any errors.
There are legible parts in the trace-file:

I'm not at all familiar with w3wp.exe, but if you had some isolated repro that I could use to debug, that would be helpful.

w3wp.exe is the worker process created by IIS application pools. It's a bootstrap for my actual web-app. I'm able to collect the readable trace-file when running my webapp on my workstation without the IIS application pool but I need to collect these data on the server where the CPU usage issue I'm investigating is occurring