The World Ends with ETW

How to Investigate ETL (Event Trace Log) Files

Introduction

When it comes to Windows logs, the first thing that usually comes to mind is Event Logs in EVTX format. But there's another type of log called ETL (Event Trace Log), too (though it's not nearly as well known).

ETL files are used for Windows diagnostics, application debugging, performance analysis, and more. They can contain detailed records of system activity, including processes, file operations, and network communication, making them another useful source of information for Windows forensics.

If you've ever investigated Windows Event Logs, you probably know that the information you want is often missing.

By default, detailed process activity, file operations, and network communication may not be recorded. Logs may also have been deleted by an attacker or overwritten due to rotation settings, leaving only the last hour or two of activity. None of this is particularly unusual.

On the other hand, ETL files may contain more detailed information than Event Logs. Sometimes, even when the Event Logs have been deleted, the ETL files are still there.

In this article, I'll go over what ETW and ETL actually are and share some tips for investigating ETL files.

1. ETW Overview

What is ETW?

ETW (Event Tracing for Windows) is a Windows mechanism for collecting events emitted by the operating system and applications. ETL (Event Trace Log) is the file format used to store the collected events.

ETW was introduced in Windows 2000 and significantly expanded in Windows Vista.

With Windows Vista, the event publishing model was integrated with Windows Event Log, allowing events to be handled through a shared infrastructure.

This article focuses on ETW in Windows Vista and later.

How Does It Differ from Windows Event Logs?

Windows Event Log and ETW share parts of the underlying event publishing infrastructure. The main difference is how events are collected and stored.

Windows Event Log saves events assigned to channels in EVTX files. ETW, on the other hand, collects events through sessions, which can write them to ETL files or deliver them to consumers in real time.

For example, a system might record an error message in an Event Log while collecting the beginning and end of the corresponding operation in a separate ETW session. In that case, the EVTX file would contain the error, while the ETL file would contain information about the operations surrounding it.

The same event may appear in both, depending on the collection configuration. This means that an ETL file may contain process activity, file operations, or network destinations that are absent from the EVTX files.

That said, tools such as Sysmon can also record detailed information in EVTX files, so ETL is not necessarily more detailed in every case.

ETW Components

ETW consists of the following components.

The following diagram provides a simplified overview of these components and how events flow between them. Solid arrows represent event flow, while dashed arrows represent configuration operations. The ETL file highlighted in blue is what we'll be investigating in this article.

First, the Controller starts and stops sessions, enables or disables Providers, and configures collection settings. Typical Controllers include logman.exe and Windows Performance Recorder.

When a Controller enables a Provider, the Provider emits events. These events are filtered according to the conditions configured by the Controller, such as Level (severity), Keyword (category), and the Providers selected for collection, before being delivered to a Session.

The events are first collected in memory buffers, which are then used to write ETL files or deliver events to Consumers in real time.

A single Session can collect events from multiple Providers. Multiple Sessions can also run at the same time, and events emitted by the same Provider may be delivered to several Sessions and saved in separate ETL files.

Each Session has its own collection and storage settings. In circular logging, older data is overwritten. When investigating an ETL file, it is therefore worth checking not only its filename but also the Providers it contains and the time range it covers.

For a more detailed diagram, see Microsoft's ETW Architecture.

2. Preserving ETL Files

ETL files can be stored in all sorts of locations, depending on the application and its configuration.

When collecting evidence, you can search the system and copy the files directly, or use an acquisition tool that supports ETL collection.

Live Acquisition

Using Built-in Windows Tools

You can use Windows' built-in robocopy to copy all files matching *.etl* from C:\ to E:\Cases\etls, preserving their relative paths.

Run the following commands with administrator privileges.

Check etl-copy.log for files that could not be copied due to access restrictions, open handles, or other errors.

Note that /COPY:DAT does not preserve access control lists (ACLs), ownership, or other security metadata. This example is intended for collecting ETL files rather than performing a complete forensic acquisition.

After copying the files, it's a good idea to calculate and record SHA-256 hashes so their integrity can be verified later.

As mentioned earlier, events are initially collected in memory buffers. This method only captures data that has already been written to disk.

If you also want to investigate events that have not yet been written to an ETL file, another option is to acquire a memory image and use etw-scan, a Volatility 3 plugin.

However, not every event can necessarily be recovered from memory.

Using Acquisition Tools

Ideally, I'd prefer to collect everything with an acquisition tool, but tools with comprehensive ETL collection support still seem somewhat limited.

KAPE can collect related files using predefined Targets. Depending on the selected Targets, it can collect not only ETL files but also registry information and other artifacts from the source system.

Be aware that KAPE has restrictions on commercial use.

Again, run this with administrator privileges.

The --tdd false option prevents files with identical contents but different paths from being deduplicated.

Of course, these Targets won't collect every ETL file on the system. You may need to add custom collection definitions or acquire certain files separately.

Velociraptor also provides Windows.Triage.Targets, which includes collection definitions derived from KapeFiles.

I haven't used it myself yet, but it may be a good alternative if you're already familiar with Velociraptor.

Acquiring ETL Files from Disk Images

One of the most reliable approaches is still to acquire a disk image.

Memory-resident information will be lost unless it is collected separately, but having a disk image means you can return to the evidence and extract additional artifacts later. It's worth considering, especially when the number of target systems is relatively small.

To search for and extract files directly from disk images, you can use ntfsfind and ntfsdump.

As an example, I used ntfsfind to search for .etl files in the disk image from NIST's CFReDS Data Leakage Case.

Here are the results.

That's quite a few files.

To extract them, simply pipe the results from ntfsfind into ntfsdump.

For more details, see NTFS Spinal Cord Sword.

Other Information Worth Preserving

To maintain the evidentiary value of collected logs, it's important to document which system they came from and how they were acquired.

From a practical standpoint, preserving information about the source environment also saves you from having to investigate the same details again later.

At a minimum, I'd recommend keeping the following information.

You can collect some of this information using PowerShell.

During live acquisition, you can also use logman query to record information about the currently available sessions and Providers.

Of course, these commands only show the state of the system at the time of acquisition. Historical session settings or definitions may no longer be available.

Ideally, you'd also preserve .man files and DLLs or EXEs containing the relevant Provider definitions. Doing all of that during initial triage may not always be realistic, though. I'll return to this later.

3. Converting ETL Files

ETL files can be read on a different computer from the one they were collected from. However, if the analysis environment lacks the required Provider definitions, application metadata, or matching versions, some events may not be decoded correctly.

Parsing the files in an environment matching the source system, or preserving files containing the relevant definitions, may allow you to recover more information.

Event Metadata Formats

An event's payload contains its event-specific fields. Without the appropriate metadata, the field names and data types may not be interpreted correctly.

For more details, see Microsoft's Event Metadata Overview.

Some of the main metadata formats are listed below.

If an ETL file cannot be decoded, first determine whether the problem is missing metadata or a limitation of the tool itself.

Converting ETL Files with Built-in Windows Tools

You can inspect ETL files using tracerpt or Get-WinEvent.

For us mere humans, CSV is usually easier to read. But if you need to inspect field names or the underlying structure in more detail, XML is also an option.

Here's an example of the information you can get.

For more information, see the documentation for tracerpt and Get-WinEvent.

When you move an ETL file to another computer, you may need Provider definitions that are only available on the original system.

If the source environment is still accessible, you can use wevtutil gp to identify the associated files registered on that system.

The DLLs or EXEs referenced by resourceFileName and messageFileName may provide useful clues.

You can also look at Microsoft\Windows\CurrentVersion\WINEVT\Publishers in the SOFTWARE registry hive.

Even with the original definitions, however, the analysis tool may not support the relevant metadata format. If all else fails, you could try examining the binary data directly—or throwing it at an AI and seeing what happens.

Converting ETL Files with ETW2JSON

If you want to search and aggregate events across multiple files, converting them into a common format makes life much easier.

Microsoft's ETW2JSON can convert ETL files into JSON.

However, the original version has a few limitations. For example, you cannot directly specify Manifest files or DLLs recovered from the source system, and it can be difficult to inspect the original data or identify which metadata was used to decode an event.

Also, having to compile it yourself because no executable is distributed is a bit of a hassle.

So I forked it and made some changes.

Main changes

• Added --allow-local-metadata to control whether metadata registered on the analysis system can be used, and added metadata provenance tracking.
• Added --manifest, --resource, and --provider-info to allow external metadata sources, including Manifest files and DLLs, to be specified.
• Preserved original event IDs, versions, raw payloads, and decode status even when an event cannot be decoded.
• Added --scan and --report-missing to identify missing metadata before conversion.
• Added JSONL output support.
• Added safeguards to prevent existing files from being overwritten.

By default, the fork does not use metadata registered on the analysis computer.

If you just want to take a quick look at the contents, you can enable it with --allow-local-metadata.

Keep in mind that the definitions on the analysis computer may differ from those on the source system.

Here's an example of the output.

Note that timestamp is a raw ETW timestamp, not a UTC date and time. If you want to correlate these events with other logs, you need to convert the timestamps using the trace clock information.

You can also specify a DLL or EXE using --resource. The file is only read as a metadata source; it is not executed.

Use --manifest if you want to provide a .man file instead.

You can check the decode_status field to see whether an event was decoded successfully.

• decoded: Successfully decoded
• metadata_missing: Required metadata was not found
• unsupported: The event format is not supported by the tool

If decoding fails, --report-missing can help identify which metadata definitions are missing.

Using Tools Designed for Specific Purposes

For commonly investigated logs, such as Windows Update logs, it is often easier to use tools or commands designed specifically for them.

The following two Microsoft GUI tools are also well known.

Both provide detailed views of trace data, including graphical representations of events.

Their primary use cases seem to be debugging and performance tuning rather than forensics, though.

The following Japanese-language article explains how to use them in more detail.

How to Use PerfView and Windows Performance Analyzer (WPA)

Analyzing ETL Files on Linux

If you want to analyze ETL files outside Windows, you can use tools such as Airbus CERT etl-parser and Dissect.

etl-parser is still actively maintained and is quite useful.

For example, you can convert ETL files into XML.

Dissect is another well-known open-source digital forensics framework. It provides ETL-related plugins that can be useful depending on what you're investigating.

The example above extracts events from Boot and Shutdown ETL files in a disk image.

4. Putting the Information Together

Once you understand the basic concepts and how to convert ETL files, the rest is much like investigating Windows Event Logs.

The fields recorded in each log can vary considerably. The trick is to organize the information around a common timeline, then keep forming hypotheses, testing them against the evidence, and refining them.

Knowledge and intuition. And, above all, a whole lot of patience.

For more about investigating Windows Event Logs, see my earlier article:

Appendix: Tips for Investigating ETL Files

As mentioned earlier, ETL files are scattered all over the system. I've put together some notes on where to look, depending on what you're trying to investigate.

There is still a lot I don't understand about ETL files, and the available information seems to vary considerably depending on the system and its configuration. It may take some time before these findings can be turned into broadly applicable investigation techniques.

I'll add more as I verify things myself.

Program Execution

According to Uncovering Hidden Forensic Evidence in Windows: The Mystery of AutoLogger-Diagtrack-Listener.etl, an investigation of AutoLogger-Diagtrack-Listener.etl on Windows Server 2016 revealed the path and command line of an executable that had already been deleted.

However, the conditions under which this particular ETL file records such information remain unclear.

File Operations During Startup and Shutdown

BootCKCL.etl and ShutdownCKCL.etl may contain information about file operations, process activity, and DLL loading during system startup and shutdown.

Interesting events include FileIo/FileCreate, FileIo/FileDelete, Process/Start, and Process/End.

However, FileIo/FileCreate alone does not prove that a new file was created. You shouldn't determine whether an operation succeeded based solely on its event name. Check related events and correlate them with other artifacts.

ETW Event Tracing for Windows and ETL Files walks through an investigation involving a suspicious executable.

Disk and Volume Configuration

energy-ntkl.etl may contain information about disk and device configurations at the time the trace was recorded.

Events such as SystemConfig/PhyDisk, SystemConfig/LogDisk, and SystemConfig/PnP look useful when investigating disk details. It is worth correlating them with registry artifacts and other sources.

Reference: ETW Event Tracing for Windows and ETL Files

Paths and Run Keys Handled by Explorer

ExplorerStartupLog.etl may contain events related to paths, network shares, and Run keys handled by Windows Explorer.

The etw-providers-docs repository contains Provider definitions, including the following events.

As with other artifacts, make sure to correlate the findings with LNK files, Jump Lists, ShellBags, and other available evidence.

Remote Desktop Connections

According to Remote Desktop Application vs MSTSC Forensics: The RDP Artifacts You Might Be Missing, ETL files from the legacy Store version of Remote Desktop may reveal connection destinations, usernames, successful logins, and disconnections.

Related .model files may also contain information that can be used to identify saved credentials.

Wi-Fi Connections

Wifi.etl may contain events related to wireless network connections.

However, finding an SSID does not necessarily mean that a connection was successful. Check the related events to determine how far the connection process progressed—authentication, successful connection, disconnection, and so on.

Reference: Advanced troubleshooting wireless network connectivity

External IP Addresses

Delivery Optimization (DoSvc), which is used to distribute content such as Windows updates, may record external IP addresses in ETL files.

According to What was my IP? Ask DoSvc on Windows 10, the ExternalIpAddress keyword can reveal information about external IP addresses, including those observed when using a VPN.

On Windows 10 version 2004 and later, Get-DeliveryOptimizationLogAnalysis can also be used to inspect connection destinations, peer types, transferred data volumes, and other information.

Windows Update

The Get-WindowsUpdateLog command can be used to extract information from Windows Update ETL files.

You can use an update GUID to follow the operations surrounding a particular update.

For example, ProtocolTalker relates to communication with update servers, DownloadManager handles downloads, and Handler and Setup are associated with installation.

UpdateSessionOrchestration.etl contains information about update and restart coordination, while NotificationUxBroker.etl contains information related to notifications.

Closing Thoughts

ETL files sometimes contain useful information that isn't recorded in Windows Event Logs. However, the information available varies considerably between environments, making ETL feel like even more of a mysterious rabbit hole than Event Logs.

I haven't accumulated enough knowledge to confidently decide which ETL files are worth keeping and which can be ignored. For now, perhaps the best approach is to collect everything and analyze it later.

Or maybe just collect everything with the intention of throwing it all at an LLM.

The end.