Market Data Packet Captures (PCAP) are the complete historical raw data feeds on all channels for both futures and options from the channels' MBO release. The Settlements and Valuations channel is a new channel available in PCAP format.
Packet Capture data is available on four exchanges from January 2017 to present and all files are in FIX format. For the availability of each Channel please reference the FAQs below.
Security Definition Files
Tag Usage by Security Type
View layout guides and message specifications for the following market data formats.
For best results, use right click + Save As to download each file.
CME MDP 3.0 (Current Production Format)
The CME Group Market Data Platform (MDP) disseminates event-based bid, ask, trade, and statistical data for CME Group markets and also provides recovery and support services for market data processing. Click below to access layout guide.
What format is the file delivered in?
PCAP files are SBE raw data grouped by channel. The file contains channel number, length of message and actual payload. For example: 312 (channel), 96 (length of message), <actual payload>.
How do I best understand the timestamp(s)?
All messages will have FIX tag 60 (within payload), this represents the Globex started processing time at nanosecond granularity with microsecond accuracy.
What is the precision level for timestamping in PCAP?
PCAP is delivered in nanosecond granularity with microsecond accuracy.
Does PCAP come from the same source as Top-of-Book and Time & Sales data?
No, our PCAP files are produced from CME Globex Market Data Platform FIX/FAST feeds, while our Top-of-Book and Time & Sales data come from an internal post-trade processing system. The granularity of the timestamps for the data offerings is different due to the sources.
What is the Settlements and Valuations Channel?
With the launch of Settlements and Valuations, CME Group introduced theoretical valuations. Theoretical valuations is new type of data produced by CME Group for futures and options products without volume, open interest, or underlying pricing required for Clearing products. For more information, see here.
How far back do you maintain MBO records?
What product is a channel on?
Please reference the page posted here to find out what MDP channel each product is on. Please not the channel is listed in column AC.
How many levels of depth are available? Has this depth changed over time?
PCAP has the Full depth of book. This has not changed over time.
Are spreads included in the PCAP files?
Exchange-listed spread products are included for both futures and options spreads.
Do you have sample files available?
In order to obtain a sample PCAP file you must set up an s3 account and then contact CME Group's Data Sales team CMEDataSales@CMEGroup.com. You can find instruction on how to entitle your s3 account to DataMine data here.
Are there any anomolies with this data?
For Channel 381, NYMEX Globex Emissions Options, there is a data gap from 6/27/2021 - 7/12/2021
When are these files delivered?
CME Group’s process to distribute PCAP files begins at 5:00 p.m. Central Time (CT) and is typically complete by midnight.
How many files do I receive per day for a PCAP subscription?
You will receive 2 files per day for each channel you are subscribed to. The first will have the previous date in the file name. This is data that occurred from 4:30 PM CT to Midnight CT. The second file will have the current date in the file name and will be from Midnight CT to 4:00 PM CT. On Sundays, you will receive all data from the start of the feed. The earliest heartbeat messages that are captured have a timestamp of 5:30 AM CT.
If I purchase daily updates of PCAP data, will I get historical data as well?
No. When an order is placed for daily updates of PCAP data, the first file included will be generated for the start date of the subscription. However, files remain accessible for 30 days after purchase, enabling the customer to reference previous day’s data.
Where can I obtain Security Definitions for the data in the PCAP data?
A security definition file is published daily and sent to you with the PCAP file.
How do I interpret PCAP data?
PCAP files include all bids and offers, full depth of the book.
How large are the PCAPfiles?
The size of the file varies upon the channel and date. On our busiest days in 2018, channel 311 (Equity options) were averaging between 7.5 GB and 10 GB a day.
Will a channel be missing packet level sequence numbers?
No you should receive all packet level sequence numbers.
Should I be missing instrument level sequence numbers (Tag 83)?
No, all instrument level sequence numbers should be in each file.
Will I see heartbeat messages?
Yes, you will see these messages.
Can trade summaries be split into different files?
Yes, if a strategy is traded and the instruments are on two different channels, the summary will be split between the two different channels.
Should an Order ID (Tag 37) change during the life of an order?
No, tag 37 should stay the same for each order until it has been cancelled or filled. If there is any modification other than a cancel or fill the Order ID stays the same.
Should orders be tied to implied prices?
No, orders are not tied to implied prices.