Page tree
Skip to end of metadata
Go to start of metadata

With CME Smart Stream on Google Cloud Platform (GCP), clients can access real-time CME Group market data feeds through native Google Cloud Platform services.  CME Smart Stream on Google Cloud Platform (GCP) supports JavaScript Object Notation (JSON) formatted futures data, allowing customers to use an easy, low-cost, and highly scalable format to drive their business. Data available in this universal format includes top of book information, second market level depth (i.e., second best bid and offer), trade execution, and product statistics including settlement, total daily volume, and open interest.

This page provides an overview of CME Smart Stream on GCP JSON technical impacts. CME Smart Stream on GCP leverages Google Cloud Pub/Sub technology for market data distribution. For complete details regarding Google Cloud Pub/Sub, refer to Google Pub/Sub documentation.

CME Smart Stream on GCP also offers Simple Binary Encoded (SBE) data. For more information, refer to the CME Smart Stream on GCP SBE impact documentation.

Contents

Revision History

DateDescription
TBD
  • "Testing and Certification" - Updated section. Added note about test environment data rates.
  • "Google Cloud Pub/Sub Processing Overview" - Updated section with more links to Google's documentation.
    • "CME MDP Included Attributes" - Added note about Saturday test data.
September 16, 2021Removed parenthetical values following String datatypes in message layout
July 15, 2021Initial release.

Key Events and Dates

DateMilestone
Now AvailableNew Release
Now AvailableProduction

Testing and Certification

The following section outlines testing and certification for GCP.

Certification

Certification is not required for CME Smart Stream on GCP JSON.

CME Globex Test Environments

The New Release test environment allows product and new functionality testing prior to release in production. Use the New Release environment to perform:

  • New product testing
  • Development testing on new functionality

The New Release test environment has lower data rates than production.

Summary of Impacts

The following market data is published in JSON format via the Google Pub/Sub:

  • Top of Book (TOB)
  • Level 2 Order Book (LVL 2)
  • Market Statistics (OI, Settlement and Traded Volume)
  • Time and Sales (Trade Information)

Market Data Offerings via Google Cloud

JSON CME Smart Stream on GCP supports CME Globex sourced products from the following Designated Contract Markets (DCM):

  • CME
  • CBOT
  • COMEX
  • NYMEX
  • DME

Google Cloud Pub/Sub Processing Overview 

This section provides an overview of CME MDP processing for Google Cloud Pub/Sub. For complete details regarding Google Cloud Pub/Sub technology, refer to Google's online Pub/Sub documentation. For Pub/Sub libraries, refer to Google’s Pub/Sub Client Libraries. Common GCP SDK Pub/Sub processing concepts include the following:

Terminology

  • Topic: A named resource to which messages are sent by a publisher (CME Group).
  • Subscription: A named resource representing the stream of messages from a single, specific topic, to be delivered to the subscribing application (client system). 
  • Message: The combination of Google data and CME SBE data sent to a topic and delivered to subscribers.  

CME MDP on Google Cloud Pub/Sub Message Flow

The following diagram outlines the Publisher (CME Group) relationship with the Subscriber (client system) for Google Cloud Pub/Sub technology.   

pubsub2

  1. CME Group creates a topic in the Cloud Pub/Sub service and send messages to the topic. To process CME MDP on Google Cloud Pub/Sub, client systems must subscribe to a topic to receive data. The Google Cloud PubSub message will contain CME Group messages.
  2. Message store ensures that published messages are retained on behalf of subscriptions.
  3. Cloud Pub/Sub forwards messages from a topic to each individual subscription. Each subscription receives messages by pulling them from the service. For CME MDP on Cloud Pub/Sub there are four types of topics for a given product group that client systems can subscribe to:
    Top of Book Topic - Provides a 1-deep Market by Price (MBP) book for an instrument.
    2-Level Book Topic - Provides a 2-deep Market by Price (MBP) book for an instrument.
    Market Stats Topic - Includes Open Interest, Settlement and Traded Volume.
    Trade Information Topic - Provides trade updates for an instrument.
  4. The subscriber receives pending messages from its subscription and acknowledges each one to the Cloud Pub/Sub service. A message contains JSON messages.
  5. When a message is acknowledged by the subscriber, it is removed from the subscription's message queue.

Message Structure

The following section outlines the CME MDP messaging within a Google Pub/Sub Message structure.  

CME MDP Included Data

The Google Pub/Sub data field provides the full CME JSON message. The following diagram outlines CME data within a Google Pub/Sub message:

PubSub Message JSON2

CME MDP Included Attributes 

In addition to the full CME Group JSON message, Smart Stream on GCP provides the following quick access data in the Google Pub/Sub attributes field:

SendingTime - Sending time from CME before the message is sent to GCP.

Client systems may receive Saturday test data upon connection. Test data can be filtered using the SendingTime attribute.  

Message Ordering

Google Pub/Sub does not guarantee message order. Therefore, the client system must ensure the correct order for processing messages via SendingTime attribute. 

Topic Naming Methodology

CME Smart Stream JSON on Google Cloud Platform (GCP) has data per product. Topic names have the following methodology:  

Name

Valid Values

Description

EnvType

PROD = Production

NR = New Release

CERT = Certification 

Environment
ServiceNameSSCL = Smart Stream Cloud LinkCloud service provider
Provider

GCP = Google Cloud Products

Name of the cloud provider

ContentTypeMD = Market DataType of content
Content Timing

RT = Real Time

Time data is sent
ContentOwner

CMEG = CME Group Owned Market Data

DME = Dubai Mercantile Exchange

Reference to exchange

Format

JSON = JavaScript Object Notation

Feed Type
Version

v01000

The version of the message format for a topic
Topic Type

TOB = Top of Book

LVL2 = Top Two Book Levels

TRD = Time and Sales (Trade Information)

STAT = Statistic Information 

Defines the type of topic

SecurityExchange

Security Exchange:

XCBT = Chicago Board of Trade
XCME = Chicago Mercantile Exchange
XNYM = New York Mercantile Exchange
XCEC= COMEX (Commodities Exchange Center)
DUMX = Dubai Mercantile Exchange

Security Exchange for JSON Topics

Product Group

Examples:

CL = Crude Oil Futures 

Product Group. JSON Topics Only

Topic Name Examples

The following are topic name examples:

  • Production JSON 30-Year U.S. Treasury Bond Futures Top of Book Information: PROD.SSCL.GCP.MD.RT.CMEG.JSON.v01000.TOB.XCBT.ZB
  • New Release JSON Crude Oil Futures Trade Information: NR.SSCL.GCP.MD.RT.CMEG.JSON.v01000.TRD.XNYM.CL

Additional Product and Instrument Information

Product and instrument referential data for CME Smart Stream on GCP can be obtained via CME Reference Data API.

The attribute 'id' maps to the Reference Data API instrument field globexSecurityId.

Message Specification 

The following section outlines the full message specification for SmartStream JSON messages.

Trade Message

This message is generated for the Trade topic.

#

Key

Attribute

Type

Enumeration

Description

1headermessageTypeStringTSDenotes trade summary message
2headersentTime

String

ex "2020-05-11T11:22:12.208530531"

CME MDP gateway sends the message (UTC).

Format: "yyyy-MM-dd'T'HH:mm:ss.n"

3headerversionString1.0API version
4payloadlastUpdateTime

String


ex "2020-05-11T11:22:12.208530531"

Last update  processing time (UTC). 

Format: "yyyy-MM-dd'T'HH:mm:ss.n"

5payload.tradeSummaryaggressorSideshort 

0 = No aggressor

1 = Buy

2 = Sell

255 = Null Value

Indicates which side is aggressor of the trade. If there is a zero value present, then there is no aggressor.

Trades without aggressors occur:

  • at Market Open
  • after a Pre-Open or after a Pause
  • when the event includes customer order participation in a trade with a CME Globex-generated implied bid or offer.
6payload.tradeSummarymdTradeEntryIduInt32

Common Trade ID that links each trade execution.

7payload.tradeSummarytradePricePRICE9
Trade price
8payload.tradeSummarytradeQtyint32
Total traded quantity
9payload.tradeSummarytradeOrderCountint32
Identifies the total number of non-implied orders per instrument that participated in a match event
10payload.tradeSummarytradeUpdateActionString

NEW

CHANGE

DELETE

Trade market data update action.

11payload.tradeSummary.orderQtyorderIdint32


Unique ID assigned by CME Globex to identify orders.

12payload.tradeSummary.orderQtylastOrdQtyint32
Quantity of trade
13payload.InstrumentdefinitionSourceString

E = exchange defined

U = user defined

Identifies user-defined instruments. If the tag is not present, the instrument is not user-defined.

14payload.InstrumentexchangeMicStringXCBT = Chicago Board of Trade
XCME = Chicago Mercantile Exchange
XNYM = New York Mercantile Exchange
XCEC= COMEX (Commodities Exchange Center)
DUMX = Dubai Mercantile Exchange
Exchange used to identify a security.
15payload.Instrumentidint32

Unique instrument ID as qualified by the exchange per market segment.

The unique instrument ID value will not be reused until the next trade date following an instrument expiration or deletion.

16payload.InstrumentmarketSegmentIdint32

Identifies the market segment.

Populated for all CME Globex instruments.

17payload.InstrumentperiodCodeString ex "202106"

This field provides the calendar month reflected in the instrument symbol. Format YYYYMM (e.g., 201912) 

For futures spreads, this field contains the first leg's calendar month reflected in the instrument symbol.

18payload.InstrumentproductCodeString

String field that indicates the underlying asset code (Product Code). Example: GE (Eurodollars), ES (E-Minis).

19payload.InstrumentproductTypeString

FUT = Future or Future Spread

Product Type
20payload.InstrumentproductGroupString

Product Group Code.

21payload.InstrumentsymbolStringe.g. ‘6EN2 P1220’, 6AZ4

Instrument Name or Symbol.


Book Message

This message is generated for top of book and level 2 book topics.

#

Key

Attribute

Type

Enumeration

Description

1headermessageTypeString

OBTS = top of book

LVL2 = level 2 book depth

Denotes trade book message
2headersentTime

String


ex "2020-05-11T11:22:12.208530531"

Time CME MDP gateway sends the message (UTC).

Format: "yyyy-MM-dd'T'HH:mm:ss.n"

3headerversionString
API version
4payloadtradingStatus

String

TradingHalt

Close            

NewPriceIndication                       

ReadyToTrade (start of session)

NotAvailableForTrading

UnknownOrInvalid       

PreOpen  

PreCross

Cross

PostClose             

Identifies the trading status applicable to the instrument or product group.

5payload.InstrumentdefinitionSourceString

E = exchange defined

U = user defined

Identifies user-defined instruments. If the tag is not present, the instrument is not user-defined.

6payload.InstrumentexchangeMicStringXCBT = Chicago Board of Trade
XCME = Chicago Mercantile Exchange
XNYM = New York Mercantile Exchange
XCEC= COMEX (Commodities Exchange Center)
DUMX = Dubai Mercantile Exchang
Exchange used to identify a security.
7payload.InstrumentmarketSegmentIdint32

Identifies the market segment.

Populated for all CME Globex instruments.

8payload.instrumentperiodCodeStringex "202106"

The calendar month reflected in the instrument symbol.

For futures spreads, this field contains the first leg's calendar month reflected in the instrument symbol.

9payload.instrumentproductCodeString

An exchange-specific code assigned to a group of related securities, which are concurrently affected by market events.

10payload.instrumentproductGroupString
Product group
11payload.instrumentproductTypeString

FUT = Future or Future Spread

Product type
12payload.instrumentsymbolString
Instrument Name or Symbol.
13payload.askLevellastUpdateTime

String


ex "2020-05-11T11:22:12.208530531"

Last update time for ask price

Format: "yyyy-MM-dd'T'HH:mm:ss.n"

14payload.askLevelorderCntint32
Aggregate number of orders at the given price level.
15payload.askLevelpricelong

Price of the MD Entry. 

16payload.askLevelqtyint32
Order quantity.
17payload.bidLevellastUpdateTime

String

ex "2020-05-11T11:22:12.208530531"

Last update time for bid price.

Format: "yyyy-MM-dd'T'HH:mm:ss.n"

18payload.bidLevelorderCntint32
Aggregate number of orders at the given price level.
19payload.bidLevelpricelong

Price of the MD Entry. 

20payload.bidLevelqtyint32
Order quantity


Statistic Message

This message is generated for statistic topics.

For more information regarding settlements, see CME Group Settlements.

#

Key

Attribute

Type

Enumeration

Description

1headermessageTypeStringTSTATDenotes statistic message.
2headersentTime

String

ex "2020-05-11T11:22:12.208530531"

Time CME MDP gateway sends the message (UTC).

Format: "yyyy-MM-dd'T'HH:mm:ss.n"

3headerversionString
API version
4payloadlastUpdateTimelongex "2020-05-11T11:22:12.208530531"

Last update  processing time (UTC). 

Format: "yyyy-MM-dd'T'HH:mm:ss.n"

5payload.tradeStatisticssettlementPricePRICENULL9
Settlement price.
6payload.tradeStatisticssettlementPriceDate

String

ex "2020-05-13"

Date of trade session corresponding to a statistic entry. 


Format: "YYYY-MM-DD"

7payload.tradeStatisticssettlementPriceTimestampString ex "2020-05-11T11:22:12.208530531"

Time of trade session corresponding to a statistic entry. 

Format: "yyyy-MM-dd'T'HH:mm:ss.n"

8payload.tradeStatisticssettlementFinalboolean
Final settlement price
9payload.tradeStatisticssettlementActualboolean
Actual settlement price
10payload.tradeStatisticssettlementRoundedboolean
Rounded settlement price
11payload.tradeStatisticsopenInterestint32

The total open interest for the market at the close of the prior trading session.

12payload.tradeStatisticsopenInterestDateString ex "2020-05-13"

Open interest trade date

Format: "YYYY-MM-DD"

13payload.tradeStatisticsopenInterestTimestampString ex "2020-05-11T11:22:12.208530531"Open interest update time.
14payload.tradeStatisticsclearedVolume
int32

Cleared volume quantity.
15payload.tradeStatisticsclearedVolumeDate

String


Cleared volume date.
16payload.tradeStatisticsclearedVolumeTimestampString
Cleared volume time.
17payload.InstrumentdefinitionSourceString

E = exchange defined

U = user defined

Identifies user-defined instruments. If the tag is not present, the instrument is not user-defined.

18payload.InstrumentexchangeMicStringXCBT = Chicago Board of Trade
XCME = Chicago Mercantile Exchange
XNYM = New York Mercantile Exchange
XCEC= COMEX (Commodities Exchange Center)
DUMX = Dubai Mercantile Exchang
Exchange used to identify a security.
19payload.Instrumentidint

Unique instrument ID as qualified by the exchange per market segment.

The unique instrument ID value will not be reused until the next trade date following an instrument expiration or deletion.

20payload.InstrumentmarketSegmentIdint

Identifies the market segment.

Populated for all CME Globex instruments.

21payload.InstrumentperiodCodeString ex "202106"

The calendar month reflected in the instrument symbol. Format YYYYMM (e.g., 201912) 

For futures spreads, this field contains the first leg's calendar month reflected in the instrument symbol.

22payload.InstrumentproductCodeString

String field that indicates the underlying asset code (Product Code). Example: GE (Eurodollars), ES (E-Minis).

23payload.InstrumentproductTypeString

FUT = Future or Future Spread

Product Type.
24payload.InstrumentproductGroupString

Product Group Code.

25payload.InstrumentsymbolStringe.g. ‘6EN2 P1220’, 6AZ4

Instrument Name or Symbol.

Contact Information

For technical development support, contact Certification Support for Electronic Trading (CSET).

For production requests, please contact the Global Command Center (GCC).

For all other inquiries, please contact Global Account Management (GAM).


  • No labels