Skip to end of metadata
Go to start of metadata

Replay Processing

To provide message replay capability, each ITC channel has a separate set of gateway instances and multicast addresses to replay requested data.  If a sequence number gap is detected, a replay request can be initiated for message recovery. The requested data for the channel(s) are transmitted through the appropriate replay gateway and multicast address.

To request a market data replay, a separate TCP connection is established to the CME Group replay gateway. A request message is made to specify the desired sequence gap and channel.  After the request is made, CME Group will transmit a Replay Response message to communicate the status of the request and then will terminate the TCP connection. A separate connection must be made for each request.

The CME Group replay gateway will authenticate the user connection and validate the replay request for a valid channel value and sequence range. CME Group has specified a maximum sequence range of 2,000 messages for any given request and a maximum of 15 requests per second. Only data for the current trading session is guaranteed for message replay availability. Client applications should cache sequence state and historical data to mitigate the amount of data recovered

To mitigate the total message rate from replay requests initiated across multiple channels, the CME Group replay gateway instances throttle message replay.  The replay channel IGMP join should be left after the replay is finished. CME Group recommends that developers externalize configurable parameters to manage the number of concurrent requests and delay between requests to be able to tailor the application to specific bandwidth configurations.

Customers connected to a replay channel will also receive replay requests from other customers that will impact bandwidth utilization.

Replay Request

When initiating a replay request, the MDP client application initiates a TCP connection to the CME Group MDP Gateway.  After establishing a socket connection, a replay request is sent to the CME MDP Gateway and a response message is sent with information indicating the success or failure of the request.

To obtain a username and password for the ITC replay, contact Global Account Management (GAM).

The request and response message formats are detailed below:

Request

Example:

User=CME{SOH}Password=CME{SOH}RequestType=REPLAY{SOH}Begin=1{SOH}End=100{SOH}Channel=1{SOH}

 

Field

Value

Description

Section

User

String

CME Group-provided user name for each firm

Header

  SOH 

Timestamp

String

Time stamp of when request was processed

Header
  SOH 

Request Type

String

‘REPLAY’ for sequence number recovery

Header

  SOH 

Result

String

Indicated result of request

Result Code Meaning:

  • 0001 Success
  • 0002 System unable to accept requests
  • 0003 Number of configurable requests exceeded
  • 0010 Malformed request
  • 0011 Missing fields from request
  • 0020 Missing request Type
  • 0021 Missing user Identifier
  • 0022 Missing password
  • 0023 Unknown Request Type
  • 0024 Invalid Userid/password combination
  • 0110 Missing channel identifier
  • 0111 Unknown channel identifier
  • 0112 Missing begin field
  • 0113 Malformed begin field
  • 0114 Missing End field
  • 0115 Malformed End field
  • 0116 Invalid range
  • 0200 Request incorrectly formatted
  • 9999 System error

Body

  SOH  

Channel

String

Channel of sequence number range to be replayed

Body

  SOH 

 Replay Messages

With each request to replay missed or lost messages, a system message is sent preceding the message replay to alert applications of the dataset. System messages must be used to process replay requests efficiently.

Since system messages are sent artificially within a replay channel, all system messages have a sequence number of ‘0’.   Any message received on a replay channel with a sequence number of ‘0’ can be assumed to be a system message.  System messages are transmitted compressed following the standard compression.

System messages have a header and a message body formatted as follows:

Field

Value

Description

Section

Type

String

  • "Heartbeat” – Used in lieu of message replay
  • “Replay” – Used to alert a replay
  • “Announcement” – Used to alert replay events

Header

SOH

Channel               

String

The channel to be replayed (i.e. ‘1’)

Body

SOH

Request Begin

String

Beginning number of sequence requested

Body

SOH

Request End

String

Ending number of sequence requested

Body

SOH

Begin

String

Beginning sequence number being sent

Body

SOH

End

String

Ending sequence number being sent

Body

SOH

Timestamp

String

Timestamp of proposed request

Body

SOH

Channel

String

1

Body - Heartbeat

SOH

Channel               

String

1

Body - Announcement

SOH

Detail

String

Details of announcement

Body - Announcement

SOH

 Replay Batching

The CME Group  MDP replay mechanism leverages a batching algorithm to process and respond to replay requests more efficiently.  Each group of requests within a channel is analyzed for sequence number overlap to combine similar requests into one replay stream.  A reasonable gap is used to group requests that may be close to overlapping.

In the following example, five users for the same channel submit replay requests:

Request

Channel

Sequence Range

CME Response Timestamp

A

1

1,000 - 2,000

12345

B

1

1,500 – 3,000

12367

C

1

3,100 – 5,000

12378

D

1

10,000 - 11,100

12389

E

1

10, 250 - 10,270

12391

Batch 1 - Request A/B/C, insert a reasonable gap between 3,000 and 3,100

Batch 2 - Request D/E

The request from the table above would be grouped into following replay batches:

Batch

Channel

Sequence Range

1

1

1, 000 – 5,000

2

1

10,000 – 11,100

Before each batch of messages, a system message would be sent detailing the messages to be sent as part of the batch.  For example:

Batch 1

Field

Value

Description

Type                      

String

“Replay” – Used to alert a replay

SOH

Channel               

String

1

SOH

RequestBegin

String

1000

SOH  

RequestEnd

String

5000

SOH

Begin

String

1000

SOH

End

String

5000

SOH

Timestamp

String

12345

SOH  

 Batch 2

Field

Value

Description

Type                      

String

“Replay” – Used to alert a replay

SOH

Channel               

String

1

SOH

RequestBegin

String

10000

SOH

RequestEnd

String

11000

SOH

Begin

String

10000

SOH

End

String

11100

SOH

Timestamp

String

12345

SOH

Each system message contains the RequestBegin and RequestEnd fields along with the Begin and End sequence ranges to provide detection of unavailable data.  For example, if a request is made for sequence numbers 9,000 to 11,000 for a given channel, but the lowest available sequence number is 10,000, the requested data from 9,000 to 10,000 is unavailable.  To eliminate the potential for an unrecoverable scenario, the system message would indicate that the range of 9,000 to 10,000 was requested, but unavailable for processing.

Field

Value

Description

Type                      

String

“Replay” – Used to alert a replay

SOH

Channel               

String

1

SOH

RequestBegin

String

9000

SOH

RequestEnd

String

11000

SOH

Begin

String

10000

SOH

End

String

11100

SOH

Timestamp

String

12345

SOH

In the unlikely event that a request is made for a sequence range that is too high and not available, the Begin and End values will be 0.  For example, if a request is made for a range outside of the available range of data (i.e., 9,000 to 9,500), a system message will be sent with a Begin and End value of 0.  No additional messages will be sent.

Field

Value

Description

Type                      

String

“Replay” – Used to alert a replay

SOH

Channel               

String

1

SOH

RequestBegin

String

9000

SOH

RequestEnd

String

9500

SOH

Begin

String

0

SOH

End

String

0

SOH

Timestamp

String

12345

SOH
  • No labels