Documentation
DataMine List API
Table of Contents
1. Overview
The DataMine List API allows the querying of all entitled files, with optional criteria to limit the number of results returned. The maximum number of results from a list is 1,000 per page. If you have more results than the limit, you need to use the paging links that are returned in the "paging" section as part of the response. The API updates in real-time giving you the most up-to-date view of your accessible files, such as when a new daily file is made available.
DataMine’s List API Service uses the same API credentials as the File Download API and pairs well when you want a list of your entitled files, or which files have been made available for the given date.
Key features:
- Ability to list entitled files available for download.
- Filtering to see which files are available by dataset or other parameters.
- Real-time updates to the list of files available.
URL: All API requests should be prefixed with the following base URL:
https://datamine.new.cmegroup.com/api/list_entitlements_files
2. Authentication
All requests to the API require authentication. Both OAuth 2.0 or Basic Auth are supported per your existing entitled API ID.
OAuth 2.0 (Bearer token): To authenticate, obtain an access token and include it in the authorization header as a Bearer token. To generate the access token, do so through the OAuth 2.0 Access Token endpoint:
https://auth.cmegroup.com/as/token.oauth2
Note: Tokens expire after 30 minutes
Header Example: Authorization: Bearer <YOUR_ACCESS_TOKEN_HERE>
Basic Authentication: To authenticate, provide your username and password encoded in the Authorization header.
3. List API Data and Response Structure
The following data is available in the API:
Endpoint: GET /list_entitlements_files
Full URL: https://datamine.new.cmegroup.com/api/list_entitlements_files
| Field | Type | Description | Example |
|---|---|---|---|
| category_code | string | The dataset’s category code as listed on DataMine. | EOD |
| exchange_code | string |
The product’s listed exchange. Can be checked on DataMine or the Product Slate page. CME Group exchanges are abbreviated to: XCME (CME), XCEC (COMEX), XNYM (NYMEX), or XCBT (CBOT). |
XCEC |
| product_code | string | The listed product code, typically the Globex Code. Can be checked on DataMine or the Product Slate page for a full list of codes (i.e. Globex, Clearing, etc.). | GC |
| dataset_code | string | The dataset code as listed on DataMine. | EOD_XCEC_GC_FUT_0 |
| dataset_name | string | The dataset’s standard name. | Gold futures |
| foi_indicator | string | Product type: FUT (futures) or OPT (options). | FUT |
| num_of files | integer | The number of available files within the dataset_code for the given period_date. | 3 |
| period_date | string | The data content date in YYYYMMDD format. | 20250611 |
| total_size | integer | The total file(s) size in bytes. | 3111 |
| files | array | Files within the specified dataset_name and period_date. | |
| → api_download_link | string | The File Download API URL for the given file. | https://datamine.cmegroup.com/cme/api/v2/download?fid=20250611-EOD_XCEC_GC_FUT_0_ETH_P |
| → file_id | string | The “fid” that can be used for identifying and downloading the given file through the File Download API. | 20250611-EOD_XCEC_GC_FUT_0_ETH_P |
| → file_name | string | The given file name when downloading through the File Download API. | xcec-eode-gc-fut-20250611.csv.gz |
| → sftp_path | string | The file path if downloading through the SFTP. | EOD_XCEC_GC_FUT_0/EOD_XCEC_GC_FUT_0/2025/06/11/xcec-eode-gc-fut-20250611.csv.gz |
| → size | integer | The file size in bytes. | 882 |
| paging | object | Pagination information if the request exceeds 1000 results. | |
| → next | string | The List API request for the next page of results. It will be null if there’s no next pages. | https://datamine.cmegroup.com/api/list_entitlements_files?offset=1000&limit=1000&dataset_code=EOD_XCEC_GC_FUT_0 |
| → previous | string | The List API request for the prior page of results. It will be null if there’s no previous pages. | https://datamine.cmegroup.com/api/list_entitlements_files?offset=0&limit=1000&dataset_code=EOD_XCEC_GC_FUT_0 |
4. Parameters
Accepted Parameters (all are optional, case-sensitive):
| Parameter Name | Type | Description | Example value |
|---|---|---|---|
| category_code | string | Unique code by category. | EOD |
| exchange_code | string | The product’s listed exchange. Can be checked on DataMine or the Product Slate page. CME Group exchanges are abbreviated to: XCME (CME), XCEC (COMEX), XNYM (NYMEX), or XCBT (CBOT). | XCEC |
| product_code | string | The listed product code, typically the Globex Code. Can be checked on DataMine or the Product Slate page for a full list of codes (i.e. Globex, Clearing, etc.). | GC |
| dataset_code | string | Unique code by dataset. The dataset code can be found on the catalog product page. | EOD_XCEC_GC_FUT_0 |
| period_date | string | Date of file publishing in YYYYMMDD format. | 20250611 |
| foi_indicator | string | Product type: FUT (futures) or OPT (options). | FUT |
| limit | integer | Maximum number of items to return per page. Defaults to 1000 (Max: 1000). | 50 |
| offset | integer | The number of items to skip before starting to collect results. | 1000 |
Request Example:
None
GET
https://datamine.new.cmegroup.com/api/list_entitlements_files?category_code=EOD&period_date=20250611&dataset_code=EOD_XCEC_GC_FUT_0
Response Example:
JSON
{
"data": [
{
"category_code": "EOD",
"dataset_code": "EOD_XCEC_GC_FUT_0",
"dataset_name": "Gold Futures",
"files": [
{
"api_download_link": "https://datamine.new.cmegroup.com/cme/api/v2/download?fid=20250611-EOD_XCEC_GC_FUT_0_ETH_P",
"file_id": "20250611-EOD_XCEC_GC_FUT_0_ETH_P",
"file_name": "xcec-eode-gc-fut-20250611.csv.gz",
"sftp_path": "EOD_XCEC_GC_FUT_0/EOD_XCEC_GC_FUT_0/2025/06/11/xcec-eode-gc-fut-20250611.csv.gz",
"size": 882
},
{
"api_download_link": "https://datamine.new.cmegroup.com/cme/api/v2/download?fid=20250611-EOD_XCEC_GC_FUT_0_ETH_F",
"file_id": "20250611-EOD_XCEC_GC_FUT_0_ETH_F",
"file_name": "xcec-eodf-gc-fut-20250611.csv.gz",
"sftp_path": "EOD_XCEC_GC_FUT_0/EOD_XCEC_GC_FUT_0/2025/06/11/xcec-eodf-gc-fut-20250611.csv.gz",
"size": 1114
},
{
"api_download_link": "https://datamine.new.cmegroup.com/cme/api/v2/download?fid=20250611-EOD_XCEC_GC_FUT_0_ETH_P",
"file_id": "20250611-EOD_XCEC_GC_FUT_0_ETH_P",
"file_name": "xcec-eodp-gc-fut-20250611.csv.gz",
"sftp_path": "EOD_XCEC_GC_FUT_0/EOD_XCEC_GC_FUT_0/2025/06/11/xcec-eodp-gc-fut-20250611.csv.gz",
"size": 1115
}
],
"num_of_files": 3,
"period_date": "20250611",
"total_size": 3111
}
],
"paging": {
"next": "",
"previous": ""
}
}
5. Pagination
The following parameters/fields are used to support pagination:
- limit: Parameter that specifies the maximum number of records to return in a single response. Default is 1000, maximum is 1000.
- offset: Parameter that specifies the number of records to skip from the beginning of the result set. Useful for skipping to a particular page.
- nextPage: A field that provides the URL in the paging object of the response. Follow this URL to retrieve the next set of results.
- previousPage: A field that provides the URL in the paging object of the response. Follow this URL to retrieve the previous set of results.
How to use nextPage and previousPage: Instead of manually calculating limit and offset, it's recommended to use the nextPage and previousPage URLs provided in the response. These URLs already contain the correct limit and offset values for the subsequent or preceding pages. If the field is null, that implies there are no next or prior pages to scan respectively.
6. Error handling
The DataMine List API uses standard HTTP status codes to indicate the success or failure of an API request. In case of an error, the response body will contain a JSON object with details about the error.
HTTP status codes and error codes:
| HTTP status code | Error code | Description |
| 200 OK | (N/A) | The request was successful. |
| 400 Bad Request | INVALID_INPUT | The request was malformed or had invalid parameters/headers. |
| 401 Unauthorized | UNAUTHORIZED | Authentication credentials were missing or invalid. |
| 403 Forbidden | ACCESS_DENIED | The authenticated user does not have permission to access the resource. |
| 404 Not Found | RESOURCE_NOT_FOUND | The requested file/file path could not be found. |
| 429 Too Many Requests | RATE_LIMIT_EXCEEDED | The client has sent too many requests in a given amount of time. |
| 500 Internal Server Error | INTERNAL_ERROR | An unexpected error occurred on the server. |
7. Rate limiting
To ensure stability of the API, rate limiting is implemented. You are allowed to make up to 120 requests per minute per API ID.
If you exceed the rate limit, you will receive a 429 Too Many Requests HTTP status code. The following headers will be included in the response to help you manage your rate limit:
- X-RateLimit-Limit: The maximum number of requests allowed in the current window.
- X-RateLimit-Remaining: The number of requests remaining in the current window.
- X-RateLimit-Reset: The time at which the current rate limit window resets, in seconds.
8. Sample connection code (Python)
Python
import requests
api_id = '<API ID>' # Enter API ID here
api_password = '<API PASSWORD>' # Enter API Password here
# Function for generating OAuth token
auth_url = 'https://auth.cmegroup.com/as/token.oauth2'
def bearer_token(token_url, client_id, client_secret):
response = requests.post(
token_url,
data={"grant_type": "client_credentials"},
auth=(client_id, client_secret),
)
return response.json()["access_token"]
access_token = bearer_token(auth_url, api_id, api_password)
datamine_list_url = 'https://datamine.new.cmegroup.com/api/list_entitlements_files'
header = {"Authorization": f"Bearer {access_token}",
"User-Agent": "Python/DataMine List API"}
# Optional Parameters Example
#params = {"category_code": "EOD",
# "dataset_code": "EOD_XCEC_GC_FUT_0",
# "period_date": "20250616",
# "limit": "1000",
# "offset": "0"}
response = requests.get(datamine_list_url,
headers=header,
#params=params
)
response.json()
Example for using cURL to retrieve the List API using an OAuth API ID
None
curl -d "grant_type=client_credentials" -u '<API_ID>:<API_PASSWORD>' https://auth.cmegroup.com/as/token.oauth2
curl -H 'Authorization: Bearer <PASTE_TOKEN_HERE>' https://datamine.new.cmegroup.com/api/list_entitlements_files
9. Changelog
| Version | Date | Description |
| 1.0 | Jun 23, 2025 | Initial release of the List API documentation. |
