Explicit OpenInterest DataMappingMode on EUREX/HKFE/KRX futures leaves the continuous contract unmapped and History() empty

[AlexCatarino]

Summary

Calling AddFuture with an explicit DataMappingMode.OpenInterest on a market whose map files have no OpenInterest rows (EUREX, HKFE, KRX) produces a continuous contract that never maps. Mapped stays null for the whole backtest, and History() on the continuous symbol returns zero rows. No error or warning is raised. The same request with LastTradingDay returns full history. CME futures (e.g. NKD) work in both modes.

Cause

• The EUREX map files (e.g. future/eurex/map_files/map_files_20260922.zip → fesx.csv, fdax.csv) only contain mapping-mode 0 (LastTradingDay) and 1 (FirstDayMonth) rows.
• Extensions.GetUniverseMappingModeOrDefault (Common/Extensions.cs) already swaps OpenInterest for LastTradingDay on HKFE, EUREX and ICE ("currently no OI available data"). It only does this when OpenInterest comes from UniverseSettings. An explicit dataMappingMode argument to AddFuture/AddSecurity, or to History(..., dataMappingMode: ...), bypasses it.
• KRX is not in that market list at all. We have not tested whether KRX also fails when OpenInterest comes from the default setting.

Reproduction

Cloud backtest, LEAN v2.5.0.0.18126:

from AlgorithmImports import *
BARS = 1300
TICKERS = [('FESX', Market.EUREX), ('HSI', Market.HKFE), ('KM', Market.KRX), ('NKD', Market.CME)]
MODES = [DataMappingMode.OPEN_INTEREST, DataMappingMode.LAST_TRADING_DAY]

class Lean9764Repro(QCAlgorithm):
    def initialize(self):
        self.set_start_date(2024, 1, 2)
        self.set_end_date(2024, 3, 29)
        self.futures = {t: self.add_future(t, Resolution.DAILY, market=m,
                            data_normalization_mode=DataNormalizationMode.BACKWARDS_RATIO,
                            data_mapping_mode=DataMappingMode.OPEN_INTEREST, contract_depth_offset=0)
                        for t, m in TICKERS}

    def on_data(self, slice):
        if self.time.day == 15:
            self.log(f'{self.time.date()} mapped: ' + ', '.join(f'{t}={f.mapped}' for t, f in self.futures.items()))

    def on_end_of_algorithm(self):
        for t, f in self.futures.items():
            for mode in MODES:
                df = self.history(f.symbol, BARS, Resolution.DAILY, data_mapping_mode=mode)
                self.log(f'{t} history({mode}): ' + ('EMPTY' if df.empty else f'bars={len(df)}'))

Result:

ticker	market	History OpenInterest	History LastTradingDay	Mapped
FESX	EUREX	empty	1300 bars	None
HSI	HKFE	empty	1300 bars	None
KM	KRX	empty	1300 bars	None
NKD	CME	1300 bars	1300 bars	set

Expected

Either apply the same LastTradingDay fallback to an explicit OpenInterest request on these markets and log a warning, or throw an error saying OpenInterest mapping is not available for that market. Silently returning an unmapped contract and empty history is the worst of the options. KRX should probably be added to the fallback list too.

Secondary

Calling AddFuture a second time for the same ticker with a different DataMappingMode silently keeps the first call's mapping mode. The continuous-universe symbol does not include the mapping mode, and AddSecurity skips creating a universe when one with that symbol already exists. This is not the cause of the empty history: adding the ticker once with explicit OpenInterest reproduces it on its own.

Reference

Internal ticket: Intercom conversation 215475691979456.

[matdornela]

Hi, I'm interested in picking this up. Happy also to discuss the proposed solution

[AlexCatarino]

Correction to my earlier comment. Adding the same ticker twice with two DataMappingModes is not the root cause. A control backtest that adds FESX only once, with an explicit OpenInterest mode, is just as empty and has Mapped == null.

The actual cause is that there is no OpenInterest map data for EUREX/HKFE/KRX, and the LastTradingDay fallback only applies when OpenInterest comes from the default setting. I have rewritten the description and title with the reproduction and results.