-
-
Notifications
You must be signed in to change notification settings - Fork 1k
Description
Octobot produces different cumulative PnL results in backtesting when using mathematically equivalent position sizes with different leverage ratios.
Environment
- Version: 2.0.12 (Docker on Windows, same on MacOS)
- Mode: Daily Trading Mode
- Testing: Backtesting on identical date ranges
- Timeframe: Daily candles
- Assets: 3 cryptocurrencies (same across all tests)
- Evaluators: 4 evaluators (identical across all tests)
Bug Description
When running backtests with identical parameters except for amount_per_buy percentage and leverage, mathematically equivalent position sizes produce significantly different cumulative PnL results.
Test Cases
All tests use identical:
- Date ranges
- Assets
- Evaluators
- Strategy settings
- All other parameters
Only changed parameters:
amount_per_buy(portfolio percentage)leverage
Results
Test 1: 15% amount_per_buy × 6x leverage = 90% effective position size
Test 2: 5% amount_per_buy × 18x leverage = 90% effective position size → PnL: Significantly lower than Test 1
Test 3: 22.5% amount_per_buy × 4x leverage = 100% effective position size → PnL: Different result again
Expected Behavior
Tests 1, 2 and 3 should produce identical PnL results since:
- 15% × 6x = 90% effective position size
- 5% × 18x = 90% effective position size
- 22.5% × 4x = 90% effective position size
The mathematical exposure to price movements should be identical.
Actual Behavior
- Test 2 (5% × 18x) produces higher PnL than Test 1 (15% × 6x)
- Test 3 (25% × 4x) produces different results from both above
- Results vary significantly despite mathematical equivalence
Steps to Reproduce
- Set up Daily Trading Mode with any strategy
- Configure backtesting with identical date range
- Run Test 1: 15% amount_per_buy, 6x leverage
- Run Test 2: 5% amount_per_buy, 18x leverage
- Run Test 2: 22.5% amount_per_buy, 4x leverage
- Compare PnL results
Additional Information
- This issue was discovered during strategy optimization
- Multiple test runs confirm the inconsistency
- The bug appears to be systematic, not random