Skip to content

Commit ef1a519

Browse files
committed
increased the starting ID to avoind conflicts
1 parent cbe1077 commit ef1a519

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

engine/base_client/search.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -174,7 +174,9 @@ def cycling_query_generator(queries, total_count):
174174
interval_pbar = None
175175

176176
# Initialize global doc_id offset to ensure uniqueness across intervals
177-
global_doc_id_offset = 0
177+
# Start from a high offset to avoid conflicts with uploaded dataset doc_ids
178+
# Most datasets have < 100M records, so starting from 100M should be safe
179+
global_doc_id_offset = 100000000
178180

179181
# Overall accumulators
180182
overall_results = []

0 commit comments

Comments
 (0)