Skip to content

Commit 0139bd2

Browse files
committed
ContextSpellChecker: fix unpredictable behavior of mutable.HashMap modification
genVocab would not iterate over all vocab in the mutable.HashMap. This is due to the iterator state of the mutable.HashMap being changed during iteration. Fixed by using mapValuesInPlace.
1 parent cbbed3d commit 0139bd2

File tree

1 file changed

+1
-3
lines changed

1 file changed

+1
-3
lines changed

src/main/scala/com/johnsnowlabs/nlp/annotators/spell/context/ContextSpellCheckerApproach.scala

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -621,9 +621,7 @@ class ContextSpellCheckerApproach(override val uid: String)
621621

622622
// compute frequencies - logarithmic
623623
totalCount = math.log(totalCount)
624-
for (key <- vocab.keys) {
625-
vocab.update(key, math.log(vocab(key)) - totalCount)
626-
}
624+
vocab.mapValuesInPlace { case (_, v) => math.log(v) - totalCount }
627625
logger.info(s"Vocabulary size: ${vocab.size}")
628626
(vocab, classes)
629627
}

0 commit comments

Comments
 (0)