Lesson plan

Hard negative mining

Hard negative mining, also referred to as hard negative sampling, is a family of techniques in machine learning for selecting or constructing negative samples more difficult to distinguish from a corresponding positive example with respect to uniformly-sampled negatives, leading to a more informative training signal for discriminative classifiers and similarity models. The technique is used when the pool of possible negative examples is far larger than the set of positives and most randomly chosen negatives are easy to classify by the model, contributing little to learning. Concentrating training on the small fraction of difficult negatives is intended to provide stronger gradient signal and faster convergence than uniform sampling. Hard negative mining originated in object detection in the 1990s, where it was referred to as bootstrapping (not to be confused with statistical bootstrap resampling), and was later formalized for support vector machine models. Closely related ideas appear in deep metric learning, information retrieval, recommender systems, self-supervised contrastive learning, and vision-language models. Surveys distinguish hard negative mining from the broader practice of negative sampling, in which negatives are drawn at random or in proportion to frequency rather than by difficulty.

Back to article

Article summary

Hard negative mining, also referred to as hard negative sampling, is a family of techniques in machine learning for selecting or constructing negative samples more difficult to distinguish from a corresponding positive example with respect to uniformly-sampled negatives, leading to a more informative training signal for discriminative classifiers and similarity models. The technique is used when the pool of possible negative examples is far larger than the set of positives and most randomly chosen negatives are easy to classify by the model, contributing little to learning. Concentrating training on the small fraction of difficult negatives is intended to provide stronger gradient signal and faster convergence than uniform sampling. Hard negative mining originated in object detection in the 1990s, where it was referred to as bootstrapping (not to be confused with statistical bootstrap resampling), and was later formalized for support vector machine models. Closely related ideas appear in deep metric learning, information retrieval, recommender systems, self-supervised contrastive learning, and vision-language models. Surveys distinguish hard negative mining from the broader practice of negative sampling, in which negatives are drawn at random or in proportion to frequency rather than by difficulty.

Key claims

No claims recorded.

Discussion questions

  1. Summarise the article in your own words. Which single claim would you verify first, and why?
  2. Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
  3. If two sources disagreed about one of these facts, what would you need in order to trust one over the other?

Bibliography

No sources recorded.

Key claims and bibliography are generated from the same tracked claims and sources as the article page.