top of page

Neural Search Talks [3] — Learning to Retrieve Passages without Supervision

In this third episode of the Neural Search Talks podcast, Andrew Yates and Sergi Castella discuss the paper "Learning to Retrieve Passages without Supervision" by Ori Ram et al.

Despite the massive advances in Neural Information Retrieval in the past few years, statistical models still overperform neural models when no annotations are available at all. This paper proposes a new self-supervised pertaining task for Dense Information Retrieval that manages to beat BM25 on some benchmarks without using any label.


00:00 Introduction

00:36 "Learning to Retrieve Passages Without Supervision"

02:20 Open Domain Question Answering

05:05 Related work: Families of Retrieval Models

08:30 Contrastive Learning

11:18 Siamese Networks, Bi-Encoders and Dual-Encoders

13:33 Choosing Negative Samples

17:46 Self supervision: how to train IR models without labels.

21:31 The modern recipe for SOTA Retrieval Models

23:50 Methodology: a new proposed self supervision task

26:40 Datasets, metrics and baselines

33:50 Results: Zero-Shot performance

43:07 Results: Few-shot performance

47:15 Practically, is not using labels relevant after all?

51:37 How would you "break" the Spider model?

53:23 How long until Neural IR models outperform BM25 out-of-the-box robustly?

54:50 Models as a service: OpenAI's text embeddings API

13 views0 comments

Recent Posts

See All


bottom of page