[TOC]

  1. Title: Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interaction
  2. Author: Federico Bianchi
  3. Publish Year: 2021
  4. Review Date: Jan 2022

Summary of paper

the author investigated grounded language learning through the natural interaction between users and the shopping website search engine.

How they do it

  1. convert the shopping object dataset into a Latent Grounded Domain

    • image-20220103212726450
    • related products end up closer in the embedding space

  2. train the mapping model (mapping from text query to a portion of product space) based on the user click behaviour (In the training dataset, the users queries about “Nike” and the they would click relevant Nike Product)

    1. This approach can be seen as a neural generalisation of model-theoretic semantics, where the ex-tension of “shoes” is not a discrete set of objects, but a region in the grounding space.

  3. train the functional composition model

    • image-20220103221637240
    • (looks like) this compositional model is designed for solving zero-shot learning problems
    • this is a model that input two DeepSet embedding and output another DeepSet embedding
    • e.g., “Nike” + “Shoes” = “Nike Shoes”

Example

image-20220103220532146

The contributions are the following

  1. converting into a grounded latent domain allows for better generalisation performance

Comments about the paper (one paragraph)

The user behaviour dataset is very valuable. but they are not open sourced any more