[TOC]

  1. Title: Does Vision Accelerate Hierarchical Generalisation of Neural Language Learners
  2. Author: Tatsuki Kuribayashi
  3. Publish Year: 1 Feb 2023
  4. Review Date: Fri, Mar 3, 2023
  5. url: https://arxiv.org/pdf/2302.00667.pdf

Summary of paper

Motivation

  • we want to know if the visual information improves hierarchical generalisaiton of the language model

  • image-20230303153510788

  • image-20230303153540288

  • image-20230303153621365

Contribution

  • our results have exhibited that vision accelerated a proper linguistic generlisation in the simplified, artificial setting,
  • but LMs struggled with the proper generalisation in the noisy, realistic setting. These mixed results have indicated several possibilities; for example, an image can potentially boost language acquisition, but learners’ additional visual/linguistic **prior knowledge should be needed t**o robustly make use of raw images for efficient language acquisition.