Tatsuki_kuribayashi Does Vision Accelerate Hierarchical Generalisation of Neural Language Learners 2023

[TOC]

1. Title: Does Vision Accelerate Hierarchical Generalisation of Neural Language Learners
2. Author: Tatsuki Kuribayashi
3. Publish Year: 1 Feb 2023
4. Review Date: Fri, Mar 3, 2023
5. url: https://arxiv.org/pdf/2302.00667.pdf

Summary of paper

Motivation

• we want to know if the visual information improves hierarchical generalisaiton of the language model

• 

• 

• 

Contribution

• our results have exhibited that vision accelerated a proper linguistic generlisation in the simplified, artificial setting,
• but LMs struggled with the proper generalisation in the noisy, realistic setting. These mixed results have indicated several possibilities; for example, an image can potentially boost language acquisition, but learners’ additional visual/linguistic **prior knowledge should be needed t**o robustly make use of raw images for efficient language acquisition.