[TOC]
- Title: BABYAI++: Towards Grounded-Language Learning Beyond Memorization
- Author: Tianshi Cao et. al.
- Publish Year: 2020 ICLR
- Review Date: Jan 2022
Summary of paper
The paper introduced a new RL environment BabyAI++ that can investigate whether RL agents can extract knowledge from descriptive text and eventually increase generalisation performance.
BabyAI++ environment example
- the descriptive text describe the feature of the object.
- notice that the feature of object can easily change as we change the descriptive text.
Model
they studied three architecture for processing descriptive text and scene representation into a combined visual+text embedding
- concat-fusion
- FiLM
- attention fusion
their attention fusion model obtained the best result
Contribution
Their results shows that attention based text-image grounded language learning allows the agents to extract knowledge from descriptive text and obtain better generalisation performance.