arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · cs.CL

Compositional Languages Emerge in a Neural Iterated Learning Model

Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B. Cohen, Simon Kirby

· ICLR-2020

See on the map

Abstract

The principle of compositionality, which enables natural language to represent complex concepts via a structured combination of simpler ones, allows us to convey an open-ended set of messages using a limited vocabulary. If compositionality is indeed a natural property of language, we may expect it to appear in communication protocols that are created by neural agents in language games. In this paper, we propose an effective neural iterated learning (NIL) algorithm that, when applied to interacting neural agents, facilitates the emergence of a more structured type of language. Indeed, these languages provide learning speed advantages to neural agents during training, which can be incrementally amplified via NIL. We provide a probabilistic model of NIL and an explanation of why the advantage of compositional language exist. Our experiments confirm our analysis, and also demonstrate that the emerged languages largely improve the generalizing power of the neural agent communication.

References

The reference list for this paper has not been collected yet.

Cited by 5

5 papers in the index cite this (Semantic Scholar counts 124 in total).

The cited-by list is still filling in while reference lists are collected.

More like this