The recent successes of neural networks in generating human-like language have sparked significant discussions in cognitive science, with many researchers arguing that classical puzzles about human cognition and challenges to artificial intelligence are being addressed by neural networks. A notable aspect is the argument from systematicity by Jerry Fodor and Zenon Pylyshyn, which posits that humans exhibit systematic biconditional dependencies. For instance, understanding the sentence "John saw Mary" implies understanding "Mary saw John."
Symbolic systems explain this systematicity in language and thought, while neural networks fail to provide immediate explanations. Despite recent claims that neural networks have met this challenge, particularly by Brenden Lake and Marco Baroni who argue that their meta-learning for compositionality protocol matches and explains human systematicity, we find these conclusions to be premature. Our results indicate that their model struggles to learn rules that deviate even slightly from its training data distribution. Moreover, the model behaves unsystematically on many within-distribution problems. Thus, we conclude that Fodor and Pylyshyn's challenge to neural networks remains unmet.
Blogger's Review: This article provides a thorough analysis of the limitations of neural networks in understanding linguistic systematicity, emphasizing the importance of traditional cognitive models. While neural networks excel in certain tasks, they still require rigorous theoretical frameworks to address complex cognitive phenomena. This research serves as a reminder that advancements in technology do not necessarily resolve fundamental challenges in cognitive science.