{"id":12179,"date":"2019-05-13T06:00:03","date_gmt":"2019-05-13T10:00:03","guid":{"rendered":"http:\/\/whamit.mit.edu\/?p=12179"},"modified":"2019-05-12T21:40:41","modified_gmt":"2019-05-13T01:40:41","slug":"complang-5-13-ethan-wilcox-harvard","status":"publish","type":"post","link":"https:\/\/ling-phil.mit.edu\/whamit\/2019\/05\/13\/complang-5-13-ethan-wilcox-harvard\/","title":{"rendered":"CompLang 5\/13 - Ethan Wilcox (Harvard)"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><strong>Speaker<\/strong>: <a href=\"https:\/\/wilcoxeg.github.io\/\">Ethan Wilcox<\/a> (Harvard University)<br><strong>Title<\/strong>: Neural Network Language Models as Psycholinguistic Subjects: The Case of Filler\u0097Gap Dependencies<br><strong>Time<\/strong>: Thursday, 5\/13, 5-6pm <br><strong>Location<\/strong>: 46-5165<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Abstract<\/strong>:<br>Recurrent Neural Networks (RNNs) are one type of neural model that has been able to achieve state-of-the-art scores on a variety of natural language tasks, including translation and language modeling (which is used in, for example, text prediction). However, the nature of the representations that these &#8216;black boxes&#8217; learn is poorly understood, raising issues of accountability and controllability of the NLP system. In this talk, I will argue that one way to assess what these networks are learning is to treat like subjects in a psycholinguistic experiment. By feeding them hand-crafted sentences that belie the model&#8217;s underlying knowledge of language I will demonstrate that they are able to learn the filler&#8212;gap dependency, and are even sensitive to the hierarchical constraints implicated in the dependency. Next, I turn to &#8220;island effects&#8221;, or structural configurations that block the filler&#8212;-gap dependency, which have been theorized to be unlearnable. I demonstrate that RNNs are able to learn some of the &#8220;island&#8221; constraints and even recover some of their pre-island gap expectation. These experiments demonstrate that linear statistical models are able to learn some fine-grained syntactic rules, however their behavior remains un-humanlike in many cases. <\/p>\n","protected":false},"excerpt":{"rendered":"<p>Speaker: Ethan Wilcox (Harvard University)Title: Neural Network Language Models as Psycholinguistic Subjects: The Case of Filler\u0097Gap DependenciesTime: Thursday, 5\/13, 5-6pm Location: 46-5165 Abstract:Recurrent Neural Networks (RNNs) are one type of neural model that has been able to achieve state-of-the-art scores on a variety of natural language tasks, including translation and language modeling (which is used [&hellip;]<\/p>\n","protected":false},"author":21,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-12179","post","type-post","status-publish","format-standard","hentry","category-announcements"],"_links":{"self":[{"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/posts\/12179","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/comments?post=12179"}],"version-history":[{"count":1,"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/posts\/12179\/revisions"}],"predecessor-version":[{"id":12180,"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/posts\/12179\/revisions\/12180"}],"wp:attachment":[{"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/media?parent=12179"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/categories?post=12179"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ling-phil.mit.edu\/whamit\/wp-json\/wp\/v2\/tags?post=12179"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}