Psych 156A/ Ling 150: Acquisition of Language II
Transcription
Psych 156A/ Ling 150: Acquisition of Language II
Psych156A/Ling150: AcquisitionofLanguageII Announcements HW3isduebytheendofclasstoday Reviewquestionsareavailableforstructure Lecture16 StructureI Onlinecourseevaluationsareavailableforthisclass-please fillthemout!:) ComputationalProblem: Figureouttheorderofwords(syntax) ComputationalProblem: Figureouttheorderofwords(syntax) Jarethjugglescrystals Jarethjugglescrystals SubjectVerbObject SubjectVerbObject NounVerbNoun NPNP DependsongrammaticalcategorieslikeNounsandVerbs(and theirassociatedphrases(NP)),butalsoonmoreprecise distinctionslikeSubjectsandObjects. SomeNounPhrasedistinctions: Subject=usuallytheagent/actoroftheaction,“doer”:Jareth Object=usuallytherecipientoftheaction,“doneto”:crystals Importantidea:Theobservablewordorderspeakersproduce(like SubjectObjectVerb)istheresultofasystemofwordorderrulesthat speakersunconsciouslyusewhentheyspeak.Thissystemofwordorder rulesiscalledsyntax. ComputationalProblem: Figureouttheorderofwords(syntax) Jarethjugglescrystals Jarethjugglescrystals SubjectVerbObject SubjectVerbObject OnewaytogenerateSubjectVerbObjectorder: Thelinguisticsystemspecifiesthatorderasthegeneralpatternof thelanguage.AnexampleofthiskindofsystemisEnglish. English ComputationalProblem: Figureouttheorderofwords(syntax) AnotherwaytogenerateSubjectVerbObjectorder: ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but theVerbinmainclausesmovestothesecondpositionandsomeotherphrase (liketheSubject)movestothefirstposition.Anexamplelanguagelikethisis German. SubjectVerbObject German ComputationalProblem: Figureouttheorderofwords(syntax) SubjectObjectVerb ComputationalProblem: Figureouttheorderofwords(syntax) Jarethjugglescrystals Jarethjugglescrystals SubjectVerbObject SubjectVerbObject AnotherwaytogenerateSubjectVerbObjectorder: ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but theVerbinmainclausesmovestothesecondpositionandsomeotherphrase (liketheSubject)movestothefirstposition.Anexamplelanguagelikethisis German. AnotherwaytogenerateSubjectVerbObjectorder: ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but theVerbinmainclausesmovestothesecondpositionandsomeotherphrase (liketheSubject)movestothefirstposition.Anexamplelanguagelikethisis German. movementrules German ____ VerbSubjectObjectVerb movementrules German SubjectVerbSubjectObjectVerb ComputationalProblem: Figureouttheorderofwords(syntax) ComputationalProblem: Figureouttheorderofwords(syntax) Jarethjugglescrystals Jarethjugglescrystals SubjectVerbObject SubjectVerbObject AthirdwaytogenerateSubjectVerbObjectorder: ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but theObjectmovesaftertheVerbincertaincontexts(theObjectisunexpected information).Kannadaisalanguagelikethis. AthirdwaytogenerateSubjectVerbObjectorder: ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but theObjectmovesaftertheVerbincertaincontexts(theObjectisunexpected information).Kannadaisalanguagelikethis. movementrule Kannada SubjectObjectVerb ComputationalProblem: Figureouttheorderofwords(syntax) Kannada SubjectObjectVerbObject ComputationalProblem: Figureouttheorderofwords(syntax) Jarethjugglescrystals Jarethjugglescrystals SubjectVerbObject SubjectVerbObject German English SubjectVerbObject SubjectVerbSubjectObjectVerb Kannada SubjectObjectVerbObject Thelearningproblem:Howdochildrenknowwhich systemtheirlanguageuses? German English SubjectVerbObject SubjectVerbSubjectObjectVerb Kannada SubjectObjectVerbObject Thisisahardquestion! Childrenonlyseetheoutputofthesystem(theobservableword orderofSubjectVerbObject). Translationisnotsoeasy: morethanjustword-by-word Syntax:Onereasonwhytranslationissohard http://www.nbc.com/nbc/The_Tonight_Show_with_Jay_Leno/headlines/ Translationisnotsoeasy: morethanjustword-by-word Translationisnotsoeasy: morethanjustword-by-word translate.google.com translate.google.com Hebrew HaitianCreole Literally: Throughdangersimmenseanddifficultiesnotnumbered,there-istomefightingthroughmyherecastletransitioncitygoblintakebackyou childthere-wasto-youstolen. Literally: Throughdangercountlessanddifficultiescountless,Iwasfighthow meheretheymansionthemorefarthancitiestheGoblintheytake backchildrenofthatyouwasthiefit. Translationisnotsoeasy: morethanjustword-by-word translate.google.com Abouthumanknowledge: Language&variation Hindi Literally: Untoldanduncountabledifficultiesthreatsmediumthrough,Iyou stoleisthatchildrenbacktaketheghostcitybeyondpalacethehere yourmethodsfromfightfought. NavajoCodeTalkers Crucialcryptographicmethodusedin WorldWarII NavajoCodeTalkerParadox(Baker2001) EnglishmustbeverydifferentfromNavajo JapanesecoulddecodeEnglish,but couldn’tdecodeNavajowhentheydidn’t knowitwasNavajo. http://en.wikipedia.org/wiki/Code_talker#Use_of_Navajo “…JohnstonsawNavajoasansweringthemilitaryrequirementforan undecipherablecode.NavajowasspokenonlyontheNavajolandsofthe AmericanSouthwest,anditssyntaxandtonalqualities,nottomentiondialects, madeitunintelligibletoanyonewithoutextensiveexposureandtraining.One estimateindicatesthatattheoutbreakofWorldWarIIfewerthan30nonNavajoscouldunderstandthelanguage….” https://www.youtube.com/watch?v=5rSvm3m8ZUA (~3minvideo) EnglishmustbesimilartoNavajo EnglishcanbetranslatedintoNavajoandbackwithnolossof meaning.(Languagesarenotjustaproductoftheculture-pastoral Arizonalifestylecouldn’thavepreparedthecodetalkersforPacific Islandhightechwarfare.Yet,translationwasstillpossible.) Typesofvariation Morphology(wordforms) English:invariantwordforms “thegirliscrying”,“Iamcrying” Navajo:noinvariantforms(theremaybe100-200prefixesforverb stems) At’éédyicha.“Girlcrying” Yishcha.“Iamcrying” (yi+sh+cha) Ninááhwiishdlaad.“Iamagainplowing” (ni+náá+ho+hi+sh+l+dlaad) Typesofvariation Wordorder(syntax) English:SubjectVerbObject(invariantwordorder) “Theboysawthegirl” Navajo:SubjectObjectVerb,ObjectSubjectVerb (varyingwordorders,meaningdependsonlyonverb’sform) Ashkiiat’éédyiyiiltsá boygirlsaw “Theboysawthegirl” Ashkiiat’éédbiilstá boygirlsaw “Thegirlsawtheboy” Typesofvariation wals.info:TheWorldAtlasofLanguageStructures Typesofvariation Let’slookatsyntax… Jacklaughs. Thistellsusthatmostlanguages havetheSubjectcomebeforethe Verb…butnotalldo. LaughsJack. Typesofvariation Let’slookatsyntax… Typesofvariation Let’slookatsyntax… WhatvaluedoesEnglishhave? WhataboutFijian? WhataboutSpanish? HowarethedifferentSubjectand Verbordersdistributedaroundthe world? Similarities&differences:Parameters Chomsky:Differentcombinationsofdifferentbasic elements(parameters)wouldyieldtheobservable languages(similartothewaydifferentcombinationsof differentbasicelementsinchemistryyieldmany different-seemingsubstances). Thinkingaboutsyntacticvariation Similarities&differences:Parameters BigIdea:Arelativelysmallnumberofsyntax parametersyieldsalargenumberofdifferent languages’syntacticsystems. Similarities&differences:Parameters BigIdea:Arelativelysmallnumberofsyntax parametersyieldsalargenumberofdifferent languages’syntacticsystems. 5different parametersof variation Similarities&differences:Parameters BigIdea:Arelativelysmallnumberofsyntax parametersyieldsalargenumberofdifferent languages’syntacticsystems. Similarities&differences:Parameters BigIdea:Arelativelysmallnumberofsyntax parametersyieldsalargenumberofdifferent languages’syntacticsystems. 2different parameter valuesofone parameter Totallanguages thatcanbe represented =2*2*2*2*2 =25 =32 Similarities&differences:Parameters Learninglanguagestructure Chomsky:Childrenarebornknowingtheparameters ofvariation.ThisispartofUniversalGrammar.Input fromthenativelinguisticenvironmentdetermines whatvaluestheseparametersshouldhave. BigIdea:Arelativelysmallnumberofsyntax parametersyieldsalargenumberofdifferent languages’syntacticsystems. English Japanese Tagalog Navajo French … Learninglanguagestructure Learninglanguagestructure Chomsky:Childrenarebornknowingtheparameters ofvariation.ThisispartofUniversalGrammar.Input fromthenativelinguisticenvironmentdetermines whatvaluestheseparametersshouldhave. Chomsky:Childrenarebornknowingtheparameters ofvariation.ThisispartofUniversalGrammar.Input fromthenativelinguisticenvironmentdetermines whatvaluestheseparametersshouldhave. English Japanese Learninglanguagestructure Chomsky:Childrenarebornknowingtheparameters ofvariation.ThisispartofUniversalGrammar.Input fromthenativelinguisticenvironmentdetermines whatvaluestheseparametersshouldhave. Navajo Generalizationsaboutlanguagestructure Greenberg’swordordergeneralizations Greenberg’swordordergeneralizations Navajo Navajo Japanese Japanese Basicwordorder: SubjectObjectVerb Basicwordorder: SubjectObjectVerb Ashkiiat’éédyiyiiltsá boygirlsaw Jareth-gaHoggle-obutta JarethHogglehit “Theboysawthegirl” “JarethhitHoggle” Greenberg’swordordergeneralizations Greenberg’swordordergeneralizations Navajo Navajo Postpositions: NounPhrasePostposition ‘éé’biihnáásdzá clothingintoI-got-back “Igotbackinto(my)clothes.” Japanese Postpositions: NounPhrasePostposition Jareth-gaSarahtokurumada JarethSarahwithcarby Londonniitta Londontowent Japanese PossessorbeforePossessed PossessorbeforePossessed PossessorPossession PossessorPossession Chidíbi-jáád Carits-leg Toby-noimooto-ga Toby’ssister “thecar’swheel” “Toby’ssister” “JarethwenttoLondonwithSarahby car.” Greenberg’swordordergeneralizations Greenberg’swordordergeneralizations Navajo English Japanese Basicwordorder: SubjectObjectVerb Basicwordorder: SubjectObjectVerb Postpositions: NounPhrasePostposition Postpositions: NounPhrasePostposition PossessorbeforePossessed PossessorPossession PossessorbeforePossessed PossessorPossession Despitethedifferencesinthelanguages(andtheirculturalhistories), bothJapaneseandNavajoareverysimilarwhenviewedthroughthese threestructuraldescriptions. Edo(Nigeria) Greenberg’swordordergeneralizations Greenberg’swordordergeneralizations English English Edo(Nigeria) Basicwordorder: SubjectVerbObject Basicwordorder: SubjectVerbObject SarahfoundToby ÒzómiénAdésuwá OzofoundAdesuwa Edo(Nigeria) Prepositions: PrepositionNounPhrase Prepositions: PrepositionNounPhrase JarethgavethecrystaltoSarah ÒzórhiénénéebénéAdésuwá OzogavethebooktoAdesuwa Greenberg’swordordergeneralizations Greenberg’swordordergeneralizations English English Edo(Nigeria) PossessedbeforePossessor PossessedbeforePossessor PossessionPossessor PossessionPossessor questofSarah OmoOzó childOzo (alternative:Sarah’squest) “childofOzo” Edo(Nigeria) Basicwordorder: SubjectVerbObject Basicwordorder: SubjectVerbObject Prepositions: PrepositionNounPhrase Prepositions: PrepositionNounPhrase PossessedbeforePossessor PossessionPossessor PossessedbeforePossessor PossessionPossessor Again,despitethedifferencesinthelanguages(andtheircultural histories),bothEnglishandEdoareverysimilarwhenviewedthrough thesethreestructuraldescriptions. Greenberg’swordordergeneralizations Greenbergfoundforty-five“universals”oflanguages-patterns overwhelminglyfollowedbylanguageswithunsharedhistory(Navajo &Japanese,English&Edo) Notallcombinationsarepossible-somepatternsrarelyappear Ex:SubjectVerbObjectlanguage(English/Edo-like)+postpositions (Navajo/Japanese-like) Onepotentialparameter English Italian SubjectVerb SubjectVerb Jarethverrá Jarethwill-come “Jarethwillcome.” “Jarethwillcome.” grammatical grammatical Moral:Languagesmaybemoresimilarthantheyfirstappear“onthe surface”,especiallyifweconsidertheirstructuralproperties. Onepotentialparameter English Italian *VerbSubject VerbSubject VerráJareth Will-arriveJareth *WillarriveJareth Onepotentialparameter English *Verb Willcome grammatical Verb Verrá He-will-come “Hewillcome” “Jarethwillarrive” ungrammatical Italian ungrammatical grammatical Onepotentialparameter Onepotentialparameter Expletivesubjects:wordswithoutcontent (maybemoredifficulttonotice) English Italian SubjectVerb SubjectVerb English Italian *VerbSubject *Verb VerbSubject Verb Raining. Piove. It-rains. “It’sraining.” “It’sraining.” Notokaytoleaveout expletivesubject“it”. Okaytoleaveout expletivesubject“it”. Thesewordorderpatternsmightbefairlyeasytonotice.They involvethecombinationsofSubjectandVerbthatare grammaticalinthelanguage.Achildmightbeabletonoticethe prevalenceofsomepatternsandtheabsenceofothers. Onepotentialparameter That-traceeffectforsubjectquestions English Whodoyouthink(*that)willcome? Requiresno“that”inembeddedclause,despite allowing“that”indeclarativesandobject questions Ithink(that)HogglewillsaveSarah. Whodidyouthink(that)Hogglewouldsave? Italian Onepotentialparameter That-traceeffectforsubjectquestions English Italian CredicheJarethverrá. YouthinkthatJarethwill-come. “YouthinkthatJarethwillcome.” Checrediche__verrá? Whothink-youthatwill-come? “Whodoyouthinkwillcome?” Allows“that”intheembedded clauseofasubjectquestion(and declarativeclauses). Onepotentialparameter English Italian SubjectVerb SubjectVerb *VerbSubject VerbSubject *Verb TheValueofParameters:Learningthehardstuffby noticingtheeasypatterns Verb Notokaytoleaveout expletivesubject“it”. Okaytoleaveout expletivesubject“it”. Requiresspecialactionfor embeddedsubjectquestions. Doesnotrequirespecialaction forembeddedsubject questions. Alltheseinvolvethesubjectinsomeway-coincidence? Idea:No!There’salanguageparameterinvolvingthesubject. TheValueofParameters:Learningthehardstuffby noticingtheeasypatterns Englishvs.Italian:SubjectParameter Bigidea:Ifallthesestructuralpatternsaregeneratedfromthesame linguisticparameter(e.g.a“subject”parameter),thenchildrencan learnthehard-to-noticepatterns(likethepatternsofembeddedsubject questions)bybeingexposedtotheeasy-to-noticepatterns(likethe optionaluseofsubjectswithverbs).Thehard-to-noticepatternsare generatedbyonesettingoftheparameter,whichchildrencanlearn fromtheeasy-to-noticepatterns. Children’sknowledgeoflanguagestructurevariationisbelievedby linguisticnativiststobepartofUniversalGrammar,whichchildrenare bornwith. Englishvs.Italian:SubjectParameter English SubjectVerb Italian SubjectVerb Easierto notice *VerbSubject VerbSubject Hardtonotice *Verb Verb Expletives Itrains Piove. It-rains. EmbeddedSubject-questionformation(easytomiss) Whodoyouthink(*that)willcome? Checrediche__verrá? Whothink-youthatwill-come? Anotherpossibleparameter Syntax:theHeadDirectionalityparameter(Baker2001,Cook&Newson 1996):headsofphrases(ex:NounsofNounPhrases,VerbsofVerb Phrases,PrepositionsofPrepositionPhrases)areconsistentlyineither theleftmostorrightmostposition Japanese/Navajo:Head-Last VerbPhrase: ObjectVerb Postpositions: NounPhrasePostposition VP NP Object Verb PP NP Object P postposition UniversalGrammar:Parameters Anotherpossibleparameter Syntax:theHeadDirectionalityparameter(Baker2001,Cook&Newson 1996):headsofphrases(ex:NounsofNounPhrases,VerbsofVerb Phrases,PrepositionsofPrepositionPhrases)areconsistentlyineither theleftmostorrightmostposition Edo/English:Head-First VerbPhrase: VerbObject Prepositions: PrepositionNounPhrase NP Object Edo/English NP Subject S VP VP NP Subject NP Verb Object PP PP P Preposition Japanese/Navajo S VP Verb Atthislevelofstructuralanalysis(parameters),languagesdiffervaryminimally fromeachother.Thismakeslanguagestructuremucheasierforchildrento learn.Alltheyneedtodoissettherightparametervaluesfortheirlanguage, basedonthedatathatareeasytoobserve. NP Object Verb NP Object PP NP P Object postposition P NP preposition Object Parameters Butwhatarelinguisticparametersreally?Howdotheywork?What exactlyaretheysupposedtodo? Aparameterismeanttobesomethingthatcanaccountformultiple observationsinsomedomain. Parameterforastatisticalmodel:determineswhatthemodel predictswillbeobservedintheworldinavarietyofsituations Parameterforourmental(andlinguistic)model:determineswhat wepredictwillbeobservedintheworldinavarietyofsituations Statisticalparameters Thenormaldistributionisa statisticalmodelthatuses twoparameters: -µ forthemean -σ forthestandarddeviation Statisticalparameters Supposethisisamodelofhow manyminuteslateyou’llbe toclass. Let’susethemodelwithµ = 0,andσ2=0.2.(blueline) Ifweknowthevaluesoftheseparameters,wecanmakepredictionsaboutthe likelihoodofdatawerarelyorneversee. Statisticalparameters Supposethisisamodelofhow manyminuteslateyou’llbe toclass. Let’susethemodelwithµ = 0,andσ2=0.2.(blueline) Howlikelyareyoutobe5minuteslate,giventheseparameters? Notverylikely!Wecantellthisjustbyknowingthevaluesofthetwo statisticalparameters.Theseparametervaluesallowustoinferthe probabilityofsomeobservedbehavior. Statisticalparameters Observingdifferentquantitiesof datawithparticularvalues cantelluswhichvaluesofμ andσ2aremostlikely,ifwe knowwearelookingto determinethevaluesofμ andσ2infunctionφ(X) Observingdatapointsdistributedlikethegreencurvetellsusthatμislikelyto bearound-2,forexample. Linguisticprinciplesvs.linguisticparameters Statisticalvs.Linguisticparameters Importantsimilarity: Wedonotseetheprocessthat generatesthedata,butonly thedatathemselves.This meansthatinordertoform ourexpectationsaboutX, weare,ineffect,reverse engineeringtheobservable data. Ourknowledgeoftheunderlyingfunction/principlethatgeneratesthesedata- φ(X)-aswellastheassociatedparameters-μ,andσ2-allowsustorepresentan infinitenumberofexpectationsaboutthebehaviorofvariableX. Linguisticprinciplesvs.linguisticparameters Bothprinciplesandparametersareoftenthoughtofasinnatedomain-specific abstractionsthatconnecttomanystructuralpropertiesaboutlanguage. Linguisticparameterscorrespondtothepropertiesthatvaryacrosshuman languages.Comparison:μandσ2determinetheexactformofthecurvethat representsthelikelihoodofobservingcertaindata.Whiledifferentvaluesfor theseparameterscanproducemanydifferentcurves,thesecurvessharetheir underlyingformduetothecommoninvariantfunction. Bothprinciplesandparametersareoftenthoughtofasinnatedomain-specific abstractionsthatconnecttomanystructuralpropertiesaboutlanguage. Linguisticprinciplescorrespondtothepropertiesthatareinvariantacrossall humanlanguages.Comparison:theequation’sform–itisthestatistical “principle”thatexplainstheobserveddata. Theutilityofconnectingtomultipleproperties Thefactthatparametersconnecttomultiplestructuralproperties thenbecomesaverygoodthingfromtheperspectiveofsomeone tryingtoacquirelanguage.Thisisbecauseachildcanlearnabout thatparameter’svaluebyobservingmanydifferentkindsof examplesinthelanguage. “Thericherthedeductivestructureassociatedwithaparticular parameter,thegreatertherangeofpotential‘triggering’datawhich willbeavailabletothechildforthe‘fixing’oftheparticular parameter”–Hyams(1987) Theutilityofconnectingtomultipleproperties Whyhard-to-learnstructuresareeasier Let’sassumeanumberofpropertiesareallconnectedtoparameter P,whichcantakeoneoftwovalues:aorb. Parameterscanbeespeciallyusefulwhenachildistryingtolearn thethingsaboutlanguagestructurethatareotherwisehardto learn,perhapsbecausetheyareverycomplexproperties themselvesorbecausetheyappearveryinfrequentlyinthe availabledata. P aorb? P1 P2 P3 P4 P5 Whyhard-to-learnstructuresareeasier HowdowelearnwhetherP4showsbehavioraorb? OnewayistoobservemanyinstancesofP4. P P5 ButwhatifP4occursveryrarely?Wemightneverseeanyexamples ofP4. aorb? P1 P2 P3 P4 Whyhard-to-learnstructuresareeasier P P1 P2 P3 aaaaaaaaaa… P4 P5 ??? aorb? Whyhard-to-learnstructuresareeasier Fortunately,ifP4isconnectedtoP,wecanlearnthevalueforP4by learningthevalueofP.Alsofortunately,PisconnectedtoP1,P2, P3,andP5. P Whyhard-to-learnstructuresareeasier Step1:ObserveP1,P2,P3,orP5.Inthiscase,alltheobserved examplesofthesestructuresarebehaviora. aorb? P P1 P2 P3 P1 P2 P3 P4 ??? P4 ??? P5 P5 Whyhard-to-learnstructuresareeasier Step2:UsethisknowledgetosetthevalueofparameterPtoa. P P1 P2 P3 P4 P5 ??? aaa aaaaaaaaaaa… Whyhard-to-learnstructuresareeasier Step3:SinceparameterPisconnectedtoP4,wecanpredictthatP4will alsoshowbehaviora-eventhoughwe’veneverseenanyexamplesof it!(WecanalsoinferP3andP5thesameway.) a aaa aaaaaaaaaaa… aorb? P P1 P2 P3 P4 P5 a aaa aaaaaaaaaaa… a HierarchicalBayesianlearninglinks: Overhypotheses Whyacquisitioniseasier P P1 P2 P3 P4 P5 a aaa aaaaaaaaaaa… a a Thishighlightsanotherbenefitof parameters-wedon’thavetolearn thebehaviorofeachstructure individually.Instead,wecanobserve somestructures(ex:P1andP2)and infertherightbehaviorforthe remainingstructures(P3,P4,andP5). OverhypothesesinhierarchicalBayesianlearningare generalizationsmadeatamoreabstractlevel,whichcovermany differentdatatypes. Inthisway,theyaresimilarinspirittolinguisticparameters. Thatis,insteadofhavingtomake5 decisions(oneforP1,P2,P3,P4,and P5),weactuallyonlyneedtomake onedecision-isPaorb? a HierarchicalBayesianlearninglinks: Overhypotheses HierarchicalBayesianlearninglinks: Overhypotheses Overhypothesisexample Overhypothesisexample Supposeyouareobservingthecontentsofmarblebags. Thefirstbagyoulookathas20blackmarbles. 20 HierarchicalBayesianlearninglinks: Overhypotheses HierarchicalBayesianlearninglinks: Overhypotheses Overhypothesisexample Overhypothesisexample Thesecondbagyoulookathas20whitemarbles. Thethirdandfourthbagsyoulookathave20blackmarbles. 20 20 20 HierarchicalBayesianlearninglinks: Overhypotheses 20 20 20 HierarchicalBayesianlearninglinks: Overhypotheses Overhypothesisexample Overhypothesisexample Yougetafifthbagandpulloutasinglemarble.It’swhite.Whatdoyou predictaboutthecolordistributionoftherestofthemarblesinthe bag? Mostadultspredictthisbagwillcontain19otherwhitemarbles,fora totalof20whitemarbles. 1 20 20 20 20 1 20 20 20 20 20 HierarchicalBayesianlearninglinks: Overhypotheses HierarchicalBayesianlearninglinks: Overhypotheses Overhypothesisexample Overhypothesisexample Whatifyouthengetasixthbagandpulloutasinglepurplemarble fromit? Mostadultswouldpredictthattheother19marblesinthatbagare purpletoo,for20purplemarblestotal. 1 20 20 20 20 1 20 1 20 HierarchicalBayesianlearninglinks: Overhypotheses Overhypothesisexample 20 20 1 20 20 HierarchicalBayesianlearninglinks: Overhypotheses Overhypothesisexample Whydoesthishappen?Itseemslikeyou’relearningsomethingabout thecolordistributioningeneral(notjustforaparticularbag):all marblesinabaghavethesamecolor.Thisallowsyoutomake predictionswhenyou’veonlyseenasinglemarbleofwhatevercolor fromabag. 1 20 1 20 20 20 20 20 20 observed 20 20 1 20 20 20 1 20 HierarchicalBayesianlearninglinks: Overhypotheses HierarchicalBayesianlearninglinks: Overhypotheses Overhypothesisexample Overhypothesisexample overhypothesis allthesamecolor allblack 20 allwhite 20 allblack 20 overhypothesis allthesamecolor allblack 1 20 1 20 20 HierarchicalBayesianlearninglinks: Overhypotheses allblack allwhite 20 20 allblack 20 makepredictions allblack 1 20 1 20 20 HierarchicalBayesianlearninglinks: Overhypotheses Linguisticoverhypothesisexample Linguisticoverhypothesisexample Supposeyouareobservingthewordorderofsentencesyouhear. ThefirstsentenceyouhearhastheVerbbeforetheObject(“Seethe penguin?”) VerbObject HierarchicalBayesianlearninglinks: Overhypotheses HierarchicalBayesianlearninglinks: Overhypotheses Linguisticoverhypothesisexample Linguisticoverhypothesisexample ThesecondsentenceyouhearhasthePrepositionbeforetheObject (“Ilikethepenguinontheiceberg”)andalsotheVerbbeforethe Object(“Ilikethepenguinontheiceberg”). Thesedatatellyouaboutwordorderforverbsandobjectsandalso aboutwordorderforprepositionsandtheirobjects. observed VerbObject PrepositionObject VerbObject VerbObject HierarchicalBayesianlearninglinks: Overhypotheses PrepositionObject VerbObject HierarchicalBayesianlearninglinks: Overhypotheses Linguisticoverhypothesisexample Linguisticoverhypothesisexample Inaddition,theyarerelatedviatheheaddirectionalityparameter, whichfunctionsasanoverhypothesis. Knowingthevalueofthisparameterallowsyoutopredictotherword orderpropertiesofthelanguage. Headdirectionality=headfirst headfirst VerbObject headfirst PrepositionObject VerbObject Headdirectionality=headfirst headfirst VerbObject makepredictions headfirst PrepositionObject VerbObject PossessionPossessor HierarchicalBayesianlearninglinks: Overhypotheses Learningoverhypotheses Learningoverhypotheses:Dewar&Xu(2010) 9-month-olds Question: Whenprovidedwithpartialevidenceaboutafew objectsinafewcategories,caninfantsforma moreabstractgeneralization(anoverhypothesis) thatthenappliestoanewcategory? Bayesianlearnercomputationalmodelsareabletolearn overhypotheses,providedtheyknowwhattheparametersare andtherangeofvaluesthoseparameterscantake(ex:Kemp, Perfors,&Tenenbaum2006). Whataboutreallearners? Learningoverhypotheses:Dewar&Xu(2010) 9-month-olds Trainingtrials: Observefourdifferentobjects pulledoutbyexperimenterwho hadhereyesclosed-theobjects aredifferentcolorsbutalways havethesameshape. Learningoverhypotheses:Dewar&Xu(2010) 9-month-olds Experimentaltrials: Expectedoutcome(assuming infantshadtheoverhypothesis thatalltheobjectsfromasingle boxshouldbethesameshape) = Experimenterpullsouttwo objectswiththesamenew shape. Infantsshouldnotbesurprised. Learningoverhypotheses:Dewar&Xu(2010) 9-month-olds 9-month-olds Experimentaltrials: Unexpectedoutcome(assuming infantshadtheoverhypothesis thatalltheobjectsfromasingle boxshouldbethesameshape) = Experimenterpullsouttwo objectswithdifferentshapes, onewhichisnewandonewhich isold. Infantsshouldbesurprised. Learningoverhypotheses:Dewar&Xu(2010) 9-month-olds Controltrials: Expectedoutcome(assuming infantshadtheoverhypothesis thatalltheobjectsfromasingle boxshouldbethesameshape) = Experimenterpullsouttwo objectswiththesamenew shape. Infantsshouldnotbesurprised. Learningoverhypotheses:Dewar&Xu(2010) 9-month-olds Controltrials: Unexpectedoutcomecontrol= Experimenterpullsouttwo objects,onewithanewshape thatcamefromthenewboxand onewithanoldshapethatcame fromanoldboxthatcontained thatshape. Learningoverhypotheses:Dewar&Xu(2010) Infantsshouldnotbesurprised thisoutcomeiscompatiblewith theoverhypothesis.(The overhypothesisisactually irrelevanthere.) Results: Infantsintheexperimental conditionlookedlongeratthe unexpectedoutcome(~14.28s) whencomparedtotheexpected outcome(~11.32s). Theyweresurprisedatthe evidencethatdidn’tsupportthe overhypothesis! Learningoverhypotheses:Dewar&Xu(2010) 9-month-olds 9-month-olds Results: Infantsinthecontrolcondition didnotlooklongeratthe expectedoutcomeascompared totheunexpectedoutcome controlthathadthesame objectspresent(~10.3-11.0s). Learningoverhypotheses:Dewar&Xu(2010) Theywerenotsurprisedatthe evidencethatwascompatible withtheoverhypothesis,evenif theevidenceinvolvedtwo differentlyshapedobjects. Overallresult: 9-month-oldsappearableto formoverhypothesesfromvery limiteddatasets. Hopefully,thismeanstheycan alsouselinguisticparametersto learn,sinceparametersare similartooverhypothesesabout language! Summary:Linguisticparameters Parametersmakeacquisitioneasierbecausehard-to-learn structurescanbelearnedbyobservingeasy-to-learnstructuresthat areconnectedtothesameparameters. Questions? Linguisticparametersaresimilartostatisticalparametersinthat theyareabstractionsabouttheobservabledata.Forlinguistic parameters,theobservabledataarelanguagedata. Parametersmaybesimilartooverhypotheses,whichBayesian learnersand9-month-oldsarecapableoflearning. Youshouldbeabletodoupthroughquestion11on thestructurereviewquestions. ExtraMaterial Syntax:Onereasonwhynaturallanguage comprehensionissohardforcomputers SolvingtheLanguageProblem (ArtificialIntelligence) SolvingtheLanguageProblem (ArtificialIntelligence) HAL9000from2001:ASpaceOdyssey(1968) 2012:Apple’sSiriisgettingcloser,thoughstillhasproblems… PerfectproductionandcomprehensionofEnglish. http://bits.blogs.nytimes.com/2012/07/15/with-apple’s-siri-aromance-gone-sour/?_php=true&_type=blogs&_r=0 1960s:Languagenotconsideredoneofthe“hard”problemsofartificialintelligence. 2010:Gettingbetterbutstillnotperfect. http://www.research.att.com/~ttsweb/tts/demo.php SolvingtheLanguageProblem (ArtificialIntelligence) SolvingtheLanguageProblem (ArtificialIntelligence) Contrast:Chess-playing. Updatefor2011onamachine’sabilitiestodowhathumansdo: In1997,aprogramnamedDeepBluebeat thereigningworldchampioninchess.Itdid thisbyhavingenoughcomputational resourcestoinvestigateeverymoveoption beforeitactuallymadethechessmove.This showsthatcomputers’poorperformanceon languageisnotaboutinsufficient computationalpower,sincethereisenough computationalpowertosolvethechessplayingproblem(whichsomepeoplemight consideraverydifficultproblem). Manvs.Machine(Watson)inJeopardy &howhardaproblemlanguagecomprehensionandproductionis SolvingtheLanguageProblem (ArtificialIntelligence) 2013:Trueon-the-flylanguagecomprehensionisstillprettyhard,aswellas determiningtheanswerto“commonsense”questionsthatarephrased naturally. http://www.sciencedaily.com/releases/2013/07/130715151059.htm “Oneofthehardestproblemsinbuildinganar€ficialintelligence,Sloansaid,isdevising acomputerprogramthatcanmakesoundandprudentjudgmentbasedonasimple percep€onofthesitua€onorfacts-thedic€onarydefini€onofcommonsense. CommonsensehaseludedAIengineersbecauseitrequiresbothaverylargecollec€on offactsandwhatSloancallsimplicitfacts—thingssoobviousthatwedon'tknowwe knowthem.Acomputermayknowthetemperatureatwhichwaterfreezes,butwe knowthaticeiscold.”-JeanneGalatzer-Levy “We'res€llveryfarfromprogramswithcommonsense-AIthatcananswer comprehensionques€onswiththeskillofachildof8,"saidSloan.Heandhis colleagueshopethestudywillhelptofocusa~en€ononthe"hardspots"inAI research. http://www.youtube.com/watch?v=dr7IxQeXr7g (approximately9minvideo) Watsonvs.allhumanity h~ps://www.youtube.com/watch?v=WFR3lOm_xhE (approximately4minvideo) Typesofvariation Vocabulary English“think”verbs:think,know,wonder,suppose,assume,… Multipletypesoftheactionverb“think”.Eachhascertainusesthatare appropriate. “Iwonderwhetherthegirlsavedherlittlebrotherfromthe goblins.”[grammatical] *“Isupposewhetherthegirlsavedherlittlebrotherfromthe goblins.”[ungrammatical] Typesofvariation Vocabulary English“think”verbs:think,know,wonder,suppose,assume,… Navajo“carry”verbs:dependsonobjectbeingcarried aah(carryasolidround-ishobject) kaah(carryanopencontainerwithcontents) lé(carryaflexibleobject) Typesofvariation Sounds:EachlanguageusesaparticularsubsetofthesoundsintheInternational PhoneticAlphabet,whichrepresentsallthesoundsusedinallhumanlanguages. There’softenoverlap(ex:“m”,“p”areusedinmanylanguages),butlanguagesalso maymakeuseofthelesscommonsounds. lesscommonEnglishsounds:“th”[T], “th”[D] lesscommonNavajosounds:“whisperedl”,“nasalizeda”,…