Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
course:esslli2021:start [2021/08/11 22:58]
schtepf [Schedule & handouts]
course:esslli2021:start [2022/08/11 12:26] (current)
schtepf [Hands-on Distributional Semantics (ESSLLI 2021 / 2022)]
Line 1: Line 1:
-====== Hands-on Distributional Semantics (ESSLLI 2021)  ======+====== Hands-on Distributional Semantics (ESSLLI 2021 / 2022)  ======
  
 **Hands-on Distributional Semantics – From first steps to interdisciplinary applications** **Hands-on Distributional Semantics – From first steps to interdisciplinary applications**
 +[[https://2022.esslli.eu/|{{ :course:esslli2021:esslli2022_logo.png?150|ESSLLI 2022 (Galway)}}]]
 [[https://esslli2021.unibz.it/|{{ :course:esslli2021:esslli21.png?250|ESSLLI 2021 (online)}}]] [[https://esslli2021.unibz.it/|{{ :course:esslli2021:esslli21.png?250|ESSLLI 2021 (online)}}]]
 \\ \\
 //Foundational course at [[https://esslli2021.unibz.it/page/course/hands_on_distributional_semantics_from_first_steps_to_interdisciplinary_applications_introductory_course/|ESSLLI 2021]], online, August 9–13, 2021// //Foundational course at [[https://esslli2021.unibz.it/page/course/hands_on_distributional_semantics_from_first_steps_to_interdisciplinary_applications_introductory_course/|ESSLLI 2021]], online, August 9–13, 2021//
 +\\
 +
 +**Hands-on Distributional Semantics for Linguistics using R**
 +\\
 +//Foundational course at [[https://2022.esslli.eu/courses-workshops-accepted/week-1-and-2-schedule.html#W1|ESSLLI 2022]], Galway, Ireland, August 8–12, 2022//
 +\\
 +
  
   * [[course:material|Software & data sets]]   * [[course:material|Software & data sets]]
Line 10: Line 18:
  
 {{:icon_star.png?24 }} {{:icon_star.png?24 }}
-  * **schedule change**: we will continue with part 2 on Wednesday, then cover parts 3–5 on Thursday and Friday +  * update of all materials for the 2022 edition of the course has been completed 
-  * combined **presentation slides for parts 3 & 4** now available +  * //Thanks for attending our course! It's a pleasure working with you.//
  
 ===== Course description ===== ===== Course description =====
  
-Distributional semantic models (DSM) are based on the assumption that the meaning of a word can (at least to a certain extent) be inferred from its usage, i.e. its distribution in text. Therefore, these models dynamically build semantic representations through a statistical analysis of the contexts in which words occur. DSMs are a promising technique for solving the lexical acquisition bottleneck by unsupervised learning, and their distributed representation provides a cognitively plausible, robust and flexible architecture for the organisation and processing of semantic information.+Distributional semantic models (DSM) – also known as “word space”, “distributional similarity”, or more recently “word embeddings” – are based on the assumption that the meaning of a word can (at least to a certain extent) be inferred from its usage, i.e. its distribution in text. Therefore, these models dynamically build semantic representations – in the form of high-dimensional vector spaces – through a statistical analysis of the contexts in which words occur. DSMs are a promising technique for solving the lexical acquisition bottleneck by unsupervised learning, and their distributed representation provides a cognitively plausible, robust and flexible architecture for the organisation and processing of semantic information.
  
-In this introductory course we will highlight the interdisciplinary potential of DSM beyond standard semantic similarity tasks, with applications in cognitive modeling and theoretical linguistics. This course aims to equip participants with the background knowledge and skills needed to build different kinds of DSM representations and apply them to a wide range of tasks. There will be a particular focus on practical exercises with the user-friendly [[http://www.r-project.org/|R]] software package [[http://wordspace.r-forge.r-project.org/|wordspace]] and various pre-built models.+In this introductory course we will highlight the interdisciplinary potential of DSM beyond standard semantic similarity tasks, with applications in cognitive modeling and theoretical linguistics. This course aims to equip participants with the background knowledge and skills needed to build different kinds of DSM representations – from traditional “count” models to neural word embeddings – and apply them to a wide range of tasks. The hands-on sessions will be conducted in [[http://www.r-project.org/|R]] with the user-friendly [[http://wordspace.r-forge.r-project.org/|wordspace]] package and various pre-built models.
  
-**Lecturers:** [[http://www.stefan-evert.de/|Stefan Evert]] (FAU Erlangen-Nürnberg) & [[https://www.ims.uni-stuttgart.de/en/institute/team/Lapesa/|Gabriella Lapesa]] (IMS, U Stuttgart)+ 
 +**Lecturers:** [[https://www.stephanie-evert.de/|Stephanie Evert]] (FAU Erlangen-Nürnberg) & [[https://www.ims.uni-stuttgart.de/en/institute/team/Lapesa/|Gabriella Lapesa]] (IMS, U Stuttgart)
  
 ===== Organizational information ===== ===== Organizational information =====
Line 26: Line 35:
 Additional instructions will be given in the first session on Monday. In particular, you will be asked to download and install the ''wordspaceEval'' package using a password provided in the course. Additional instructions will be given in the first session on Monday. In particular, you will be asked to download and install the ''wordspaceEval'' package using a password provided in the course.
  
-We will answer questions during lectures and in the afternoon via the course's [[https://esslli21.slack.com/archives/C028KS92G3Z|Slack channel]]. Registered participants of ESSLLI 2021 should have access to this channel.+/* We will answer questions during lectures and in the afternoon via the course's [[https://esslli21.slack.com/archives/C028KS92G3Z|Slack channel]]. Registered participants of ESSLLI 2021 should have access to this channel. */
  
  
Line 35: Line 44:
 === Day 1: Introduction === === Day 1: Introduction ===
  
-[[http://www.collocations.de/data/esslli2021/esslli2021_part1.slides.pdf|presentation slides]] (PDF, 2.1 MB) – [[http://www.collocations.de/data/esslli2021/esslli2021_part1.handout.pdf|handout]] (PDF, 1.MB) – R code: [[http://www.collocations.de/data/esslli2021/hands_on_day1.R|hands_on_day1.R]]+[[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part1.slides.pdf|presentation slides]] (PDF, 2.1 MB) – [[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part1.handout.pdf|handout]] (PDF, 1.MB) – R code: [[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:hands_on_day1.R|hands_on_day1.R]]
  
   * motivation and geometric intuition   * motivation and geometric intuition
   * distributional vs. semantic similarity   * distributional vs. semantic similarity
   * outline of the course   * outline of the course
-  * practice: //software setup, first practical exercises with the ''wordspace'' package//+  * practice: //software setup, first steps with the ''wordspace'' package//
  
 === Day 2: Building a DSM === === Day 2: Building a DSM ===
  
-[[http://www.collocations.de/data/esslli2021/esslli2021_part2.slides.pdf|presentation slides]] (PDF, 1.MB) – [[http://www.collocations.de/data/esslli2021/esslli2021_part2.handout.pdf|handout]] (PDF, 1.2 MB) – R code: [[http://www.collocations.de/data/esslli2021/hands_on_day2.R|hands_on_day2.R]] – bonus material: [[http://www.collocations.de/data/esslli2021/hands_on_day2_input_formats.R|hands_on_day2_input_formats.R]], [[http://www.collocations.de/data/esslli2021/hands_on_day2_matrix_factorization.R|hands_on_day2_matrix_factorization.R]] +[[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part2.slides.pdf|presentation slides]] (PDF, 1.MB) – 
 +[[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part2.handout.pdf|handout]] (PDF, 1.2 MB) – 
 +R code: [[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:hands_on_day2.R|hands_on_day2.R]] – 
 +bonus material: [[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:hands_on_day2_input_formats.R|hands_on_day2_input_formats.R]]
  
   * formal definition of a DSM, taxonomy of parameters   * formal definition of a DSM, taxonomy of parameters
Line 54: Line 66:
 === Day 3: Which aspects of meaning does a DSM capture? === === Day 3: Which aspects of meaning does a DSM capture? ===
  
-[[http://www.collocations.de/data/esslli2021/esslli2021_part3_4.slides.pdf|presentation slides for days 3 & 4]] (PDF, 6.MB) – [[http://www.collocations.de/data/esslli2021/esslli2021_part3_4.handout.pdf|handout for days 3 & 4]] (PDF, 6.MB) – R code: [[http://www.collocations.de/data/esslli2021/hands_on_day3.R|hands_on_day3.R]], [[http://www.collocations.de/data/esslli2021/hands_on_day3_exercise_1.R|hands_on_day3_exercise_1.R]], [[http://www.collocations.de/data/esslli2021/hands_on_day3_exercise_2.R|hands_on_day3_exercise_2.R]] +[[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part3.slides.pdf|presentation slides]] (PDF, 3.MB) – 
 +[[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part3.handout.pdf|handout]] (PDF, 2.MB) – 
 +R code: [[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:hands_on_day3_exercise_1.R|hands_on_day3_exercise_1.R]], [[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:hands_on_day3_exercise_2.R|hands_on_day3_exercise_2.R]]
  
   * evaluation: conceptual coordinates   * evaluation: conceptual coordinates
Line 63: Line 77:
 === Day 4: DS beyond NLP – Linguistic theory === === Day 4: DS beyond NLP – Linguistic theory ===
  
-//presentation slides integrated into day above//+[[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part4.slides.pdf|presentation slides]] (PDF, 3.6 MB) – 
 +[[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part4.handout.pdf|handout]] (PDF, 3.5 MB) – 
 +R code: [[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:hands_on_day4.R|hands_on_day4.R]] – 
 +bonus material: [[http://www.collocations.de/data/esslli2021/schuetze1998.R|schuetze1998.R]] 
  
   * linguistic exploitation of DSM representations   * linguistic exploitation of DSM representations
Line 72: Line 90:
  
 === Day 5: DS beyond NLP – Cognitive modelling === === Day 5: DS beyond NLP – Cognitive modelling ===
 +
 +[[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part5.slides.pdf|presentation slides]] (PDF, 1.6 MB) –
 +[[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:esslli2022_part5.handout.pdf|handout]] (PDF, 1.4 MB) –
 +R code: [[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:hands_on_day5.R|hands_on_day5.R]] –
 +bonus task: [[http://www.collocations.de/data/CogALex4.rda|CogALex4.rda]] (0.2 MB) –
 +bonus material: [[http://wordspace.collocations.de/lib/exe/fetch.php/course:esslli2021:hands_on_day5_matrix_factorization.R|hands_on_day5_matrix_factorization.R]] 
  
   * DSMs for cognitive modelling   * DSMs for cognitive modelling
Line 77: Line 101:
   * predicting free associations with DSMs   * predicting free associations with DSMs
   * practice: //combining DSMs with first-order co-occurrence for the FAST task//   * practice: //combining DSMs with first-order co-occurrence for the FAST task//
- 
-<!-- 
- 
-=== Day 1: Introduction === 
- 
-{{:course:esslli2018:dsm_tutorial_part1.slides.pdf|Presentation slides}} (PDF, 1.1 MB) – {{:course:esslli2018:dsm_tutorial_part1.handout.pdf|handout}} (PDF, 0.9 MB) – {{:course:esslli2018:part1_examples.R|R code}} 
- 
-  * motivation and brief history of distributional semantics 
-  * common DSM architectures & prototypical applications 
-  * first practical exercises with the ''wordspace'' package 
- 
- 
-=== Day 2: The parameters of a DSM === 
- 
-{{:course:esslli2018:dsm_tutorial_part2.slides.pdf|Presentation slides}} (PDF, 1.3 MB) – {{:course:esslli2018:dsm_tutorial_part2.handout.pdf|handout}} (PDF, 1.0 MB) – {{:course:esslli2018:part2_examples.R|R code}} – {{:course:esslli2018:part2_input_formats.R|practice: input formats}} – {{:course:esslli2018:part2_exercise.R|exercise (DSM parameters)}} 
- 
- 
-  * taxonomy of DSM parameters: context representation, feature scaling, normalization and standardization, distance/similarity measures, dimensionality reduction 
-  * overview of common parameter settings & best-practice recommendations 
-  * practical exercises: building DSMs and exploring their parameters 
- 
- 
-=== Day 3: Applications and evaluation === 
- 
-{{:course:esslli2018:dsm_tutorial_part3.slides.pdf|Presentation slides}} (PDF, 2.0 MB) – {{:course:esslli2018:dsm_tutorial_part3.handout.pdf|handout}} (PDF, 1.8 MB) – {{:course:esslli2018:part3_examples.R|R code}} – {{:course:esslli2018:part3_exercise.R|exercise (evaluation)}} 
- 
-  * attributional and relational similarity, clustering and semantic categorization, multiple-choice tasks 
-/*  * supervised & unsupervised classification based on DSM vectors */ 
-  * insights from recent parameter evaluation studies 
-  * practical exercises: implementation and evaluation of selected tasks 
- 
- 
-=== Day 4: Elements of matrix algebra === 
- 
-{{:course:esslli2018:dsm_tutorial_part4.slides.pdf|Presentation slides}} (PDF, 0.7 MB) – {{:course:esslli2018:dsm_tutorial_part4.handout.pdf|handout}} (PDF, 0.6 MB) – {{:course:esslli2018:part4_examples.R|R code}} – {{:course:esslli2018:schuetze1998.R|bonus practice: Schütze-style WSD}} – {{:course:esslli2018:part4_exercise.R|exercise (roll your own DSM)}} 
- 
-  * basic matrix and vector operations, orthogonal projection & dimensionality reduction 
-  * singular value decomposition (SVD) 
-  * practical exercises: roll your own DSM with matrix operations 
- 
- 
-=== Day 5: Making sense of DSMs === 
- 
-  * mathematical properties of and relations between different types of DSM 
-  * singular value decomposition (SVD) as a latent class model 
-  * comparison with neural vector embeddings 
- 
--->