Distributional Semantic Models (ESSLLI 2009)

Schedule & handouts

Credits: Some of the slides in the course are based on an earlier DSM course taught by Marco Baroni

Foundations of DSMs (Mon, 27 July)
  • general introduction & background
  • DSMs in a nutshell
  • the DSM parameter space: a taxonomy

Slides, Handout

Matrix algebra & vector spaces (Tue, 28 July)
  • matrix representation, foundations of matrix algebra
  • geometric analysis (norms, distances, similarity & angles)
  • implementation: R as an interactive DSM laboratory

Slides, HandoutDSM lab (R script), bnc_vobj_filtered.txt.gz (15 MB) – Homework (R script), potter_tokens.txt.gz (8.9M)

DSMs in computational linguistics and cognitive science (Wed, 29 July)
  • What can we do with DSM vectors?
  • Do DSMs capture interesting aspects of word meaning?

Slides, Handout, Concrete nouns clustering, Concrete nouns MDS

Geometric interpretation of DSM spaces (Thu, 30 July)
  • more advanced matrix algebra
  • distances, norms and angles
  • principal components analysis (PCA) as noise reduction

Slides, Handout

SVD (Fri, 31 July, part A)
  • singular value decomposition (SVD)
  • managing high-dimensional DSMs (SVD, random indexing)

Slides, Handout

Challenges for DSMs & current research (Fri, 31 July, part B)
  • looking for a unified model
  • DSMs and compositionality

Slides, Handout