Table of Contents

Courses and Tutorials on DSM

ESSLLI 2009 – NAACL-HLT 2010 – ESSLLI '16 & '18 – ESSLLI 2021 – Software & data sets – Bibliography

Software for the course

Practical examples and exercises for these courses and tutorials are based on the user-friendly software package wordspace for the interactive statistical computing environment R. If you want to follow along, please bring your own laptop and set up the required software as follows:

  1. Install up-to-date versions of R (4.0 or newer) and the RStudio GUI
  2. Use the installer built into RStudio (or the standard R GUI) to install the following packages from the CRAN archive:
    • sparsesvd (v0.2)
    • wordspace (v0.2-6)
    • recommended: e1071, rsparse, Rtsne, uwot
    • optional: tm, quanteda, data.table, wordcloud, shiny, spacyr, udpipe, coreNLP (don't worry if some of these fail to install)
    • optional: NMF (also install biocManager, then run command BiocManager::install("bioBase"))
  3. During the course, you will be asked to install a further package with additional evaluation tasks (wordspaceEval) from a password-protected Web page:
    • wordspaceEval v0.2: Source/Linux – MacOS – Windows (login required)
    • if you are stuck with R v3.x, please use the older package version 0.1: Source/Linux – MacOS – Windows (login required)
    • download a suitable version and select “Install from: Package Archive File” in RStudio
  4. Download the sample data files listed below
  5. Download one or more of the pre-compiled DSMs listed below

Scaling R to large data sets

Most of our hands-on examples work reasonably well in a standard R installation, even on a moderately powerful laptop computer. However, if you intend to work on real-life tasks and process large DSMs, it is important to enable multi-threaded computation in R. Since DSMs build on matrix operations, a multi-threaded linear algebra library (“BLAS”) is key.

  1. In Linux, it should be sufficient to install the OpenBLAS package, e.g. in Ubuntu: sudo apt install libopenblas-dev
  2. In MacOS, follow these instructions to enable the VecLib BLAS built into MacOS. You may also want to enable OpenMP for an additional speed boost on expensive distance metrics (but this is less important).
  3. In Windows, you can try installing Microsoft R Open or do a Web search for alternative solutions.

Example data sets

Pre-compiled DSMs

Pre-compiled DSMs for use with the wordspace package for R. Each model is contained in an .rda file, which can be loaded into R with the command load("model.rda") and creates an object with the same name (model).

DSMs based on the English Wikipedia

These models were compiled from WP500, a 200-million word subset of the Wackypedia corpus comprising the first 500 words of each article. Each model covers a vocabulary of the 50,000 most frequent content words (lemmatized) in the corpus and has at least 50,000 feature dimensions. The latent SVD dimensions are based on log-transformed sparse simple-ll scores with L2-normalization. Power scaling with Caron $P = 0$ (i.e. equalization of the latent dimensions) has been applied, but the reduced vectors are not re-normalized.

Neural word embeddings

Some publicly available pre-trained neural embeddings, converted into .rda format for use with the wordspace package.

Web interfaces