Introduction

Interaction between formal and distributional semantics

This workshop aims to elicit interaction between the fields of formal semantics and distributional semantics. Formal semantics is the discipline that tries to produce a very precise theory of meaning by representing natural language within a logical framework. The principle of compositionality has been explored in depth within the domain of formal semantics. However, formal semanticists mostly focus on functional elements, such as determiners, quantification, modality, and tense morphemes, inter alia. The standard representation of lexical items within a formal semantic framework has been somewhat neglected: the meaning of a content word is usually represented by its denotation, which is difficult to implement within a computational framework. Distributional semantics, on the other hand, is very successful at inducing the meaning of individual content words, but less so with regard to function words and compositionality. In this respect, the frameworks of formal semantics and distributional semantics are complementary. Still, there has not been a lot of interaction between both communities throughout the years. This workshop brings together researchers from both fields of formal and distributional semantics, in order to promote interaction between both communities.

Speakers

Marianna Apidianaki (LIMSI-CNRS)

Lexical similarity and substitution in translation

Semantic knowledge acquisition from parallel corpora relies on distributional and translational regularities observed in the data. These serve to cluster word usages into senses and to identify sets of semantically related words and larger text segments. The resulting synonym and paraphrase sets offer a generalisation across similar meanings which can reduce sparseness and improve evaluation in translation applications. In this talk I will look into the relations encoded in semantic representations commonly used in multilingual tasks. I will demonstrate that meaning distinctions present in resources derived from parallel data might lead to erroneous judgments of semantic relatedness. Furthermore, I will explore the correlation of semantic adequacy judgments based on source and translation context. I will end with an account of the factors defining semantic relatedness and substitution in a multilingual setting.

Nicholas Asher (CNRS, IRIT)

TBD

Marco Baroni (CIMeC, Università di Trento)

Vector-based semantics from joint linguistic and visual evidence

The development of increasingly more effective ways to induce vector-based representations of words and visual concepts from natural data opens the way to empirical studies of how language and vision are combined to perform various semantic tasks. Building on this recent tradition, I will present the multimodal skip-gram model, an algorithm that induces vectors capturing both linguistic and visual knowledge in a realistic learning regime in which the two sources of information are presented jointly. I will focus on evaluating the algorithm on tasks that should shed some light on interesting aspects of human semantic knowledge: predicting different kinds of similarity judgments, predicting visual properties of concepts in lack of direct visual evidence (including visual properties associated to abstract words) and simulating, more specifically, how subjects induce properties of the visual referents of a new word from limited linguistic evidence.

Joint work with: Angeliki Lazaridou, Nghia The Pham, Marco Marelli

Stephen Clark (University of Cambridge)

The Theory and Practice of Compositional Distributed Semantics

This talk will describe a complete mathematical framework for deriving distributed representations compositionally using Combinatory Categorial Grammar (CCG). The tensor-based framework extends that of Coecke et al., which was previously applied to pregroup grammars, and is based on the observation that tensors are functions (multi-linear maps) and hence can be manipulated by the combinators of CCG, including type-raising and composition. The existence of robust, broad-coverage CCG parsers opens up the possibility of applying the tensor-based framework to naturally occurring text.

I will also describe our ongoing efforts to implement the framework, for which there are considerable practical challenges. I will describe some of the sentence spaces we are exploring; some of the datasets we are developing; and some of the machine learning techniques we are using in an attempt to learn the values of the tensors from corpus data.

This work is being carried out with Luana Fagarasan, Douwe Kiela, Jean Maillard, Tamara Polajnar, Laura Rimell, Eva Maria Vecchi, and involves collaborations with Mehrnoosh Sadrzadeh (Queen Mary) and Ed Grefenstette and Bob Coecke (Oxford).

Tim Van de Cruys (CNRS, IRIT)

An Exploration of Formal Semantic Phenomena within a Distributional Framework

Many semantic generalizations have been amply investigated within the field of formal semantics, such as the mass-count noun distinction, or the telicity of verb phrases, inter alia. However, few studies have looked at these phenomena from an empirical perspective. In this talk, we explore whether empirical data coming from distributional semantics is able to support semantic generalizations that are known from formal semantic theories. Secondly, we explore whether distributional methods are able to model certain theoretical concepts known from formal semantics (such as coercion) in a fully automatic way.

Cécile Fabre, Nabil Hathout, Ludovic Tanguy (CLLE/ERSS, Université de Toulouse II)

Some aspects of distributional semantics in specialized corpora

In this talk we focus on the application of distributional semantics methods to a small specialized language corpus of 586 papers presented at TALN, the French NLP annual conference. This corpus has two interesting characteristics. By its size and genre, it is representative of the data available for some applicative tasks (resource building for NLP or knowledge engineering), and its particular domain allows us to perform an expert and rigorous evaluation.

We will discuss the constraints this type of data imposes on the current distributional methods in terms of parameter setting (choice of measures, threshold, etc.). We will also adress a number of linguistic topics by studying the distributional behavior of different types of words (varying in POS, frequency, field-specificity, etc.)

Alexander Koller (University of Potsdam)

Top-down questions for distributional semantics

Much of the literature in distributional semantics (DS) claims that DS is a “success story”. In my talk, I will challenge this claim by asking what the measure of success should be. I will distinguish between bottom-up and top-down views on linguistic theories, and argue that we count truth-conditional semantics as failed (for computational linguistics) for top-down reasons, and DS as a success for bottom-up reasons. I will propose that we should define a clear top-down goal for DS; I believe we need to think about this so we can compare DS more fairly to truth-conditional semantics, and to make DS-based linguistic theories falsifiable.

In the second part of the talk, I will focus on one obvious candidate for a top-down goal, namely similarity of arbitrary phrases. I will ask whether our evaluation methods for similarity are appropriate, and whether similarity of larger phrases is a meaningful concept if the task and context are left unspecified. Time permitting, I will also explore the relationship between the Distributional Hypothesis and the Late-Wittgenstein “meaning as use” view of semantics, and discuss some examples of how the “use” of natural-language expressions in communication between actual people goes beyond the distribution of words with respect to other words or images.

Louise McNally (Universitat Pompeu Fabra)

Distributional representations, Carlsonian kinds, and Roots

This talk has two goals. First, I will argue that interesting results might be obtained by using distributional semantic representations to model Carlsonian kinds or kind descriptions as used in linguistic work that posits them as the denotation of the innermost layer in a so-called "layered DP" (e.g. Zamparelli 1995, McNally and Boleda 2004). I will sketch a tentative approach to connecting these representations to DRT, such that their advantages for the analysis of complex nominal descriptions can be exploited (see Garrette, et al. 2011 for an earlier attempt to do this with other goals in mind; see also Kamp 2013 for work that is much in the same spirit, though it does not use distributional representations). Second, I will point to some parallelisms between the proposed approach to the layered DP and syntactic frameworks that build on a deep distinction between roots devoid of interesting grammatical properties and functional structure, such as Distributed Morphology (Halle and Marantz 1993). The larger program is to develop semantic representations that better reflect the articulation between "root" and functional structure for which there is increasing evidence from syntactic theory and, in addition, better cope with the role of general world knowledge in the construction of linguistic meaning.

Sebastian Padó (University of Stuttgart)

Type and Typicality: Human Sentence Processing Uses Both

The concept of (discrete) types and type hierarchies is a mainstay of formal semantics. In contrast, distributional semantics is generally based on a notion of (graded) similarity and, correspondingly, (graded) typicality.

In my talk, I will report on recent psycholinguistic work, both experimental and computational, on logical metonymy ("Peter began the hike/book"). While the phenomenon has traditionally been explained in terms of type (longer reading times for entities compared to events), we there is evidence that these results are nevertheless consistent with a distributional model.

In a recent experiment, however, we manipulated type and typicality independently of one another, and found effects for both variables – indicating that human sentence processing makes use of both types of information. This raises interesting challenges for computational models of human sentence processing.

Benjamin Spector (CNRS, IJN, ENS)

TBD

Programme

Friday, February 27
09:00 Registration
09:15 Introduction
09:45 Marco Baroni (CIMeC, Università di Trento)
Vector-based semantics from joint linguistic and visual evidence
10:40 Coffee
11:05 Marianna Apidianaki (LIMSI-CNRS)
Lexical similarity and substitution in translation
12:00 Lunch
14:00 Stephen Clark (University of Cambridge)
The Theory and Practice of Compositional Distributed Semantics
14:55 Nicholas Asher (CNRS, IRIT)
TBD
15:50 Coffee
16:15 Cécile Fabre, Nabil Hathout, Ludovic Tanguy (CLLE/ERSS, Université de Toulouse II)
Some aspects of distributional semantics in specialized corpora
17:10 End for the day
 
20:00 Dinner
at La Table de William
90 Rue Saint-Roch, 31400 Toulouse
 
Saturday, February 28
09:00 Coffee
09:15 Sebastian Padó (University of Stuttgart)
Type and Typicality: Human Sentence Processing Uses Both
10:10 Alexander Koller (University of Potsdam)
Top-down questions for distributional semantics
11:05 Coffee
11:30 Benjamin Spector (CNRS, IJN, ENS)
TBD
12:25 Lunch
14:25 Louise McNally (Universitat Pompeu Fabra)
Distributional representations, Carlsonian kinds, and Roots
15:20 Tim Van de Cruys (CNRS, IRIT)
An Exploration of Formal Semantic Phenomena within a Distributional Framework
16:15 Coffee
16:40 Discussion session
17:30 The end

Location

The workshop will take place at the “open space”, which is located on the second floor of building Q at the Manufacture des Tabacs, 21, Allée de Brienne, Toulouse.


View Larger Map

To locate building Q, you can look at a map of the Manufacture des Tabacs.

Registration

Registration is closed. Please contact us directly if you still want to participate.

Contact

This workshop is organized by Marta Abrusan, Nicholas Asher, Sylvie Doutre, and Tim Van de Cruys. For questions, please contact marta.abrusanatirit.fr.