Despite impressive advances in Natural Language Generation (NLG) and Large Language Models (LLMs), researchers are still unclear about important aspects of NLG evaluation. To substantiate this claim, I examine current classifications of hallucination and omission in data-text NLG, and I propose a logic-based synthesis of these classfications. I conclude by highlighting some remaining limitations of all current thinking about hallucination and by discussing implications for LLMs.

Performance on a dataset is often regarded as the key criterion for assessing NLP models. I argue for a broader perspective, which emphasizes scientific explanation. I draw on a long tradition in the philosophy of science, and on the Bayesian approach to assessing scientific theories, to argue for a plurality of criteria for assessing NLP models. To illustrate these ideas, I compare some recent models of language production with each other. I conclude by asking what it would mean for institutional policies if the NLP community took these ideas onboard.

This article offers a survey of computational research on referring expression generation (REG). It introduces the REG problem and describes early work in this area, discussing what basic assumptions lie behind it, and showing how its remit has widened in recent years. We discuss computational frameworks underlying REG, and demonstrate a recent trend that seeks to link REG algorithms with well-established Knowledge Representation techniques. Considerable attention is given to recent efforts at evaluating REG algorithms and the lessons that they allow us to learn. The article concludes with a discussion of the way forward in REG, focusing on references in larger and more realistic settings.

It is often desirable that referring expressions be chosen in such a way that their referents are easy to identify. This article focuses on referring expressions in hierarchically structured domains, exploring the hypothesis that referring expressions can be improved by including logically redundant information in them if this leads to a significant reduction in the amount of search that is needed to identify the referent. Generation algorithms are presented that implement this idea by including logically redundant information into the generated expression, in certain well-circumscribed situations. To test our hypotheses, and to assess the performance of our algorithms, two controlled experiments with human subjects were conducted. The first experiment confirms that human judges have a preference for logically redundant expressions in the cases where our model predicts this to be the case. The second experiment suggests that readers benefit from the kind of logical redundancy that our algorithms produce, as measured in terms of the effort needed to identify the referent of the expression.

This article examines the role of gradable properties in referring expressions from the perspective of natural language generation. First, we propose a simple semantic analysis of vague descriptions (i.e., referring expressions that contain gradable adjectives) that reflects the context-dependent meaning of the adjectives in them. Second, we show how this type of analysis can inform algorithms for the generation of vague descriptions from numerical data. Third, we ask when such descriptions should be used. The article concludes with a discussion of salience and pointing , which are analyzed as if they were gradable adjectives.

This article challenges the received wisdom that template-based approaches to the generation of language are necessarily inferior to other approaches as regards their maintainability, linguistic well-foundedness, and quality of output. Some recent NLG systems that call themselves “template-based” will illustrate our claims.

This paper brings a logical perspective to the generation of referring expressions, addressing the incompleteness of existing algorithms in this area. After studying references to individual objects, we discuss references to sets, including Boolean descriptions that make use of negated and disjoined properties. To guarantee that a distinguishing description is generated whenever such descriptions exist, the paper proposes generalizations and extensions of the Incremental Algorithm of Dale and Reiter (1995).

In this paper, it is argued that “coreference” annotations, as performed in the MUC community for example, go well beyond annotation of the relation of coreference proper. As a result, it is not always clear what semantic relation these annotations are encoding. The paper discusses a number of problems with these annotations and concludes that rethinking of the coreference task is needed before the task is expanded. In particular, it suggests a division of labor whereby annotation of the coreference relation proper is separated from other tasks such as annotation of bound anaphora and of the relation between a subject and a predicative NP.