This year's Microsoft BCS/BCS IRSG Karen Spärck Jones Award was awarded to:
Krisztian Balog
Professor at the University of Stavanger, Norway
Titel: On Entities and Evaluation
Abstract:
In this presentation, I will address two broad topics, entities and evaluation, which have been the main focus of my research for the last decade or so. Over this period, we have witnessed entities becoming first-class citizens in many information access scenarios. With this has also come an increased reliance on knowledge bases, which organize information about entities in a structured and semantically meaningful way. I will review progress made on two specific retrieval problems, entity retrieval and entity linking, and illustrate how these can be utilized as building blocks for solving more complex tasks. Further, I will highlight some open challenges that remain in this space. The second part of the talk will concentrate on evaluation, which has been a central theme in IR since the inception of the field. For a long time, system-oriented evaluation has primarily been performed using offline test collections, following the Cranfield paradigm. While this rigorous methodology ensures the repeatability and reproducibility of experiments, it is inherently limited by abstracting the actual user, to a large extent, away. I will argue for the (complementary) need of online evaluation. Specifically, I will introduce the "living labs" evaluation methodology along with past and present efforts to implement it as a collaborative R&D scheme.
Short bio:
Krisztian Balog is a full professor at the University of Stavanger in Norway, where he leads the Information Access & Interaction research group. Currently, he is spending his sabbatical as a Visiting Staff Faculty Researcher at Google, London. Balog obtained a PhD in Computer Science from the University of Amsterdam in 2008 under the supervision of Prof. Maarten de Rijke, and holds MSc degrees in Computer Science from the Eötvös Loránd University, Budapest (2005) and from the Vrije Universiteit, Amsterdam (2004). His general research interests lie in the use and development of information retrieval, information extraction, and machine learning techniques for intelligent information access tasks. Balog serves as a senior programme committee member at SIGIR, CIKM and ECIR, as an Associate Editor of the ACM Transactions on Information Systems, and as a former coordinator of information retrieval benchmarking efforts at TREC and CLEF. He has recently authored the (open access) book Entity-Oriented Search.