Dr. Giuseppe Carenini is a Professor of Computer Science at the University of British Columbia. He has broad interdisciplinary interests and his work on Natural Language Processing and Information Visualization to support decision making has been published in over 100 peer-reviewed papers (including best papers at UMAP-14 and ACM-Tiis-14).
What are your current research interests?
I am interested in Intelligence Augmentation, in essence, enabling humans and AI to collaborate on solving challenging data-intensive problems by leveraging their complementary abilities. I believe that this requires AI systems to be able to effectively communicate with humans by processing (understanding and generating) both natural language and information visualizations. To this end, I have been working on discourse processing, a critical natural language processing (NLP) task which involves dealing with language beyond single sentences, from uncovering how clauses and sentences are related to each other in the source documents, to planning and generating coherent multi-sentential text starting for communicative goals. In my research, I have applied the resulting computational models to extremely useful tasks involving large number of documents, like summarization and topic modeling. Remarkably, these tasks are outside the scope of recent popular language models (like ChatGPT) because such models can only deal with rather limited textual input. To enable complex human AI communication, I am also very interested in combining NLP with information visualization. For instance, while NLP techniques can extract useful information form large collection of documents (e.g., topics and sentiment), information visualization can enable the interactive analysis of the extracted information. Conversely, automatically generated language prompts and captions can help analysts, while they explore complex datasets using sophisticated interactive visualization interfaces.
How does your work intersect with Language Sciences?
For many years, I have been teaching a Computer Science graduate course in NLP, which has been taken by several students in Linguistics and form the iSchool. This has fostered collaboration between the two departments and arguably planted one of the seeds for developing a vision for Language Sciences. Last year, I designed a new undergraduate course on NLP, that will benefit students interested in computational models of language across campus. In my research, I develop NLP methods to enable AI system to understand and generate language. Often, I rely on theories from linguistics covering aspects of the meaning of words and sentences (i.e., semantics), as well as how language is used in communication to help users to achieve their goals (i.e., pragmatics). Although most of my work is on the English language, we strive to develop general multilingual approaches that can be applied to other human languages. Finally, my research truly reflects the diversity and cross-disciplinarily of language sciences. I engage in successful collaborations to apply my research to health, business intelligence, and economics. I work with epidemiologists on monitoring social media and analyzing doctor-patient on-line conversations, with neurologists on predicting mild cognitive impairment from speech, with IT companies on text analytics of conversations (e.g., mtgs summarization), and with economist on analyzing the responsiveness of the U.S. Regulatory System to public comments.
What is a project that you’re currently working on that you’re particularly proud of?
Modern language models, like GPT, are very powerful, but we do not really know how they encode specific knowledge about language. In one project, we are looking under the hood of these large models to identify what discourse information they capture and where they store it. Once this will be clarified, we will be able to develop techniques to inject the missing discourse knowledge into such models. By doing so, we can significantly enhance their ability to generate longer and more coherent text, paving the way for more advanced language generation capabilities.
Preliminary results on this research have been published in two recent papers:
- Chuyuan Li, Patrick Huber, Wen Xiao, Maxime Amblard, Chloe Braud, Giuseppe Carenini: Diiscourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues. EACL (Findings) 2023: 2517-2534
- Patrick Huber, Giuseppe Carenini: Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models. NAACL-HLT 2022: 2376-2394
What inspires you to pursue your line of research/work?
Language is the pinnacle of human intelligence, essentially defining our species. Uncovering the incredibly complex computational mechanisms underlying our language abilities is not only a formidable intellectual endeavour, but will also enable us to create AI agents that can effectively communicate and collaborate with humans, helping us in making better high-stakes decisions. This is already generating enormous benefits for our society by transforming and improving entire sectors like healthcare, finance, and law. On a more personal level, the quest to build better NLP models constantly challenges me to keep learning across multiple disciplines and to creatively explore new ideas with my collaborators.
If you had to choose a different career path, what would you be interested in pursuing and why?
I often tell my friends that if I had not pursued a career as an AI scientist, I would have really enjoyed being a detective. I think there are many similarities between the two jobs. Both demand a careful examination of all the available evidence, the ability to ask the right questions and often to think out of the box, connecting seemingly unrelate events and ideas. Patience and persistence are also essential qualities. In both jobs, rushing to conclusions can be detrimental; instead, you must be open to revising your assumptions and reasoning. Nevertheless, I still find greater fulfillment in teaching, advising students, uncovering new knowledge and building AI prototypes, than in solving crimes and pursuing culprits. So, no career change is on the horizon for me!