PhD
Transparent analysis of multi-modal embeddings
My PhD Thesis lies in the intersection of machine learning, natural language processing, computer vision and multi-modal semantic grounding. Its focus is on a deeper analysis of data sources and models, unearthing how different meaning representations "understand" concepts. The aim is to make a step towards interpretable models and to move beyond standard evaluation metrics such as accuracy.
See related publications here.
Scroll down for software packages and tools related to the thesis.
-
EmbEval
A Python toolkit for transparent and interpretable evaluation of Embeddings of different sources and modalities.
-
MMFeat Flickr API
I developed a Python Flickr API for image search for the mmfeat github project. It is a Python toolkit aiming to make it easier for researchers to use multi-modal features. Both image and sound (i.e., visual and auditory representations) are supported.
-
ConceptGame
Concept Game is a two-player collaborative game, where players guess the concept for a list of words. It is similar to the famous gamified data collection game ESP Game but with word clusters instead of images.
The method used to generate the puzzles is part of the PhD Thesis.