Fabio Cuzzolin

Università degli Studi di Padova

Ph.D., Industrial Electronics and Computer Science

January 1, 1998 – January 1, 2001

Università degli Studi di Padova

Master's, Computer Engineering

January 1, 1990 – January 1, 1997

UDrive

President and Founder

August 1, 2025 – Present

Remote

Oxford Brookes Institute for Artificial Intelligence, Data Analysis and Systems (AIDAS)

Director

January 1, 2024 – Present

Oxford, England, United Kingdom · On-site

Aiuta

Scientific Advisor

April 1, 2023 – July 1, 2023

Remote

Oxford Brookes University

Steering Committee Member, Institute for Ethical AI

March 1, 2020 – January 1, 2024

Huawei Technologies - Simon Fraser University Joint Visual Computing Centre

Executive Committee Member

September 1, 2018 – August 1, 2020

Vancouver, Canada Area

Oxford Brookes University

Professor of Artificial Intelligence

January 1, 2016 – Present

Oxford Brookes University

Director of the Visual Artificial Intelligence Laboratory

September 1, 2012 – March 1, 2022

Visual Artificial Intelligence Laboratory @ Oxford Brookes

Research Lead

January 1, 2012 – Present

Oxford, England, United Kingdom

INRIA

Marie Curie Fellow

September 1, 2006 – September 1, 2008

Grenoble Area, France

UCLA

Postdoctoral Researcher

October 1, 2004 – April 1, 2006

Greater Los Angeles Area

Politecnico di Milano

Giovane Ricercatore

January 1, 2003 – December 1, 2004

Università degli Studi di Padova

Postdoctoral Researcher

June 1, 2001 – May 1, 2003

Padova Area, Italy

Washington University in St. Louis

Visiting Scholar

June 1, 2000 – December 1, 2000

Greater St. Louis Area

Epistemic Generative AI

January 1, 2026 – Present

Generative AI (GenAI) is an ongoing revolution induced by artificial intelligence algorithms capable of producing novel and realistic content autonomously, transforming fields such as drug discovery, materials science and astronomy, to name a few. Generative AI is largely responsible for the latest wave of enthusiasm towards AI, having quickly achieved massive real-world impact. However,there is a growing recognition that uncertainty quantification is one of the major stumbling blocks towards a new generation of AI approaches. This may prevent an area such as AI for science (very strongly on the rise, as attested by Deepmind’s recent Nobel prize in chemistry) from unfolding its full potential. A ground-breaking new paradigm for generative AI in the presence of uncertainty would undoubtedly unleash a new wave of scientific discoveries and advances powered by AI. Our research hypothesis is that properly modelling the epistemic uncertainty in the generation process can lead to more robust, reliable and creative models. Accordingly, our overall goal is to lay the foundations for an epistemic generative AI where epistemic uncertainty is embedded in the generation process. This translates into three concrete Objectives: (1)Devising a general framework for modelling epistemic uncertainty in the parameter space of models. (2) Based upon this framework, designing anew class of Random-Set Large Language Models outputting an epistemic uncertainty measure, in the form of a random set, in the token space, capable of mitigating hallucination problems by teaching the network the true extent of the possible ramifications of a sentence. (3) Leveraging second-order probability distributions, develop a class of epistemic diffusion models featuring a much-improved diversity of generation.

Intrinsically-aligned AI

July 1, 2025 – Present

This is a £300,000 collaborative project with Carlo Cordasco, from the University of Manchester. Artificial Intelligence (AI) is increasingly used in public sectors like healthcare and justice, making explainability critical to building trust and enabling accountability. This project addresses the "black box" problem of AI systems by examining the trade-off between accuracy and explainability in both human and AI decision-making. It seeks to establish whether there should be a Right to Explanation, balancing societal benefits like informed self-advocacy with potential efficiency costs. By combining normative analysis, empirical studies and algorithmic development, the project aims to create a new explainable AI paradigm aligned with public trust and ethical values, guiding AI regulation and practical adoption. The overall objective is to lay down the foundations for both theory and practice of AI algorithms able to discount between accuracy and explainability depending on user/s preferences, in a whole new “intrinsically-aligned” paradigm.

Evolving AI

March 1, 2024 – Present

Deep worries exist about a number of potentially existential challenges to the human race. Among those, the dangers posed by anthropic climate change to our civilisation and the risks associated with the uncontrolled development of artificial intelligence (AI) are at the forefront of the public discourse. What is currently arguably overlooked, partly because of the siloed nature of the two debates, is the enormous untapped potential for a positive use of a new generation of artificial intelligence techniques, in a holistic context in which humans, machines and the environment converge towards an optimal, desirable equilibrium. We believe such a tantalising prize can be achieved, once the foundations for a new form of machine intelligence, one able to dynamically adapt to, understand and align with evolving environments in the presence of uncertainty, are laid out. The pillars of this novel “Evolving AI” framework are: (1) A paradigm for an evolving intelligence based on time-varying non-stationary optimisation, applied to both the learning process and the emergence in time of new goals, superseding current machine and continual learning paradigms to create machines able to truly adapt to and make reliable predictions in dynamic environments; (2) Resilience, achieved by intrinsically modelling the “epistemic” uncertainty induced by lack of data of sufficient quantity and quality via techniques from higher-order uncertainty theory; (3) The ability to understand complex and highly dynamic systems, achieved by studying the role of causality as a means of both grounding AI models into the real-world, and of allowing them to understand the inner workings of the world itself. This project tests these new capabilities for climate impact prediction in hydrology, tackling the three major challenges of variability, uncertainty and causality, while leveraging the Evolving AI concept to make the enormous datasets involved easily sharable in an innovative generative approach.

AI for science

March 1, 2024 – Present

The AI revolution is reshaping scientific research, impacting fields like drug discovery, physics, materials science, and astronomy. While many AI models address one-off regression problems, scientific and engineering problems often require modeling time-varying physical processes, typically represented by nonlinear differential equations. Coupled with high computational costs, this has given rise to the development of neural partial differential equation models, which offer cost-effective surrogate solutions to complex simulation problems by integrating machine learning techniques. Neural operator methods, in particular, have claimed advantages in terms of efficiency and extrapolation capabilities as compared to contemporary methods. However, the pervasiveness of uncertainty in these models poses a major challenge when employing neural operators in science and engineering. This uncertainty stems from insufficient data quality and quantity for training neural models, as well from the misrepresentation of physical processes and observational noise. The compound effect of these sources creates a situation where we are faced with uncertainty, without having the tools to exactly quantify the extent to which we are ignorant about our ignorance. In its turn, this ignorance undermines the trust in the results. Our overarching goal is therefore to develop a next generation of AI methods for science and engineering that allow for efficiently and effectively learning relations between function spaces, while being aware of their own uncertainty. To reach this ambitious, breakthrough goal, we will develop a class of new neural operators by making them aware of their own uncertainty by incorporating state-of-the-art neural operator schemes with advanced imprecise probabilistic tools for uncertainty quantification. Furthermore, we will deploy these developments to the extremely relevant but challenging issue of building digital twins of nuclear fusion power plants.

Universal AI

February 1, 2023 – Present

Current AI methods are monads, incapable of truly “evolving” as the learning process itself is rigidly assumed to be fixed, despite some attempts. Further, learning is arguably only one component of intelligence. Reasoning with facts should be an integral part of any holistic approach to intelligence, but much remains to be done in terms of defining what “understanding” truly means. Inspired by human intelligence, we thus propose a vision for the future articulated around the notion of “Universal” Artificial Intelligence, seamlessly incorporating all the above elements: versatility, robustness to uncertainty, compositionality, ability to evolve, semantic understanding. Our vision is articulated into the following concepts: Evolving intelligence: the design of a new class of machines able to adapt to evolving environments by modelling and solving time-varying, non-stationary, optimisation problems able to describe how learning processes can evolve over time to meet new needs. Compositional intelligence: a new compositional, “universal” form of AI, delivering versatile machines able to juggle multiple tasks and truly transfer knowledge across them, powered by radically new neurosymbolic approaches and a rethinking of the role of language. Holistic learning: a new approach to learning, piecing together all the elements of the learning and reasoning experience in a formal framework delineating causal relationships, composing grammars and Bayesian graphical models to supersede existing, siloed paradigms for learning from examples or interactions. Epistemic robustness, achieved via an “epistemic” approach to AI based on a Socratic ‘know that you don’t know’ stance (supported in our Epistemic AI and EPIMP projects), allowing us to radically tackle bias and model adaptation via the mathematics of uncertainty.

Neurosymbolic AI

January 1, 2023 – Present

This project is a collaboration with Oxford University (Lukasiewicz), Imperial College London (Giunchiglia), but also the University of Edinburgh (Vergari). The concept is to study ways of constraining neural networks to obey to common sense constraints. Our work so far has won awards at AI4AD @ IJCAI 2022 and IJCLR 2022. https://link.springer.com/article/10.1007/s10994-023-06322-z where we introduced the ROad event Awareness Dataset with logical Requirements (ROAD-R), the first publicly available dataset for autonomous driving with requirements expressed as logical constraints. Given ROAD-R, we show that current state-of-the-art models often violate its logical constraints, and that it is possible to exploit them to create models that (i) have a better performance, and (ii) are guaranteed to be compliant with the requirements themselves. Our ROAD-R dataset, the first of its kind, was released via a NeurIPS 2023 challenge: https://neurips.cc/virtual/2023/competition/66596

KTP with Supponor LTD

November 1, 2021 – Present

The KTP aims to substitute Supponor's current method with a software (computer vision) focused system that uses AI and machine learning, which doesn't currently exist. The considerable technical challenges will be addressed in developing a solution which can replicate the sub-pixel accuracy, image quality and processing speed of the current system, thus maintaining our current standards. The new system will be also be better able handle environmental complexities such as reflections and weather.

Epistemic Artificial Intelligence

March 1, 2021 – Present

The Visual AI Lab has won funding for a €3M Future Emerging Technologies project, funded by the EU Horizon 2020 programme, entitled “Epistemic AI”. Prof Fabio Cuzzolin will be the Coordinator of the project. The other two partners are KU Leuven (Belgium), led by Senior Researcher Dr Keivan Shariatmadar, and TU Delft (Netherlands), led by Associate Professor Neil Yorke-Smith. The project started in March 2021 and will have a duration of 60 months. Although artificial intelligence (AI) has improved remarkably over the last years, its inability to deal with fundamental uncertainty severely limits its application. This proposal re-imagines AI with a proper treatment of the uncertainty stemming from our forcibly partial knowledge of the world. As currently practised, AI cannot confidently make predictions robust enough to stand the test of data generated by processes different (even by tiny details, as shown by ‘adversarial’ results able to fool deep neural networks) from those studied at training time. While recognising this issue under different names (e.g. ‘overfitting’), traditional ML seems unable to address it in nonincremental ways. As a result, AI systems suffer from brittle behaviour, and find difficult to operate in new situations, e.g. adapting to driving in heavy rain or to other road users’ different styles of driving, e.g. deriving from cultural traits. Epistemic AI’s overall objective is to create a new paradigm for a next-generation artificial intelligence providing worst-case guarantees on its predictions thanks to a proper modelling of real-world uncertainties.

MAESTRO

January 1, 2021 – Present

While performing complex tasks in the challenging environment of an operating room (OR), staff must interact and communicate at different levels. Potential sources of human error are manifold and serious patient harm may occur because of pressure and cognitive overload. Today’s operating rooms do heavily rely on technology, but with little or no functional integration between devices. Our MAESTRO concept aims at laying the foundations for the operating room of the mid-21st Century, as a surgical environment powered by trustable, human-understanding artificial intelligence able to continually adapt and learn the best way to optimise safety, efficacy, teamwork, economy, and clinical outcomes. MAESTRO'S objectives are therefore (i) to deploy and test a platform integrating multiple sensing modalities as a sandpit for further research, (ii) to validate the multi-sensor fusion sensing of escalating cognitive load, (iii) to perform multimodal situation awareness for automated surgical check-listing, and (iv) to study the feasibility of multi-objective reinforcement learning for autonomous intervention, as the pillar of an AI-assisted OR of the future. We are currently working to release our MAESTRO dataset, the first holistic benchmark for smart operating rooms of the future.

Theory of mind at the interface of neuroscience and AI

July 1, 2019 – December 1, 2023

The Visual AI Laboratory has secured, in partnership with the University of Cambridge, a Leverhulme Trust Research Project Grant entitled "Theory of mind at the interface of neuroscience and AI". The duration of the project is 30 months, starting September 2019. The Cambridge co-investigator is Professor Barbara Sahakian, a leading figure in the field of neuroscience with more than 60,000 scholar citations. Emerging applications of artificial intelligence are highlighting the limitations of established approaches in situations involving humans. The integration of neuroscience and machine learning has the potential to enable significant advances in both fields. Theory of Mind capabilities, i.e., the ability to 'read' other sentient beings' mental states, are crucial for the development of a next generation, "human-centric" artificial intelligence aimed to understand the behaviour of complex agents. In a mutually beneficial process, computational models developed within artificial intelligence could provide new insights about how these mechanisms work in the human brain.

Some novel paradigms for analyzing human actions in complex videos

May 1, 2019 – Present

The Visual AI Laboratory has secured, in partnership with the Indian Institute of Technology (IIT) Bombay funding from UKIERI (the UK-India Education and Research Initiative) for a project on "Analysis of Human Action in Unconstrained Videos". IIT Bombay Director Subhasis Chaudhuri will lead the Indian side of the effort. Human action detection and recognition from videos are two of the most challenging tasks in computer vision. These problems become even more severe while dealing with fine-grained action categories. An exploration of the evolution of salient bodyparts’ (local motion) is needed in this respect to better discriminate such similar-looking human activities. Dominant action detection paradigms work by locating actions of interest on a frame by frame basis, and linking them up in time to form ‘action tubes’. Moreover, given the vast category of possible actions, it is very hard to annotate labelled training videos in a cost-effective manner. The notion of ‘zero-shot’ classification, which we explain below, can be adopted in such situations for the categorization of previously unexplored human activities. In this perspective, we propose in this project to explore the notion of mid-level feature mining from video data for the sake of: (1) Automatically discovering from videos action tubes capturing the evolution of motion salient parts efficiently. (2) Recognizing the detected human activities under two learning scenarios: i) zero-shot, ii) multi-shot.

Knowledge Transfer Partnership with Createc and Sportlight

May 1, 2019 – Present

A Knowledge Transfer Partnership (KTP) with Createc and Sportlight, two successful spinoffs of Oxford University, was funded at the latest round by Innovate UK. The project is split into two key phases each taking approximately 12 months, aiming to demonstrate a simple proof of concept at the mid-point with the second year focused on maturation, refinement and steps to commercialisation. The first phase will consist of the Associate reviewing the state of the art and conducting a literature review, understanding the hardware and system architecture and capturing further datasets for algorithmic training, in addition to the following technical work packages: Sensor fusion: The company's system provides not only video imagery from multiple viewpoints but also data providing depth, dynamic data and point cloud overlays over the imagery. This enables a novel approach to action identification where this extra information can be integrated with the video to enhance performance Person segmentation: The first task ahead of person or action identification is to segment the person from the background which due to the tracking system is highly dynamic. This is a key enabling task but there are multiple existing techniques for performing this task Person identification: It is important for all applications to associate an action with an individual. In the crowd monitoring case, single actions may be inconsequential but an individual carrying out multiple actions may be of more interest Single person action identification: This task will develop algorithms for identifying single person actions from the video data.

Artificial intelligence for autonomous driving

April 1, 2019 – Present

The Visual AI Laboratory, in partnership with Dr Matthias Rolf of the Cognitive Robotics group and the Autonomous Driving group led by Dr Andrew Bradley, has secured funding for £100,000 from the the School of Engineering, Computing and Mathematics to support a Research Fellow in Artificial Intelligence for Autonomous Driving, for a period of two years. The project concerns the design and development of novel ways for robots and autonomous machines to interact with humans in a variety of emerging scenarios, including: human-robot interaction, autonomous driving, personal (virtual or robotic) assistants. In particular, we believe novel, disruptive applications of AI require much more sophisticated forms of communication between humans and machines, something that goes far beyond conventional explicit and linguistic exchange of information towards implicit non-verbal communication and understanding of each other's behaviour. For example, smart cars need to understand that children and construction workers have different reasoning processes that lead to very different observable behaviour, in order to blend in with the road as a human-centered environment. Empathetic machines have the potential to revolutionise healthcare, by providing better care catering for the psychological needs of patients. Morally and socially appropriate behaviour is key in all such scenarios, to build trust and lead to acceptance from the public.

Continual learning

January 1, 2019 – Present

We proposed a Continual (semi-supervised) learning paradigm for continual improvement of deployed models, based on unsupervised data stream. We organised the CSSL @ IJCAI 2021 workshop, released the related CAR and CCC continual learning datasets, edited volume with Springer, Frontiers topic editor, PAMI paper in preparation https://sites.google.com/view/sscl-workshop-ijcai-2021/ We collaborated with ContinualAI on new Avalanche library for continual learning and upcoming conference series https://avalanche.continualai.org/

Road event detection for autonomous driving (ROAD)

January 1, 2018 – Present

The accurate detection and anticipation of actions performed by multiple road agents (pedestrians, vehicles, cyclists, and so on) is a crucial problem to solve for enabling autonomous vehicles with the capability to support reliable and safe autonomous decision-making. While the task of teaching an autonomous vehicle how to drive can be tackled in a brute-force, direct reinforcement learning approach, a sensible and attractive alternative is first to provide the vehicle with situation awareness capabilities, to then feed the resulting intermediate representations of road scenarios (in terms of agents, events and scene configuration) to a suitable decision-making strategy. This, in particular, has the advantage of allowing the modeling of the reasoning process of road agents in a theory-of-mind approach, inspired by the behavior of the human mind in similar contexts. We propose a change of paradigm towards action recognition/detection in that the focus is not on the objects/actors themselves and their appearance, but on what they do and the meaning of their behaviour. Following our previous successful experience in hosting ROAD workshop in ICCV2021 and ROAD++ workshop in ICCV2023, our goal is to put to the forefront of the research in autonomous driving the topic of situation awareness, intended as the ability to create semantically useful representations of dynamic road scenes in terms of the notion of ‘road event’ with the present of domain adaptation.

SARAS - Smart Autonomous Robotic Assistant Surgeon

January 1, 2018 – December 1, 2022

Horizon 2020 project Coordinator: Dr Riccardo Muradore, University of Verona, Italy Budget: €4,315,640 (Oxford Brookes' share: €596,073) Own role: Scientific Officer (SO) for the whole project, as well as WP Leader In surgical operations many people crowd the area around the operating table. The introduction of robotics in surgery has not decreased this number. During a laparoscopic intervention with the da Vinci robot, for example, the presence of an assistant surgeon, two nurses and an anaesthetist, is required, together with that of the main surgeon teleoperating the robot. The assistant surgeon needs always be present to take care of simple surgical tasks the main surgeon cannot perform with the robotic tools s/he is teleoperating (e.g. suction and aspiration during dissection, moving or holding organs in place to make room for cutting or suturing, using the standard laparoscopic tools). Another expert surgeon is thus required to play the role of the assistant, to properly support the main surgeon using traditional laparoscopic tools as shown in Figure 1. The goal of SARAS is to develop a next-generation surgical robotic platform that allows a single surgeon (i.e., without the need for an expert assistant surgeon) to execute robotic minimally invasive surgery (R-MIS), thereby increasing the social and economic efficiency of a hospital while guaranteeing the same level of safety for patients. This platform is called solo-surgeon system.

Meta Vision KTP

September 1, 2015 – August 1, 2017

Innovate UK - Knowledge Transfer Partnership Budget: £160,000 Own role: Academic supervisor Partner: Meta Vision LTD In the welding industry, we see an increasing need for automated inspection, both in partnership with automated seam tracking and as a completely separate function. The aim of this project is to develop algorithms for computer vision capable of analysing 3D scans of robotic welds, by extracting underlying geometry, identifying a range of standard defects, and classifying the welds as acceptable or not according to geometrical definitions. Three key stages of the project can be identified: Performing automatic analysis of 3D data requires an in depth understanding and application of the underlying mathematics involved. It will be necessary to use this knowledge to define the basis for the operation of the algorithms. The second step will be to use the mathematical development in the form of a set of algorithms for matching the 3D datasets of actual parts to be inspected to either theoretical models of good and bad welds or stored, processed 3D models of good and bad welds, and thereby making a determination of the overall quality of the weld in question and identifying any particular defects. To support the first two items above, it may be necessary to have a database which extracts key geometric information about good and bad shapes and makes that available to the inspection algorithms themselves. The database will also store basic 3D representations of complete parts.

The Total Probability Theorem for Belief Functions

September 1, 2013 – Present

The PhD student's work will fit in the uncertainty theory branch of my research, in which I am a world leader. The problem is the following. In scenarios such as climate change or disaster risk analysis making predictions is difficult, since the available data are either scarce, incomplete, or missing. In such cases "cautious" approaches to uncertainty modeling have an edge over classical probability theory, as they can come up with robust (albeit imprecise) predictions under partial data. Those based on "belief functions" are arguably among the most powerful. However, the necessary mathematical tools for prediction and estimation in this framework have only partially been developed yet, due to their inherent complexity: in particular, the generalization of the classical total probability theorem to belief functions is crucially needed. I propose to fully develop the theory of total belief, in order to make belief calculus a viable tool for practitioners, with potentially enormous repercussions in all fields of applied science.

Belief Modeling Regression for example-based pose estimation

February 1, 2013 – Present

In example-based pose estimation, the configuration or “pose” of an evolving object is sought given visual evidence, having to rely uniquely on a set of examples. We assume here that, in a training stage, a number of feature measurements is extracted from the available images, while an “oracle” provides us with the true object pose at each instant. In this scenario, a sensible approach consists in learning maps from features to poses, using the information provided by the training set. In particular, multi-valued mappings linking feature values to set of training poses can be easily constructed. A probability measure on any feature space is then naturally mapped to a convex set of probabilities on the set of training poses, in a form of a “belief function”. Given a test image, its feature measurements translate into a collection of belief functions on the set of training poses, which when combined yield there an entire family of probability distributions. From the latter, both a single, central pose estimate and a set of extremal estimates can be computed, together with a measure of how reliable the estimate is. We call this technique “Belief Modeling Regression”.

Dynamical Generative and Discriminative Models for Action and Activity Localization and Recognition

April 1, 2012 – January 1, 2014

Action and activity recognition are intuitive but extremely difficult tasks, which lie at the root of a panoply of scenarios of human machine interaction, ranging from gaming, mobile computing and video retrieval to health monitoring, surveillance, robotics and biometrics.

Recognising and localising human actions

October 1, 2011 – Present

Current state-of-the-art action classification methods derive action representations from the entire video clip in which the action unfolds, even though this representation may include parts of actions and scene context which are shared amongst multiple classes. For example, different actions involving the movement of the hands may be performed whilst walking, against a common background. In this work, we propose an action classification framework in which discriminative action subvolumes are learned in a weakly supervised setting, owing to the difficulty of manually labelling massive video datasets. The learned sub-action models are used to simultaneously classify video clips and to localise actions in space-time. Each subvolume is cast as a BoF instance in an MIL framework, which in turn is used to learn its class membership. We demonstrate quantitatively that the classification performance of our proposed algorithm is comparable and in some cases superior to the current state-of-the-art on the most challenging video datasets, whilst additionally estimating space-time localisation information.

Tensorial modeling of dynamical systems for gait and activity recognition

August 1, 2011 – January 1, 2014

EPSRC - First Grant Budget: £122,000 Own role: Principal Investigator (PI) Biometrics such as face, iris, or fingerprint recognition for surveillance and security have received growing attention in the last decade. They suffer, however, from two major limitations: they cannot be used at a distance, and require user cooperation. For these reasons, originally driven by an initiative of US’s DARPA, identity recognition from gait has been proposed as a novel behavioral biometrics, based on people’s distinctive gait pattern. Despite its attractive features, though, gait identification is still far from being ready to be deployed in practice, as in real-world scenarios recognition is made extremely difficult by the presence of nuisance factors such as viewpoint, illumination, clothing, etcetera. Similar issues are shared by other applications such as action and activity recognition. This proposal concerns the problem of classifying video sequences by attributing to each sequence a label, such as the type of event recorded or the identity of the person performing a certain action. It proposes a novel framework for motion recognition capable of dealing in a principled way with the issue of nuisance factors in both gait and activity recognition. The goal is pushing towards a more widespread diffusion of gait identification, as a concrete contribution to enhancing the security levels in the country in the current, uncertain scenarios. However, as the techniques devised in this proposal are extendable to action and identity recognition, their commercial exploitation potential in, for instance, video indexing or interactive video games is also enormous.