Our project, REGARD, funded through the ANR-NSERC Joint Grant on Artificial Intelligence, aims to develop theory and methods for the evaluation of generative AI in real-world environments. In particular, we are designing a human-in-the-loop platform to facilitate the evaluation of LLMs based on criteria deemed important by domain experts, including those related to the security and safety of AI systems geared towards specific deployment contexts.

The core innovation of the REGARD project is to create a framework based on adaptive human guidance and simulation through probabilistic restricted-world models. Initially, human expertise will be leveraged to provide high-level descriptions of both desired and undesired system behaviors—covering task performance, ethical considerations, and potential harms. These descriptions will serve as the foundation for a probabilistic inference model that constructs a restricted-world model. This model will be used to generate synthetic data for benchmarking systems under rare but critical edge cases, and simulate interactions with users exhibiting potentially risky profiles.