Abstract

Building on previous work on incorporating large language models (LLM) in gaming, we investigate the possibility of implementing LLM as evaluating agents of open-ended challenges in serious games and its potential to facilitate a meaningful experience for the player. We contribute with a sustainability game prototype in a single natural language prompt about energy communities and we tested it with 13 participants inside ChatGPT-3.5. Two participants were already aware of energy communities before the game, and eight of the remaining 11 gained valuable knowledge about the specific topic. Comparing ChatGPT-3.5 evaluations of players’ interaction with an expert’s assessment, ChatGPT-3.5 correctly evaluated 81% of player’s answers. Our results are encouraging and show the potential of using LLMs as mediating agents in educational games, while also allowing easy prototyping of games through natural language prompts.

Video Preview

Abstract:
This paper presents the results of an empirical exploration of 10 cursor movement metrics designed to measure respondent hesitation in online surveys. As a use case, this work considers an online survey aimed at exploring how people gauge the electricity consumption of domestic appliances. The cursor metrics were derived computationally from the mouse trajectories when rating the consumption of each appliance and analyzed using Multidimensional Scaling, Jenks Natural Breaks, and the Jaccard Similarity Index techniques. The results show that despite the fact that the metrics measure different aspects of the mouse trajectories, there is an agreement with respect to the appliances that generated higher levels of hesitation. The paper concludes with an outline of future work that should be carried out in order to further understand how cursor trajectories can be used to measure respondent hesitation.

Poster:

Reproducible Research:

Data (scripts will be added soon)

Abstract

In order to promote a more effective understanding of electricity consumption in households, and leverage the potential of current and future energy conservation initiatives, we have developed an online survey in which visual icons of various electrical devices coupled with a Likert scale are used to investigate people’s perception of the amount of electricity consumed by a variety of individual domestic appliances. In this paper we present initial results from a pilot study conducted with 293 participants from over 50 countries and outline our plans for future work. Our initial results suggest that despite a considerable number of respondents was able to give the correct answers, many others seem to lack an understanding of appliance electricity consumption, and thus further work should be carried in this direction.

Reproducible Research:

Data