Publications
See all our releases and publications
2021 |
Arévalo-Arboleda, Stephanie; Ruecker, Franziska; Dierks, Tim; Gerken, Jens Assisting Manipulation and Grasping in Robot Teleoperation with Augmented Reality Visual Cues ConferenceForthcomingMIA CHI Conference on Human Factors in Computing Systems (CHI '21), Forthcoming, ISBN: 978-1-4503-8096-6/21/05. Abstract | BibTeX | Links: @conference{Arevalo-Arboleda2021, title = {Assisting Manipulation and Grasping in Robot Teleoperation with Augmented Reality Visual Cues}, author = {Stephanie Arévalo-Arboleda and Franziska Ruecker and Tim Dierks and Jens Gerken}, doi = {10.1145/3411764.3445398}, isbn = {978-1-4503-8096-6/21/05}, year = {2021}, date = {2021-05-03}, booktitle = {CHI Conference on Human Factors in Computing Systems (CHI '21)}, abstract = {Teleoperating industrial manipulators in co-located spaces can be challenging. Facilitating robot teleoperation by providing additional visual information about the environment and the robot affordances using augmented reality (AR), can improve task performance in manipulation and grasping. In this paper, we present two designs of augmented visual cues, that aim to enhance the visual space of the robot operator through hints about the position of the robot gripper in the workspace and in relation to the target. These visual cues aim to improve the distance perception and thus, the task performance. We evaluate both designs against a baseline in an experiment where participants teleoperate a robotic arm to perform pick-and-place tasks. Our results show performance improvements in different levels, reflecting in objective and subjective measures with trade-offs in terms of time, accuracy, and participants’ views of teleoperation. These findings show the potential of AR not only in teleoperation, but in understanding the human-robot workspace.}, keywords = {augmented reality, depth perception, hands-free interaction, human-robot interaction, teleoperation, visual cues}, pubstate = {forthcoming}, tppubtype = {conference} } Teleoperating industrial manipulators in co-located spaces can be challenging. Facilitating robot teleoperation by providing additional visual information about the environment and the robot affordances using augmented reality (AR), can improve task performance in manipulation and grasping. In this paper, we present two designs of augmented visual cues, that aim to enhance the visual space of the robot operator through hints about the position of the robot gripper in the workspace and in relation to the target. These visual cues aim to improve the distance perception and thus, the task performance. We evaluate both designs against a baseline in an experiment where participants teleoperate a robotic arm to perform pick-and-place tasks. Our results show performance improvements in different levels, reflecting in objective and subjective measures with trade-offs in terms of time, accuracy, and participants’ views of teleoperation. These findings show the potential of AR not only in teleoperation, but in understanding the human-robot workspace. |
2020 |
Pascher, Max Praxisbeispiel Digitalisierung konkret: Wenn der Stromzähler weiß, ob es Oma gut geht. Beschreibung des minimalinvasiven Frühwarnsystems „ZELIA“ Book Chapter Vilain, Michael (Ed.): Wege in die digitale Zukunft - Was bedeuten Smart Living, Big Data, Robotik & Co für die Sozialwirtschaft?, S. 137-148, Nomos Verlagsgesellschaft mbH & Co. KG, 2020, ISBN: 78-3-8487-6621-5. BibTeX | Links: @inbook{Pascher2020, title = {Praxisbeispiel Digitalisierung konkret: Wenn der Stromzähler weiß, ob es Oma gut geht. Beschreibung des minimalinvasiven Frühwarnsystems „ZELIA“}, author = {Max Pascher}, editor = {Michael Vilain}, doi = {10.5771/9783748907008-137}, isbn = {78-3-8487-6621-5}, year = {2020}, date = {2020-10-06}, booktitle = {Wege in die digitale Zukunft - Was bedeuten Smart Living, Big Data, Robotik & Co für die Sozialwirtschaft?}, pages = {137-148}, publisher = {Nomos Verlagsgesellschaft mbH & Co. KG}, keywords = {AAL, consumption, data science, elderly, monitoring, wellbeing}, pubstate = {published}, tppubtype = {inbook} } |
Arévalo-Arboleda, Stephanie; Pascher, Max; Lakhnati, Younes; Gerken, Jens Understanding Human-Robot Collaboration for People with Mobility Impairments at the Workplace, a Thematic Analysis InproceedingsMIA RO-MAN 2020 - IEEE International Conference on Robot and Human Interactive Communication, 2020, ISBN: 978-1-7281-6075-7. Abstract | BibTeX | Links: @inproceedings{Arévalo-Arboleda2020b, title = {Understanding Human-Robot Collaboration for People with Mobility Impairments at the Workplace, a Thematic Analysis}, author = {Stephanie Arévalo-Arboleda and Max Pascher and Younes Lakhnati and Jens Gerken}, url = {https://hci.w-hs.de/pub_understanding_hrc_ta/, PDF Download}, doi = {10.1109/RO-MAN47096.2020.9223489}, isbn = {978-1-7281-6075-7}, year = {2020}, date = {2020-07-31}, booktitle = {RO-MAN 2020 - IEEE International Conference on Robot and Human Interactive Communication}, abstract = {Assistive technologies, in particular human-robot collaboration, have the potential to ease the life of people with physical mobility impairments in social and economic activities. Currently, this group of people has lower rates of economic participation, due to the lack of adequate environments adapted to their capabilities. We take a closer look at the needs and preferences of people with physical mobility impairments in a human-robot cooperative environment at the workplace. Specifically, we aim to design how to control a robotic arm in manufacturing tasks for people with physical mobility impairments. We present a case study of a shelteredworkshop as a prototype for an institution that employs people with disabilities in manufacturing jobs. Here, we collected data of potential end-users with physical mobility impairments, social workers, and supervisors using a Participatory Design technique (Future-Workshop). These stakeholders were divided into two groups, end-users and supervising personnel (social workers, supervisors), which were run across two separate sessions. The gathered information was analyzed using thematic analysis to reveal underlying themes across stakeholders. We identified concepts that highlight underlying concerns related to the robot fitting into the social and organizational structure, human-robot synergy, and human-robot problem management. In this paper, we present our findings and discuss the implications of each theme when shaping an inclusive humanrobot cooperative workstation for people with physical mobility impairments.}, keywords = {Assistive Robotics, Creating Human-Robot Relationships, HRI and Collaboration in Manufacturing Environments}, pubstate = {published}, tppubtype = {inproceedings} } Assistive technologies, in particular human-robot collaboration, have the potential to ease the life of people with physical mobility impairments in social and economic activities. Currently, this group of people has lower rates of economic participation, due to the lack of adequate environments adapted to their capabilities. We take a closer look at the needs and preferences of people with physical mobility impairments in a human-robot cooperative environment at the workplace. Specifically, we aim to design how to control a robotic arm in manufacturing tasks for people with physical mobility impairments. We present a case study of a shelteredworkshop as a prototype for an institution that employs people with disabilities in manufacturing jobs. Here, we collected data of potential end-users with physical mobility impairments, social workers, and supervisors using a Participatory Design technique (Future-Workshop). These stakeholders were divided into two groups, end-users and supervising personnel (social workers, supervisors), which were run across two separate sessions. The gathered information was analyzed using thematic analysis to reveal underlying themes across stakeholders. We identified concepts that highlight underlying concerns related to the robot fitting into the social and organizational structure, human-robot synergy, and human-robot problem management. In this paper, we present our findings and discuss the implications of each theme when shaping an inclusive humanrobot cooperative workstation for people with physical mobility impairments. |
Dierks, Tim Visual Cues: Integration of object pose recognition with an augmented reality system as means to support visual perception in human-robot control Masters ThesisMIA Westfälische Hochschule, 2020. Abstract | BibTeX | Links: @mastersthesis{Dierks2020, title = {Visual Cues: Integration of object pose recognition with an augmented reality system as means to support visual perception in human-robot control}, author = {Tim Dierks}, url = {https://hci.w-hs.de/pub_dierks_tim_masterthesis/, PDF Download}, year = {2020}, date = {2020-05-28}, address = {Neidenburger Straße 43, 45897 Gelsenkirchen}, school = {Westfälische Hochschule}, abstract = {Autonomy and self-determination are fundamental aspects of living in our society. Supporting people for whom this freedom is limited due to physical impairments is the fundamental goal of this thesis. Especially for people who are paralyzed, even working at a desk job is often not feasible. Therefore, in this thesis a prototype of a robot assembly workstation was constructed that utilizes a modern Augmented Reality (AR)-Head-Mounted Display (HMD) to control a robotic arm. Through the use of object pose recognition, the objects in the working environment are detected and this information is used to display different visual cues at the robotic arm or in its vicinity. Providing the users with additional depth information and helping them determine object relations, which are often not easily discernible from a fixed perspective. To achieve this a hands-free AR-based robot-control scheme was developed, which uses speech and head-movement for interaction. Additionally, multiple advanced visual cues were designed that utilize object pose detection for spatial-visual support. The pose recognition system is adapted from state-of-the-art research in computer vision to allow the detection of arbitrary objects with no regard for texture or shape. Two evaluations were performed, a small user study that excluded the object recognition, which con firms the general usability of the system and gives an impression on its performance. The participants were able to perform difficult pick and place tasks with a high success rate. Secondly, a technical evaluation of the object recognition system was conducted, which revealed an adequate prediction precision, but is too unreliable for real-world scenarios as the prediction quality is highly variable and depends on object orientations and occlusion.}, keywords = {augmented reality, hands-free interaction, human-robot interaction, pose recognition}, pubstate = {published}, tppubtype = {mastersthesis} } Autonomy and self-determination are fundamental aspects of living in our society. Supporting people for whom this freedom is limited due to physical impairments is the fundamental goal of this thesis. Especially for people who are paralyzed, even working at a desk job is often not feasible. Therefore, in this thesis a prototype of a robot assembly workstation was constructed that utilizes a modern Augmented Reality (AR)-Head-Mounted Display (HMD) to control a robotic arm. Through the use of object pose recognition, the objects in the working environment are detected and this information is used to display different visual cues at the robotic arm or in its vicinity. Providing the users with additional depth information and helping them determine object relations, which are often not easily discernible from a fixed perspective. To achieve this a hands-free AR-based robot-control scheme was developed, which uses speech and head-movement for interaction. Additionally, multiple advanced visual cues were designed that utilize object pose detection for spatial-visual support. The pose recognition system is adapted from state-of-the-art research in computer vision to allow the detection of arbitrary objects with no regard for texture or shape. Two evaluations were performed, a small user study that excluded the object recognition, which con firms the general usability of the system and gives an impression on its performance. The participants were able to perform difficult pick and place tasks with a high success rate. Secondly, a technical evaluation of the object recognition system was conducted, which revealed an adequate prediction precision, but is too unreliable for real-world scenarios as the prediction quality is highly variable and depends on object orientations and occlusion. |
Ruecker, Franziska Visuelle Helfer: Ein Augmented Reality Prototyp zur Unterstützung der visuellen Wahrnehmung für die Steuerung eines Roboterarms Masters ThesisMIA Westfälische Hochschule, 2020. Abstract | BibTeX | Links: @mastersthesis{Ruecker2020, title = {Visuelle Helfer: Ein Augmented Reality Prototyp zur Unterstützung der visuellen Wahrnehmung für die Steuerung eines Roboterarms}, author = {Franziska Ruecker}, url = {https://hci.w-hs.de/pub_rueckermasterarbeit_komprimiert/, PDF Download}, year = {2020}, date = {2020-05-14}, address = {Neidenburger Straße 43, 45897 Gelsenkirchen}, school = {Westfälische Hochschule}, abstract = {Körperliche Behinderungen können einen Menschen soweit einschränken, dass für sie ein autonomes und selbstbestimmtes Leben, trotz intakter mentaler und kognitiven Fähigkeiten, nicht mehr möglich ist. Daher ist für Menschen, die beispielsweise vom Hals abwärts gelähmt sind, sogenannten Tetraplegikern, jede Zurückgewinnung von Autonomie eine Steigerung der Lebensqualität. In dieser Masterarbeit wird ein Augmented Reality Prototyp entwickelt, der es Tetraplegikern oder Menschen mit einer ähnlichen körperlichen Einschränkung erlaubt, an einem Mensch-Roboter Arbeitsplatz Montageaufgaben durchzuführen und ihnen somit eine Integration ins Arbeitsleben ermöglichen kann. Der Prototyp erlaubt es den Benutzer ohne die Nutzung der Hände, einen Kuka iiwa Roboterarm mit der Microsoft HoloLens zu steuern. Dabei wird ein Fokus darauf gelegt, das Blickfeld des Benutzers mit speziellen virtuellen Visualisierungen, sogenannten visuellen Helfern, anzureichern, um die Nachteile, die durch die Bewegungseinschränkungen der Zielgruppe ausgelöst werden, auszugleichen. Diese visuellen Helfer sollen bei der Steuerung des Roboterarms unterstützen und die Bedienung des Prototyps verbessern. Eine Evaluation des Prototyps zeigte Tendenzen, dass das Konzept der visuellen Helfer den Benutzer den Roboterarm präziser steuern lässt und seine Bedienung unterstützt.}, keywords = {augmented reality, evaluation, hands-free interaction, human-robot interaction}, pubstate = {published}, tppubtype = {mastersthesis} } Körperliche Behinderungen können einen Menschen soweit einschränken, dass für sie ein autonomes und selbstbestimmtes Leben, trotz intakter mentaler und kognitiven Fähigkeiten, nicht mehr möglich ist. Daher ist für Menschen, die beispielsweise vom Hals abwärts gelähmt sind, sogenannten Tetraplegikern, jede Zurückgewinnung von Autonomie eine Steigerung der Lebensqualität. In dieser Masterarbeit wird ein Augmented Reality Prototyp entwickelt, der es Tetraplegikern oder Menschen mit einer ähnlichen körperlichen Einschränkung erlaubt, an einem Mensch-Roboter Arbeitsplatz Montageaufgaben durchzuführen und ihnen somit eine Integration ins Arbeitsleben ermöglichen kann. Der Prototyp erlaubt es den Benutzer ohne die Nutzung der Hände, einen Kuka iiwa Roboterarm mit der Microsoft HoloLens zu steuern. Dabei wird ein Fokus darauf gelegt, das Blickfeld des Benutzers mit speziellen virtuellen Visualisierungen, sogenannten visuellen Helfern, anzureichern, um die Nachteile, die durch die Bewegungseinschränkungen der Zielgruppe ausgelöst werden, auszugleichen. Diese visuellen Helfer sollen bei der Steuerung des Roboterarms unterstützen und die Bedienung des Prototyps verbessern. Eine Evaluation des Prototyps zeigte Tendenzen, dass das Konzept der visuellen Helfer den Benutzer den Roboterarm präziser steuern lässt und seine Bedienung unterstützt. |
Arévalo-Arboleda, Stephanie; Dierks, Tim; Ruecker, Franziska; Gerken, Jens There’s More than Meets the Eye: Enhancing Robot Control through Augmented Visual Cues InproceedingsMIA HRI 2020 - ACM/IEEE International Conference on Human-Robot Interaction, 2020, ISBN: 978-1-4503-7057. Abstract | BibTeX | Links: @inproceedings{Arévalo-Arboleda2020, title = {There’s More than Meets the Eye: Enhancing Robot Control through Augmented Visual Cues}, author = {Stephanie Arévalo-Arboleda and Tim Dierks and Franziska Ruecker and Jens Gerken}, url = {https://hci.w-hs.de/pub_lbr1017_visualcues_arevalo_cameraready/, PDF Download}, doi = {10.1145/3371382.3378240}, isbn = {978-1-4503-7057}, year = {2020}, date = {2020-03-23}, booktitle = {HRI 2020 - ACM/IEEE International Conference on Human-Robot Interaction}, abstract = {In this paper, we present the design of a visual feedback mechanism using Augmented Reality, which we call augmented visual cues, to assist pick-and-place tasks during robot control. We propose to augment the robot operator’s visual space in order to avoid attention splitting and increase situational awareness (SA). In particular, we aim to improve on the SA concepts of perception, comprehension, and projection as well as the overall task performance. For that, we built upon the interaction design paradigm proposed by Walker et al.. On the one hand, our design augments the robot to support picking-tasks; and, on the other hand, we augment the environment to support placing-tasks. We evaluated our design in a first user study, and results point to specific design aspects that need improvement while showing promise for the overall approach, in particular regarding user satisfaction and certain SA concepts.}, keywords = {augmented reality, human-robot interaction, visualization}, pubstate = {published}, tppubtype = {inproceedings} } In this paper, we present the design of a visual feedback mechanism using Augmented Reality, which we call augmented visual cues, to assist pick-and-place tasks during robot control. We propose to augment the robot operator’s visual space in order to avoid attention splitting and increase situational awareness (SA). In particular, we aim to improve on the SA concepts of perception, comprehension, and projection as well as the overall task performance. For that, we built upon the interaction design paradigm proposed by Walker et al.. On the one hand, our design augments the robot to support picking-tasks; and, on the other hand, we augment the environment to support placing-tasks. We evaluated our design in a first user study, and results point to specific design aspects that need improvement while showing promise for the overall approach, in particular regarding user satisfaction and certain SA concepts. |
Sandkühler, Christian Konzeption, Implementierung und Evaluation eines Metadaten-Repositorys in Form einer Web-Applikation Bachelor Thesis Westphalian University of Applied Sciences, 2020. BibTeX | Tags: bachelor thesis @mastersthesis{Sandkuehler2020, title = {Konzeption, Implementierung und Evaluation eines Metadaten-Repositorys in Form einer Web-Applikation}, author = {Christian Sandkühler}, year = {2020}, date = {2020-02-22}, school = {Westphalian University of Applied Sciences}, keywords = {bachelor thesis}, pubstate = {published}, tppubtype = {mastersthesis} } |
2019 |
Lakhnati, Younes; Springer, Raphael; White, Edward; Gerken, Jens Is It Real?: Understanding Interaction Mechanics Within the Reality-virtuality Continuum Inproceedings Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia, ACM, New York, NY, USA, 2019, ISBN: 978-1-4503-7624-2. Abstract | BibTeX | Links: @inproceedings{Lakhnati2019, title = {Is It Real?: Understanding Interaction Mechanics Within the Reality-virtuality Continuum}, author = {Younes Lakhnati and Raphael Springer and Edward White and Jens Gerken}, url = {https://hci.w-hs.de/pub_isitrealunderstandinteractionmechanics/, PDF Download}, doi = {10.1145/3365610.3365634}, isbn = {978-1-4503-7624-2}, year = {2019}, date = {2019-11-26}, booktitle = {Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia}, publisher = {ACM}, address = {New York, NY, USA}, abstract = { The concept of Mixed Reality has existed in research for decades but has experienced rapid growth in recent years, mainly due to technological advances and peripherals such as the Microsoft HoloLens reaching the market. Despite this, certain design aspects of Mixed Reality experiences, such as the different nuances of real and virtual elements, remain largely unexplored. This paper presents an explorative study with 15 participants which aims to investigate and gain a better understanding of the different qualities of real and virtual objects. To that end, we developed a Mixed Reality board game that offered different combinations of real and virtual game components, such as the board, the pieces and the dice. Our analysis shows that the participants generally preferred the completely virtual variant but appreciated different qualities of real and virtual elements. The results also indicate that virtual interaction elements work better on a real background than vice versa. However, this conflicts with some participants' preference of using physical pieces for the haptic experience, creating a design trade-off. This study represents a first step in exploring how the experience changes when swapping elements of differing realities for one another and identifying these trade-offs. }, keywords = {augmented reality, augmented virtuality, board game, mixed reality, reality-virtuality trade-off, user study}, pubstate = {published}, tppubtype = {inproceedings} } The concept of Mixed Reality has existed in research for decades but has experienced rapid growth in recent years, mainly due to technological advances and peripherals such as the Microsoft HoloLens reaching the market. Despite this, certain design aspects of Mixed Reality experiences, such as the different nuances of real and virtual elements, remain largely unexplored. This paper presents an explorative study with 15 participants which aims to investigate and gain a better understanding of the different qualities of real and virtual objects. To that end, we developed a Mixed Reality board game that offered different combinations of real and virtual game components, such as the board, the pieces and the dice. Our analysis shows that the participants generally preferred the completely virtual variant but appreciated different qualities of real and virtual elements. The results also indicate that virtual interaction elements work better on a real background than vice versa. However, this conflicts with some participants' preference of using physical pieces for the haptic experience, creating a design trade-off. This study represents a first step in exploring how the experience changes when swapping elements of differing realities for one another and identifying these trade-offs. |
Lakhnati, Younes; Springer, Raphael; Gerken, Jens Mensch ARgere Dich Nicht: A Board Game Testbed for Mixed Reality Interactions Incollection Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia, ACM, New York, NY, USA, 2019, ISBN: 978-1-4503-7624-2. Abstract | BibTeX | Links: @incollection{Lakhnati2019b, title = {Mensch ARgere Dich Nicht: A Board Game Testbed for Mixed Reality Interactions}, author = {Younes Lakhnati and Raphael Springer and Jens Gerken}, url = {https://hci.w-hs.de/pub_aboardgametestbed/, PDF Download}, doi = {10.1145/3365610.3368471}, isbn = {978-1-4503-7624-2}, year = {2019}, date = {2019-11-26}, booktitle = {Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia}, publisher = {ACM}, address = {New York, NY, USA}, abstract = {"Mensch ARgere Dich Nicht" is a Mixed Reality (MR) game based on a popular German board game "Mensch Ärgere Dich Nicht", from the early 1900s, similar to Pachisi. Developed to be a test bed for investigating mixed interactions, the application offers real and virtual variations of all core game elements (board, dice, pieces) for a fully modifiable experience. Using the Microsoft HoloLens, players can interact with the virtual game elements and receive additional information about the current game state. The game also recognizes interactions with real game elements and translates them into the virtual world.}, keywords = {augmented reality, augmented virtuality, board game, mixed reality, reality-virtuality trade-off}, pubstate = {published}, tppubtype = {incollection} } "Mensch ARgere Dich Nicht" is a Mixed Reality (MR) game based on a popular German board game "Mensch Ärgere Dich Nicht", from the early 1900s, similar to Pachisi. Developed to be a test bed for investigating mixed interactions, the application offers real and virtual variations of all core game elements (board, dice, pieces) for a fully modifiable experience. Using the Microsoft HoloLens, players can interact with the virtual game elements and receive additional information about the current game state. The game also recognizes interactions with real game elements and translates them into the virtual world. |
Köckerling, Jonas Empathische Musik: Verwenden des emotionalen Zustands des Benutzers zur Optimierung von KI-generierten Playlisten Bachelor Thesis Westphalian University of Applied Sciences, 2019. BibTeX | Tags: bachelor thesis @mastersthesis{Koeckerling2019, title = {Empathische Musik: Verwenden des emotionalen Zustands des Benutzers zur Optimierung von KI-generierten Playlisten}, author = {Jonas Köckerling}, year = {2019}, date = {2019-11-13}, school = {Westphalian University of Applied Sciences}, keywords = {bachelor thesis}, pubstate = {published}, tppubtype = {mastersthesis} } |
Arévalo-Arboleda, Stephanie; Miller, Stanislaw; Janka, Martha; Gerken, Jens What's behind a choice? Understanding Modality Choices under Changing Environmental Conditions InproceedingsMIA ICMI '19 2019 International Conference on Multimodal Interaction, S. 291-301, 2019, ISBN: 978-1-4503-6860-5. Abstract | BibTeX | Links: @inproceedings{Arévalo-Arboleda2019, title = {What's behind a choice? Understanding Modality Choices under Changing Environmental Conditions}, author = {Stephanie Arévalo-Arboleda and Stanislaw Miller and Martha Janka and Jens Gerken}, url = {https://hci.w-hs.de/pub_whatsbehindachoiceunderstandingmodalitychoicesunderchangingenvironmentalconditions/, PDF Download}, doi = {10.1145/3340555.3353717}, isbn = { 978-1-4503-6860-5}, year = {2019}, date = {2019-10-14}, booktitle = {ICMI '19 2019 International Conference on Multimodal Interaction}, pages = {291-301}, abstract = {Interacting with the physical and digital environment multimodally enhances user flexibility and adaptability to different scenarios. A body of research has focused on comparing the efficiency and effectiveness of different interaction modalities in digital environments. However, little is known about user behavior in an environment that provides freedom to choose from a range of modalities. That is why, we take a closer look at the factors that influence input modality choices. Building on the work by Jameson & Kristensson, our goal is to understand how different factors influence user choices. In this paper, we present a study that aims to explore modality choices in a hands-free interaction environment, wherein participants can choose and combine freely three hands-free modalities (Gaze, Head movements, Speech) to execute point and select actions in a 2D interface. On the one hand, our results show that users avoid switching modalities more often than we expected, particularly, under conditions that should prompt modality switching. On the other hand, when users make a modality switch, user characteristics and consequences of the experienced interaction have a higher impact on the choice, than the changes in environmental conditions. Further, when users switch between modalities, we identified different types of switching behaviors. Users who deliberately try to find and choose an optimal modality (single switcher), users who try to find optimal combinations of modalities (multiple switcher), and a switching behavior triggered by error occurrence (error biased switcher). We believe that these results help to further understand when and how to design for multimodal interaction in real-world systems.}, keywords = {hands-free interaction, modality choices, multimodality, point and select}, pubstate = {published}, tppubtype = {inproceedings} } Interacting with the physical and digital environment multimodally enhances user flexibility and adaptability to different scenarios. A body of research has focused on comparing the efficiency and effectiveness of different interaction modalities in digital environments. However, little is known about user behavior in an environment that provides freedom to choose from a range of modalities. That is why, we take a closer look at the factors that influence input modality choices. Building on the work by Jameson & Kristensson, our goal is to understand how different factors influence user choices. In this paper, we present a study that aims to explore modality choices in a hands-free interaction environment, wherein participants can choose and combine freely three hands-free modalities (Gaze, Head movements, Speech) to execute point and select actions in a 2D interface. On the one hand, our results show that users avoid switching modalities more often than we expected, particularly, under conditions that should prompt modality switching. On the other hand, when users make a modality switch, user characteristics and consequences of the experienced interaction have a higher impact on the choice, than the changes in environmental conditions. Further, when users switch between modalities, we identified different types of switching behaviors. Users who deliberately try to find and choose an optimal modality (single switcher), users who try to find optimal combinations of modalities (multiple switcher), and a switching behavior triggered by error occurrence (error biased switcher). We believe that these results help to further understand when and how to design for multimodal interaction in real-world systems. |
Becker, Marvin; Borsum, Florian; Ihmig, Maike; Terbeck, Clara Wheelchair Traffic Assistant - Ein Konzept für mehr Sicherheit durch Usability Inproceedings Mensch und Computer 2019 - Workshopband, Gesellschaft für Informatik e.V., Bonn, 2019. Abstract | BibTeX | Links: @inproceedings{Becker2019, title = {Wheelchair Traffic Assistant - Ein Konzept für mehr Sicherheit durch Usability}, author = {Marvin Becker and Florian Borsum and Maike Ihmig and Clara Terbeck}, url = {https://hci.w-hs.de/pub_wheelchair_traffic_assistant-2/, PDF Download}, doi = {10.18420/muc2019-ws-379}, year = {2019}, date = {2019-09-08}, booktitle = {Mensch und Computer 2019 - Workshopband}, publisher = {Gesellschaft für Informatik e.V.}, address = {Bonn}, abstract = {Barrierefreiheit hat sich über die letzten Jahre zu einem wichtigen Prinzip in der Gesellschaft und im Bauwesen entwickelt. Trotzdem ist Barrierefreiheit in vielen Bereichen noch ungenügend umgesetzt. Gehbehinderungen oder Lähmungen (z.B. Tetraplegie) machen sogar das Überqueren der Straße zu einer gefährlichen Situation. Personen im Rollstuhl können aufgrund der sitzenden Position nur schlecht um parkende Autos herum oder über sie hinwegsehen, um die Gefahrensituation einzuschätzen. Dieses Paper stellt ein Konzept für ein Assistenzsystem zur sicheren Straßenüberquerung vor, das Personen, die einen Rollstuhl benutzen, insbesondere Tetraplegiker, dabei unterstützen soll, die Straße an schlecht einsehbaren Stellen zu überqueren und diese bei der Einschätzung der Situation zu unterstützen und vor Gefahren zu warnen. Das Konzept basiert auf der Idee, Daten von am Rollstuhl befestigten Sensoren und Kameras in Echtzeit auszuwerten und diese auf einem Bildschirm darzustellen. Der Nutzer soll durch ein barrierefreies Navigationskonzept zwischen drei Ansichten (Kamerasicht, Vogelperspektive, Rückspiegel) navigieren können.}, keywords = {assistant, traffic, wheelchair}, pubstate = {published}, tppubtype = {inproceedings} } Barrierefreiheit hat sich über die letzten Jahre zu einem wichtigen Prinzip in der Gesellschaft und im Bauwesen entwickelt. Trotzdem ist Barrierefreiheit in vielen Bereichen noch ungenügend umgesetzt. Gehbehinderungen oder Lähmungen (z.B. Tetraplegie) machen sogar das Überqueren der Straße zu einer gefährlichen Situation. Personen im Rollstuhl können aufgrund der sitzenden Position nur schlecht um parkende Autos herum oder über sie hinwegsehen, um die Gefahrensituation einzuschätzen. Dieses Paper stellt ein Konzept für ein Assistenzsystem zur sicheren Straßenüberquerung vor, das Personen, die einen Rollstuhl benutzen, insbesondere Tetraplegiker, dabei unterstützen soll, die Straße an schlecht einsehbaren Stellen zu überqueren und diese bei der Einschätzung der Situation zu unterstützen und vor Gefahren zu warnen. Das Konzept basiert auf der Idee, Daten von am Rollstuhl befestigten Sensoren und Kameras in Echtzeit auszuwerten und diese auf einem Bildschirm darzustellen. Der Nutzer soll durch ein barrierefreies Navigationskonzept zwischen drei Ansichten (Kamerasicht, Vogelperspektive, Rückspiegel) navigieren können. |
Pascher, Max; Schneegass, Stefan; Gerken, Jens SwipeBuddy: A Teleoperated Tablet and Ebook-Reader Holder for a Hands-Free Interaction InproceedingsMobILe Lamas, David; Loizides, Fernando; Nacke, Lennart; Petrie, Helen; Winckler, Marco; Zaphiris, Panayiotis (Ed.): Human-Computer Interaction – INTERACT 2019, S. 568-571, Springer, Cham, 2019, ISBN: 978-3-030-29390-1. Abstract | BibTeX | Links: @inproceedings{Pascher2019b, title = {SwipeBuddy: A Teleoperated Tablet and Ebook-Reader Holder for a Hands-Free Interaction}, author = {Max Pascher and Stefan Schneegass and Jens Gerken}, editor = {David Lamas and Fernando Loizides and Lennart Nacke and Helen Petrie and Marco Winckler and Panayiotis Zaphiris}, url = {https://hci.w-hs.de/pub_SwipeBuddy/, PDF Download}, doi = {10.1007/978-3-030-29390-1_39}, isbn = {978-3-030-29390-1}, year = {2019}, date = {2019-08-23}, booktitle = {Human-Computer Interaction – INTERACT 2019}, issuetitle = {Lecture Notes in Computer Science}, volume = {11749}, pages = {568-571}, publisher = {Springer, Cham}, abstract = {Mobile devices are the core computing platform we use in our everyday life to communicate with friends, watch movies, or read books. For people with severe physical disabilities, such as tetraplegics, who cannot use their hands to operate such devices, these devices are barely usable. Tackling this challenge, we propose SwipeBuddy, a teleoperated robot allowing for touch interaction with a smartphone, tablet, or ebook-reader. The mobile device is mounted on top of the robot and can be teleoperated by a user through head motions and gestures controlling a stylus simulating touch input. Further, the user can control the position and orientation of the mobile device. We demonstrate the SwipeBuddy robot device and its different interaction capabilities.}, keywords = {hands-free, head gestures, head-based interaction, human-robot collaboration, human-robot interaction, robot control, self-determined life}, pubstate = {published}, tppubtype = {inproceedings} } Mobile devices are the core computing platform we use in our everyday life to communicate with friends, watch movies, or read books. For people with severe physical disabilities, such as tetraplegics, who cannot use their hands to operate such devices, these devices are barely usable. Tackling this challenge, we propose SwipeBuddy, a teleoperated robot allowing for touch interaction with a smartphone, tablet, or ebook-reader. The mobile device is mounted on top of the robot and can be teleoperated by a user through head motions and gestures controlling a stylus simulating touch input. Further, the user can control the position and orientation of the mobile device. We demonstrate the SwipeBuddy robot device and its different interaction capabilities. |
Pascher, Max; Baumeister, Annalies; Klein, Barbara; Schneegass, Stefan; Gerken, Jens Little Helper: A Multi-Robot System in Home Health Care Environments InproceedingsMobILe Proceedings of the 2019 International workshop on Human-Drone Interaction (iHDI) as part of the ACM Conference on Human Factors in Computing Systems, ACM 2019. Abstract | BibTeX | Links: @inproceedings{Pascher2019, title = {Little Helper: A Multi-Robot System in Home Health Care Environments}, author = {Max Pascher and Annalies Baumeister and Barbara Klein and Stefan Schneegass and Jens Gerken}, url = {https://hci.w-hs.de/pub_little_helper/, PDF Download https://hal.archives-ouvertes.fr/hal-02128382}, year = {2019}, date = {2019-05-04}, booktitle = {Proceedings of the 2019 International workshop on Human-Drone Interaction (iHDI) as part of the ACM Conference on Human Factors in Computing Systems}, organization = {ACM}, abstract = {Being able to live independently and self-determined in once own home is a crucial factor for social participation. For people with severe physical impairments, such as tetraplegia, who cannot use their hands to manipulate materials or operate devices, life in their own home is only possible with assistance from others. The inability to operate buttons and other interfaces results also in not being able to utilize most assistive technologies on their own. In this paper, we present an ethnographic field study with 15 tetraplegics to better understand their living environments and needs. Results show the potential for robotic solutions but emphasize the need to support activities of daily living (ADL), such as grabbing and manipulating objects or opening doors. Based on this, we propose Little Helper, a tele-operated pack of robot drones, collaborating in a divide and conquer paradigm to fulfill several tasks using a unique interaction method. The drones can be tele-operated by a user through gaze-based selection and head motions and gestures manipulating materials and applications.}, keywords = {activities of daily living, ethnographic study, healthcare environment, multi-robot system, tetraplegics}, pubstate = {published}, tppubtype = {inproceedings} } Being able to live independently and self-determined in once own home is a crucial factor for social participation. For people with severe physical impairments, such as tetraplegia, who cannot use their hands to manipulate materials or operate devices, life in their own home is only possible with assistance from others. The inability to operate buttons and other interfaces results also in not being able to utilize most assistive technologies on their own. In this paper, we present an ethnographic field study with 15 tetraplegics to better understand their living environments and needs. Results show the potential for robotic solutions but emphasize the need to support activities of daily living (ADL), such as grabbing and manipulating objects or opening doors. Based on this, we propose Little Helper, a tele-operated pack of robot drones, collaborating in a divide and conquer paradigm to fulfill several tasks using a unique interaction method. The drones can be tele-operated by a user through gaze-based selection and head motions and gestures manipulating materials and applications. |
Auda, Jonas; Pascher, Max; Schneegass, Stefan Around the (Virtual) World: Infinite Walking in Virtual Reality Using Electrical Muscle Stimulation Inproceedings Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, ACM 2019, ISBN: 978-1-4503-5970-2/19/05. Abstract | BibTeX | Links: @inproceedings{Auda2019, title = {Around the (Virtual) World: Infinite Walking in Virtual Reality Using Electrical Muscle Stimulation}, author = {Jonas Auda and Max Pascher and Stefan Schneegass }, url = {https://hci.w-hs.de/pub_around_the_virtual_world/, PDF Download}, doi = {10.1145/3290605.3300661}, isbn = {978-1-4503-5970-2/19/05}, year = {2019}, date = {2019-04-18}, booktitle = {Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems}, organization = {ACM}, abstract = {Virtual worlds are infinite environments in which the user can move around freely. When shifting from controller based movement to regular walking as input, the limitation of the real world also limits the virtual world. Tackling this challenge, we propose the use of electrical muscle stimulation to limit the necessary real world space to create an unlimited walking experience. We thereby actuate the users' legs in a way that they deviate from their straight route and, thus, walk in circles in the real world world while still walking straight in the virtual world. We report on a study comparing this approach to vision shift - the state of the art approach - as well as combining both approaches. The results show that particularly combining both approaches yield high potential to create an infinite walking experience.}, keywords = {electrical muscle stimulation, Locomotion, Virtual Reality, walking}, pubstate = {published}, tppubtype = {inproceedings} } Virtual worlds are infinite environments in which the user can move around freely. When shifting from controller based movement to regular walking as input, the limitation of the real world also limits the virtual world. Tackling this challenge, we propose the use of electrical muscle stimulation to limit the necessary real world space to create an unlimited walking experience. We thereby actuate the users' legs in a way that they deviate from their straight route and, thus, walk in circles in the real world world while still walking straight in the virtual world. We report on a study comparing this approach to vision shift - the state of the art approach - as well as combining both approaches. The results show that particularly combining both approaches yield high potential to create an infinite walking experience. |
Kronhardt, Kirill Konzeption, prototypische Umsetzung und Evaluation alternativer Benutzeroberflächen für die Arbeitssuche am Beispiel der Stellenplattform jobvector.de Bachelor Thesis Westphalian University of Applied Sciences, 2019. BibTeX | Tags: bachelor thesis @mastersthesis{Kronhardt2019, title = {Konzeption, prototypische Umsetzung und Evaluation alternativer Benutzeroberflächen für die Arbeitssuche am Beispiel der Stellenplattform jobvector.de}, author = {Kirill Kronhardt }, year = {2019}, date = {2019-02-25}, school = {Westphalian University of Applied Sciences}, keywords = {bachelor thesis}, pubstate = {published}, tppubtype = {mastersthesis} } |
Hülscher, Nico Konzeption und Entwicklung eines Sprachassistenten für den Einsatz in der Pflege Bachelor Thesis Westphalian University of Applied Sciences, 2019. BibTeX | Tags: bachelor thesis @mastersthesis{Huelscher2019, title = {Konzeption und Entwicklung eines Sprachassistenten für den Einsatz in der Pflege}, author = {Hülscher, Nico}, year = {2019}, date = {2019-01-30}, school = {Westphalian University of Applied Sciences}, keywords = {bachelor thesis}, pubstate = {published}, tppubtype = {mastersthesis} } |
2018 |
Pascher, Max; Wöhle, Lukas SwipeTank - A teleoperated tablet and ebook-reader holder for a hands-free interaction PresentationMobILe 14.10.2018. Abstract | BibTeX | Links: @misc{Pascher2018, title = {SwipeTank - A teleoperated tablet and ebook-reader holder for a hands-free interaction}, author = {Max Pascher and Lukas Wöhle}, url = {https://hci.w-hs.de/pub_swipetank/, PDF Download}, year = {2018}, date = {2018-10-14}, abstract = {This extended abstracts provides a contribution for the Student Innovation Contest of the UIST'18. We will build a tank-like tablet and ebook-reader that can be teleoperated by a user through headmotion and gestures to provide a hands-free interaction mode.}, keywords = {hands-free, head gestures, human-robot collaboration, robot control}, pubstate = {published}, tppubtype = {presentation} } This extended abstracts provides a contribution for the Student Innovation Contest of the UIST'18. We will build a tank-like tablet and ebook-reader that can be teleoperated by a user through headmotion and gestures to provide a hands-free interaction mode. |
Baumeister, Annalies; Pascher, Max; Thietje, Roland; Gerken, Jens; Klein, Barbara Anforderungen an die Interaktion eines Roboterarms zur Nahrungsaufnahme bei Tetraplegie – Eine ethnografische Analyse InproceedingsMobILe Kongress und Ausstellung zu Alltagsunterstützenden Assistenzlösungen / Active Assisted Living (AAL) - Tagungsband, S. 100-101, Karlsruher Messe- und Kongress GmbH Karlsruhe, 2018. Abstract | BibTeX | Links: @inproceedings{Baumeister2018, title = {Anforderungen an die Interaktion eines Roboterarms zur Nahrungsaufnahme bei Tetraplegie – Eine ethnografische Analyse}, author = {Annalies Baumeister and Max Pascher and Roland Thietje and Jens Gerken and Barbara Klein}, url = {https://hci.w-hs.de/pub_aal_2018_tagungsband_web/, PDF Download}, year = {2018}, date = {2018-10-11}, booktitle = {Kongress und Ausstellung zu Alltagsunterstützenden Assistenzlösungen / Active Assisted Living (AAL) - Tagungsband}, pages = {100-101}, address = {Karlsruhe}, organization = {Karlsruher Messe- und Kongress GmbH}, abstract = {Selbstständig und selbstbestimmt essen und trinken zu können gehört zu den Grundbedürfnissen des Menschen und wird den Aktivitäten des täglichen Lebens (ATLs) zugeordnet. Körperliche Beeinträchtigungen, die mit Funktionsverlusten in Armen, Händen und ggf. der Beweglichkeit des Oberkörpers einhergehen, schränken die selbstständige Nahrungszufuhr erheblich ein. Die Betroffenen sind darauf angewiesen, dass ihnen Getränke und Mahlzeiten zubereitet, bereitgestellt und angereicht werden. Zu dieser Personengruppe gehören Menschen mit querschnittbedingter Tetraplegie, Multiple Sklerose, Muskeldystrophie und Erkrankungen mit ähnlichen Auswirkungen. Derzeit gibt es verschiedene assistive Technologien, die das selbstständige Essen und Trinken wieder ermöglichen sollen – darunter Essapparate, die auf einem Tisch platziert werden (z.B. iEat , Obi), sowie an einem E-Rollstuhl befestigte Roboterarme (z.B. Jaco, iArm). Diesen Produkten ist gemeinsam, dass für die Bedienung Restfunktionen in Armen, Händen und Oberkörper benötigt werden. Wie aber muss ein Interaktionsdesign für einen Roboterarm gestaltet sein, damit er von den Betroffenen zur Nahrungsaufnahme genutzt werden kann? Welche Anforderungen gibt es und welche Aspekte sind in Bezug auf die Akzeptanz eines Roboterarms zu berücksichtigen? }, keywords = {AAL, interview, observing, tetraplegics, user study}, pubstate = {published}, tppubtype = {inproceedings} } Selbstständig und selbstbestimmt essen und trinken zu können gehört zu den Grundbedürfnissen des Menschen und wird den Aktivitäten des täglichen Lebens (ATLs) zugeordnet. Körperliche Beeinträchtigungen, die mit Funktionsverlusten in Armen, Händen und ggf. der Beweglichkeit des Oberkörpers einhergehen, schränken die selbstständige Nahrungszufuhr erheblich ein. Die Betroffenen sind darauf angewiesen, dass ihnen Getränke und Mahlzeiten zubereitet, bereitgestellt und angereicht werden. Zu dieser Personengruppe gehören Menschen mit querschnittbedingter Tetraplegie, Multiple Sklerose, Muskeldystrophie und Erkrankungen mit ähnlichen Auswirkungen. Derzeit gibt es verschiedene assistive Technologien, die das selbstständige Essen und Trinken wieder ermöglichen sollen – darunter Essapparate, die auf einem Tisch platziert werden (z.B. iEat , Obi), sowie an einem E-Rollstuhl befestigte Roboterarme (z.B. Jaco, iArm). Diesen Produkten ist gemeinsam, dass für die Bedienung Restfunktionen in Armen, Händen und Oberkörper benötigt werden. Wie aber muss ein Interaktionsdesign für einen Roboterarm gestaltet sein, damit er von den Betroffenen zur Nahrungsaufnahme genutzt werden kann? Welche Anforderungen gibt es und welche Aspekte sind in Bezug auf die Akzeptanz eines Roboterarms zu berücksichtigen? |
Schrader, Verena; Gerken, Jens; Heinecke, Andreas M Smart Cooking in der Praxis - Eine Feldstudie über das Kochen mittels Smartwatch Inproceedings Dachselt, Raimund; Weber, Gerhard (Ed.): Mensch und Computer 2018 - Tagungsband, Gesellschaft für Informatik e.V., Bonn, 2018. Abstract | BibTeX | Links: @inproceedings{Schrader2018, title = {Smart Cooking in der Praxis - Eine Feldstudie über das Kochen mittels Smartwatch}, author = {Verena Schrader and Jens Gerken and Andreas M. Heinecke}, editor = {Raimund Dachselt and Gerhard Weber}, url = {https://hci.w-hs.de/pub_smartcooking/, PDF Download}, doi = {10.18420/muc2018-mci-0259}, year = {2018}, date = {2018-09-04}, booktitle = {Mensch und Computer 2018 - Tagungsband}, publisher = {Gesellschaft für Informatik e.V.}, address = {Bonn}, abstract = {In der wissenschaftlichen Literatur gibt es kaum Studien, die sich mit der konkreten Alltagstauglichkeit von Smartwatches beschäftigen, um zu verstehen, warum die Klasse von wearables eher ein Nischendasein führt. In diesem Beitrag wird daher die Verwendung einer Smartwatch am Beispiel Kochen untersucht. Hierzu wurde eine Koch-App mit Rezeptinformationen für eine Smartwatch entwickelt, welche über Hand- und Armbewegungen in Form von Gesten bedient werden kann. In einer Feldstudie mit acht Probanden wurde ermittelt, inwieweit diese Interaktionsform den Kochprozess verändert. Die Ergebnisse zeigen, dass die unmittelbare Verfügbarkeit der Uhr sowohl Effizienz- als auch Effektivitätsvorteile gegenüber klassischen Kochhilfen bietet. Die Steuerung via Freihandgesten erlaubte zudem die Nutzung in einem Szenario, in welchem die Hände oft belegt oder verschmutzt sind und somit eine Bedienung per Finger problematisch sein kann. Die Uhr wurde von den Probanden dabei als nützliches Werkzeug erachtet, obwohl diese bislang keinerlei Erfahrung mit einem solchen Gerät hatten.}, keywords = {experiment, field study, gesture interaction, interaction design, Smartwatch}, pubstate = {published}, tppubtype = {inproceedings} } In der wissenschaftlichen Literatur gibt es kaum Studien, die sich mit der konkreten Alltagstauglichkeit von Smartwatches beschäftigen, um zu verstehen, warum die Klasse von wearables eher ein Nischendasein führt. In diesem Beitrag wird daher die Verwendung einer Smartwatch am Beispiel Kochen untersucht. Hierzu wurde eine Koch-App mit Rezeptinformationen für eine Smartwatch entwickelt, welche über Hand- und Armbewegungen in Form von Gesten bedient werden kann. In einer Feldstudie mit acht Probanden wurde ermittelt, inwieweit diese Interaktionsform den Kochprozess verändert. Die Ergebnisse zeigen, dass die unmittelbare Verfügbarkeit der Uhr sowohl Effizienz- als auch Effektivitätsvorteile gegenüber klassischen Kochhilfen bietet. Die Steuerung via Freihandgesten erlaubte zudem die Nutzung in einem Szenario, in welchem die Hände oft belegt oder verschmutzt sind und somit eine Bedienung per Finger problematisch sein kann. Die Uhr wurde von den Probanden dabei als nützliches Werkzeug erachtet, obwohl diese bislang keinerlei Erfahrung mit einem solchen Gerät hatten. |
