Collaborating with computer vision systems: an exploration of audio feedback

2014. Cecily Morrison, Neil Smyth, Robert Corish, Kenton O'Hara & Abigail Sellen
Proceedings of the 2014 Conference on Designing Interactive Systems.

Abstract

Computer visions (CV) systems are increasingly finding new roles in domains such as healthcare. These collaborative settings are a new challenge for CV systems, requiring the design of appropriate interaction paradigms. The provision of feedback, particularly of what the CV system can 'see,' is a key aspect, and may not always be possible to present visually. We explore the design space for audio feedback for a scenario of interest, the clinical assessment of Multiple Sclerosis using a CV system. We then present a mixed-methods experimental study aimed at providing some first insights into the challenges and opportunities of designing audio feedback of this kind. Specifically, we compare audio feedback that differentiates which body parts the CV system can see to audio feedback that is undifferentiated. The findings reveal that it is not enough to simply convey that something might be out of view of the camera as what the camera can 'see' depends on the specific configuration of participants and the peculiarities of the skeleton inference algorithms. The results highlight the importance of providing feedback which more naturally conveys spatial information in developing CV systems for collaborative use.

Read the full paper here ›

Citation

Morrison, C., Smyth, N., Corish, R., O'Hara, K., & Sellen, A. (2014). Collaborating with computer vision systems: an exploration of audio feedback. Proceedings of the 2014 Conference on Designing Interactive Systems (pp. 229–238). New York, NY, USA: Association for Computing Machinery. URL: https://doi.org/10.1145/2598510.2598519, doi:10.1145/2598510.2598519

BibTeX

@inproceedings{10.1145/2598510.2598519,
author = {Morrison, Cecily and Smyth, Neil and Corish, Robert and O'Hara, Kenton and Sellen, Abigail},
title = {Collaborating with computer vision systems: an exploration of audio feedback},
year = {2014},
isbn = {9781450329026},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/2598510.2598519},
doi = {10.1145/2598510.2598519},
abstract = {Computer visions (CV) systems are increasingly finding new roles in domains such as healthcare. These collaborative settings are a new challenge for CV systems, requiring the design of appropriate interaction paradigms. The provision of feedback, particularly of what the CV system can 'see,' is a key aspect, and may not always be possible to present visually. We explore the design space for audio feedback for a scenario of interest, the clinical assessment of Multiple Sclerosis using a CV system. We then present a mixed-methods experimental study aimed at providing some first insights into the challenges and opportunities of designing audio feedback of this kind. Specifically, we compare audio feedback that differentiates which body parts the CV system can see to audio feedback that is undifferentiated. The findings reveal that it is not enough to simply convey that something might be out of view of the camera as what the camera can 'see' depends on the specific configuration of participants and the peculiarities of the skeleton inference algorithms. The results highlight the importance of providing feedback which more naturally conveys spatial information in developing CV systems for collaborative use.},
booktitle = {Proceedings of the 2014 Conference on Designing Interactive Systems},
pages = {229–238},
numpages = {10},
keywords = {kinect, computer vision technologies, co-present interaction, audio feedback},
location = {Vancouver, BC, Canada},
series = {DIS '14}
}