hh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
PastVision+: Thermovisual Inference of Recent Medicine Intake by Detecting Heated Objects and Cooled Lips
Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.ORCID iD: 0000-0002-4998-1685
Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.ORCID iD: 0000-0002-4929-1262
2017 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 4, article id 61Article in journal (Refereed) Published
Abstract [en]

This article addresses the problem of how a robot can infer what a person has done recently, with a focus on checking oral medicine intake in dementia patients. We present PastVision+, an approach showing how thermovisual cues in objects and humans can be leveraged to infer recent unobserved human-object interactions. Our expectation is that this approach can provide enhanced speed and robustness compared to existing methods, because our approach can draw inferences from single images without needing to wait to observe ongoing actions and can deal with short-lasting occlusions; when combined, we expect a potential improvement in accuracy due to the extra information from knowing what a person has recently done. To evaluate our approach, we obtained some data in which an experimenter touched medicine packages and a glass of water to simulate intake of oral medicine, for a challenging scenario in which some touches were conducted in front of a warm background. Results were promising, with a detection accuracy of touched objects of 50% at the 15 s mark and 0% at the 60 s mark, and a detection accuracy of cooled lips of about 100 and 60% at the 15 s mark for cold and tepid water, respectively. Furthermore, we conducted a follow-up check for another challenging scenario in which some participants pretended to take medicine or otherwise touched a medicine package: accuracies of inferring object touches, mouth touches, and actions were 72.2, 80.3, and 58.3% initially, and 50.0, 81.7, and 50.0% at the 15 s mark, with a rate of 89.0% for person identification. The results suggested some areas in which further improvements would be possible, toward facilitating robot inference of human actions, in the context of medicine intake monitoring.

Place, publisher, year, edition, pages
Lausanne: Frontiers Media S.A., 2017. Vol. 4, article id 61
Keywords [en]
thermovisual inference, touch detection, medicine intake, action recognition, monitoring, near past inference
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
URN: urn:nbn:se:hh:diva-35592DOI: 10.3389/frobt.2017.00061ISI: 000415716600001Scopus ID: 2-s2.0-85061971533OAI: oai:DiVA.org:hh-35592DiVA, id: diva2:1162903
Available from: 2017-12-05 Created: 2017-12-05 Last updated: 2020-05-07Bibliographically approved

Open Access in DiVA

fulltext(6189 kB)140 downloads
File information
File name FULLTEXT01.pdfFile size 6189 kBChecksum SHA-512
0dd6d6fae30203b32e9f29fef3fec0a2f0357387b8577cee1c02a3f8f4b0f65063800d1acccb813ee85877b8ee432598e96d1444c121204bcf259be7f731b361
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Cooney, MartinBigun, Josef

Search in DiVA

By author/editor
Cooney, MartinBigun, Josef
By organisation
CAISR - Center for Applied Intelligent Systems Research
In the same journal
Frontiers in Robotics and AI
Computer Vision and Robotics (Autonomous Systems)

Search outside of DiVA

GoogleGoogle Scholar
Total: 140 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 190 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf