hh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Multimodal Detection and Classification of Robot Manipulation Failures
Istanbul Technical University, Maslak, Turkey.ORCID iD: 0000-0001-8259-3986
Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research (CAISR).ORCID iD: 0000-0002-5712-6777
Istanbul Technical University, Maslak, Turkey.ORCID iD: 0000-0003-2993-6681
2024 (English)In: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 9, no 2, p. 1396-1403Article in journal (Refereed) Published
Abstract [en]

An autonomous service robot should be able to interact with its environment safely and robustly without requiring human assistance. Unstructured environments are challenging for robots since the exact prediction of outcomes is not always possible. Even when the robot behaviors are well-designed, the unpredictable nature of the physical robot-object interaction may lead to failures in object manipulation. In this letter, we focus on detecting and classifying both manipulation and post-manipulation phase failures using the same exteroception setup. We cover a diverse set of failure types for primary tabletop manipulation actions. In order to detect these failures, we propose FINO-Net (Inceoglu et al., 2021), a deep multimodal sensor fusion-based classifier network architecture. FINO-Net accurately detects and classifies failures from raw sensory data without any additional information on task description and scene state. In this work, we use our extended FAILURE dataset (Inceoglu et al., 2021) with 99 new multimodal manipulation recordings and annotate them with their corresponding failure types. FINO-Net achieves 0.87 failure detection and 0.80 failure classification F1 scores. Experimental results show that FINO-Net is also appropriate for real-time use. © 2016 IEEE.

Place, publisher, year, edition, pages
Piscataway, NJ: IEEE, 2024. Vol. 9, no 2, p. 1396-1403
Keywords [en]
Robot sensing systems, Robots, Task analysis, Monitoring, Hidden Markov models, Collision avoidance, Real-time systems, Deep learning methods, data sets for robot learning, failure detection and recovery, sensor fusion
National Category
Robotics and automation Computer graphics and computer vision
Identifiers
URN: urn:nbn:se:hh:diva-52942DOI: 10.1109/lra.2023.3346270ISI: 001136735400012Scopus ID: 2-s2.0-85181561810OAI: oai:DiVA.org:hh-52942DiVA, id: diva2:1846426
Note

Funding: The Scientific and Technological Research Council of Türkiye under Grant 119E-436.

Available from: 2024-03-22 Created: 2024-03-22 Last updated: 2025-02-05Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Aksoy, Eren

Search in DiVA

By author/editor
Inceoglu, ArdaAksoy, ErenSariel, Sanem
By organisation
Center for Applied Intelligent Systems Research (CAISR)
In the same journal
IEEE Robotics and Automation Letters
Robotics and automationComputer graphics and computer vision

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 158 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf