Muhammad Baihaqi, Angel Garcia Contreras, Seiya Kawano, Koichiro Yoshino. "Rapport-Driven Virtual Agent: Rapport Building Dialogue Strategy for Improving User Experience at First Meeting" In Proceedings of the 25th Interspeech Conference (INTERSPEECH), Kos, Greece, September 2024.
Shun Inadumi, Seiya Kawano, Akishige Yuguchi, Yasutomo Kawanishi, Koichiro Yoshino. "A Gaze-grounded Visual Question Answering Dataset for Clarifying Ambiguous Questions." In Proceedings of The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), Torino, Italy, May. 2024.
Nobuhiro Ueda, Hideko Habe, Yoko Matsui, Akishige Yuguchi, Seiya Kawano, Yasutomo Kawanishi, Sadao Kurohashi, Koichiro Yoshino. "J-CRe3: A Japanese Conversation Dataset for Real-world Reference Resolution," In Proceedings of The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), Torino, Italy, May 2024.
Shang-Chi Tsai, Seiya Kawano, Angel Garcia Contreras, Koichiro Yoshino, Yun-Nung Chen. "ASMR: Augmenting Life Scenario using Large Generative Models for Robotic Action Reflection." In Proceedings of International Workshop on Spoken Dialogue Systems Technology (IWSDS) 2024, Sapporo, Japan, March. 2024, Best Paper Award.
Shohei Tanaka, Konosuke Yamasaki, Akishige Yuguchi, Seiya Kawano, Satoshi Nakamura, Koichiro Yoshino. "Do as I Demand, Not as I Say: A Dataset for Developing a Reflective Life Support Robot." IEEE Access, Vol. 12, pp.11774-11784, January 2024.
Koichiro Yoshino, Yun-Nung Chen, Paul Crook, Satwik Kottur, Jinchao Li, Behnam Hedayatnia, Seungwhan Moon, Zhengcong Fei, Zekang Li, Jinchao Zhang, Yang Feng, Jie Zhou, Seokhwan Kim, Yang Liu, Di Jin, Alexandros Papangelis, Karthik Gopalakrishnan, Dilek Hakkani-Tur, Babak Damavandi, Alborz Geramifard, Chiori Hori, Ankit Shah, Chen Zhang, Haizhou Li, Jo?o Sedoc, Luis F. D'Haro, Rafael Banchs and Alexander Rudnicky. "Overview of the Tenth Dialog System Technology Challenge: DSTC10." IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), Vol. 32, pp.765-768, July 2023,
Seiya Kawano, Koichiro Yoshino, David Traum and Satoshi Nakamura. "End-to-End Dialogue Structure Parsing on Multi-Floor Dialogue Based on Multi-Task Learning." Frontiers in Robotics and AI, Vol.10, May 2023.
Taiki Nakamura, Seiya Kawano, Akishige Yuguchi, Yasutomo Kawanishi and Koichiro Yoshino. "Operative Action Captioning for Estimating System Actions." In Proceedings of 2023 IEEE International Conference on Robotics and Automation (ICRA2023), London, UK, May 2023.
Seiya Kawano, Masahiro Mizukami, Koichiro Yoshino and Satoshi Nakamura. "Entrainable Neural Conversation Model based on Reinforcement Learning" IEEE Access, Vol.8, pp.178283-178294, September 2020.
Koichiro Yoshino, Kohei Wakimoto, Yuta Nishimura and Satoshi Nakamura. "Caption Generation of Robot Behaviors based on Unsupervised Learning of Action Segments." In Proceedings of International Workshop on Spoken Dialogue Systems Technology (IWSDS2020) , Madrid (virtual), Spain, Nov. 2020, Best Paper Award.
We will research robots operating in human living environments and using spoken language to interact with such robots.
A variety of research topics are required: speech recognition, recognizing various things in the actual living environment, understanding the situation based on the recognition results, making decisions (reasoning, planning, control) that match the system's goal, and generating speech and movement for realizing such robots.
To realize spoken dialogue in the real world, we will work on all related research issues and basic research using a language-centered system.
Current Research Projects (AY)
2024 -- 2027, Japan Science and Technology Agency, PRESTO, "Research on Natural Language Processing Formalizing Tacit Knowledge in Research and Development" (PRESTO), Main Proposer
2024 -- 2025, Google Gift 2024, Real-World Understanding and Grounding for Embodied Robot Dialogue, Main Proposer
2022 -- 2025, Ministry of Education, Culture, Sports, Science and Technology - JAPAN, KAKEN, Scientific Research (B), "Knowledge Inference System for Robots by Bridging Text Commonsense and Real-world Observation" , Main Proposer
2024 -- 2028, Ministry of Education, Culture, Sports, Science and Technology - JAPAN, KAKEN, (A), "Knowledge Extension and Environment Recognition Infrastructure through Active Open-world Recognition" , Collaboration (Main Proposer: Yasutomo Kawanishi)
2024 -- 2028, Ministry of Education, Culture, Sports, Science and Technology - JAPAN, KAKEN, (Challenging Research: Pioneering), "A human-centered approach to cognition and behavior based on the fusion of psychology, AI, and robotics" , Collaboration (Main Proposer: Takatsune Kumada)
Past Projects
2022 -- 2023, Ministry of Education, Culture, Sports, Science and Technology - JAPAN, KAKEN, Innovative Areas, "Dialogue Management and Response Expressions in Persuasive Dialogue toward Building Long-term Relationships", Main Proposer
2020 -- 2021, Ministry of Education, Culture, Sports, Science and Technology - JAPAN, KAKEN, Innovative Areas, "Multimodal Dialogue Robot for Persuading Users", Main Proposer
2019 -- 2019, Adobe Research Gift, Image Editing System with Natural Language, Main Proposer
2017 -- 2020, Ministry of Education, Culture, Sports, Science and Technology - JAPAN, KAKEN, Scientific Research (S), Next Generation Speech Translation, Collaboration (Main Proposer: Satoshi Nakamura)
2017 -- 2020, Ministry of Education, Culture, Sports, Science and Technology - JAPAN, KAKEN, Scientific Research (A), Simultaneous Speech Translation with Paralinguistic Information,Collaboration (Main Proposer: Satoshi Nakamura)
2016 -- 2019, Japan Science and Technology Agency, PRESTO, "Spoken Dialogue System based on Incremental Language Understanding and Knowledge Acquisition" (PRESTO ), Main Proposer
2016 -- 2018, NICT, IoT-to-Human System for Kyoto Inbound Tourism, Collaboration (Main Proposer: Satoshi Nakamura)
2015 -- 2017, Ministry of Internal Affairs and Communications - JAPAN, Strategic Information and Communications R&D Promotion Programme (SCOPE), Lifestyle Support Dialogue System for Watching Over the Elderly, Collaboration (Main Proposer: Satoshi Nakamura)
2017 -- 2018, Microsoft Research Asia Collaborative Research, Sentence Generation based on Integration of Various Observations, Main Proposer
2015 -- 2016, Ministry of Education, Culture, Sports, Science and Technology - JAPAN, KAKEN, Start-up, Research on Practical Spoken Language Processing, Main Proposer
2013 -- 2014, Ministry of Education, Culture, Sports, Science and Technology - JAPAN, KAKEN, Research Fellow Fund, User and Domain Adaptive Multimodal Dialogue Systems Based on Information Extraction, Main Proposer