针对目前虚拟现实(VR)人机交互技术在雷达领域应用较少,且主要集中在单模态交互上,导致VR 显控体验感不佳的问题,文中基于视觉、听觉、手势等多模态开展雷达显控VR 人机交互技术研究,提出了多模态VR 交互系统架构、交互设计流程与方法、交互体验提升策略。在雷达VR 显控中集成应用了人因手势、自然语音、智能眼控等多模态交互;同时基于多传感器设备,构建了多模态融合的VR 自然交互体系,实现将手势、语音、眼控等行为数据融合,充分发挥多模态交互优势,有效增强雷达显控人机交互通道和效率,带来更加直观、自然、沉浸的用户体验。

At present, VR human-computer interaction technology is seldom applied in radar field, and mainly focuses on singlemodal interaction, resulting in poor experience of display and control. Based on multi-modal of vision, hearing and gesture, a study on VR interaction technology of radar display and control is carried out. VR multi-modal interaction system architecture, design process and methods, experience promotion strategies are proposed. And the human gesture, natural voice and intelligent eye control are innovatively integrated in radar VR multi-modal interaction. In addition, based on the multi-sensor equipment, VR multimodal natural interaction system is built to realize the fusion of gesture, voice and eye control data. That gives full play to the advantages of multi-modal interaction to effectively enhance the interaction channel and efficiency, and bring more intuitive, natural and immersive user experience.

TN957. 7