基于骨架的双模注意力时空图卷积网络人体动作识别方法

石祥滨 刘宏哲

PDF(661 KB)
PDF(661 KB)
沈阳航空航天大学学报 ›› 2023, Vol. 40 ›› Issue (1) : 58-66.
信息科学与工程

基于骨架的双模注意力时空图卷积网络人体动作识别方法

  • 石祥滨1,2,刘宏哲2
作者信息 +
History +

摘要

人体动作识别已成为当今热门研究领域之一,基于骨架的人体动作识别方法因其能够明确展现人体动作而备受关注。针对手工设计的人体拓扑图在提取特征时无法获取全局信息、存在大量冗余信息等问题,提出一个双模注意力时空图卷积网络,充分利用对动作识别起关键作用的节点信息。首先,提出SGSAE模块,使用自注意力机制对所有关节点之间的关系进行建模,实现对节点信息全局特征的提取,并且在网络的训练过程中优化图的拓扑结构,最终获得适应各种数据样本的图拓扑结构。其次,按不同权重融合节点的全局特征与局部特征。最后,将通道注意力机制引入到网络模型中,提出MCA模块融合通道特征,减少大量冗余信息,提高动作识别精度。实验结果表明,本文提出的双模注意力时空图卷积网络在NTU-RGB+D和Kinetics数据集上取得了较好的动作识别效果。

Abstract

Human action recognition has become one of the hot areas today. The skeleton-based human action recognition method has been widely noticed for its ability to show human action clearly. In response to the problem that the manually designed human topology graph cannot obtain global information and has a lot of redundant information when extracting features. A Dual-mode Attention Spatio-Temporal Graph Convolutional Network is proposed to achieve the full utilization of node information that plays a key role in action recognition. First, the SGSAE module is proposed to model the relationship between all nodes using a self-attention mechanism to achieve the extraction of global features of node information, and to optimize the topology of the graph during the training process of the network to finally obtain a graph topology that adapts to various data samples. Second, the global and local features of the nodes are fused by different weights. Finally, the channel attention mechanism is introduced into the network, and the MCA module is proposed to fuse channel features to reduce a large amount of redundant information and improve action recognition accuracy. The experimental results show that the proposed Dual-mode Attention Spatio-Temporal Graph Convolutional Network achieves better performance on NTU-RGB+D and Kinetics datasets.

引用本文

导出引用
石祥滨 刘宏哲. 基于骨架的双模注意力时空图卷积网络人体动作识别方法. 沈阳航空航天大学学报. 2023, 40(1): 58-66
PDF(661 KB)

14193

Accesses

0

Citation

Detail

段落导航
相关文章

/