Approximation of Weight Matrices using Hierarchical Matrices

Abstract: Deep neural networks have shown their high performance accuracy in many areas such as image and speech recognition. Further they are also associated with high computational costs. In this paper, the approximation of a weight matrices is investigated in terms of time and prediction accuracy. Hierarchical matrices (H-matrices) are used for the approximation. In order to find a suitable H-matrix approximation, submatrices which are approximately low rank must be found in the original matrix. Various variations in algorithm such as the low-rank approximation method and the rank were investigated. From Pre-trained Pytorch models such as ResNet, GoogLeNet and MobileNetV2 the last layers were extracted and approximated. The ImageNet dataset was used for testing. For all tested models it was shown that the time required for a matrix vector operation is be significantly smaller for an approximated matrix. By using GoogLeNet with an approximation rank of 30, 43.89 % of the computing time was saved with a percentage accuracy loss of 5.69 %.

FINISHED