A Low-cost Artificial Neural Network Model for Raspberry Pi
Abstract
In this paper, a ternary neural network with complementary binary arrays is proposed for representing the signed synaptic weights. The proposed ternary neural network is deployed on a low-cost Raspberry Pi board embedded system for the application of speech and image recognition. In conventional neural networks, the signed synaptic weights of –1, 0, and 1 are represented by 8-bit integers. To reduce the amount of required memory for signed synaptic weights, the signed values were represented by a complementary binary array. For the binary inputs, the multiplication of two binary numbers is replaced by the bit-wise AND operation to speed up the performance of the neural network. Regarding image recognition, the MINST dataset was used for training and testing of the proposed neural network. The recognition rate was as high as 94%. The proposed ternary neural network was applied to real-time object recognition. The recognition rate for recognizing 10 simple objects captured from the camera was 89%. The proposed ternary neural network with the complementary binary array for representing the signed synaptic weights can reduce the required memory for storing the model’s parameters and internal parameters by 75%. The proposed ternary neural network is 4.2, 2.7, and 2.4 times faster than the conventional ternary neural network for MNIST image recognition, speech commands recognition, and real-time object recognition respectively.
Keywords:
artificial neural network, deep learning, speech recognition, image recognition, ternary neural networksDownloads
References
A. Krizhevsky, I. Sutskever, G. E. Hinton, “Imagenet classification with deep convolutional neural networks”, Advances in Neural Information Processing Systems, Lake Tahoe, USA, December 3-8, 2012
K. He, X. Zhang, S. Ren, J. Sun, “Deep residual learning for image recognition”, in: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, IEEE, 2016 DOI: https://doi.org/10.1109/CVPR.2016.90
A. Graves, N. Jaitly, “Towards end-to-end speech recognition with recurrent neural networks”, International Conference on Machine Learning, Beijing, China, June 21-26, 2014
P. B. Patil, “Multilayered network for LPC based speech recognition”, IEEE Transactions on Consumer Electronic, Vol. 44, No. 2, pp. 435-438, 1998 DOI: https://doi.org/10.1109/30.681960
B. M .Zahran, “Using neural networks to predict the hardness of aluminum alloys”, Engineering, Technology & Applied Science Research, Vol. 5, No. 1, pp. 757-759, 2015 DOI: https://doi.org/10.48084/etasr.529
G. S. Fesghandis, A. Pooya, M. Kazemi, Z. N. Azimi, “Comparison of multilayer perceptron and radial basis function neural networks in predicting the success of new product development”, Engineering, Technology & Applied Science Research, Vol. 7, No. 1, pp. 1425-1428, 2015 DOI: https://doi.org/10.48084/etasr.936
H. Jang, A. Park, K. Jung, “Neural network implementation using CUDA and Open MP”, in: Proceedings - Digital Image Computing: Techniques and Applications, pp. 155-161, IEEE, 2008 DOI: https://doi.org/10.1109/DICTA.2008.82
Y. Wang, J. Lin, Z. Wang, “An energy-efficient architecture for binary weights convolution neural networks”, IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 26, No. 2, pp. 280-293, 2017 DOI: https://doi.org/10.1109/TVLSI.2017.2767624
T. Simons, D. J. Lee, “A review of binarized neural networks”, Electronics, Vol. 8, No. 6, pp. 1-25, 2019 DOI: https://doi.org/10.3390/electronics8060661
M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, Y. Bengio, “BinaryNet: Training deep neural networks with weights and activations constrained to +1 or −1”, available at: https://arxiv.org/abs/1602.02830, 2016
C. Baldassi, A. Braunstein, N. Brunel, R. Zecchina, “Efficient supervised learning in networks with binary synapses”, Proceedings of the National Academy of Science of the USA, Vol. 104, No. 26, pp. 11079-11084, 2007 DOI: https://doi.org/10.1073/pnas.0700324104
K. Hwang, W. Sung, “Fixed-point feedforward deep neural network design using weights +1, 0, and −1”, 2014 IEEE Workshop on Signal Processing Systems, Belfast, UK, October 20–22, 2014 DOI: https://doi.org/10.1109/SiPS.2014.6986082
H. Yonekawa, S. Sato, H. Nakahara, “A ternary weight binary input convolutional neural network: Realization on the embedded processor”, IEEE 48th International Symposium on Multiple-Valued Logic, Linz, Austria, May 16-18, 2018 DOI: https://doi.org/10.1109/ISMVL.2018.00038
S. Yin, P. Ouyang, J. Yang, T. Lu, X. Li, L. Liu, S. Wei, “An energy-efficient reconfigurable processor for binary-and ternary-weight neural networks with flexible data bit width”, IEEE Journal of Solid-State Circuits, Vol. 54, No. 4, pp. 1120-1136, 2018 DOI: https://doi.org/10.1109/JSSC.2018.2881913
L. Deng, P. Jiao, J. Pei, Z. Wu, G. Li “GXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework”, Neural Networks, Vol. 100, pp. 49-58, 2018 DOI: https://doi.org/10.1016/j.neunet.2018.01.010
L. F. Abbott, W. G. Regehr, “Synaptic computation”, Nature, Vol. 431, pp. 796-803, 2004 DOI: https://doi.org/10.1038/nature03010
R. S. Zucker, W. G. Regehr, “Short-term synaptic plasticity”, Annual Review of Physiology, Vol. 64, pp. 355–405, 2002 DOI: https://doi.org/10.1146/annurev.physiol.64.092501.114547
R. Lamprecht, J. LeDoux, “Structural plasticity and memory”, Nature Reviews, Neuroscience, Vol. 5, No. 1, pp. 45-54, 2004 DOI: https://doi.org/10.1038/nrn1301
T. Mitchell, Machine learning, McGraw-Hill, 1997
L. Deng, “The MNIST database of handwritten digit images for machine learning research [Best of the Web]”, IEEE Signal Processing Magazine, Vol. 29, No. 6, pp. 141-142, 2012 DOI: https://doi.org/10.1109/MSP.2012.2211477
P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition”, available at: https://arxiv.org/abs/1804.03209, 2018
Downloads
How to Cite
License
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
