Hardware Implementation of a Deep Learning-based Model for Image Quality Assessment
Received: 4 March 2024 | Revised: 13 March 2024 | Accepted: 14 March 2024 | Online: 24 March 2024
Corresponding author: Yahia Said
Abstract
Image quality assessment is very important for accurate analysis and better interpretation. In reality, environmental effects and device limitations may degrade image quality. Recently, many image quality assessment algorithms have been proposed. However, these algorithms require high computation overhead, making them unsuitable for mobile devices, such as smartphones and smart cameras. This paper presents a hardware implementation of an image quality assessment algorithm based on a Lightweight Convolutional Neural Network (LCNN) model. Many advances have been made in the construction of high-accuracy LCNN models. The current study used EfficientNet V2. The model achieved state-of-the-art image classification performance on many famous benchmark datasets while having a smaller size than other models with the same performance. The model was utilized to learn human visual behavior through understanding dataset information without prior knowledge of target visual behavior. The proposed model was implemented employing a Field Programmable Gate Array (FPGA) for possible integration into mobile devices. The Xilinx ZCU 102 board was implemented to evaluate the proposed model. The results confirmed the latter’s efficiency in image quality assessment compared to existing models.
Keywords:
deep learning, artificial intelligence, embedded systems, FPGA, efficientNet v2, image quality assessmentDownloads
References
G. Zhai and X. Min, "Perceptual image quality assessment: a survey," Science China Information Sciences, vol. 63, no. 11, Apr. 2020, Art. no. 211301.
M. Tan and Q. Le, "EfficientNetV2: Smaller Models and Faster Training," in Proceedings of the 38th International Conference on Machine Learning, Jul. 2021, pp. 10096–10106.
Q. Huynh-Thu and M. Ghanbari, "Scope of validity of PSNR in image/video quality assessment," Electronics Letters, vol. 44, no. 13, pp. 800–801, Jun. 2008.
S. Rani, Y. Chabrra, and K. Malik, "An Improved Denoising Algorithm for Removing Noise in Color Images," Engineering, Technology & Applied Science Research, vol. 12, no. 3, pp. 8738–8744, Jun. 2022.
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, Apr. 2004.
Z. Wang, E. P. Simoncelli, and A. C. Bovik, "Multiscale structural similarity for image quality assessment," in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, Nov. 2003, vol. 2, pp. 1398-1402.
Z. Wang and Q. Li, "Information Content Weighting for Perceptual Image Quality Assessment," IEEE Transactions on Image Processing, vol. 20, no. 5, pp. 1185–1198, Nov. 2010.
C. Galkandage, J. Calic, S. Dogan, and J.-Y. Guillemaut, "Full-Reference Stereoscopic Video Quality Assessment Using a Motion Sensitive HVS Model," IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 2, pp. 452–466, Mar. 2021.
H. T. R. Kurmasha, A. F. H. Alharan, C. S. Der, and N. H. Azami, "Enhancement of Edge-based Image Quality Measures Using Entropy for Histogram Equalization-based Contrast Enhancement Techniques," Engineering, Technology & Applied Science Research, vol. 7, no. 6, pp. 2277–2281, Dec. 2017.
L. Zhang, Y. Shen, and H. Li, "VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment," IEEE Transactions on Image Processing, vol. 23, no. 10, pp. 4270–4281, Jul. 2014.
L. Zhang, L. Zhang, X. Mou, and D. Zhang, "FSIM: A Feature Similarity Index for Image Quality Assessment," IEEE Transactions on Image Processing, vol. 20, no. 8, pp. 2378–2386, Dec. 2011.
W. Xue, L. Zhang, X. Mou, and A. C. Bovik, "Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index," IEEE Transactions on Image Processing, vol. 23, no. 2, pp. 684–695, Dec. 2014.
A. Maalouf, M.-C. Larabi, and C. Fernandez-Maloigne, "A grouplet-based reduced reference image quality assessment," in 2009 International Workshop on Quality of Multimedia Experience, Jul. 2009, pp. 59–63.
I. P. Gunawan and M. Ghanbari, "Reduced-reference picture quality estimation by using local harmonic amplitude information," in London Communications Symposium, 2003, pp. 353–358.
E. C. Larson and D. M. Chandler, "Most apparent distortion: full-reference image quality assessment and the role of strategy," Journal of electronic imaging, vol. 19, no. 1, 2010, Art. no. 011006.
N. Ponomarenko et al., "Image database TID2013: Peculiarities, results and perspectives," Signal Processing: Image Communication, vol. 30, pp. 57–77, Jan. 2015.
R. Vadhi, V. S. Kilari, and S. S. Kumar, "An Image Fusion Technique Based on Hadamard Transform and HVS," Engineering, Technology & Applied Science Research, vol. 6, no. 4, pp. 1075–1079, Aug. 2016.
D. Ghadiyaram and A. C. Bovik, "Massive Online Crowdsourced Study of Subjective and Objective Picture Quality," IEEE Transactions on Image Processing, vol. 25, no. 1, pp. 372–387, Nov. 2015.
V. Hosu, H. Lin, T. Sziranyi, and D. Saupe, "KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment," IEEE Transactions on Image Processing, vol. 29, pp. 4041–4056, 2020.
Z. Ying, H. Niu, P. Gupta, D. Mahajan, D. Ghadiyaram, and A. Bovik, "From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, Jun. 2020, pp. 3572–3582.
B. Yan, B. Bare, and W. Tan, "Naturalness-Aware Deep No-Reference Image Quality Assessment," IEEE Transactions on Multimedia, vol. 21, no. 10, pp. 2603–2615, Mar. 2019.
H. Zhu, L. Li, J. Wu, W. Dong, and G. Shi, "MetaIQA: Deep Meta-Learning for No-Reference Image Quality Assessment," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, Jun. 2020, pp. 14131–14140.
S. A. Golestaneh, S. Dadsetan, and K. M. Kitani, "No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency," in 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, Jan. 2022, pp. 3989–39994.
Downloads
How to Cite
License
Copyright (c) 2024 Yahia Said, Yazan A. Alsariera
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
