Deep Model Compression for Efficient Edge Inference Under Resource Constraints
Keywords:
model compression, structured pruning, 8-bit quantization, knowledge distillation, edge computing, efficient inference, Jetson NanoAbstract
Deep neural networks often need large memory, high computation, and high energy use, which limits their use on embedded and edge devices. This study proposes a multi-stage compression method for resource-limited inference. The method uses structured pruning to remove redundant channels, 8-bit quantization to reduce memory and computation cost, and teacher-student distillation to recover accuracy after compression. Experiments were conducted on MobileNet and ResNet and deployed on Jetson Nano. The results showed that the method reduced model parameters by about 72%, lowered inference latency to 38% of the original model, and reduced energy use by about 45%. Top-1 accuracy decreased by only 1.6%, showing that the compressed model maintained stable recognition performance. These results indicate that the method can improve deployment efficiency with limited accuracy loss. The method provides a practical approach for low-power vision tasks, mobile sensing, and edge monitoring systems.
References
Zhu, W., Yao, Y., & Yang, J. (2025). Optimizing Financial Risk Control for Multinational Projects: A Joint Framework Based on CVaR-Robust Optimization and Panel Quantile Regression.
Swaminathan, T. P., Silver, C., Akilan, T., & Kumar, J. (2025). Benchmarking deep learning models on nvidia jetson nano for real-time systems: An empirical investigation. Procedia Computer Science, 260, 906-913.
Yan, B. (2026). A Comparative Study: Greedy versus GAN-Based Compressed Sensing for Fast MRI Reconstruction. International Journal of Advance in Applied Science Research, 5(3), 36-42.
Hernández, N., Toledo, P., Blanco, V., & Almeida, F. (2025, April). 2: 4 Pruning on Edge Devices: Performance, Energy Efficiency and Accuracy. In European Conference on Parallel Processing (pp. 278-291). Cham: Springer Nature Switzerland.
Feng, H. (2024, October). Design of Intelligent Charging System for Portable Electronic Devices Based on Internet of Things (IoT). In 2024 5th International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE) (pp. 568-571). IEEE.
Tripathi, K., Malik, D., Akshat, A., & Lata, K. (2025). A survey on 1-bit quantized large language models. Neural Computing and Applications, 37(25), 20823-20878.
Chen, F., Li, S., Liang, H., Xu, P., & Yue, L. (2025). Optimization Study of Thermal Management of Domestic SiC Power Semiconductor Based on Improved Genetic Algorithm.
Lee, S., Lee, D., Hyun, M., Kim, H., & Song, B. C. (2025). Compressing Neural Networks on Limited Computing Resources. IEEE Access.
Liang, R., Feifan, F. N. U., Liang, Y., & Ye, Z. (2025). Emotion-Aware Interface Adaptation in Mobile Applications Based on Color Psychology and Multimodal User State Recognition. Frontiers in Artificial Intelligence Research, 2(1), 51-57.
Liu, S., & Yim, J. (2025). Research on Generative AI Creation Systems Based on Visual Language Modeling: Human-Machine Collaboration and Cognitive Feedback Mechanisms. Available at SSRN 6139770.
Dritsas, E., & Trigka, M. (2026). Deployment-Aware Compression of Large Language Models for Edge Intelligence: A System-Level Survey. IEEE Transactions on Artificial Intelligence.
Suchý, I., & Turčaník, M. (2026). Review of large YOLOv8 and RT-DETR energy efficiency on edge devices for real-time detection. Scientific Reports.
Chen, X., Xiao, H., Zeng, Z., Zhang, S., & Xiao, R. (2025). Fine-Grained Multivariate Time Series Anomaly Detection via Causal Inference. Knowledge-Based Systems, 114765.
Mejia-Cajicá, F. A., Hernández, C. J. B., Riveill, M., & Henao, J. A. G. (2026). Optimizing Deep Neural Networks via Knowledge Distillation for Efficient and Sustainable Medical Image Classification. Revista UIS Ingenierías, 25(1), 49-70.
Gao, X., Chen, J., & Huang, M. (2026, January). Research on Risk Dependency Structures and Resource Allocation Optimization in New Energy Technology Collaboration within Enterprise Distributed Innovation. In 2026 International Conference on Electrical Engineering, Intelligent Control and Artificial Intelligence (EEICAI) (pp. 43-47). IEEE.
Raza, S. M., Abidi, S. M. H., Masuduzzaman, M., & Shin, S. Y. (2025). Survey on the application with lightweight deep learning models for edge devices. Authorea Preprints.
Xu, S., Ma, Q., Liu, H., & Yue, L. (2026). Continuous Reorganization and Performance Preservation of Agent Memory Structure Under Distributed Change Environments.
Rasheed, S. (2026). Lightweight Deep Learning Models for Face Mask Detection in Real-Time Edge Environments: A Review and Future Research Directions. Machine Learning and Knowledge Extraction, 8(4), 102.
Gao, G., Ma, X., Lu, C., & Gao, R. (2026). Reliability Analysis and Application Research of SMS Communication Systems in Medical Notification Scenarios.
Kim, G. I., Hwang, S., & Jang, B. (2025). Efficient compressing and tuning methods for large language models: A systematic literature review. ACM Computing Surveys, 57(10), 1-39.
Wang, Y., Yin, X., Chen, J., & Wang, Y. (2026). Evidence-Based Study on Low-Burden Digital Phenotyping for Precision Screening of Oral Anti-Obesity Drug Efficacy.
Kim, G. I., Hwang, S., & Jang, B. (2025). Efficient compressing and tuning methods for large language models: A systematic literature review. ACM Computing Surveys, 57(10), 1-39.
Jiao, Y., Wang, A., Zhao, B., & Shi, T. (2026). Quantitative Study on the Construction and Application Effectiveness of Graffiti Wall Painting Teaching Models in Public Space Contexts.
Aruna Gladys, A., Aishwarya, R., Vetriselvi, V., Daphne, S., Baskar, P. N., & Amaithi Rajan, A. (2026). Building Expert Small Models: A Comprehensive Survey of Model Compression, Knowledge Distillation, and Augmented Inference. Archives of Computational Methods in Engineering, 1-42.
Liu, S., Liu, X., & Feng, H. (2025, November). Research on AI-Driven Visual Design and Immersive Interactive Experiences Based on Multimodal Cognition and User. In Proceedings of the 2025 International Conference on Digital Society and Intelligent Computing (pp. 734-740).
Gu, X. (2026, January). Analysis of the Synergistic Role of User Natural Language Feedback and Behavioral Data in Adjusting Recommendation Strategies for Mental Health Platforms. In Proceedings of the 2026 5th International Conference on Big Data, Information and Computer Network (pp. 198-203).
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Software Engineering and Artificial Intelligence Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
