Hybrid Partitioning for Embedded and Distributed CNNs Inference on Edge Devices - Université Grenoble Alpes Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Hybrid Partitioning for Embedded and Distributed CNNs Inference on Edge Devices

Partitionnement hybride pour une inférence distribuée et embarquée des CNNs sur les équipements en bordure de réseau.

Nihel Kaboubi
  • Fonction : Auteur
  • PersonId : 1222096
Loïc Letondeur
  • Fonction : Auteur
  • PersonId : 1022843
Thierry Coupaye
Frédéric Desprez
Denis Trystram
  • Fonction : Auteur
  • PersonId : 1129029

Résumé

Convolutional Neural Networks (CNNs) and Deep Neural Networks (DNNs) are ubiquitously utilized in many Internet of Things applications, especially for real-time image-based analysis. In order to cope with concerns such as resiliency, privacy and near real time analysis, these models must be deployed on edge devices. Particularly for large models, the large number of parameters becomes a bottleneck for the inference process because edge devices are resource constrained, subjects to failures and/or hardware faults. New solutions to cope with these issues are required. This paper proposes a hybrid partitioning strategy, architecture and implementation (called HyPS), which identifies the best positions in the model structure to split the network structure into small partitions that fit resources constraints of edge devices noticeably by decreasing instantaneous memory needs. The generated partitions consume less memory than the original network and each partition can be processed almost separately, resulting in new ways to process CNN's execution at the edge. Thanks to this partitioning strategy, large CNNs inference can be run without modifying the main model architecture. The proposed approach is assessed on the well-known neural network structure of VGG16 for image classification. The results of the experimental campaign show that the partitioning method allows for the successful inference of large models on devices with limited overhead and high accuracy.
Fichier principal
Vignette du fichier
Conference_paper_2022HAL.pdf (1.75 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03967388 , version 1 (08-03-2023)

Licence

Paternité

Identifiants

  • HAL Id : hal-03967388 , version 1

Citer

Nihel Kaboubi, Loïc Letondeur, Thierry Coupaye, Frédéric Desprez, Denis Trystram. Hybrid Partitioning for Embedded and Distributed CNNs Inference on Edge Devices. ANTIC 2022 - International conference on advanced network technologies and intelligent computing, Department of Computer Science Institute of Science Banaras Hindu University, Dec 2022, Varanasi, India. ⟨hal-03967388⟩
50 Consultations
84 Téléchargements

Partager

Gmail Facebook X LinkedIn More