hh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
MOHAQ: Multi-Objective Hardware-Aware Quantization of recurrent neural networks
Halmstad University, School of Information Technology.ORCID iD: 0000-0002-4674-3809
Umeå University, Umeå, Sweden.
KTH University, Stockholm, Sweden.
Halmstad University, School of Information Technology.ORCID iD: 0000-0002-4932-4036
Show others and affiliations
2022 (English)In: Journal of systems architecture, ISSN 1383-7621, E-ISSN 1873-6165, Vol. 133, article id 102778Article in journal (Refereed) Published
Abstract [en]

The compression of deep learning models is of fundamental importance in deploying such models to edge devices. The selection of compression parameters can be automated to meet changes in the hardware platform and application. This article introduces a Multi-Objective Hardware-Aware Quantization (MOHAQ) method, which considers hardware performance and inference error as objectives for mixed-precision quantization. The proposed method feasibly evaluates candidate solutions in a large search space by relying on two steps. First, post-training quantization is applied for fast solution evaluation (inference-only search). Second, we propose the ”beacon-based search” to retrain selected solutions only and use them as beacons to estimate the effect of retraining on other solutions. We use speech recognition models on TIMIT dataset. Experimental evaluations show that Simple Recurrent Unit (SRU)-based models can be compressed up to 8x by post-training quantization without any significant error increase. On SiLago, we found solutions that achieve 97% and 86% of the maximum possible speedup and energy saving, with a minor increase in error on an SRU-based model. On Bitfusion, the beacon-based search reduced the error gain of the inference-only search on SRU-based models and Light Gated Recurrent Unit (LiGRU)-based model by up to 4.9 and 3.9 percentage points, respectively.

Place, publisher, year, edition, pages
Amsterdam: Elsevier, 2022. Vol. 133, article id 102778
Keywords [en]
Simple recurrent unit, Light gated recurrent unit, Quantization, Multi-objective optimization, Genetic algorithms
National Category
Embedded Systems
Identifiers
URN: urn:nbn:se:hh:diva-48679DOI: 10.1016/j.sysarc.2022.102778ISI: 000892114100006Scopus ID: 2-s2.0-85141919627OAI: oai:DiVA.org:hh-48679DiVA, id: diva2:1712924
Available from: 2022-11-23 Created: 2022-11-23 Last updated: 2023-08-21Bibliographically approved
In thesis
1. Deep Learning on the Edge: A Flexible Multi-level Optimization Approach
Open this publication in new window or tab >>Deep Learning on the Edge: A Flexible Multi-level Optimization Approach
2022 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Recent advances in Deep Learning (DL) research have been adopted in a wide variety of applications, including autonomous driving, AI in health care, and smart homes. In parallel, research in high-performance embedded computing has resulted in advanced hardware platforms that offer enhanced performance and energy efficiency for demanding computations. However, the high demands of DL models for computational and memory resources are still a challenge for embedded computing. Algorithmic optimizations can be used to reduce the computational and memory requirements of DL models. Hardware implementations and architectures can also be tuned to support DL applications’ requirements. This thesis identifies that insufficient coordination between hardware implementations and models’ optimizations limits the efficiency of the resulting implementations. In addition, the implementation methods themselves suffer from poor flexibility in adapting to changes in the model and application constraints. The overarching theme of this thesis is to study and propose methods for the efficient and flexible implementation of DL models on embedded platforms. The work in this thesis bridges the gap between DL models’ algorithmic optimizations and embedded platforms’ hardware-specific optimizations, and investigates the features that need support from DL domain-specific architectures. In addition, a method for multi-objective quantization of DL models is proposed to address both the model error and platform performance metrics. Post-training optimization techniques are employed to facilitate the multiobjective optimization of the models because they do not require retraining after model optimization. This thesis also reviews the optimization methods that are known to have been applied to improve the implementation efficiency of DL models. It highlights the most fruitful optimizations found in existing, highly efficient implementations, and applies them in the proposed methods. A method for mapping Convolution Neural Networks (CNN) on Epiphany, a manycore architecture, is proposed and evaluated. A method for quantization and approximation for RNN models in a post-training fashion is also proposed, and evaluated on four RNN models. The proposed quantization method is used in a hardware-aware multi-objective optimization for RNN models to be deployed on SiLago and Bit fusion architectures.

Place, publisher, year, edition, pages
Halmstad: Halmstad University Press, 2022. p. 65
Series
Halmstad University Dissertations ; 95
National Category
Embedded Systems
Identifiers
urn:nbn:se:hh:diva-48680 (URN)978-91-89587-03-8 (ISBN)978-91-89587-02-1 (ISBN)
Public defence
2022-12-15, Halda, Visionen, Kristian IV:s, väg 3, Halmstad, 11:58 (English)
Opponent
Supervisors
Note

Kompletteras med LibrisID när tillgängligt.

Available from: 2022-11-24 Created: 2022-11-23 Last updated: 2022-11-24Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Rezk, NesmaUl-Abdin, ZainAksoy, Eren

Search in DiVA

By author/editor
Rezk, NesmaUl-Abdin, ZainAksoy, Eren
By organisation
School of Information Technology
In the same journal
Journal of systems architecture
Embedded Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 50 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf