ALF: Autoencoder-Based Low-Rank Filter-Sharing for Efﬁcient Convolutional Neural Networks

ALF: Autoencoder-based Low-rank Filter-sharing for Efﬁcient Convolutional Neural Networks

Alexander Frickenstein1*, Manoj-Rohit Vemparala1*, Nael Fasfous2*, Laura Hauenschild2*, Naveen-Shankar Nagaraja1, Christian Unger1, Walter Stechele2 1Autonomous Driving, BMW Group, Munich, Germany 2Department of Electrical and Computer Engineering, Technical University of Munich, Munich, Germany 1{< firstname >.< lastname >}@bmw.de, 2{< firstname >.< lastname >}@tum.de

Abstract—Closing the gap between the hardware requirements function and the time spent for model exploration render of state-of-the-art convolutional neural networks and the limited learning-based compression policies as a complex method, resources constraining embedded applications is the next big requiring domain expertise. challenge in deep learning research. The computational complexity and memory footprint of such neural networks are typically In this work, we propose the autoencoder-based low-rank daunting for deployment in resource constrained environments. filter-sharing technique (ALF) which generates a dense, com- Model compression techniques, such as pruning, are emphasized pressed CNN during task-specific training. ALF uses the among other optimization methods for solving this problem. inherent properties of sparse autoencoders to compress data Most existing techniques require domain expertise or result in in an unsupervised manner. By introducing an information irregular sparse representations, which increase the burden of deploying deep learning applications on embedded hardware bottleneck, ALF-blocks are used to extract the most salient accelerators. In this paper, we propose the autoencoder-based features of a convolutional layer during training, in order to low-rank filter-sharing technique (ALF). When applied to various dynamically reduce the dimensionality of the layer’s tensors. networks, ALF is compared to state-of-the-art pruning methods, To the best of our knowledge, ALF is the first method where demonstrating its efficient compression capabilities on theoretical autoencoders are used to prune CNNs. After optimization, the metrics as well as on an accurate, deterministic hardware-model. In our experiments, ALF showed a reduction of 70% in network resulting model consists of fewer filters in each layer, and parameters, 61% in operations and 41% in execution time, with can be efficiently deployed on any embedded hardware due to minimal loss in accuracy. its structural consistency. The contributions of this work are summarized as follows: I.INTRODUCTION • Approximation of weight filters of convolutional layers In recent years, deep learning solutions have gained popu- using ALF-blocks, consisting of sparse autoencoders. larity in several embedded applications ranging from robotics • A two player training scheme allows the model to learn to autonomous driving [1]. This holds especially for computer the desired task while slimming the neural architecture. vision based algorithms [2]. These algorithms are typically • Comparative analysis of ALF against learning and rule- demanding in terms of their computational complexity and based compression methods w.r.t. known metrics, as well their memory footprint. Combining these two aspects empha- as layer-wise analysis on hardware model estimates. sizes the importance of constraining neural networks in terms II.RELATED WORK of model size and computations for efficient deployment on embedded hardware. The main goal of model compression Efforts from both industry and academia have focused on techniques lies in reducing redundancy, while having the reducing the redundancy, emerging from training deeper and desired optimization target in mind. wider network architectures, with the aim of mitigating the arXiv:2007.13384v1 [cs.LG] 27 Jul 2020 Hand-crafted heuristics are applied in the field of model challenges of their deployment on edge devices [6]. Compres- compression, however, they severely reduce the search space sion techniques such as quantization, low-rank decomposition and can result in a poor choice of design parameters [3]. and pruning can potentially make CNNs more efficient for Particularly for convolutional neural networks (CNNs) with the deployment on embedded hardware. Quantization aims numerous layers, such as ResNet1K [4], hand-crafted op- to reduce the representation redundancy of model parame- timization is prone to result in a sub-optimal solution. In ters and arithmetic [7]–[9]. Quantization and binarization are contrast, a learning-based policy would, for example, lever- orthogonal to this work and can be applied in conjunction age reinforcement learning (RL) to automatically explore the with the proposed ALF method. In the following sections, we CNN’s design space [5]. By evaluating a cost function, the RL- classify works which use low-rank decomposition and pruning agent learns to distinguish between good and bad decisions, techniques into rule-based and learning-based compression. thereby converging to an effective compression strategy for a A. Rule-based Compression given CNN. However, the effort of designing a robust cost Rule-based compression techniques are classified as hav- *Corresponding Authors ing static or pseudo-static rules, which are followed when compressing a given CNN. Low-rank decomposition or rep- ing, a much needed synergy between efficient CNN design and resentation techniques have been used to reduce the number the target hardware platform is achieved. Tan et al. [15] pro- of parameters (Params) by creating separable filters across pose MNAS-Net, which performs NAS using an RL-agent for the spatial dimension or reducing cross-channel redundancy in mobile devices. Cai et al. [16] propose ProxylessNAS, which CNNs [10], [11]. Hand-crafted pruning can be implemented derives specialized, hardware-specific CNN architectures from based on heuristics to compute the saliency of a neuron. Han an over-parameterized model. et al. [3] determined the saliency of weights based on their Differently, Guo et al. [17] dynamically prune the CNN magnitude, exposing the superfluous nature of state-of-the- by learnable parameters, which have the ability to recover art neural networks. Pruning individual weights, referred to when required. Their scheme incorporates pruning in the as irregular pruning, leads to inefficient memory accesses, training flow, resulting in irregular sparsity. Zhang et al. [18] making it impractical for general purpose computing plat- incorporate a cardinality constraint into the training objective forms. Regularity in pruning becomes an important criterion to obtain different pruning regularity. Bagherinezhad et al. [19] towards accelerator-aware optimization. Frickenstein et al. [12] propose Lookup-CNN (LCNN), where the model learns a propose structured, kernel-wise magnitude pruning along with dictionary of shared filters at training time. During inference, a scalable, sparse algorithm. He et al. [13] prunes redundant the input is then convolved with the complete dictionary filters using a geometric mean heuristic. Although the filter profiting from lower computational complexity. As the same pruning scheme is useful w.r.t. hardware implementations, it is accounts for filter-sharing, this weight-sharing approach is challenging to remove filters as they directly impact the input arguably closest to the technique developed in this work. channels of the subsequent layer. Rule-based compression However, the methodology itself and the training procedure techniques overly generalize the problem at hand. Different are still fundamentally different from one another. CNNs vary in complexity, structure and target task, making it hard to set such one-size-fits-all rules when considering the TABLE I: Classification of model compression methods. No Pre-trained Learning No Extensive different compression criteria. Method\Advantage Model Policy Exploration

B. Learning-based Compression Rule-based Compression: Recent works such as [5] and [14] have demonstrated that Low-Rank Dec. [10], [11] Prune (Handcrafted) [3], [12], [13] it is difficult to formalize a rule to prune networks. Instead, Learning-based Compression: they expose the pruning process as an optimization problem Prune (RL-Agent) [5], [14] to be solved through an RL-agent. Through this process, the NAS [15], [16] Prune (Automatic) [17]–[19] RL-agent learns the criteria for pruning, based on a given cost ALF [Ours] function. Huang et al. [5] represent a CNN as an environment for III. AUTOENCODER-BASEDLOW-RANK FILTER-SHARING an RL-agent. An accuracy term and an efficiency term are The goal of the proposed filter-sharing technique is to combined to formulate a non-differentiable policy to train the replace the standard convolution with a more efficient alter- agent. The target is to maximize the two contrary objectives. native, namely the autoencoder-based low-rank filter-sharing Balancing the terms can eventually vary for different models (ALF)-block. An example of the ALF-block is shown in Fig. 1. and tasks. An agent needs to be trained individually for each Without loss of generality, Al−1 ∈ RHi×Wi×Ci is consid- layer. Moreover, layers with multiple channels may slow down ered as an input feature map to a convolutional layer l ∈ [1,L] the convergence of the agent, rendering the model exploration of an L-layer CNN, where Hi and Wi indicate the height and as a slow and greedy process. In the work proposed by width of the input, and Ci is the number of input channels. He et al. [14], an RL-agent prunes channels without fine- The weights W l ∈ RK×K×Ci×Co are the trainable parameters tuning at intermediate stages. This cuts down the time needed of the layer l, where K and Co are the kernel dimensions and for exploration. Layer characteristics such as size, stride and the number of output channels respectively. operations (OPs), serve the agent as input. These techniques In detail, the task is to approximate the filter bank W in require carefully crafted cost functions for the optimization a convolutional layer during training by a low-rank version K×K×Ci×Ccode agent. The formulation of the cost functions is non-trivial, Wcode ∈ R , where Ccode < Co. The low-rank requiring expert knowledge and some iterations of trial-and- version of the weights Wcode is utilized later in the deployment error. Furthermore, deciding what the agent considers as the stage for an embedded-friendly application. environment can present another variable to the process, mak- In contrast to previous structured pruning approaches [12], ing it challenging to test many configurations of the problem. [13], [19], this method does not intend to alter the structure of As mentioned earlier, each neural network and target task the model in a way which results in a changed dimensionality l Ho×Wo×Co combination present a new and unique problem, for which of the output feature maps A ∈ R , where Ho this process needs to be repeated. and Wo indicate the height and width of the output. This More recently, Neural Architectural Search (NAS) tech- is done by introducing an additional expansion layer [20]. niques have been successful in optimizing CNN models at The advantages are twofold. First, each layer can be trained design-time. Combined with Hardware-in-the-Loop (HIL) test- individually without affecting the other layers. Second, it K×K×Co×Ccode Input: I Autoencoder: (Training only) W and the encoder filters Wenc ∈ R . Mprune Encoder zeroizes elements of W˜ and σ refers to a non-linear W code ae activation function, i.e. tanh(). Different configurations of σae, σinter and initialization schemes are studied in Sec. IV-A. ˜ Wcode = σae(Wcode Mprune) = σae((W · Wenc) Mprune) (3) CNN: W enc Eq. 4 provides the corresponding formula for the recon- ... STE! ALF Block: structed filters Wrec of the decoding stage. The symbol · stands ~ M M for a matrix multiplication and for a Hadamard product W prune l-1 c code M A code respectively. The pruning mask prune acts as a gate, allowing only the most salient filters to appear as non-zero values c STE! Clip i in Wcode, in the same manner as sparse autoencoders. The Conv c STE! decoder must, therefore, learn to compensate for the zeroized c → c i o code filters to recover a close approximate of the input filter bank. Wcode l Decoder Ã c Pruned o Wrec = σae(Wcode · Wdec) (4) c W code dec In order to dynamically select the most salient filters, an ad- Conv 1×1×1×Co co ditional trainable parameter, denoted mask M ∈ R , Wexp is introduced with its individual elements mi ∈ M. By Wrec Al exploiting the sparsity-inducing property of L1 regularization, individual values in the mask M are driven towards zero Autoencoder Training ... during training. Since the optimizer usually reaches values ~ close to zero, but not exactly zero, clipping is performed to Prediction Y Task-related Training zero out values that are below a certain threshold t. Further, Fig. 1: ALF block in training mode showing the forward and Clip the clipping function Mprune = (M, t) = I{|mi| >t}mi backward path for the task-related and autoencoder training. allows the model to recover a channel when required. simplifies the end-to-end training and allows comparison of B. Training Procedure the learned features. For understanding the training procedure of ALF, it is The expansion layer is comprised of point-wise convolutions important to fully-comprehend the training setup, including with weights W ∈ 1×1×Ccode×Co , for mapping the inter- exp R the two player game of the CNN and the ALF-blocks. mediate feature maps after an ALF-block A˜l ∈ Ho×Wo×Ccode , R Task-related Training: The weights W are automatically to the output feature map Al, as expressed in Eq. 1. compressed by the ALF-blocks. Allowing the weights W to re- l ˜l l−1 A = σ(A ∗ Wexp) = σ(σinter(A ∗ Wcode) ∗ Wexp) (1) main trainable, instead of using fixed filters from a pre-trained model, is inspired by binary neural networks (BNNs) [8]. As the point-wise convolution introduces a certain overhead The motivation is that weights from a full-precision CNN with regard to operations and weights, it is necessary to might not be equally applicable when used in a BNN and, analyze the resource demands of the ALF-block compared thus, require further training. Analogously, the weights from to the standard convolution and ensure Ccode < Ccode,max, a pre-trained model might not be the best fit for the instances where Ccode,max denotes the number of filters which have to of W in the filter-sharing use-case. Therefore, training these be removed to attain an efficiency improvement, see Eq. 2. variables is also part of the task optimizer’s job. It’s objective C C K2 C C K2 is the minimization of the loss function Ltask, which is the i o → C = b i o c (2) 2 code,max 2 accumulation of the cross-entropy loss LCE, of the model’s Ccode(CiK + Co) CiK + Co prediction Y˜ and the label Y of an input image I, and A. Autoencoder-based Low-rank Filter-sharing Block the weight decay scaling factor νwd multiplied with the L2 As stated before, the autoencoder is required to identify regularization loss Lreg. correlations in the original weights W and to derive a com- It is important to point out that no regularization is applied pressed representation Wcode from them. The autoencoder is to the instances of either W or Wcode. Even though each only required in the training stage and is discarded in the Wcode contributes to a particular convolution operation with a deployment stage. considerable impact on the task loss Ltask, the task optimizer Referring back to Fig. 1, the autoencoder setup including the can influence this variable only indirectly by updating W . As 1×1×1×Co pruning mask Mprune ∈ R is illustrated. According neither of the variables Wenc, Wdec, or M of the appended to the design of an autoencoder, Eq. 3 gives the complete autoencoder are trained based on the task loss, they introduce expression for calculating the compressed weights Wcode. The a sufficiently high amount of noise, which arguably makes any encoder performs a matrix multiplication between the input further form of regularization more harmful than helpful. In fact, this noise introduced by autoencoder variables does corresponding to the variables updated by the autoencoder affect the gradient computation for updating variable W . As optimizer, are given in Eq. 6. this might hamper the training progress, Straight-Through- ∂Lae ∂Lae ∂Wrec ∂Wcode ∂Mprune Estimator (STE) [8] is used as a substitute for the gradients gM = = · · · ∂M ∂W ∂W ∂M ∂M of Hadamard product with the pruning mask, as well as for rec code prune (6) ∂L ∂W ∂W multiplication with the encoder filters. This ensures that the STE= ae · rec · code gradients for updating the input filters can propagate through ∂Wrec ∂Wcode ∂Mprune the autoencoder without extraneous influence. C. Deployment This measure is especially important in case of the The utility of the autoencoders is limited to the training Hadamard product, as a significant amount of weights in process, since the weights are fixed in the deployment stage. Mprune might be zero. When including this operation in the gra- The actual number of filters is still the same as at the beginning dient computation, a correspondingly large proportion would of the training (Ccode = Co). However, the code comprises of be zeroized as a result, impeding the information flow in the a certain amount of filters containing only zero-valued weights backward pass. Nevertheless, this problem can be resolved by which are removed. For the succeeding expansion layer, fewer using the STE. In Eq. 5, the gradients for the variables gW input channels imply that the associated channels in Wexp are for a particular ALF-block are derived. not used anymore and can be removed as well (Fig.1 pruned ˜ ˜ ∂Ltask ∂Ltask ∂A ∂Wcode ∂Wcode gW = = · · · channels (gray)). After post-processing, the model is densely ∂W ∂A˜ ∂Wcode ∂W˜ ∂Worig code (5) compressed and is ready for efficient deployment. ˜ STE ∂Ltask ∂A = · IV. EXPERIMENTAL RESULTS ∂A˜ ∂Wcode We evaluate the proposed ALF technique on CIFAR-10 [22] Autoencoder Training: Each autoencoder is trained indi- and ImageNet [23] datasets. The 50k train and 10k test images vidually by a dedicated SGD optimizer, referred to as an of CIFAR-10 are used to respectively train and evaluate ALF. autoencoder optimizer. The optimization objective for such an The images have a resolution of 32 × 32 pixel. ImageNet optimizer lies in minimizing the loss function Lae = Lrec + consists of ∼ 1.28 Mio. train and 50K validation images with νprune ·Lprune. The reconstruction loss is computed using the a resolution of (256 × 256) px. If not otherwise mentioned, MSE metric and can be expressed by Lrec = MSE(W, Wrec). all hyper-parameters specifying the task-related training were In the field of knowledge distillation [21], similar terms are adopted from the CNN’s base implementation. For ALF, the studied. The decoder must learn to compensate for the zeroized hyperparameters m = 8 and pr = 0.85 are set. filters to recover a close approximate of the input filter. If a A. Configuration Space Exploration lot of values in Mprune are zero, a large percentage of filters are pruned. To mitigate this problem, the mask regularization Crucial design decisions are investigated in this section. P function Lprune = 1/Co |m| is multiplied with a scaling The aforementioned novel training setup includes a number (m∗(θ−pr )) factor νprune = max(0, 1 − e max ), which decays with of new parameters, namely the weight initialization scheme increasing zero fraction in the mask and slows down the for Wexp,Wenc and Wdec, consecutive activation functions and pruning rate towards the end of the training. In detail, νprune batch normalization layers. For that purpose, Plain-20 [4] is adopts the pruning sensitivity [3] of convolutional layers, trained on CIFAR-10. Experiments are repeated at least twice where m ∈ [1, 10] is the slope of the sensitivity, prmax ∈ [0, 1] to provide results with decent validity (bar-stretching). is the maximum pruning rate and θ = Ccode,zero/Ccode is the Setup 1: The effect of additional expansion layers is studied in zero fraction, with Ccode,zero referring to the number of zero Fig. 2a taking the initialization (He [24] and Xavier [25]), extra filters in Wcode, As a consequence, the regularization effect non-linear activations σinter (i.e. ReLU) and batch normaliza- decreases and fewer filters are zeroized in the code eventually. tion (BN) into account. The results suggest that expansion In other words, the task of Lrec is to imitate W by Wrec by layers can lead to a tangible increase in accuracy. In general, learning Wdec, Wenc and M while Lprune steadily tries to prune the Xavier initialization yields slightly better results than the further channels. He initialization and is chosen for the expansion layer of the The autoencoder optimizer updates the variables Wenc, Wdec ALF blocks. In addition, the incorporation of the BNinter layer and M, based on the loss derived from a forward pass seems to not have perceivable advantages. The influence of through the autoencoder network. Since Mprune and Wcode σinter is confirmed in the next experiment. are intermediate feature maps of the autoencoder, they are Setup 2: In this setup, the ALF block’s pruning mask M not updated by the optimizer. While the gradient calculation is not applied, thus, no filters are pruned. This is applicable for the encoder and decoder weights is straight forward, the to select a weight initialization scheme for Wenc and Wdec gradients for updating the mask M require special handling. (referred as Wae,init) and an activation function σae. In case no ˜l The main difficulty lies in the clipping function which is activation function σinter (blue) is applied to A , the accuracy is non-differentiable at the data points t and −t. As previously higher than with ReLU layers. Based on the results the Xavier mentioned, for such cases the STE can be used to approximate initialization is selected for Wae,init. Moreover, tanh() out- the gradients. The mathematical derivation for the gradients, performs other non-linear activation functions σae. As the TABLE II: Pruned CNNs on CIFAR-10, for Conv layers only. 90 Method Policy Params OPs[106] Acc[%] 85 Plain-20 [4] — 0.27M 81.1 90.5

90 advantages on real hardware execution. Here, we analyze the 85 real advantage of ALF by investigating its implications on 80 hardware. The deterministic execution of CNNs on target Accuracy [%] 75 hardware platforms eases the creation of hardware models with

he|relu highly accurate estimates of normalized latency and energy. rand|tanh he|tanhxavier|tanh he|sigmoid rand|relu xavier|relu rand|sigmoid xavier|sigmoid We use an accurate modeling framework, Timeloop [26], to (b) Config. [W |σ ], σ =none), σ =ReLU. ae,init ae inter inter replicate the execution and scheduling scheme of the Eye- 100 90 riss [27] accelerator. The Eyeriss model used in the experiments consists of a 16 × 16 array of processing elements 80 (PEs). Each PE contains three separate register files (RFs), 80 60 Plain20 ALF(lr=1e-3,t=5e-4) one for each datatype (inputs, weights and outputs). The word- ALF(lr=1e-5, t=1e-4) ALF(lr=1e-3,t=1e-4) ALF(lr=1e-4, t=1e-4) ALF(lr=1e-3,t=5e-5) Accuracy [%] width of all datatypes is fixed to 16-bits. The combined RFs

Remaining Filters [%] 40 70 in a single PE add up to 220 words. The global buffer has a 50 100 150 200 total size of 128KB and can hold output and input datatypes. (c) Training epochs. ALF trained on different lr and t. Weights bypass the global buffer and are directly fed to the Fig. 2: Design space exploration for Plain-20 on CIFAR-10. weight RFs of the PEs. The mapper’s search algorithm follows an exhaustive method with a timeout of 100K iterations and pruning mask M is not active, the regularization property of victory condition of 1K iterations per thread. We simulate the autoencoder is absent causing a noticeable accuracy drop. the compressed configurations achieved by the ALF-block on Setup 3: Five variants of ALF, resulting in different sparsity the Plain-20 and ResNet-20 models. The batch size for all rates, are explored in Fig 2c and compared to the uncom- experiments is set to 16 and the energy values are normalized pressed Plain-20 (90.5% accuracy). The first three variants against the energy cost of a single register file read [27]. −5 −4 −4 differ in terms of the threshold t ∈ {5·10 , 1·10 , 5·10 } The latency is normalized to the bandwidth of a register −3 while the learning rate lrae = 1 · 10 . We observe that the (2 bytes/cycle). pruning gets more aggressive when the threshold t is increased. Fig. 3 shows the advantages of the ALF technique. The The number of non-zero filters remaining are 40.17%, 38.6% results show a trend of high contributions from the RF and 35.71% respectively (see green, pink and blue curves). We memory, particularly in the deeper layers of each CNN. This is select the threshold t = 1e−4 as a trade-off choice between due to the efficient dataflow of the accelerator, which increases sparsity and accuracy. The behaviour of ALF is also accessed the data reuse at the lowest-level memory, reducing the number by changing the learning rate of the autoencoder. The learning −5 −4 of redundant data accesses between the higher memory levels. rate lrae ∈ {1 · 10 , 1 · 10 } is explored in the consecutive variants (see turquoise and purple curves). The number of By analyzing the energy breakdown results of both ALF remaining non-zero filters increases as there are less updates models, an increase in DRAM energy is observed. This is −4 due to additional off-chip data movement introduced by the to the sparsity mask M. In case of lrae = 1·10 (purple), the resulting network has less number of non-zero filters with a expansion layer. This is highlighted in the initial layers, as significant accuracy drop. However, considering the trade-off their inputs are larger. Practically, such codependent layers between accuracy and pruning rate, we choose the learning can be fused with some advanced scheduling techniques [28], rate lr = 1 · 10−3 for the autoencoder. eliminating this overhead. Nevertheless, the strong improve- ae ments in the deeper layers offset this degradation, leading to B. Comparison with State-of-the-Art an overall 29% lower energy consumption and 41% latency CIFAR-10: Tab. II compares ALF against its full precision reduction over the vanilla model. counterpart (ResNet-20) and state-of-the-art pruned models The latency line-plots reveal an anomaly for layer [13], [14]. The ALF model consists of the least number of conv312 on the ALF-Plain-20 compressed model. This com- operations and training parameters compared to other pruning pressed layer requires more energy and execution cycles than works, with an accuracy drop of 1.9% compared to its full the vanilla Plain-20 execution. Taking a closer look, ALF- precision model. Plain-20 utilizes only 8% of the PEs for processing this layer. Hardware-Model Estimates: Improvements in the number of Although ALF is a structured pruning technique, the resulting OPs and Params of a CNN do not always indicate tangible non-sparse model could have reduced parallelism opportunities 6 ·108 ·10 4 2 Register Global Buffer Off-Chip DRAM 3.5 1.8 ALF-ResNet-20 ALF-Plain-20 Plain-20/ResNet-20 1.6 3 1.4 ALF-Plain-20 2.5 ALF-ResNet-20 1.2 Plain-20 / ResNet20 2 1

1.5 0.8 0.6 1 Normalized Energy 0.4 Normalized Latency 0.5 0.2 0 0 CONV1 CONV211 CONV212 CONV221 CONV222 CONV231 CONV232 CONV311 CONV312 CONV321 CONV322 CONV331 CONV332 CONV411 CONV412 CONV421 CONV422 CONV431 CONV432 Fig. 3: Energy consumption breakdown and latency of vanilla and ALF-compressed ResNet/Plain-20 execution. under the restrictions enforced by the row-stationary dataflow REFERENCES on the hardware. These results cannot be intuitively concluded [1] A. Frickenstein, M.-R. Vemparala, J. Mayr, et al., “Binary DAD-Net: by simply considering OPs and Params as metrics for a Binarized driveable area detection network for autonomous driving,” in compression technique. This emphasizes the importance of ICRA, 2020. [2] C. Szegedy, Wei Liu, Yangqing Jia, et al., “Going deeper with convo- hardware-aware validation of pruning advantages, as per- lutions,” in CVPR, June 2015. formed in this subsection. [3] S. Han, J. Pool, J. Tran, et al., “Learning both weights and connections for efficient neural networks,” in NeurIPS, 2015. TABLE III: Benchmarking results on ImageNet, comparing [4] K. He, X. Zhang, S. Ren, et al., “Deep residual learning for image models and methods with regard to MACs and Params. recognition,” in CVPR, 2016. [5] Q. Huang, K. Zhou, S. You, et al., “Learning to prune filters in Method Policy Params OPs [106] Acc [%] convolutional neural networks,” in WACV, 2018. [6] A. Frickenstein, C. Unger, and W. Stechele, “Resource-Aware Optimiza- SqueezeNet [20] — 1.23M 1722 57.2 % tion of DNNs for Embedded Applications,” in CRV, 2019. GoogleNet [2] — 6.80M 3004 66.8% [7] S. Vogel, M. Liang, A. Guntoro, et al., “Efficient hardware acceleration ResNet-18 [4] — 11.83M 3743 69.8 % of CNNs using logarithmic data representation with arbitrary log-base,” Pruned ResNet-18: in ICCAD, 2018. LCNN [19] Automatic – 749 62.2 % [8] I. Hubara, M. Courbariaux, D. Soudry, et al., “Binarized Neural Net- FPGM [13] Handcrafted – 2178 67.8 % works,” in NeurIPS, 2016. AMC [14] RL-Agent 8.9M 1874 67.7 % [9] N. Fasfous, M.-R. Vemparala, A. Frickenstein, et al., “Orthruspe: ALF (ours) Automatic (t = 10−4) 4.24M 1239 64.3 % Runtime reconfigurable processing elements for binary neural networks,” in DATE, 2020. [10] X. Zhang, J. Zou, K. He, et al., “Accelerating very deep convolutional ImageNet: Tab. III compares ALF against other pruned networks for classification and detection,” TPAMI, 2016. models, i.e. LCNN [19] and FPGM [13]. For comparison, [11] O. A. Malik and S. Becker, “Low-rank tucker decomposition of large GoogleNet [2], SqueezeNet [20] and ResNet-18 [4] are also tensors using tensorsketch,” in NeurIPS, 2018. [12] A. Frickenstein, M. R. Vemparala, C. Unger, et al., “DSC: Dense-sparse detailed in the table. Compared to SqueezeNet, GoogleNet and convolution for vectorized inference of cnns,” in CVPR-W, 2019. ResNet, ALF requires ×1.4, ×2.4, ×3.0 less OPs, respectively. [13] Y. He, P. Liu, Z. Wang, et al., “Filter pruning via geometric median for Moreover, ALF outperforms GoogleNet and ResNet w.r.t. the deep convolutional neural networks acceleration,” in CVPR, 2019. [14] Y. He, J. Lin, Z. Liu, et al., “AMC: Automl for model compression and parameters. Compared to other pruned models, ALF lies on the acceleration on mobile devices,” in ECCV, 2018. pareto-front for the number of parameters, the operations and [15] M. Tan, B. Chen, R. Pang, et al., “MnasNet: Platform-aware neural the accuracy. On one hand, LCNN obtained a very competitive architecture search for mobile,” arXiv:1807.11626, 2018. [16] H. Cai, L. Zhu, and S. Han, “ProxylessNAS: Direct neural architecture computational complexity (−40%), at the cost of an accuracy search on target task and hardware,” in ICLR, 2019. degradation of −2.1% compared to ALF. On the other hand, [17] Y. Guo, A. Yao, and Y. Chen, “Dynamic network surgery for efficient AMC and FPGM gain +3.4%, +3.5% accuracy improvement, DNNs,” in NeurIPS, 2016. [18] T. Zhang, S. Ye, K. Zhang, et al., “Structadmm: A systematic, high- but has +51%, +76%, more OPs than ALF respectively. efficiency framework of structured weight pruning for dnns,” 2018. [19] H. Bagherinezhad, M. Rastegari, and A. Farhadi, “LCNN: Lookup-based V. CONCLUSION convolutional neural network,” in CVPR, 2017. [20] F. N. Iandola, S. Han, M. W. Moskewicz, et al., “SqueezeNet: AlexNet- Pruning is a promising compression technique, ranging from level accuracy with 50× fewer parameters and <0.5MB model size,” rule-based to handcrafted and learning-based methods. Most arXiv:1602.07360, 2016. approaches require either a pre-trained model or an extensive [21] Y. Tian, D. Krishnan, and P. Isola, “Contrastive Representation Distil- lation,” in ICLR, 2020. model exploration, making the compression a time consuming [22] A. Krizhevsky, V. Nair, and G. Hinton, “Cifar-10 (canadian institute for process. In this paper, we propose the autoencoder-based low- advanced research),” rank filter-sharing technique (ALF) to dynamically prune a [23] J. Deng, W. Dong, R. Socher, et al., “ImageNet: A Large-Scale Hierar- chical Image Database,” in CVPR, 2009. given CNN during task-specific training. ALF employs a [24] K. He, X. Zhang, S. Ren, et al., “Delving deep into rectifiers: Surpassing sparse autoencoder to approximate the weight filters of the human-level performance on imagenet classification,” in ICCV, 2015. CNN for an embedded-friendly application. The novel method [25] X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in PMLR, 2010. is applied to computer vision tasks, CIFAR-10 and ImageNet. [26] A. Parashar, P. Raina, Y. S. Shao, et al., “Timeloop: A systematic Comparison to state-of-the-art methods is performed on well approach to dnn accelerator evaluation,” in ISPASS, 2019. known theoretical metrics (OPs and Params), as well as an [27] Y. Chen, J. Emer, and V. Sze, “Eyeriss: A spatial architecture for energy- efficient dataflow for convolutional neural networks,” in ISCA, 2016. analysis on hardware-model estimates, i.e. normalized latency [28] M. Alwani, H. Chen, M. Ferdman, et al., “Fused-layer CNN accelera- and energy. tors,” in MICRO, 2016.