IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1

A Survey of Memristive Threshold Logic Circuits Akshay Kumar Maan, Deepthi Anirudhan Jayadevi, and Alex Pappachen James, Senior Member, IEEE

Abstract—In this paper, we review the different memristive Threshold logic (TL) itself was first introduced by Warren threshold logic (MTL) circuits that are inspired from the synaptic McCulloch and Walter Pitts in 1943 [17] in the early model action of flow of neurotransmitters in the biological brain. of an artificial neuron based on the basic property of firing Brain-like generalisation ability and area minimisation of these threshold logic circuits aim towards crossing the Moore’s law of the biological neuron. The logic computed the sign of the boundaries at device, circuits and systems levels. Fast - weighted sum of its inputs. Modelling a neuron as a threshold ing memory, signal processing, control systems, programmable logic gate (TLG) that fires when input reaches a threshold has logic, image processing, reconfigurable computing, and pattern been the basis of the research on neural networks (NNs) and recognition are identified as some of the potential applications of their hardware implementation in standard CMOS logic [18]. MTL systems. The physical realization of nanoscale devices with However, to continue geometrical scaling of semiconductor memristive behaviour from materials like TiO2, ferroelectrics, silicon, and polymers has accelerated research effort in these components as per Moore’s Law (and beyond), semiconductor application areas inspiring the scientific community to pursue industry needs to investigate a future beyond CMOS, and design of high speed, low cost, low power and high density designs beyond the fetch-decode-execute paradigm of von neuromorphic architectures. Neumann architecture. Memristive devices lend themselves Index Terms—, Threshold Logic, Neural Circuits, nicely to this “neuromorphic” computing paradigm making use Neurocomputing of very-large-scale integration (VLSI) systems to mimic neuro- biological architectures present in the nervous system that need I. INTRODUCTION no initialising software, run on negligibly low power, and EARCH for scalable hardware architectures for emulating are able to perform many cognitive tasks rapidly, effortlessly neuron circuits in the biological brain is one of major and in real-time. Memristive threshold logic is thus a step Sresearch areas to realising silicon based artificial intelligence in the direction of building neuromorphic architectures while devices. The implementation of such circuits in standard de- overcoming the design and scalability issues presented by vices such as CMOS has limitations of design and scalability. CMOS NNs. Memristive devices provide an interesting alternative not only In the following section, we give a brief background of by way of their high packing density, but also the way they memristive systems and threshold logic. Section III provides can enable a profoundly different approach to large-scale a review of various implementations of Threshold computing inspired by the principle of firing of neurons in Logic (MTL) detailing the different architectures employed to the biological brain. In biological brain a neuron fires only achieve thresholding behaviour with memristive circuits and if the total weight of the synapses that receive impulses in a their applications, followed by discussion and open problems short period (called the period of latent summation) exceeds in section IV. a threshold. A threshold logic implemented in memristive devices thus offers (a) a synaptic action in that the weight (memristance of the device) can be incrementally modified by II. BACKGROUND controlling charge or flux through it, and (b) a thresholding A. Memristive Devices and Systems system that governs the firing of the output. Memristor, for example, is one such device [1]–[6] in Memristors are two-terminal circuit elements exhibiting the ‘more-than-Moore (MtM)’ era of device integration [7]– passivity property characterized by a relationship between the ( ) ( ) [11], that has a high packing density with features like charge q t and the flux-linkage φ t . The memristance (M) of low operational voltage, non-volatility, and high switching memristor has been defined by the relation M = dφm/dq and speed. Memristor (a portmanteau of MEMory ) is is expressed in Wb/C, or ohm. The characteristic behaviour currently being described as the nanoscale device capable of of a memristor was proposed by L. O. Chua in 1971 [19]. He emulating synaptic behaviour in the brain [12]–[14] because developed a circuit model built with at least 15 it can ‘remember’ the charge flown through it by changing its and other passive elements [16] to emulate the behaviour resistance. A wide range of device models and applications of a single memristor. The memristor model satisfied the have sprung up in the recent years since Hewlett-Packard (HP) passivity criterion and was characterized by a monotonically increasing curve [20], [21]. Fig. 1(a) depicts the Labs in 2008 [15], [16] described the behaviour of physical φ − q relationship between the four fundamental circuit elements TiO2-TiO2 devices by a memristor model. −x (Resistor (R), (C), (L) and Memristor A.P. James is an Associate Professor in Electrical and Electronic Engi- (M)) and the typical voltage-current behaviour of a TiO2- neering Nazarbayev University. Corresponding author email: [email protected]; TiO2−x memristive device. In 1976, L. O. Chua et al. [22] www.biomicrosystems.info A.J. Deepthi is a research staff member with Enview R&D labs. defined any nonlinear dynamical systems with memristors as A.K. Maan is a research staff at Griffith University, Australia Memristive systems . The memristive system was found to

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2

(a) (b)

Fig. 1. (a)The four fundamental two-terminal circuit elements: Resistor (R), Capacitor (C), Inductor (I) and Memristor (M). R, C, L and M can be functions of the independent variable in their defining equations, yielding nonlinear elements. For example, a charge-controlled memristor is defined by a single- valued function M(q).(b)Voltage-Current behaviour in T iO2 − T iO2−x memristor model [16]

possess memory and behaved like resistive devices endowed with a variety of dynamic characteristics. The memristive Fig. 2. A taxonomy of memrisors based on memristive behaviour observed systems were incapable of energy discharge. Yet, they were in fabrication materials [60] found to exhibit small-signal inductive or capacitive effects without introducing phase shift between the input and output waveforms. in many nanoscale electronic devices that involved the motion A physical solid-state device using nanoscale Titanium diox- of charged atomic or molecular species; in particular certain ide films exhibiting the memristive properties was invented in titanium dioxide cross-point [16], [19]–[21], [29]– 2008 by a team from HP Labs [15]. The memristive device was [55]. In 2008, G. Chen [56] recounted the impact of inven- found to be equipped with an ability to function like synapses tion of memristive devices in technology as recognition to in a biological brain [12]. The research team proposed a the promising discovery of memristors by L.O. Chua. The crossbar architecture which was a fully connected mesh of modeling aspect of engineering that involved memristors and perpendicular platinum with memristive switches made memristive systems were discussed in-depth by I. C. Goknar in of TiO2-TiO2−x to connect any two crossing wires. These 2008. In 2010, the HP Labs introduced practical memristors nanoscale switches were found to exhibit Lissajoux voltage- of size 3nm by 3nm found operable at a switching time of current behaviour shown in Fig. 2 (b). Till then, similar 1 ns (∼ 1 GHz) [57]. Memristors have thus paved way for dynamics were only common in relatively larger devices further miniaturization of integrated electronic circuits [58], [23], [24]. The memristive dynamics mapped by the pinched- promising a future in technology beyond Moore’s Law [59]. hysteresis loop [25] explains switching behaviour [26] of the The memristive systems based on device properties can be device beginning with a high resistance. As the applied voltage broadly grouped into those based on molecular and ionic thin increases, the charge flow inside the device increases slowly films, and into those based on spin and magnetic effects. An at first owing to the drop in resistance value. This behaviour overall taxonomy of the memristors based on their memristive is followed by a rapid increase in the device current up to material properties [60], [61] has been presented in Fig. 2. the maximum increase in applied voltage. When the voltage The Ionic Thin Film and Molecular memristors mostly rely was decreased, the current decreased more slowly resulting in on different material properties of the thin film atomic lat- an on-switching loop. The off-switching loop was observed tices that display hysteresis below the application of charge. when the voltage turned negative leading to the increase in These memristors can be classified into four distinct groups resistance of the device. It was observed that the resistance of viz. Metal Dioxide memristors, Ionic or Polymeric memris- the film as a whole was dependent on how much charge had tors, Resonant Tunneling memristors and Manganite been passed through it in a particular direction. In addition, memristors. The metal dioxide memristors, titanium oxide in the resistance value of the film was found to be reversible particular, are broadly explored for designing and modeling. on changing the direction of current [19], [27], [28]. The The ionic or polymeric memristors utilize dynamic doping of HP device was considered as a nanionic device owing to its inorganic die-electric type or polymer materials. In this type property of displaying fast conduction at nanoscale. of memristors, the ionic charge carriers move all over the In 2008, D. Strukov et al. [16] presented analytical re- solid state structure. The resonant tunneling diode memris- sults which showed that the memristance arises naturally tors use specially doped quantum well of the space in nanoscale systems in which solid-state electronics and layers between the sources and drain regions. The manganite ionic transport are coupled under an external bias voltage. memristors use a substrate of bilayer oxide films based on This paved way to set the foundation for understanding a manganite as opposed to titanium dioxide memristors. wide range of hysteretic current-voltage behaviour observed The magnetic and spin-based memristors are opposite to ionic

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 3

[46], and to the various very-large-scale-integration (VLSI) implementations in last two decades. These implementations are too many to be included in this survey, but they can be broadly divided into CMOS, capacitive, output-wired inverters, floating-gate and psuedo-nMOS solutions, each catering to a specific need or one or more performance parameters such as power dissipation, noise margins, sensitivity to process variations and fan-in. The survey in [63] concludes with an interesting observation. The authors provided a list of potential applications of threshold logic apart from hardware neurons, such as “ ... general microprocessors, DSPs, and cores where addition, multiplication, and multiply-accumulate, ...encryption/decryption, ...convolution/deconvolution, ...and compression/decompression [66].” The fundamental reason, the authors point out, is that TLGs need full custom design and have been in direct competition with Boolean gates and a lot of research effort has been spent on improving Fig. 3. A modified taxonomy of memristior circuit applications from that boolean logic gates since the 1970s. In this survey we reported in [62] to incorporting emerging applications such as in imaging and speech. will identify approaches and techniques on how to use threshold logic based on memristive systems to build different logic functionalities, which will we hope direct some of nanostructure and molecule based systems. This category of the future research effort in this direction. The authors memristive devices rely solely on the degree of electronic spin in [63] offer another insight: “Lastly, because nano (and and its polarization. This memristor type can be categorized reconfigurable) computing will probably get center-stage further into two types viz. Spintronic memristors and Spin positions in the (near) future, TL will surely benefit from that.” Torque Transfer (STT) memristors. In spintronic memristors, This observation is another motivation of the current survey the route of spin of electrons changes the magnetization state because memristive threshold logic, as we shall see, is capable of the device which consequently changes its resistance. In not only of emulating synaptic action in hardware, but it Spin Torque Transfer (STT) memristors, the comparative mag- does so in a highly dense, low power and scalable architecture. netization position of the two electrodes affect the magnetic state of a tunnel junction which in turn changes its resistance. Since HP’s announcement, interest in memristive electronics III. THRESHOLD LOGIC GATES AND APPLICATION and their applications has grown rapidly with several research CIRCUITS USING MEMRISTORS groups demonstrating memristive behavior in different devices and systems. Fig. 3 shows the taxonomy of memristor applica- The resistive switching nature exhibited by memristors tions proposed by Pinaki Mazumder [62] in 2012. The devices [67]–[69] has been utilized in realizing brain-inspired reported in literature all have different underlying physics threshold logic computing circuits. Here we present a review governing their memristive behaviour. of the different implementations of Memristor Threshold Logic (MTL) which realizes specific function pertaining to B. Threshold Logic the application under consideration. An early implementation of threshold logic using memristors The first simplified mathematical model of the biological was proposed by Rajendran J. et al. [70] in 2009. The neuron was introduced by McCulloch and Pitts [17] in 1943 proposed Programmable Threshold Logic Array (PTLA) in the form of the threshold logic gate (TLG). It computed the using memristors exhibited multiple levels of resistance sign of the weighted sum of inputs: to provide weighted inputs to each threshold gate. The

f (x1, x2, ...xn) = sgn(w1x1 + w2x2 + ... + wnxn) distinguishing feature of PTLA was the combination of n Negative Differential Resistance (NDR) based molecular = sgn(Σ wixi − T ) (1) i=1 switch and a multi-level resistance memristor leading to the where wi are the synaptic weights associated to inputs xi and programmable threshold gate. The design was extended to T the threshold that the gate needs to meet to fire, and n the implementation of an image classifier which classifies a is the fan-in of the TLG [63]. Since then a large number of 3x5 image into a rectangle or a triangle using three 5-input hardware implementations have been reported in literature, a TLG in the first level and a Goto pair based majority voter comprehensice survey of which can be found in [63]. The as shown in Fig. 4. A Goto pair or ‘twin’ [71] is a series implementations range from early electromechanical (tubes, connection of two tunnel negative resistance diodes that has motors and clutches) “neurocomputer” in 1951 [64], to the been used as a majority voting circuit. When the operating perceptron in 1957 [65], to the electrically ad- voltages are just enough to bring the pair to either of the justable memistor based adaptive linear element (ADALINE) stable states (0 or 1), a majority circuit can be built as in [71]

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 4

Fig. 6. An island of threshold gates [72]. Each column corresponds to a single threshold gate and the number of rows determine the fan-in of the threshold gate. A single threshold gate had been shown within a dotted box. IL refers to isolator, CM refers to current mirror, PE refers to period extender and PS refers to pulse shaper.

Fig. 4. TLG Implemented Using Memristors and Goto pair [70]

Fig. 7. A cascaded architecture using two rows of islands [72]. The unshaded box corresponds to an island of threshold gates. The lightly shaded box corresponds to an interconnection network. Fig. 5. Circuit diagram of a 3-input threshold gate using memristors as weights. (1) is a current mirror to prevent reverse flow of current, (2) is an isolator to prevent loading, (3) is a period extender to retain input pulse period, and (4) is a pulse shaper to retain memristance. [72] which provided signal and memristance restoration. The programming circuitry consisted of a pulse generator to program the memristor. The memristance property utilized in to output the majority value among the inputs. the architecture was able to reduce the power consumption and In 2010, Rajendran J. et al. [72] implemented different effective area footprint to approximately 75% in comparison Boolean functions by programming the weights at the input with the CMOS based LUTs (Look-up Tables). The delay of gates in the proposed threshold gate-array architecture. penalty of the programmable threshold gates was found The key idea was to use the memristors as weights of the as almost 12 times the delay of the 4-input LUTs. The inputs to a threshold gate as shown in Fig. 5. Additional memristors were utilized as weights in the realization of low- circuits were employed to avoid the problems arising from power Field Programmable Gate Arrays (FPGAs) using the straightforward implementation of the memristor based threshold logic in 2012 by Rajendran J. et al. [73]. The threshold gate, such as reverse current from output to input proposed Memristive Threshold Logic (MTL) gate shown in of the circuit, loading of the current comparator by the next Fig. 8 utilized the multiresistance property of memristors stage, partial restoration of the input and temporal change to implement the Boolean functions, which are the subsets in memristance. To avoid these problems, the authors added of threshold functions. The programmable threshold gates circuitry, namely: 1) a 2- current mirror on each consume less power and area when compared to their input line, 2) a 2-transistor isolation circuit, 3) a period implementations using CMOS, LUTs, and CTL gates. The extender circuit to restore the negative pulse immediately energy performance and area overhead of threshold logic following the positive pulse to represent the logic 1, 4) a six implementation were evaluated using the CAD tools from transistor pulse shaper circuit to generate a positive pulse Cadence and Berkeley SIS logic synthesis tool. In addition, followed by a negative pulse to represent a logic 1. In general, the essential countermeasures to combat the issues of using a N-input threshold gate required (2 x n) + 18 transistors. memristors in logic circuits in presence of memristance drift The proposed 3x3 crossbar based island architecture shown in like memristor refresh were proposed. Fig. 6 consisted of three 3-input threshold gates. A cascaded Another circuit design capable of realizing four different architecture formed using these threshold gate islands logic operations by changing the resistance of the memristive connected to each other through an interconnection network devices was proposed by T. Tran et al. in 2012 [74]. The shown in Fig. 7. prevented signal degradation in successive design exploration of reconfigurable Threshold Logic Gates stages with the help of the period extender and pulse shaper (TLG) implemented using Silver-chalcogenide memristive

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 5

Fig. 11. Fully connected Perceptron with training circuitry [76]

Fig. 8. A 3-input MTL gate with memristors as weights and Iref as the threshold [73] [77] was considered to be the unit sized configurable logic block (CLB). The proposed TTGA system was designed and implemented from trainable perceptron based threshold gates [77] in Cadence Spectre with 45 nm Berkeley predictive technology models (PTM) for the CMOS circuitry. The proposed training methodology based on the stochastic gradient descent training technique was capable of efficiently reconfiguring the memristive synapses in a trainable threshold gate array (TTGA). The technique overcame many circuit level issues such as parasitics and device variations that

Fig. 9. Circuit implementation with memristive devices with circuitry for configured memristive devices. The training and performance negative weights [74] results for the TTGA and the 1T1M (1 Transistor and 1 Memristor) multilevel memristive memory [42], [78], [79]

showed that the TTGA (minimum memristance values, i.e., devices [75] combined with CMOS circuits was presented OR pretrain) to be the most energy-delay and area effective in the work. The proposed re-programmable TLG shown in solution. The methodology was observed as robust to the Fig. 9 was realized in discrete hardware using a summing unpredictability of CMOS and nano circuits with decreasing op-amp circuit with memristive devices implementing the technology sizes. weights (wi). A feedback-based adaptive programming circuit shown in Fig. 10 was developed to program the In 2013, L.Gao et al. [80] proposed that a programmable individual memristive devices to predetermined resistance threshold logic gate can be implemented using a hybrid values to create each logic operation. According to the CMOS/memristor logic. The proposed linear threshold gate analysis carried out in Matlab-Simulink/Cadence, the TLGs (LTG) shown in Fig. 12 was comprised of memristive devices with Ag-Ch memristive devices had a fan-in of >10, in a universal gate which is much more powerful than similar switched at >1 GHz speed and dissipated lower static power fan-in single NAND or NOR gates. The memristive devices than a corresponding CMOS implementation. In 2012, H. implemented a ratioed diode-resistor logic wherein several Manem et al. [76] added another contribution to the field of (N) memristive devices connected in parallel to a single neuromorphic computing using memristor based threshold pull-down resistor RL, so that a dynamic range (i.e. the gates by introducing a variation-tolerant training methodology H L ratio RON /R ON) dictated the number of different Boolean to efficiently reconfigure memristive synapses in a Trainable functions the LTG could implement. This configuration made Threshold Gate Array (TTGA) system. The TTGA consisted LTG in-field configurable and potentially very compact. of arrays of trainable threshold gates [73] interleaved with The concept was experimentally verified by implementing switch blocks to enable the realization of the reconfigurable a 4-input symmetric linear threshold gate with an integrated logic fabric as shown in Fig. 11. A single layer 4-input circuit CMOS flip-flop, silicon diodes, and Ag/a-Si/Pt perceptron (trainable threshold gate) capable of implementing memristive devices. For their effectiveness in high-throughput all linearly separable 1, 2, 3 and 4 input Boolean functions pipelined circuits, the CMOS flip-flop was preferred over a CMOS gate (and possibly an inverter) to restore the output voltage to a clear binary. The proposed implementation claimed to be more robust as compared to approaches suggested in [81] and [70], since it does not rely on changing the state of memristive devices during the logic operation. In addition, each memristive device in the suggested threshold logic in [70], [72], [82] was served by a CMOS-based current mirror circuit, which leads to considerable overhead. However, the memristive devices in the proposed hybrid architecture Fig. 10. Circuit implementation for programming memristive devices [74] could be integrated into crossbar circuits which are patterned

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 6

(a)

Fig. 14. Circuit for 2-fan in DRTL gate [85]

(b)

Fig. 12. (a) Main idea of an LTG implemented with memristors and CMOS D flip-flop.(b) IV characteristics of memristive devices (schematically represented). The shaded area on panel (b) shows the range of possible intermediate states utilized for the implementation of the threshold gate [80] Fig. 15. Schematic for interconnect design for DRTL using resistive crossbar memory [85]

low voltage at OB . The inverse voltage allocation at OB and O happens when IB is greater than IA. The approach was found to outperform the combinational design [73] proposed earlier in terms of performance and energy consumption on three, four, and five input benchmark threshold functions. The percentage improvement were summarized as 77% for the delay, 50% for the energy and 88% for the Energy Delay Product (EDP). The delay and energy consumption

using the proposed implementation using Berkeley Predictive Fig. 13. Current-Mode Memristor based Threshold Logic (CMMTL) [84] Technology Models (PTM) for 45nm CMOS transistors were 0.44ns and 3.410, respectively. The proposed CMMTL is

capable of implementing all possible weight configurations above the CMOS layer (e.g., similar to the proposed (3-D) i.e., positive weighted gates, negative weighted gates, positive CMOL concepts [43], [83], such that the threshold gate area and negative weighted gates. The proposed method scales well would be determined mostly by the CMOS flip-flop. The over [73] as indicated by the increase in average EDP with method required only some minor additional circuitry for the increase in number of inputs to the threshold function. programming the memristive devices. Both the current mode logic of [84] and the combinatorial A new clocked design that combines memristors and CMOS design of [73] used sense to restore output voltage transistors to implement current mode logic threshold logic swing as opposed to CMOS D-flip-flops of [80]. gates was presented by C.B. Dara et al. [84] in 2013. A Dynamic Resistive Threshold-Logic (DRTL) design based The novel Current-Mode Memristor based Threshold Logic on non-volatile programmable resistive memory elements (CMMTL) shown in Fig. 13 consisted of two parts, a for reconfigurable computing was proposed by M. Sharad et differential part and a sensor part, the differential being a al. [85] in 2013. In DRTL shown in Fig. 14, the resistive series combination of a memristor and NMOS and having two memory elements are used to implement the weights and further divisions comprising a negative threshold and positive the thresholds, while a compact dynamic CMOS latch is inputs, and a positive threshold and negative inputs. Inputs used for the comparison operation. The multiple stages in a are applied to the positive/negative parts of the differential DRTL design could be connected using energy-efficient low part. When for a given input configuration the current IA swing programmable interconnect networks based on resistive through node A is greater than the current IB through node switches. The dynamic operation of the CMOS latches that B, the voltage at output node O rises faster than the voltage at minimizes static-power dissipation [43], [72], [74], [86] along output node OB resulting in high voltage at the node O and with the memory-based compact logic and interconnect design

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 7

Fig. 17. Circuit diagram of the resistive divider boolean logic cell that consists of a two input resistive divider and a variable threshold CMOS inverter [88]

Fig. 16. Neural logic block design from [87]

shown in Fig. 15 makes DRTL a dynamic, pipelined logic scheme with low power consumption and high performance.

The performance analysis of DRTLG and interconnect design (a) (b) was evaluated by comparing the performance of DRTL with 4-input LUT based CMOS FPGA [33], for some ISCAS-85 Fig. 18. Memory network cell and architecture (a) Cell structure (b) Network benchmarks. The performance results show the possibility of architecture (each node represents a cell) with no crossover wiring. Cells are 96 arranged in hierarchical manner with N1 > N0 [88] % higher energy efficiency and more than two orders of magnitude lower energy-delay product for DRTL.

if the threshold voltage of the inverter was set between 0V In 2013, Soltiz et al. [87] presented a robust and area and 1/3V, the cell would work as NOR logic and if it was efficient hardware implementation of a neural logic block between 2/3V and 1/3V the cell would work as NAND logic. (NLB) with an adaptive activation function, containing a To operate the cell with a large number of inputs (>20) the Weighting and Range Select and an Activation Function, as threshold voltage of the inverter, for example in case of NOR, shown in Fig. 16. The main motivation was to make threshold needed to be lowered to a very small range. To accommodate function adaptive so that nonlinearly separable functions such this effect, the authors introduced three inverters (Fig. 18) as XOR could be implemented within a single layer. Inspired with three different VDD s to form a universal gate structure by the principle of neuromodulators in the brain, the adpative to implement AND, NAND, OR, NOR, and NOT logic. For activation function comprised of m points can model any the cell to work as a NAND logic, the switches S2 and S4 ( 1) function with m − boundaries. Weighting and Range were closed, and the output was taken from Vout so that Select component applies an adjustable weight to each input, three inverters would be enabled. To implement AND logic, calculates the weighted summation, and determines which of the switches S1 and S3 were closed, and the output taken the m ranges the summation falls in. The Activation Function from Vout.If the switches S1 and S4 were closed, a NOR associates a digital output with each range. The inputs are logic from Vout was achieved: here only one inverter had passed through memristors that are trained to a memristance to be enabled. If both S2 and S3 were closed, OR logic M, ranging from Ron to Roff . To change the functionality could be implemented. The proposed universal logic cell was a different value of the memristance might be selected. The based on the cognitive memory network [89], a resistive authors note that while the proposed design adds significant memory network that has no crossover wiring that overcame complexity when compared to a threshold activation function, the hardware limitations to size and functional complexity the adaptive activation function provides benefits of fast train- associated with conventional analogue neural networks. The ing convergence times when compared to a neural logic block universal logic cell shown in Fig. 19 was employed to realize that only adjusts input weights, by training the shape of the in an application to implement conventional digital logic gates. activation function. On an optical character recognition (OCR) The simulation was performed in SPICE using feature size of application the work shows a 90 percent improvement in the 0.25m TSMC process BSIM models and HP memristor model EDP over lookup table (LUT)-based implementations. for comparison with the CMOS implementation using a 16 bit A universal Boolean logic cell based on an analog resistive adder and a 16 x 1 MUX. The analysis shows that the proposed divider and threshold logic circuit useful for mimicking brain cell offered advantages of smaller area and design simplicity in like large variable logic functions in VLSI was proposed by comparison with CMOS based logic circuits when the number A. P. James et al. [88] in 2014. The logic cell shown in Fig. 17 employed a CMOS - Resistance Threshold Logic co-design which successfully optimised the circuit design of conventional CMOS based large variable boolean logic prob- lems. In the proposed resistance based threshold logic family, the resistive divider was implemented using memristors. The output of resistive divider was then converted into a binary Fig. 19. Circuit diagram to implement NAND, NOR, AND, OR and value by a threshold operation implemented by CMOS inverter NOT logic functions consisting of memristive resistance divider and CMOS and/or Op-amp. For a two-input resistance divider circuit, inverters with three different power supply values [88]

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 8

(a) (b)

(a)

(c) (d)

Fig. 20. (a) A Schematic representation of a threshold logic gate (TLG), (b) memristive cross-bar array, (c) A resistive with access transistors, (d) transient change in resistance for different magnitude of programming current [90]

(b)

Fig. 22. (a)Circuit diagrams of the logic gates using proposed cell (i) OR gate (ii) AND gate (iii) XOR gate (b) 2 bit memristive threshold vedic multiplier [93]

inverters as shown in Fig. 21 in application to realizing

complex computing circuits. The developed threshold logic Fig. 21. Memristive threshold logic cell [93] claims to outperform the previous memristive-CMOS logic cells by providing lower chip area, lower THD, and control- lable leakage power; except for a higher power dissipation of input variables became very high. with respect to CMOS logic. The proposed Memristive Threshold Logic (MTL) cell was In 2014, Deliang Fan et al. [90] proposed a spintronic designed specifically for implementing FFT and multiplication threshold device which can be combined with CMOS com- circuits as shown in Fig. 22 in modern microprocessors patible Ag-Si memristors for designing ultra low energy with the desired lower power dissipation and smaller on-chip Spin-Memristor Threshold Logic (SMTL). The SMTL gates area footprint using nanoscale fabrication techniques. Some shown in Fig. 20 employ memristive cross-bar array (MCA) additional desirable features of the proposed cell include the to perform current-mode summation of binary inputs. The generalisation ability of the cell with a single cell structure low-voltage fast-switching spintronic threshold devices (STD) with multiple functionality; and robustness to process vari- shown in Fig. 28, based on magnetic domain wall, was found ability in temperature, memristances and technology lengths suitable for the design of energy efficient SMTL. The spin- indicating the fault tolerance ability of brain like logic circuits. torque switches based on magneto-metallic domain wall (DW) The paper reports the successful application of the MTL cells motion [91], [92] allows ultra-low voltage operation of mem- in the examples of FFT and vedic multiplication computing ristive TLGs leading to low energy dissipation at the gate level. circuits. Field programmable SMTL gate arrays was found to operate In 2014, Soudry et al. [94] proposed memristor-based grid to successfully at a small terminal voltage of 5˜0mV, resulting in perform multiplication operation for learning backpropogation ultra-low power consumption in gates as well as programmable algorithms in multilayer neural networks (MNNs). Synapses interconnect networks. The performance analysis done on comprising one memristor and two CMOS transistors were common benchmarks show that the proposed hardware can set in a grid formation each receiving two complimentary achieve more than 100x improvement in energy and 1000x read/write pulses and an enable signal, and outputting on a improvement in energy-delay product, as compared to the current line. The column inputs fed the training data and state of the art CMOS FPGA based TLG. Threshold logic row inputs the classification label. The two computational computing using hybrid Memristive-CMOS cell architecture bottlenecks the authors addressed in gradient descent machine designed for Fast Fourier Transform and Vedic Multiplication learning algorithms were matrix × vector and vector × vector have been proposed by James A.P. et al. in 2014 [93]. The multiplication. The matrix × vector product was implemented proposed architecture involved a memristive threshold circuit through the memristive grid by multiplication through Ohms configuration which consisted of the memristive averaging cir- law and analog summation of currents. The vector × vector cuit in combination with operational and/or CMOS product was done using time × voltage paradigm under the

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 9

Fig. 23. Schematic of a DTG circuit [95] Fig. 24. STL-Cell [96]

approximation that given a voltage pulse, the conductivity of a memristor would increment proportionally to the pulse duration multiplied by the pulse magnitude. The authors proposed that the area and power consumption were expected to be reduced by a factor of 13-50 in comparison with standard CMOS technology. (a) (b)

Yang et al. in 2014 [95] used oxide based resistive RAM Fig. 25. (a) A modular Bilevel Programmable Resistance memristive Thresh- (RRAM) devices to implement threshold logic in order to old logic (BPRT) cell that used 4-input pixel values (b)An architectural compute logic functions with low power, robust circuits at low representation of the cell arrangement for a 4 × 4 pixels image; in this configuration, each cell uses 4 analog/digital inputs, has bi-valued weights, supply voltages (0.6 V). They used a differential threshold and implements image dimensionality reduction [97] logic gate (DTG) circuit that consisted of five main com- ponents (Fig. 23): (1) a differential sense amplifier, which consists of two cross coupled NAND gates, (2) an SR latch, that could lead to near-continuous real-time object tracking (3) two discharge devices, (4) left (LIN) and right (RIN) input and even surpass human object tracking ability, outperformed networks, and (5) a network of . The resistive network CMOS equivalent implementation in terms of area, leakage was implemented using oxide-based random access memory power, power dissipation and delay. The proposed resistive to behave as as CMOS compatible nano-scale resistor. The switching based threshold logic cell comprised of a voltage circuit outputs a logic 1 based on the inequality of number divider circuit and a CMOS inverter gate which encoded the of active pFETs in the LIN and RIN networks. The benefits pixels of a template image. The presented logic provided a of robustness, area, and energy delay were demonstrated on a framework to implement brain like logic in a memory and 16-bit full adder and a 128-bit comparator. learning driven detection of multiple objects. Earlier in 2012 Nukala et al. [96] from the same group had used an STT-MTJ (Spin Torque Transfer-Magnetic Tun- TABLE I nelling Junction) device with conventional to build IMPLEMENTATIONS OF MTL - A COMPARISON threshold logic architecture. The resulting cell was used to program a large number of threshold logic functions, many Methodology Area Power (mW ) of which would require a multilevel network of conventional J. Rajendran et al. (2009) [70] 77.52 pm2 44.12 CMOS logic gates. Based on an array architecture of these J. Rajendran et al. (2010) [72] 234.46 pm2 32.56 2 cells they demonstrated the advantages of non-volatility and J. Rajendran et al. (2012) [73] 193.77 pm 31.3 T. Tran et al. (2012) [74] 4.0 nm2 30.8 zero standby power on a 16-bit carry look-ahead adder and 2 H. Manem et al. (2012) [76] 193.87 pm 45.7 compared with two conventional FPGA implementations. The L.Gao et al. (2013) [80] 0.4992 µm2 28.81 array had 12x lower transistor count (compared to CLA- C.B. Dara et al. (2013) [84] 270.55 pm2 118.22 2 FPGA) and 10x reduction (compared to Ripple Carry-Ahead- James A.P. et al. (2014) [88] 155.52 pm 45.34 James A.P. et al. (2015) [93] 2.08 nm2 30.72 FPGA) with comparable energy. The cell is shown in Fig. A. K. Maan et al. (2015) [97] 38.91 pm2 43.66 24. In the write phase of the cell, WR is asserted, so that a certain amount of current (I), flows through the STT-MTJ In Table I, the MTL implementations that were discussed device depending on the number of ON PMOS transistors. If have been compared on the basis of their area and static power I ≥ (Ic), the switching current, then the STT-MTJ device dissipation. The analyses have been carried out in circuits by switches to the low resistance state otherwise it remains in realizing a 2-input logic function using the proposed MTL high resistance state. When the WR pulse goes low, no current techniques using simulations under same technology parame- flows through the STT-MTJ device, and since the device is ters (i.e. HP memristor model with 0.25u TSMC technology). non-volatile, the state is maintained. According to Table II, [97] requires the lowest implementation In 2014, A. K. Maan et al. [97] presented a programmable area, which is 38.91 pm2, and [80] has the lowest static power Memristive Threshold Logic (MTL) circuit design for real- consumption, which is 28.81 mW. Among the 2-input cells time detection of moving objects. The proposed threshold logic implemented to process the image pixels [70], [97] , the MTL shown in Fig. 25, which was targeted at high speed imaging design proposed by A. K. Maan et al. [97] in 2015 has the

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 10

TABLE II COMPARISON OF THRESHOLD LOGIC IMPLEMENTATION USING MEMRISTORS

Methodology Area Power Leakage Power Speed Fan-in Fan-out Target application Energy Delay Energy Delay Product (EDP)

J. Rajendran et al. (2009) [70] NR 0.57- 1.56µW NR NR NR NR Image Classification NR 40 ps NR J. Rajendran et al. (2010) [72] 28* 42 µW NR NR NR NR NR NR 6.1 ns NR J. Rajendran et al. (2012) [73] 182** NR NR NR NR NR Boolean Logic Gates 6.3fJ’, 2.99ns’, NR 3.15fJ” 5.84ns” T. Tran et al. (2012) [74] NR NR NR >1 GHz >10 NR Digital Logic Gates NR NR NR H. Manem et al. (2012) [76] NR NR NR NR NR NR Memory 3.82nJ 189ps‘, 721.98zsJ‘, 687.6zsJ“ 180ns“ L.Gao et al. (2013) [80] NR NR NR NR NR NR Digital Logic Gates NR NR NR C.B. Dara et al. (2013) [84] NR NR NR NR NR NR Boolean Logic Gates 3.41fJ“‘ 0.42ns“‘ 1.43ysJ“‘ M. Sharad et al. (2013) [85] NR NR NR 2 GHz NR NR Programmable Logic Hardware, 480fJ*** 0.5ns*** 240ysJ Crossbar memory James A.P. et al. (2014) [88] 70pm2 9.2 µW 0.014 nW NR 14.498M NR Boolean Logic gates, Arithmetic NR 0.45 µs NR modules Deliang Fan et al. (2014) [90] NR NR NR NR 4 NR Programmable Logic Hardware ≈300fJ*** ≈2ns*** ≈0.6zsJ James A.P. et al. (2015) [93] 4.55mm2 3.00 µW 14.30 pW 1 GHz NR NR FFT and Vedic Multiplication 0.30pJ NR NR A. K. Maan et al. (2015) [97] 9.66mm2 12.30 µW 12.25 pW 100 MHz NR NR Object Detection NR NR NR

NR - Not Reported in the reference work. *Transistor count **Transistor count for C17-ISCAS-85 benchmark *** C432-ISCAS-85 benchmark; ’ISCAS-85 Benchmark C17-Before refresh ” ISCAS-85 Benchmark C17-After refresh ‘Writing to 1T1M memory circuit of physical array size 4 x 4; “Reading from 1T1M memory circuit of physical array size 4 x 4 “‘Implementation of 3-input function ”A+BC” using CMMTL lowest area and static power consumption. From the memory The progress in the VLSI circuits development depends also crossbar array implementations [85], [90] compared in Table on the availability of accurate models for memristors, that I, the MTL design proposed by Deliang Fan et al. [90] in 2014 can be incorporated into widely accepted simulation tools. has the lowest energy-delay product. Although there are several simulation models available in In Table II, a comparison between the various performance SPICE and VerilogA, they still do not cover the wide range parameters of the threshold logic cells implemented with of memristive devices available for use today. In addition, the memristors that have been reported in the paper have been physical level design of memristors and memristor-CMOS hy- summarized. These example applications of threshold logic brid circuits is challenged by the lack of process standards and cells does not directly imply high degree of generality but in- requires further adoption of these devices in semiconductor dicate that the threshold logic circuits are scalable and can find industry. In this sense, memristor devices and circuits are quite its use in a wide range of tasks. The circuits are also arranged early in the stages of commercial implementation. We note with respect of applications from the more general logic gate that while there are aforementioned issues, the ability of mem- applications to very recent pattern recognition applications. ristive threshold logic cells to be general and programmable Recent works indicate the power dissipation to be a concerning makes it an interesting alternative to CMOS logic family of factor in the design of these circuits and require changes in gates. In addition, the ability of the threshold logic cells to be the circuit configurations as well as memristor device design. programmed to behave like a pattern recognizer mimics the The low area footprint for on-chip implementation has been principle of neurons that can provide multiple functionalities the key attraction of these logic family of gates and circuits. with the same cell structure. The possibility that the threshold logic gates with memristors The hardware programmability opens up a wide range of can lead to more-than-Moore’s law outweighs the limitations applications for circuit designers and artificial intelligence of the power dissipation and lower speeds. researchers. Some of these are: (1) development of memories that can learn, store and forget like human memory, (2) IV. DISCUSSIONS AND OPEN PROBLEMS development of human like self-learning classifiers in silicon, The important properties of low leakage currents, ability (3) building higher level intelligent modules on programmable of switching into memorised resistor levels, and smaller on- and self-learning chips, (4) building array processing cir- chip area make memristor an attractive element to mimic the cuits and re-inventing analog sensory processing such as in principle of firing of neurons in silicon. Being a resistive computer vision, (5) developing CAD tools and systems for device the biggest challenges in designing memristor based memristive hybrid circuits, and (6) development very large circuit configurations are with reduction in power dissipation scale simulation and implementation of such threshold logic in comparison with CMOS only circuits. The resistive path in- systems. troduced by memristive devices within the designed threshold Memristive behaviour is observed in devices dating back logic circuits often drives larger currents through the circuits to electric arc experiments performed by Sir Humphry Davy resulting in higher power dissipation. Recently, the CMOS- in 1808. Although lost in the history, the resurgence of memristor hybrid circuits have gained popularity as CMOS memristive device as a possible frontrunner in implementation is a matured process technology, however, is gain-limited by of neural network is made possible mainly due to the advances higher power dissipation compared with CMOS-only coun- in nanoscale technologies in last decade. In addition, the terparts. In addition, in MOS-gated memristive arrays the limitations imposed by scaling issues of MOSFET devices, leakage currents become an important issue, that requires and the need to create large scale neural networks for imitating specialised compensated read-out techniques. Programmability silicon brain continue to inspire research in this field. of the memristive states in hybrid circuits become a chal- lenging problem often requiring complex add-on circuits. If V. CONCLUSION not designed carefully the area of programming circuits can This survey discussed the impact of memristive threshold outweigh the benefits of lower area offered by the memristors. logic by sketching the number of applications that have sprung

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 11

up utilizing the inherent nature of several devices that show [16] Dmitri B Strukov, Gregory S Snider, Duncan R Stewart, and R Stanley memristive behaviour. The paper reviewed the different circuit Williams. The missing memristor found. nature, 453(7191):80–83, 2008. [17] Warren S McCulloch and Walter Pitts. A logical calculus of the ideas implementations of threshold logic that employed the resistive immanent in nervous activity. The bulletin of mathematical biophysics, switching nature of memristors potentially leading to highly 5(4):115–133, 1943. efficient, high density neurocomputing devices in future. Even [18] International Technology Roadmap for Semiconductors 2012 updated with several new advances, this field of research is relatively edition. Technical report, 2012. [19] L.O. Chua. Memristor-the missing circuit element. Circuit Theory, IEEE very new, and offers several challenges and opportunities Transactions on, 18(5):507–519, Sep 1971. for neural networks, learning systems, circuits and systems [20] Leon O Chua. Synthesis of new nonlinear network elements. Proceed- communities to work together. The practical applications of ings of the IEEE, 56(8):1325–1340, 1968. [21] Leon O Chua. Introduction to nonlinear network theory. McGraw-Hill, memristive threshold logic design extend to real-time pro- 1969. cessing and recognition of natural signals, its differentiation [22] L.O. Chua and Sung Mo Kang. Memristive devices and systems. and efficient architectures for silicon memories. While the Proceedings of the IEEE, 64(2):209–223, Feb 1976. possibilities are many there exist several challenges as well, [23] CA Richter, DR Stewart, DAA Ohlberg, and R Stanley Williams. Electrical characterization of al/alox/molecule/ti/al devices. Applied including the need for better energy and power dissipation Physics A, 80(6):1355–1362, 2005. ratings, newer circuits for efficient programming of memristor [24] Jason J Blackstock, William F Stickle, Carrie L Donley, Duncan R arrays, accurate simulation models for memristors, and long- Stewart, and R Stanley Williams. Internal structure of a molecular junction device: chemical reduction of pto2 by ti evaporation onto an term development of foundry support. interceding organic monolayer. The Journal of Physical Chemistry C, 111(1):16–20, 2007. [25] Paul Penfield and Robert P Rafuse. Varactor applications. 1962. CKNOWLEDGEMENT A [26] J Joshua Yang, Matthew D Pickett, Xuema Li, Douglas AA Ohlberg, We would like to thank all the reviewers for the constructive Duncan R Stewart, and R Stanley Williams. Memristive switching mechanism for metal/oxide/metal nanodevices. Nature , feedback and comments, that has helped us to improve the 3(7):429–433, 2008. overall quality of the paper. [27] T. Prodromakis, K. Michelakisy, and C. Toumazou. Fabrication and electrical characteristics of memristors with tio2/tio2-x active layers. In Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International REFERENCES Symposium on, pages 1520–1522, May 2010. [28] Matthew D Pickett, Dmitri B Strukov, Julien L Borghetti, J Joshua Yang, [1] Leon Chua. Introduction to memristors. 2009. Gregory S Snider, Duncan R Stewart, and R Stanley Williams. Switching [2] Ronald Tetzlaff et al. Memristors and Memristive Systems. Springer, dynamics in titanium dioxide memristive devices. Journal of Applied 2014. Physics, 106(7):074508, 2009. [3] Chris Yakopcic, Eunsung Shin, Tarek M Taha, Guru Subramanyam, [29] L.O. Chua. Nonlinear circuit foundations for nanodevices. i. the four- P Terrence Murray, and Stanley Rogers. Fabrication and testing of element torus. Proceedings of the IEEE, 91(11):1830–1859, Nov 2003. memristive devices. In The 2010 International Joint Conference on [30] Yogesh N Joglekar and Stephen J Wolf. The elusive memristor: Neural Networks (IJCNN), 2010. properties of basic electrical circuits. European Journal of Physics, [4] Xu Hui, Tian XiaoBo, Nie HongShan, Li QingJiang, Liu Haijun, 30(4):661, 2009. and Xu Xin. On development and expectation of memristor. In [31] TW Hickmott. Low-frequency negative resistance in thin anodic oxide Intelligent System Design and Engineering Application (ISDEA), 2010 films. Journal of Applied Physics, 33(9):2669–2682, 1962. International Conference on, volume 1, pages 1035–1038. IEEE, 2010. [32] Rainer Waser and Masakazu Aono. -based resistive switching [5] Garrett S Rose. Overview: Memristive devices, circuits and systems. In memories. Nature materials, 6(11):833–840, 2007. Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on, pages 1955–1958. IEEE, 2010. [33] NB Zhitenev, A Sidorenko, DM Tennant, and RA Cirelli. Chemical [6] F. Corinto, P.P. Civalleri, and L.O. Chua. A theoretical approach modification of the electronic conducting states in polymer nanodevices. to memristor systems. In Cellular Nanoscale Networks and their Nature nanotechnology, 2(4):237–242, 2007. Applications (CNNA), 2014 14th International Workshop on, pages 1–2, [34] Jaap HA Smits, Stefan CJ Meskers, Rene´ AJ Janssen, Albert W July 2014. Marsman, and Dago M de Leeuw. Electrically rewritable memory cells [7] Guo-Qiang Zhang and A V Roosmalen. More than moore. The changing from poly (3-hexylthiophene) schottky diodes. Advanced Materials, international landscape, strategy and solutions of micro/. 17(9):1169–1173, 2005. In Electronic Packaging Technology, page 1, 2007. [35] Qianxi Lai, Zuhua Zhu, Yong Chen, Satish Patil, and Fred Wudl. Organic [8] Wolfgang Arden, Michel Brilloue¨t, Patrick Cogez, Mart Graef, Bert nonvolatile memory by dopant-configurable polymer. Applied physics Huizing, and Reinhard Mahnkopf. More than-moore white paper. letters, 88(13):133515–133515, 2006. Version, 2:14, 2010. [36] K Terabe, T Hasegawa, T Nakayama, and M Aono. Quantized conduc- [9] GQ Zhang, F van Roosmalen, and M Graef. The paradigm of ”more than tance atomic switch. Nature, 433(7021):47–50, 2005. moore”. In Electronic Packaging Technology, 2005 6th International [37] M.N. Kozicki, Mira Park, and M. Mitkova. Nanoscale memory elements Conference on, pages 17–24. IEEE, 2005. based on solid-state electrolytes. Nanotechnology, IEEE Transactions on, [10] GQ Zhang, Mart Graef, and Fred van Roosmalen. The rationale 4(3):331–338, May 2005. and paradigm of ”more than moore”. In Electronic Components and [38] S. Dietrich, M. Angerbauer, M. Ivanov, D. Gogl, H. Hoenigschmid, Technology Conference, 2006. Proceedings. 56th, pages 7–pp. IEEE, M. Kund, C. Liaw, M. Markert, R. Symanczyk, L. Altimime, S. Bournat, 2006. and G. Mueller. A nonvolatile 2-mbit cbram memory core featuring [11] Ralph K Cavin, Paolo Lugli, and Victor V Zhirnov. Science and advanced read and program control. Solid-State Circuits, IEEE Journal engineering beyond moore’s law. Proceedings of the IEEE, 100(Special of, 42(4):839–845, April 2007. Centennial Issue):1720–1749, 2012. [39] John R Jameson, Yoshiaki Fukuzumi, Zheng Wang, Peter Griffin, Koji [12] M.P. Sah, Hyongsuk Kim, and L.O. Chua. Brains are made of memris- Tsunoda, G Ingmar Meijer, and Yoshio Nishi. Field-programmable rec- tors. Circuits and Systems Magazine, IEEE, 14(1):12–36, Firstquarter tification in rutile tio2 crystals. Applied Physics Letters, 91(11):112101, 2014. 2007. [13] Michael Crupi, Laxman Pradhan, and Stuart Tozer. Modelling neural [40] Doo Seok Jeong, Herbert Schroeder, and Rainer Waser. Coexistence of plasticity with memristors. IEEE Canadian Review, (68):10–14, 2012. bipolar and unipolar resistive switching behaviors in a pt/ tio2/ pt stack. [14] C Evans-Pughe. The device with brains [electronics memristors]. Electrochemical and solid-state letters, 10(8):G51–G53, 2007. Engineering & Technology, 5(12):30–33. [41] A Beck, JG Bednorz, Ch Gerber, Ch Rossel, and D Widmer. Repro- [15] R. Williams. How we found the missing memristor. Spectrum, IEEE, ducible switching effect in thin oxide films for memory applications. 45(12):28–35, Dec 2008. Applied Physics Letters, 77(1):139–141, 2000.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 12

[42] M.M. Ziegler and M.R. Stan. Design and analysis of crossbar circuits [66] Tuli LE Dake and Ugur Cilingiroglu. Rank order filter using multiple for molecular nanoelectronics. In Nanotechnology, 2002. IEEE-NANO winners take all. In PROCEEDINGS OF THE IEEE MIDWEST 2002. Proceedings of the 2002 2nd IEEE Conference on, pages 323–327, SYMPOSIUM ON CIRCUITS AND SYSTEMS, volume 43, pages 482– 2002. 485. LIDA RAY TECHNOLOGIES INC.,, 2000. [43] Dmitri B Strukov and R Stanley Williams. Four-dimensional address [67] Stefan Slesazeck, Alon Ascoli, Hannes Ma¨hne, Ronald Tetzlaff, and topology for circuits with stacked multilayer crossbar arrays. Proceed- Thomas Mikolajick. Unfolding the threshold switching behavior of a ings of the National Academy of Sciences, 106(48):20155–20158, 2009. memristor. In Nonlinear Dynamics of Electronic Systems, pages 156– [44] Gang Liu, Liang Fang, Nan Li, Bing Cai Sui, and Zhi Kui Duan. New 164. Springer, 2014. behavioral modeling method for crossbar-based memristor. In Micro- [68] Anna N Morozovska, Eugene A Eliseev, Olexandr V Varenyk, Yunseok electronics and Electronics (PrimeAsia), 2010 Asia Pacific Conference Kim, Evgheni Strelcov, Alexander Tselev, Nicholas V Morozovsky, on Postgraduate Research in, pages 356–359. IEEE, 2010. and Sergei V Kalinin. Nonlinear space charge dynamics in mixed ionic- [45] Jun Wu and Minsu Choi. Memristor lookup table (mlut)-based asyn- electronic conductors: Resistive switching and ferroelectric-like chronous crossbar architecture. In Nanotechnology (IEEE- hysteresis of electromechanical response. Journal of Applied Physics, NANO), 2010 10th IEEE Conference on, pages 1100–1103, Aug 2010. 116(6):066808, 2014. [46] Cong Xu, Xiangyu Dong, Norman P Jouppi, and Yuan Xie. Design [69] John Paul Strachan, Dmitri B Strukov, Julien Borghetti, J Joshua Yang, implications of memristor-based rram cross-point structures. In Design, Gilberto Medeiros-Ribeiro, and R Stanley Williams. The switching lo- Automation & Test in Europe Conference & Exhibition (DATE), 2011, cation of a bipolar memristor: chemical, thermal and structural mapping. pages 1–6. IEEE, 2011. Nanotechnology, 22(25):254015, 2011. [47] M.S. Qureshi, M. Pickett, Feng Miao, and John Paul Strachan. Cmos [70] J. Rajendran, H. Manem, and G.S. Rose. Ndr based threshold logic fabric interface circuits for reading and writing memristor crossbar array. In with memristive synapses. In Nanotechnology, 2009. IEEE-NANO 2009. Circuits and Systems (ISCAS), 2011 IEEE International Symposium on, 9th IEEE Conference on, pages 725–728, July 2009. pages 2954–2957, May 2011. [71] E Goto, K Murata, K Nakazawa, K Nakagawa, T Moto-Oka, Y Mat- suoka, Y Ishibashi, H Ishida, T Soma, and E Wada. Esaki diode high- [48] Mohammad Bavandpour, Saeed Bagheri Shouraki, Hamid Soleimani, speed logical circuits. Electronic Computers, IRE Transactions on, Arash Ahmadi, and Ali Asghar Makhlooghpour. Simulation of memris- (1):25–29, 1960. tor crossbar structure on gpu platform. In Electrical Engineering (ICEE), [72] J. Rajendran, H. Manem, R. Karri, and G.S. Rose. Memristor 2012 20th Iranian Conference on, pages 178–183. IEEE, 2012. based programmable threshold logic array. In Nanoscale Architectures [49] Shuang Pi, Peng Lin, and Qiangfei Xia. Memristor crossbar arrays (NANOARCH), 2010 IEEE/ACM International Symposium on, pages 5– with junction areas towards sub-10× 10 nm 2. In Cellular Nanoscale 10, June 2010. Networks and Their Applications (CNNA), 2012 13th International [73] J. Rajendran, H. Manem, R. Karri, and G.S. Rose. An energy-efficient Workshop on, pages 1–2. IEEE, 2012. memristive threshold logic circuit. Computers, IEEE Transactions on, [50] Yuval Cassuto, Shahar Kvatinsky, and Eitan Yaakobi. Sneak-path con- 61(4):474–487, April 2012. straints in memristor crossbar arrays. In Information Theory Proceedings [74] Thanh Tran, A. Rothenbuhler, E.H.B. Smith, V. Saxena, and K.A. Camp- (ISIT), 2013 IEEE International Symposium on, pages 156–160. IEEE, bell. Reconfigurable threshold logic gates using memristive devices. In 2013. Subthreshold Microelectronics Conference (SubVT), 2012 IEEE, pages [51] Mohammed Affan Zidan, Ahmed Sultan, Hossam AH Fahmy, and 1–3, Oct 2012. Khaled N Salama. Leakage analysis of crossbar memristor arrays. In [75] Antonio S Oblea, Achyut Timilsina, David Moore, and Kristy A Cellular Nanoscale Networks and their Applications (CNNA), 2014 14th Campbell. Silver chalcogenide based memristor devices. In IJCNN, International Workshop on, pages 1–2. IEEE, 2014. pages 1–3, 2010. [52] E. Linn. Memristive nano-crossbar arrays enabling novel computing [76] H. Manem, J. Rajendran, and G.S. Rose. Stochastic gradient descent in- paradigms. In Circuits and Systems (ISCAS), 2014 IEEE International spired training technique for a /nano memristive trainable threshold Symposium on, pages 2596–2599, June 2014. gate array. Circuits and Systems I: Regular Papers, IEEE Transactions [53] A Sawa, T Fujii, M Kawasaki, and Y Tokura. Interface resistance on, 59(5):1051–1060, May 2012. switching at a few nanometer thick perovskite manganite active layers. [77] Saburo Muroga. Threshold logic and its applications. 1971. Applied physics letters, 88(23):232112–232112, 2006. [78] H. Manem and G.S. Rose. A read-monitored write circuit for 1t1m [54] M Hamaguchi, K Aoyama, S Asanuma, Y Uesu, and T Katsu- multi-level memristor memories. In Circuits and Systems (ISCAS), 2011 fuji. Electric-field-induced resistance switching universally observed in IEEE International Symposium on, pages 2938–2941, May 2011. transition-metal-oxide thin films. Applied physics letters, 88(14):142508, [79] Harika Manem, Jeyavijayan Rajendran, and Garrett S Rose. Design 2006. considerations for multilevel cmos/nano memristive memory. ACM [55] Krzysztof Szot, Wolfgang Speier, Gustav Bihlmayer, and Rainer Waser. Journal on Emerging Technologies in Computing Systems (JETC), Switching the electrical resistance of individual dislocations in single- 8(1):6, 2012. crystalline srtio3. Nature materials, 5(4):312–320, 2006. [80] Ligang Gao, F. Alibart, and D.B. Strukov. Programmable [56] G. Chen. Leon chua’s memristor [recognitions]. Circuits and Systems cmos/memristor threshold logic. Nanotechnology, IEEE Transactions Magazine, IEEE, 8(2):55–55, Second 2008. on, 12(2):115–119, March 2013. [57] Julien Borghetti, Gregory S Snider, Philip J Kuekes, J Joshua Yang, Dun- [81] S. Kvatinsky, N. Wald, G. Satat, A. Kolodny, U.C. Weiser, and E.G. can R Stewart, and R Stanley Williams. memristiveswitches enable state- Friedman. Mrl – memristor ratioed logic. In Cellular Nanoscale fullogic operations via material implication. Nature, 464(7290):873–876, Networks and Their Applications (CNNA), 2012 13th International 2010. Workshop on, pages 1–6, Aug 2012. [58] Bernard Widrow, WH Pierce, and JB Angell. Birth, life, and death [82] G.S. Rose, J. Rajendran, H. Manem, R. Karri, and R.E. Pino. Lever- in microelectronic systems. Military Electronics, IRE Transactions on, aging memristive systems in the construction of digital logic circuits. 1051(3):191–201, 1961. Proceedings of the IEEE, 100(6):2033–2049, June 2012. [59] R.R. Schaller. Moore’s law: past, present and future. Spectrum, IEEE, [83] Konstantin K Likharev. Hybrid cmos/nanoelectronic circuits: Opportu- 34(6):52–59, Jun 1997. nities and challenges. Journal of Nanoelectronics and Optoelectronics, 3(3):203–230, 2008. [60] Types of memristors. Technical report, 2015. [84] C.B. Dara, T. Haniotakis, and S. Tragoudas. Low power and high [61] Memristors and Memristive Systems. Technical report, 2015. speed current-mode memristor-based tlgs. In Defect and Fault Tolerance [62] Pinaki Mazumder, Sung Mo Kang, and Rainer Waser. Memristors: de- in VLSI and Nanotechnology Systems (DFT), 2013 IEEE International vices, models, and applications. Proceedings of the IEEE, 100(6):1911– Symposium on, pages 89–94, Oct 2013. 1919, 2012. [85] Mrigank Sharad, Deliang Fan, and Kaushik Roy. Ultra-low energy, [63] Valeriu Beiu, Jose´ M Quintana, and Mar´ıa J Avedillo. Vlsi implemen- high-performance dynamic resistive threshold logic. arXiv preprint tations of threshold logic-a comprehensive survey. Neural Networks, arXiv:1308.4672, 2013. IEEE Transactions on, 14(5):1217–1243, 2003. [86] D. Chabi, Weisheng Zhao, D. Querlioz, and J.-O. Klein. Robust neural [64] Marvin Minsky. Neural nets and the brain-model problem. Unpublished logic block (nlb) based on memristor crossbar array. In Nanoscale doctoral dissertation, Princeton University, NJ, 1954. Architectures (NANOARCH), 2011 IEEE/ACM International Symposium [65] Frank Rosenblatt. The perceptron: a probabilistic model for information on, pages 137–143, June 2011. storage and organization in the brain. Psychological review, 65(6):386, [87] Michael Soltiz, Dhireesha Kudithipudi, Cory Merkel, Garrett S Rose, 1958. and Robinson E Pino. Memristor-based neural logic blocks for

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 13

nonlinearly separable functions. Computers, IEEE Transactions on, Alex Pappachen James works in the area of 62(8):1597–1606, 2013. brain inspired circuits, algorithms and systems, [88] A.P. James, L.R.V.J. Francis, and D.S. Kumar. Resistive threshold logic. and has a PhD from Queensland Micro and Nan- Very Large Scale Integration (VLSI) Systems, IEEE Transactions on, otechnology Centre, Griffith University. He is cur- 22(1):190–195, Jan 2014. rently an Associate Professor in Electronics En- [89] A.P. James and S. Dimitrijev. Cognitive memory network. Electronics gineering at Nazarbayev University, and leads the Letters, 46(10):677–678, May 2010. Bioinspired microelectronic systems lab. His group [90] Deliang Fan, M. Sharad, and K. Roy. Design and synthesis of is actively engaged in the development of mem- ultralow energy spin-memristor threshold logic. Nanotechnology, IEEE ristive circuits and systems that can be inte- Transactions on, 13(3):574–583, May 2014. grated to realistic imaging and pattern recogni- [91] S. Fukami, T. Suzuki, K. Nagahara, N. Ohshima, Y. Ozaki, S. Saito, tion systems. He is a Senior Member of IEEE. R. Nebashi, N. Sakimura, H. Honjo, K. Mori, C. Igarashi, S. Miura, http://www.biomicrosystems.info/alex N. Ishiwata, and T. Sugibayashi. Low-current perpendicular domain wall motion cell for scalable high-speed mram. In VLSI Technology, 2009 Symposium on, pages 230–231, June 2009. [92] C. Augustine, A. Raychowdhury, B. Behin-Aein, S. Srinivasan, J. Tschanz, V.K. De, and K. Roy. Numerical analysis of domain wall propagation for dense memory arrays. In Electron Devices Meeting (IEDM), 2011 IEEE International, pages 17.6.1–17.6.4, Dec 2011. [93] A.P. James, D.S. Kumar, and A. Ajayan. Threshold logic computing: Memristive-cmos circuits for fast fourier transform and vedic multipli- cation. Very Large Scale Integration (VLSI) Systems, IEEE Transactions on, PP(99):1–1, 2015. [94] Daniel Soudry, Dotan Di Castro, Asaf Gal, Avinoam Kolodny, and Shahar Kvatinsky. Memristor-based multilayer neural networks with online gradient descent training. 2015. [95] Jinghua Yang, Niranjan Kulkarni, Shimeng Yu, and Sarma Vrudhula. Integration of threshold logic gates with rram devices for energy efficient and robust operation. In Proceedings of the 2014 IEEE/ACM Inter- national Symposium on Nanoscale Architectures, pages 39–44. ACM, 2014. [96] Nishant S Nukala, Niranjan Kulkarni, and Sarma Vrudhula. Spintronic threshold logic array (stla)a compact, low leakage, non-volatile gate array architecture. Journal of Parallel and Distributed Computing, 74(6):2452–2460, 2014. [97] A.K. Maan, D.S. Kumar, S. Sugathan, and A.P. James. Memristive threshold logic circuit design of fast moving object detection. Very Large Scale Integration (VLSI) Systems, IEEE Transactions on, PP(99):1–1, 2014.

Akshay Kumar Maan completed his Ph.D. in Memristor-based threshold logic cognitive circuits at Griffith School of Engineering, Griffith University, Brisbane, Australia in 2015. The work is inspired by the principle of firing of neurons in the pri- mate brain, and the ability of neuronal structures to perform different cognitive tasks, explores pro- grammable (mem)resistance-based threshold logic memory networks that are shown to perform basic logic functions to highly involved cognitive tasks such as face recognition and fast moving object de- tection, in a parallel, scalable hardware architecture. His main areas of interest are brain inspired circuits, algorithms and systems, to enable more-than-moor era of low power, high density VLSI circuits for cognitive applications such as pattern recognition, face recognition and speech recognition.

Deepthi Anirudhan Jayadevi received the B.Tech. degree in Electronics and Instrumentation Engineer- ing from Cochin University of Science and Technol- ogy, India in 2008. She received the M.Tech. degree in VLSI and Embedded Systems from ER&DCI In- stitute of Technology (ER&DCI-IT), an educational institution under the direct control of Centre for Development of Advanced Studies (CDAC), India in 2010. She was a research staff member at Enview R&D labs, and is a member of the IEEE and has more than five years of experience in teaching and research in the field of Electronics. Her current research interests include VLSI circuits and artificial intelligence.