Identification: New Paradigms, Challenges, and Opportunities

WANG Le-Yi1 ZHAO Wen-Xiao2 Abstract The traditional paradigm of system identification employs prior information on system structures and environments and input/output observation to derive system models. Extensive research and development on its methodologies, theoretical foundation, algorithms, verifications, and applications over the past half century have established a mature field with a rich literature and substantial benchmark applications. However, rapid advancement in science, technology, engineering, and social medias has ushered in a new era of science and control in which challenges and opportunities are abundant for system identification. In this sense, system identification remains an exciting, young, viable, and critical field that mandates new paradigms to meet such challenges. This article points out some potentially important aspects of system identification in these new paradigms, suggests some worthy areas of research focus, and most importantly opens the forum for further discussions. Key words System identification, uncertainty, information, complexity, networked system, large data processing, integration of identification and decision Citation Wang Le-Yi, Zhao Wen-Xiao. System identification: new paradigms, challenges, and opportunities. Acta Automatica Sinica, 2013, 39(7): 933−942

1 Introduction troduced to reduce online computational and memory com- plexities. Introduced in the 1950s as a method to model a dy- Despite diversification in approaches, the traditional namic system for control design[1], system identification paradigms of system identification mostly employ the typ- in the classical sense has grown into a well-defined and ical steps that cover both off-line modeling and real-time mature field[2−9]. The traditional system identification estimation: 1) specifications: comprehensively covers many aspects of the modeling pro- specify the plant that is to be identified together with con- cess, including data acquisition (from a lab, a testing fa- trollable and uncontrollable inputs, measured and unmea- cility, or during system operation), model structure selec- sured outputs, signal ranges, system operating conditions, tion, model parameter estimation, and model validation. and targeted applications of the model. 2) Model class se- System identification was inherently control oriented, of- lection: select model structures, parameterizations, system ten treated as an integral part of feedback control de- orders, and other complexity entities. 3) Input design and sign procedures and adaptive control algorithms. Its typ- : design input signals, scenarios for data col- ical formulation models a process as a parameterized sys- lection, schemes and rates, quantization, and time tem with observations corrupted by random noises. Its horizons. Typically, input signals must be sufficiently rich early development in this stochastic framework employed to meet persistent excitation conditions that ensure its ca- many results in stochastic analysis, , pability to extract parameter information. Based on the de- analysis, convergence types and properties, etc. However, signed and implemented inputs, one performs , its inherent connection with feedback control introduced collects data, either online or off line, and characterizes data many distinct and profound features. Many algorithms errors either in statistical terms or bounds. 4) Parameter have been introduced and their properties rigorously es- or function estimation: design estimation algorithms, either tablished, including the major milestones and fundamen- off-line bulk algorithms or online recursive ones, to extract tal results on least-squares algorithms[8, 10], prediction er- model functions or by using as much informa- ror methods[8], stochastic approximations[11−14], ordinary [11] tion as possible from the structural information and data. differential equation (ODE) approaches ,Akaikesin- 5) Model validation: evaluate the model s reliability by ap- formation criteria[15−16], Rissanens minimum length data [17] plying it to a variety of operating conditions, comparing description , and many others. Unique system fea- model predictions and actual measurement data, and eval- tures in input/output observation noises and system feed- uating model reliability, accuracy, robustness, convergence, back structures led to errors-in-variables identification [18−21] [22] etc. The above steps are often iterated until a satisfactory scenarios , closed-loop identification ,modelor- model is obtained. In this line of study, a vast literature der reduction[23], set-valued observations[24−25], - [26] [27−29] has been generated, and most aspects have been carefully domain feature extractions , and adaptation ,as and rigorously studied. In this sense, the classical system well as many successful application benchmark cases. To identification is a highly mature field and provides a rich accommodate real time data acquisition and integration treatise of understanding and schemes to support related with feedback control, many recursive algorithms were in- usages of the modeling process, including monitoring, con- Manuscript received July 4, 2012; accepted November 7, 2012 trol, diagnosis, decision, and system improvement. Supported by National Natural Science Foundation of China Naturally, when a field is so heavily studied, new but re- (61134013, 61104052, 61273193) Recommended by Academician HUANG Lin lated problems will emerge, motivated by applications and 1. Department of Electrical and Computer Engineering, Wayne also mathematical extension. Most notably the school of State University, Detroit, Michigan 48202, USA 2. Institute of Systems Science, Chinese Academy of Sciences, Beijing 100190, non-stochastic system identification, often called “worst- China 934 ACTA AUTOMATICA SINICA Vol. 39 case identification”, that regarded noises as “unknown- organisms feature development. Recently introduced con- but-bounded” was active in the 1990s[30−32]. In seeking a cepts of cyber-physical systems emphasize coordinations of set-membership-type characterization of system models in computer, control, and communications (3C) to study sys- relation to this worst-case formulation, this school linked tems holistically. These technology growths have motivated system identification to its utility with robust control, es- the system identification community to reevaluate its tradi- pecially H∞, L1, and structured uncertainty. Thanks tional frameworks, look more broadly for new application to this connection with the robust control community, areas, formulate new problems, recognize new issues, un- it often used explicitly the term “control-oriented” iden- derstand new constraints, and explore new solutions. This tification to emphasize its connection to robust control trend has provided exciting research opportunities, intro- and the needs of a set-type model characterization[33−35]. duced daunting challenges in seeking new techniques, and This development borrowed several important concepts prompted an urgent need for new tools with practical util- and findings from approximation theory and information- ity. based complexity[36−38], including model approximation In light of new system structures and diversified ap- [39−45] under the H∞ and L1 metrics , worst case iden- plication domains, system identification must adapt itself tification under time-domain data and unknown-but- into broader paradigms. In this broad and holistic doc- bounded noise characterization, frequency-domain H∞ trine, system identification aims to reduce uncertainties identification[46−52], and related model validation[53−55]. on signals, systems, and environment, to extract infor- These methodologies characterize model structures and mation from data and knowledge databases, to compress modeling error bounds to be compatible with the robust information into models, in order to support related in- control frameworks, especially the H∞ and L1 theory de- formation processing procedures such as control, monitor- veloped in the 1980s. Emphasis of complexity analysis in ing, diagnosis, decision, and coordination[59]. Information this school[52, 56−58] has introduced new tools into the field processing, storage, transportation, and sharing take valu- of complexity analysis and enhanced our understanding of able resources from data, communications, computation, fundamental limitations of identification methods and data and measurements; and as such optimal resource utility structures. On the other hand, certain emerging concerns and complexity analysis become mandatory. While sys- with the methodologies, such as conservativeness of the tem identification enjoys distinct features, it shares similar bounds, time complexity in reducing model uncertainty, goals to feedback mechanism, robust control frameworks, computational complexity in finding optimal models, and and adaptation procedures, which employ different simple and practical algorithms for adaptation, provided to process information and counteract uncertainty. As a much-needed food for thought for a holistic viewpoint of result, studies of system identification can benefit from an modeling processes, and as such energized the field of sys- integrated and interdisciplinary approach in which coordi- tem identification towards developing broader frameworks nation of information processing and resource distributions and accommodating new application advancement. can be optimized. Technology advancement during the past two decades in Mathematically, system identification is an inverse map- many frontier fields, especially in information technology, ping problem: seeking an inverse of the suitable static, has fundamentally changed the landscape for control sys- dynamic, or functional mapping from unknown internal tem science and engineering. Classical industrial process parameters or states to measurable variables. Data min- control systems were mostly single-loop and individually- ing, , machine learning, and statistics, designed controllers, and only sporadically considered sys- among others, are pursuing similar objectives. Conse- tem interactions and coordinations. With penetration of quently, system identification can absorb ideas, methods, networking concepts, communication systems, and complex and algorithms from these fields to enrich its own tool- systems into many application areas, systems have experi- boxes. enced profound structural expansion and become increas- In the following sections, we describe some areas and ingly interconnected. For example, automotive systems are issues in which system identification can be studied and linked by communication networks both intra-vehicle and expanded to, hopefully, assist and impact technology devel- in regional V2V (vehicle to vehicle) and V2I (vehicle to opment. As expected, the descriptions of future directions infrastructure) highway systems; airplanes and space shut- are always influenced by the writers limited knowledge and tles experience thrust into more extreme conditions in their bias. Many predictions may turn out to be irrelevant in speeds, power, temperature, and system complexity; sys- their parts or entirety. The purpose of this article is to point tems biology departs from traditional reductionism and fo- out some promising and important aspects of system iden- cuses on biological interconnections of cellular networks; tification in some new paradigms, suggest some focus areas, medical devices are increasingly employing micro-electro- identify key technical obstacles and potential approaches, mechanical systems (MEMS) and NANO technologies; dis- and most importantly stimulate innovative ideas from oth- tributed renewable generators and smart grids have cre- ers and open the forum for further discussions. ated many microgrids with communication networks; ad- The remaining part of the paper is organized into the vanced sensor networks are inherently team operated; inter- following sections. Section 2 discusses broader types of net search and information processing engines have entered uncertainties that system identification should consider. cloud computing age; and the list goes on. Communication systems play a pivotal role in networked These new technology progresses have intrinsic needs for systems. Section 3 highlights some new issues that are system identification. For instance, a key aspect of systems unique to identification of systems that contain commu- biology is mathematical modeling of cellular networks and nication channels. When systems become interconnected their interactions, such as gene regulatory networks, to un- and more complex, they naturally introduce nonlinearity derstand mechanism of inheritance, mutation, and living and random factors. Although these have been extensively No. 7 WANG Le Yi et al.: System Identification: New Paradigms, Challenges, and Opportunities 935 studied in traditional system identification, numerous new resent mappings of unknown structures can produce model issues arise that present new challenges. These are sum- mismatching; and clustering high-complexity system com- marized in Section 4. Section 5 is devoted to emergent ponents will inevitably leave structural uncertainties. Such phenomena for system identification caused by new data uncertainties are not of random nature since they do not types and patterns from the information age. Section 6 change randomly with each observation point, and hence do elaborates reasons why complexity analysis should draw not demonstrate “averaging” effects of random phenomena. more attention in relation to smart usage of resources. Em- A fundamental consequence is that their effects cannot be bedding system identification within intended applications, attenuated by applying the laws of large numbers or cen- objective-oriented system identification is delineated in Sec- tral limit theorems, nor appropriately described by statis- tion 7 such that more diversified performance indices and tical analysis. These uncertainties directly impact estima- criteria can be germinated. Section 8 encourages software tion accuracy and characterization, and hence they need tool development that is user-friendly and efficient. Finally, to be included in system identification. Such uncertainties Section 9 summarizes the key viewpoints of this article and are more suitably represented by deterministic worst-case suggests several pathways to nurture new paradigms of sys- types[31, 64−68]. Other than some limited explorations[66], tem identification. incorporation of both worst-case and stochastic uncertain- This article is based on its earlier Chinese version[60], ties remains an open field. with some alterations. For example, a battery system involves complicated chemical and electrical processes. Nano- and micro-level 2 Accommodation of broader types of models exist, but are too complicated for real-time system uncertainties identification. Typically, focusing on their macro behavior means that the model structure we choose involves errors Traditional system identification was mostly focused on from clustering system components, model mismatching random additive observation noises and actuator noises, as due to unknown model structures, and unmodeled dynam- well as some randomly jumping processes affecting system ics when a low-order differential equation model is used. parameters. Observation noises typically arise from sensor 3) Uncertainties from lack of data and informa- thermal noise, measurement errors, and lower-level com- tion. There are uncertainties that may be of random na- munication errors. A typical paradigm of system identifi- ture, but their statistical properties cannot be obtained due cation states that when an input is applied to a dynamic to lack of data or related information. For example, when system and the system output is measured with additive we model a communication network for a mobile system random noise corruptions, system identification aims to (such as autonomous highway vehicles in platoons, or un- acquire as much information on the systems characteris- manned aerial, ground, and underwater vehicles), the ef- tic mappings or parameters as possible from such noise- fects of “mobility” and “terrain conditions” are extremely infested observations. In this framework, stochastic anal- difficult to model. Unless the terrains are repeatedly used, ysis methods[61−63] have played important roles. It is well the data we receive cannot be accumulated to suffice a sta- perceived that they will retain their prominent positions in tistical analysis of the network model. Similarly, reliable new paradigms of system identification. However, practi- dynamic models for cellular networks require real-time and cal systems encounter much broader types of uncertainties. large-throughput data in next-generation DNA sequencing, Incorporating more diversified uncertainty types in system which are still unavailable at present due to limitations identification will lead to new directions and methodologies in instrumentation. Vague and imprecise descriptions and of system identification beyond classical stochastic analysis. partial information must be used to facilitate system iden- 1) Expanded stochastic noise types. While addi- tification and information processing under these circum- tive random observation noises form the central scenario stances. in classical system identification, new applications will 4) Uncertainties due to lack of computational ca- much broaden the varieties of noises. In new applications, pability. Even for systems that have well established stochastic uncertainties will stem from more diversified sce- model structures, computational capability limitations may narios, such as simplified sensors like binary-valued sensors, not allow us to build such a model for system identifica- data coding and decoding, compression, random transmis- tion. For example, a weather forecasting system involves sion delays in communication systems, or even artificially many factors and their histories. While historical data may added for facilitating information treatment. be available, factoring them into the model will render an Correlated noises with time- and space-dependent statis- enormous computational burden that makes “forecasting” tic characterizations, network structure-dependent noise irrelevant. Eliminating many factors which are deemed types, random sampling schemes, asynchronous and un- “minor” so that the model building process becomes feasi- coordinated random streaming of data in different parts of ble must be compensated with a description of “truncated” a networked system, and multiplicative noises are some ex- subsystems and their impact, which is usually not random. amples of new types of stochastic noises that need to be Studies of complexity issues in such cases will become in- accommodated. creasingly important for quantitatively understanding how 2) Deterministic worst-case types. Muchofa to use computational resources smartly. systems structural uncertainties arise from model simpli- 5) Uncertainties due to structural switching. fication. For instance, representing an infinite dimensional Complex systems are characterized by interactions of their system by a finite dimensional system or a higher-order subsystems. Such interactions are exemplified by their system by a lower-order one may introduce unmodeled dy- network topologies, which often change with time. Com- namics; using a to approximate locally a non- munication channels are assigned according to data traffic linear function or using simpler nonlinear functions to rep- and priorities, and hence are always dynamically changing 936 ACTA AUTOMATICA SINICA Vol. 39 their routing topologies. In internet searching and com- modules, whose real-time characterization requires a new puting engines such as Google search, parallel processing, tool set of networked identification. Biological systems and and cloud computing, data are streamed from multiple and human-patient modeling must include interactions of many ever-changing pathways and reconfigured back to their orig- subsystems, from blood circulation to metabolism, from inal structures at the receiving site. When such changes immune systems to neural signal transmissions. Highway cannot be directly observed, they must be viewed as uncer- vehicle control, as well as unmanned aerial, ground, and tainties. This is a vastly open field in which the questions of underwater vehicles often form mobile teams to accom- topology dynamics, their impact on data accuracy and re- plish a coordinated mission. Distributed computing and liability, and information processing robustness under such information processing, such as cloud computing, will take dynamic networks remain largely unknown at present. over dedicated computer systems to become the new norm. What is needed in this direction is to include multi- Systems biology, material science, patient management, ple types of uncertainty descriptions in designated identifi- weather forecasts, and finance frequently employ the multi- cation problems. Coexistence of multi-uncertainty types scale modeling structures that seek to seamlessly move from is easily understood in practical systems. For instance, molecular or even atom level subsystems to congregated while unmodeled dynamics (from order reduction) and higher level models for targeted information processing. model mismatch (from function simplification) should be These problems demand new identification methods[70] described in an uncertainty set, modeling noises in stochas- and have already started to attract attention[71−72].The tic frameworks is more reasonable and less conservative main new challenges derived from the networks include − than the worst-case formulation[66 67]. In addition, partial communication constraints and characterizations, dynamic information on the system will add another piece of struc- switching network topologies, data flow constraints, and tural information. Potential approaches include: 1) within network complexity issues. To accommodate such new de- the framework of the ODE approaches for convergence velopments, system identification must consider network- , analysis of algorithms[9 12],one related issues including, but certainly not limited to, the may use differential inclusions to accommodate the impact following aspects[73−74]. of worst-case-type uncertainties; 2) in convergence anal- 1) Local information. Networked systems can be of ysis using martingale convergence theorems[69], one may enormous scales. Information traffic through the network consider the worst-case scenarios when the underlying sys- creates overhead costs, uses precious network resources, tems have deterministic uncertainties; 3) in estimation er- and causes congestions. It is desirable that data acqui- ror characterization, one may use worst-case probabilistic sition be confined to neighborhood data flows. How to use errors when applying standard bounds and large deviations the network structures to propagate processed information, principles; 4) in regime-switching systems, one may deal rather than centralized raw data storage and processing, for with integrated models of hybrid systems and formulate networked system identification will be of great utility in joint identification problems. In this pursuit, it is necessary such systems. to study network topology uncertainties so that networked 2) Communication constraints and uncertainties. system identification can be properly studied. At present, Unlike dedicated wired grids, wireless communications are research effort in integrating different types of uncertainties far more dynamic and uncertain. Communications are con- in system identification is increasing but remains sporadic. strained by power limits and available bandwidths. Typi- More persistent and organized team efforts will be needed cal communication systems involve sampling, quantization, to advance this research frontier. data compression, and source coding to reduce data sizes and increase transmission reliability. Wireless communi- 3 System identification under net- cations are usually shared by many users. Packet loss, worked systems and communications transmission delays, errors in transmission are common. The impact of such uncertainties on system identification System identification has become a mature field in stan- must be carefully studied. These uncertainties are influ- dard system settings, mostly in small-scale and single-loop enced by data flow throughput and power level, and can be structures. On the other hand, one of the dominant trends managed by transmission protocols and coding/decoding in physical system developments is that systems become schemes. Such new elements demand a fresh look at sys- increasingly interconnected. More often than not, system tem identification. The recent development of quantized interconnections are provided by communication networks, identification and sampling schemes for state estimation is [24−25, 75] which include typical wireless communication systems and an effort in this direction . The inclusion of com- signaling systems of living organisms such as cellular signal- munication design as part of system identification will be ing. Unlike traditional process control systems in which one of great interest. concentrates on the control of one process at a time, nowa- 3) Reliability of system identification under net- days system and coordination are of substantial work topology variations. Unique to internet and wire- proportions that demand a new look at system identifica- less communications, communication networks that link tion frameworks. subsystems are changing all the time. Especially in mo- Examples are numerous. In smart grids, renewable and bile systems or networks of high traffic volume, channels distributed energy generators, controllable loads, smart me- can be lost and connected randomly due to signal path ters, phaser measurements, and the concepts of micro- changes, terrain conditions, and competition for channels grids have broken the legacy notion of the standard power by users of different priorities. Burst-type traffic, such as grid structures and mandated new methods for information conversations, uses network resources for a short time in- gathering and characterization of subsystems. Large-scale terval with large demands and then releases them. On the battery systems consist of thousands of battery cells and other hand, system identification is usually a steady task, No. 7 WANG Le Yi et al.: System Identification: New Paradigms, Challenges, and Opportunities 937 requiring persistent data to maintain its functionality. Con- versified. Current research on nonlinear system identifica- sequently, networked system identification under randomly tion is highly active, fruitful, and exciting on the one hand, switching network topologies and other uncertainties will but lacks inter-connections and collaborations on the other. be a worthy area for exploration. Relatively extensive results exist for nonlinear systems of 4) Identification of network structures. Unlike tra- special structures, such as Wiener systems[11, 82−83],Ham- ditional system identification in which model structures can merstein systems[84−88], and their extensions[89−90],non- be selected from a few typical choices, complex and in- linear autoregressive model with exogenous input (ARX) terconnected systems must identify system structures[76]. systems[91], systems with quantized observations[92−93], − This is especially true in gene regulation networks[77 79]. kernel and subspace methods[94−95], and regime switching Understanding cell-cell interactions and signaling mecha- systems[96−97], among others. nism from molecular and cell biology will form a founda- System identification will encounter more and more time tion for developing potential networked model structures varying, stochastic, and nonlinear components. First, if to explain emergent properties of proteins, but present a system parameters vary with time or operating conditions, major challenge for network structure identification. or experience sudden changes, their behavior resembles in- To deal with the above challenges, some recent efforts ternal states. Joint estimation of system parameters and appear to be promising and worth further investigation. states inevitably introduces nonlinear problems. Essential We list a few of them here. 1) We need a theory of system properties of such systems, such as joint observability, iden- identification under irregular and asynchronous sampling tifiability, and signal persistent excitation conditions are times[75]. Communication networks introduce packet losses largely unknown at present. When identified models are and delays. Consequently, data arrive at unpredictable used in control, decision, and optimization in real time, ran- times. Input excitation signals can be commanded but will dom noises will enter the system through nonlinear path- be executed at some unknown times due to the same rea- ways, leading to stochastic nonlinear systems. Typical en- son. 2) We need a theory of system identification for de- vironment information such as noise stochastic character- lay systems. Delays introduce infinite dimensional systems izations is difficult to obtain a priori, and hence must be and force consideration of functional stochastic or regular extracted from observation data also. Combined identifi- differential-integral equations. To facilitate control design cation of system parameters and environmental conditions and adaptation, the delay time needs also to be estimated. is a nonlinear problem. Finally, time variation will result 3) We need a theory of system identification using localized in irreducible errors, similar to the famous “Uncertainty or neighborhood information. This information-structure Principle” in physics[98]. Since integration of system iden- constraint may be formulated within the identification algo- tification and control in real-time implementation often in- rithms (adaptive filtering, recursive least-squares, stochas- troduces time varying components, irreducible errors and tic approximation, etc.) as a structural matrix that defines time/space complexity in stochastic nonlinear identification the information flow topology. Convergence analysis with problems are of essential importance. such constraints will lead to a meaningful new area of net- Along with the tremendous scientific progress in the worked system identification. 4) We need a comprehensive last few decades, boundaries among different research do- theory of joint system identification and state estimation mains become increasingly blurred, and correspondingly of hybrid systems that involve system uncertainty and un- new identification and estimation problems for stochastic known events. Such a theory will include communication nonlinear systems emerge from diverse areas. These are networks as part of identification problems. exemplified by system modeling and identification of smart- material actuators, financial data modeling and prediction, 4 Identification of stochastic and non- the structural and functional inference of genetic regula- linear systems tory networks, and the key node detection of world wide web and social networks, among others. These topics are The identification of stochastic systems has a long his- challenging, require collaboration among experts with dif- tory and now is in a new development stage, for instance: ferent backgrounds, and also open great opportunities. from linear systems to nonlinear systems; from a single pro- New schemes that can potentially promote expanded so- cess to networked systems; from control-oriented focus to lutions to stochastic nonlinear identification may encom- interdisciplinary research, such as systems biology, avia- pass cascaded and hierarchical recursive algorithms, set- tion and aerospace technology, quantum and nano science, valued system identification, stochastic subspace methods, finance, etc. kernel identification, etc. After decades of extensive research, stochastic identifi- cation frameworks have shown clear signs of maturity in 5 System identification in the era of theoretical foundation, algorithm development and refine- data explosion ment, accuracy and convergence properties, and applica- tions. Typical tools include Chebyshev-Markov-Stieltjes Historically, acquiring information is expensive since it inequalities, Chernoff bound, martingale convergence the- involves sensors, measurement devices, and wiring and orems, laws of large numbers, central limit theorems, large packaging. On the other hand, due to the standard wired deviations principles, Markov chains, stochastic differential structure and readily available on-board computers, infor- equations, etc. mation transformation is easy and information process- Nonlinear system identification has been pursued by ing in a computer is relatively cheap. Under such sce- many researchers[80−81]. Generally speaking, the nonlinear narios, data acquisition is well-specified, -designed, and - phenomenon is the rule rather than the exception. Con- controlled, and the collected data are all useful. Conse- sequently, nonlinear system identification is inherently di- quently, in typical system identification experiments, we 938 ACTA AUTOMATICA SINICA Vol. 39 focus on choosing correct input forms to make them “per- rors that are typically associated with low-power transmis- sistently exciting” and the corresponding system outputs sion. To reduce bandwidth usage, one needs to reduce are well defined. In the process, almost no unnecessary sampling rate, use low-resolution quantization, apply data information is generated. compression, decrease data redundancy, employ partial in- The new scenarios are emerging: systems are increas- formation, etc. All of these considerations are relatively ingly inter-connected by wireless networks; advanced data new to system identification and form promising grounds acquisition devices are placed in many ports to serve mul- for new research directions. tiple purposes. For example, smart meters are now in- In traditional system settings, resources are commonly stalled at millions of end users to support smart grid de- related to data acquisition costs, including sensors, sensor velopments. Traffic conditions are monitored using V2V types, sensor locations, numbers of scenarios to be cov- and V2I communication systems that collect data on traf- ered in identification experiments, data duration, sampling fic flows, weather conditions, and accidents, among others. rates, and data storage needs, among others. In this sense, Such devices indiscriminately collect data in fast sampling if one can eliminate a sensor, use a cheaper sensor, or rely schemes and in large volume before storing them in desig- on smaller data sizes, resource utility is reduced. Such re- nated databases. For system identification, such data flow duction will have impact on identification quality (such as schemes represent a new scenario in which data acquisi- the accuracy or sizes of uncertainty sets) and speed (such as tion is cheap, but data transformation, transportation, and convergence rates, Cramer-Rao (CR) bounds, information processing become very expensive. This trend has gener- criteria). ated a hot buzz around “data mining” in the computing Rigorous studies of such impact are of fundamental na- community[99]. ture and become more and more important. Historically, System identification development has not adapted itself worst-case identification employed information-based com- to be comfortable with such new information structures plexity and complexity notions in approximation theory yet. In such scenarios, useful data must be intelligently to study model complexity, identification speed, and irre- extracted from the databases, compressed, quantized, and ducible errors[36−38]. Similarly, pursuit of irreducible iden- coded for reduced complexity to facilitate transformation tification errors by using CR lower bounds and informa- through networks, then reconstructed and processed for tion criteria in model complexity reduction has been well system identification. There is a vast opportunity to de- accepted as essential to system identification in stochastic velop new paradigms of system identification in this do- frameworks[8, 12, 93]. main. There are tremendous opportunities in exploring com- Thanks to their data specificity, massive data carry in- plexity issues in system identification and its connection formation in vastly different structures and contents. As with control[52, 104]. Due to technical difficulties, this has such the generic study of massive data processing and algo- not been a common pursuit in the past. In new paradigms rithms may not yield efficient tools. From this viewpoint, of system identification, one should look into building new it is necessary to take an interdisciplinary approach, by complexity results from the profound theoretical founda- which one identifies information structure, designates data tions in approximation theory, statistics, information the- formats, describes “useful” information for targeted appli- ory, and computational complexity. The following four pil- cations, derives causal data relationships to model struc- lars can be integrated to develop a new complexity theory tures and parameters, devises efficient algorithms, and even for system identification in a cross-disciplinary platform. develops new embedded or distributed systems for system 1) Approximation theory. Identification time identification with massive and sparse-information data. It and sampling complexity[52, 56, 68, 98], model complexity[57], is always desirable to treat (or pre-treat) data at local data- feedback complexity[58] can be viewed as different types generating sites to avoid unnecessary data shuffling. Con- of complexity results in approximation theory. In fact, sequently, parallel, asynchronous, distributed data process- in a broader view, a feedback systems capability in pro- ing will naturally be characterizing features in large-data viding robustness can be expressed in approximation the- system identification paradigms. In this respect, system ory as well. Consequently, when we describe unmodeled identification can borrow ideas from many branches of data dynamics, model mismatch, model uncertainty set, re- mining, machine learning, distributed computing, and pat- duction of uncertainty by feedback, or uncertainty reduc- tern recognition communities, which started to deal with tion by acquired data, our complexity analysis can ben- − massive data issues decades ago[99 103]. efit greatly from the complexity results in approximation theory[36−37, 103]. 6 Complexity and smart utility of re- 2) Statistics. The celebrated Cramer-Rao lower [105] [106] sources bound and Fisher information define precisely the information contents contained in noise-corrupted data about system parameters. Fisher information matrices Studies of complexity intend to understand fundamental [106−107] relationships between resources and goals. To achieve un- have found diversified applications . However, it re- certainty reduction via system identification, one encoun- mains a puzzle how to combine this characterization with ters issues of resource consumptions and limitations. De- communications and approximation theory to derive a com- plexity theory of broader appeals. pending on applications, resources can take different forms. For example, using a cheap binary-valued sensor can reduce 3) Information theory. Shannon s information the- cost, but increases data uncertainty. In networked system ory is a fundamental foundation for information. Its im- identification, the most commonly cited network resources pact on source coding, channel coding, data distortion, and data compression are well recognized[108−111].Inad- are power and bandwidth. To reduce power consumption, system identification needs to tolerate more delays and er- dition, Shannon s sampling theory characterizes sampling No. 7 WANG Le Yi et al.: System Identification: New Paradigms, Challenges, and Opportunities 939 complexity in . In system identification smart design of control structures and functions in assisting of networked systems, the information theory should be a identification is highly interesting and promising. foundation as well. However, it must be expanded to in- 3) Networked systems interact, and subsystems influence corporate other complexity measures. and possess information about their neighbors. Also, sub- 4) Computational complexity. Computational com- systems have different objectives and specifications. Using plexity was traditionally studied in computer logic, au- networked information to help identification of subsystems tomata, and computer languages[112]. It focused on classi- will yield better solutions than centralized processing or fication of computational procedures in terms of P-class vs. uniform commitment of resources to all subsystems. At NP-class. Generally, a P-type problem is viewed as “com- present, this direction has not been actively researched. putable” and an NP-type problem is “intractable”. How- Goal-oriented information processing takes more promi- ever, for system identification and control applications, this nent roles in some related fields, such as machine learning, fundamental characterization of computational complexity data mining, congregation of Markov chains, and Petri nets, is not sufficiently detailed to provide a viable constraint perhaps due to their encounters with resource limitations, on system identification algorithms. At present, it is not curse of dimensions, and computational complexity. Con- clear how this can be resolved. On the other hand, the con- cepts and ideas from these fields can be of help for system cepts that have been used in physics, quantum computing, identification. molecular modeling, may provide some guidance, including algorithmic complexity and information, entropy, etc.[113]. 8 Customer services: user-friendly and efficient tools 7 Objective-oriented and integrated Tools for system identification have been pursued, mostly system identification in code development, such as System Identification Tool- box in Matlab/Simulink[117−120]. At present, applying sys- System identification is focused on modeling a system tem identification methods in practical systems requires ad- using real-time operational data. The terms of “control- vanced training in this field. Although it is easily observed oriented identification” and “adaptive control” represent that system identification should be part of many practi- the most pronounced effort in arguing that system identi- [114] cal systems, its online applications for real-time updating fication should be integrated with control . Integrated and controller adaptation are still limited. Currently, most system identification aims to coordinate its functions and modeling efforts are off-line. In particular, for more com- complexity with its goals in control, decision, monitors, and plicated systems, even traditional systems such as chemi- diagnosis, so that resources can be saved. cal processes and automotive systems of multi-input-multi- 1) First, system identification should be formulated output configurations, the current tools for system identi- within its usage so that it will achieve the goals without fication remain inadequate. excessive usage of its resources. Otherwise, valuable re- Learning from consumer product development, espe- sources will be wasted. For instance, for most fault detec- cially consumer electronics, one observes that while most tion and diagnosis problems, the objectives are to distin- people do not understand high technology concepts and de- guish if the system is in the “normal” or a “fault” tails in iPhones, Facebook, or cloud computing, people use has occurred. In this aspect, it is unnecessary to pursue them routinely and happily. This is due to the user-friendly convergence of parameter estimates or even high precision. tools that are added to the core technologies in these gad- Another case is vital sign monitoring for humans, including gets. To make system identification part of routine utility heart rates, respiratory rates, blood pressures, etc. For pa- in control systems, control strategy adaptations, and re- tients with normal heart and lung functions but high blood liable monitoring and diagnosis, tool packages need to be pressures, more resources should be directed toward blood developed so that engineers of very limited background in pressure estimation for closer monitoring on these higher system identification can learn to use the tool packages eas- priority variables. In other words, critical parameters and ily and quickly. Image a hand-held box of “Smart SYSID” non-essential parameters can be assigned different accuracy or a special-purpose embedded system: signal connections and priority scores in goal-oriented identification. from the physical system prompt a window of a few man- 2) Both system identification and feedback mechanisms ual items; and a few button pushes will generate system are to enhance a system s ability in dealing with uncer- characteristics and models. tainty, and as a result their functions can assist each other − in a coherent platform[115 116]. The reason for uncertainty 9 Conclusions and suggestions reduction is related to its targeted applications. Reduction of uncertainty on the system may imply more accurate con- This article summarizes promising and important direc- trol, better monitoring reliability, and better system pro- tions for system identification, within the doctrine of un- tection through diagnosis. On the other hand, there are certainty, information, and complexity, and its perceivable other mechanisms that can help to achieve these objectives. impact in several emerging and critical application areas. For example, feedback mechanism can, when used properly, We hope that this preliminary exploration of strategic di- achieve accurate control under large system uncertainties rections can stimulate further thinking in the field, and we (robust and adaptive control). Partial information from wholeheartedly welcome comments and opinions. interconnected systems can help in diagnosis (medical di- In a broad sense, system identification aims to reduce un- agnosis and deductive learning). Consequently, the task of certainties for assisting decisions. Along with new technol- system identification, when viewed as part of a larger sys- ogy advancement, the field must reposition itself to create tem structure, will vary, depending on its usage. It may new frameworks to meet emergent challenges in terms of not need to be so accurate if other means can help. The problem formulation, methodology development, theoreti- 940 ACTA AUTOMATICA SINICA Vol. 39 cal foundation, software tools and hardware modules, and References user application packages. This will require coordinated 1 Zadeh L A. On the identification problem. IEEE Transac- effort, team collaboration, and persistent and designated tions on Circuits and Systems, 1956, 3(4): 277−281 funding. To accommodate such developments, we offer a 2 Cai Ji-Bin. System Identification. Beijing: Beijing Institute few suggestions. of Technology Press, 1989 (in Chinese) 1) Critical areas. In consideration of the vast spectrum 3 Guo Lei, Cheng Dai-Zhan, Feng De-Xing. Introduction to of new application areas and limited resources, it is advis- : from Basic Concepts to Research Frontiers, able to coordinate research activity towards several thrust Beijing: Science Press, 2005 (in Chinese) areas that have been identified as the high-impact areas by 4 Ljung L, Vicino A. Guest editorial: special issue on sys- many countries, which have experienced rapid infrastruc- tem identification. IEEE Transactions on Automatic Con- trol, 2005, 50(10): 1473 ture development and vast R&D investment. In particular, 5 Gevers M. A personal view of the development of system the following three areas are worth special attention: 1) Life identification: a 30-year journey through an exciting field. science advancement. These are exemplified by systems bi- IEEE Control Systems Magazine, 2006, 26(6): 93−105 ology, medical devices, sensing and control technologies for 6 Ljung L. Perspectives on system identification. In: Proceed- drug delivery and human monitoring, and modeling and ings of the 17th IFAC World Congress. Seoul, South Korea: control for human systems. 2) Advanced transportation IFAC, 2008. 7172−7184 systems. These include electric and hybrid vehicles, bat- 7 Ljung L, Hjalmarsson H, Ohlsson H. Four encounters with teries, automated highway vehicle management, unmanned system identification. European Journal of Control, 2011, 17(5−6): 449−471 aerial, ground, and underwater vehicles, mobile robots, dis- 8 Ljung L. System Identification: Theory for the User.Engle- tributed sensing systems, etc. 3) Renewable energy and wood Cliffs, NJ: Prentice-Hall, 1987 green economy. Distributed generation, wind, photovoltaic, 9ChenHF,GuoL.Identification and Stochastic Adaptive thermal generators, smart grids, and distributed optimiza- Control. Boston, MA: Birkh¨auser, 1991 tion for power dispatching are typical examples. 10 Ljung L. Analysis of recursive stochastic algorithms. IEEE 2) Open access databases on system identifica- Transactions on Automatic Control, 1977, 22(4): 551−575 tion. Establishing openly accessible data centers for meth- 11 Chen H F. Recursive identification for Wiener model with ods, algorithms, theoretical findings, application bench- discontinuous piece-wise linear function. IEEE Transactions − marks, and software packages will facilitate research col- on Automatic Control, 2006, 51(3): 390 400 laboration for researchers, engineers, and students. To pro- 12 Kushner H J, Yin G. Stochastic Approximation and Recur- sive Algorithms and Applications (2nd edition).NewYork: mote technology transfer and enhance impact, the field s Springer-Verlag, 2003 development must be interdisciplinary with broad partici- 13 Chen H F, Zhu Y M. Stochastic approximation procedures pation. Collective development of the above databases is with randomly varying truncations. Science in China, Series one of the means to stimulate team coordination and col- A, 1986, 29(9): 914−926 lective wisdom. 14 Chen H F. Stochastic Approximation and Its Applications. 3) Advanced experimental centers. New paradigms Dordrecht, The Netherlands: Kluwer Academic, 2002 of system identification will be characterized by large data 15 Akaike H. A new look at the identification. and information processing, extensive networked systems, IEEE Transactions on Automatic Control, 1974, 19(6): 716 −723 and interdisciplinary research activity. For example, ex- 16 Burnham K P, Anderson D R. Multimodel inference: un- perimental data acquisition and system validation on high- derstanding AIC and BIC in . Sociological speed trains, space shuttles, smart grids, autonomous flying Methods and Research, 2004, 33(2): 261−304 machines, and grid-scale battery systems require expansive 17 Rissanen J. Modeling by shortest data description. Auto- testing facilities, fast computing machines, and vast com- matica, 1978, 14(5): 465−471 munication network infrastructures. These are typically 18 Zheng W X, Feng C B. Identification of stochastic time lag beyond most individual research groups or institutions. By systems in the presence of colored noise. Automatica, 1990, − providing such national facilities to all research groups, 26(4): 769 779 many more teams can pursue new strategic directions, cre- 19 Zheng W X, Feng C B. A bias-correction method for indi- rect identification of closed-loop systems. Automatica, 1995, ating a healthy and stimulating competitive environment 31(7): 1019−1024 to advance new system identification technologies. 20 Soderstrom T. Accuracy analysis of the Frisch scheme for 4) Talent development. Paradigm shifting from tra- identifying errors-in-variables systems. IEEE Transactions ditional small and local systems to networked systems of on Automatic Control, 2007, 52(6): 985−997 various technology areas must be supported by new in- 21 S¨oderstr¨omT, Soverini U, Mahata K. Perspectives on errors- terdisciplinary talents who have both strong theoretical in-variables estimation for dynamic systems. Signal Process- − training and in-depth domain knowledge. As compared ing, 2002, 82(8): 1139 1154 to traditional system identification, the new strategic di- 22 Forssel U, Ljung L. Closed-loop identification revisited. Au- tomatica, 1999, 35(7): 1215−1241 rections are far more uncertain and require new ways of 23 Obinata G, Brian D O A. Model Reduction for Control Sys- thinking in terms of problem definition, model structures, tem Design. London: Springer, 2001 constraints, tools for solving new problems, software pack- 24 Kalman R E. Design of a self-optimising control system. ages for , data acquisition, optimization, etc. In- Transactions on ASME, 1958, 80: 468−478 terdisciplinary talents are integrators who will learn mate- 25 Astr¨om K J, Wittenmark B. On self tuning regulators. Au- rial from new fields, formulate meaningful problems, con- tomatica, 1973, 9(2): 185−199 solidate teams of diversified expertise, and seek usable so- 26 Guo L. Convergence and logarithm laws of self-tuning regu- lutions in sync with technology advancement so that the lators. Automatica, 1995, 31(3): 435−450 domain experts of specific areas can actually benefit from 27 Durrett R. Probability: Theory and Examples (Cambridge the findings. Series in Statistical and Probabilistic Mathematics) (4th edi- tion). Cambridge: Cambridge University Press, 2010 No. 7 WANG Le Yi et al.: System Identification: New Paradigms, Challenges, and Opportunities 941

28 Zhou Tong. Introduction to Control-Oriented System Iden- 51 Dahleh M A, Theodosopoulos T V, Tsitsiklis J N. The sam- tification. Beijing: Tsinghua University Press, 2004 (in Chi- ple complexity of worst-case identification of FIR linear sys- nese) tems. Systems and Control Letters, 1993, 20: 157−166 29 Milanese M, Belforte G. and uncertainty 52 Chong C Y, Kumar S P. Sensor networks: evolution, op- intervals evaluation in presence of unknown but bounded er- portunities, and challenges. Proceedings of the IEEE, 2003, rors: linear families of models and . IEEE Trans- 91(8): 1247−1256 actions on Automatic Control, 1982, 27(2): 408−414 53 Wang J D, Zheng W X, Chen T W. Identification of lin- 30 Milanese M, Vicino A. Optimal estimation theory for ear dynamic systems operating in a networked environment. dynamic systems with set membership uncertainty: an Automatica, 2009, 45(12): 2763−2772 overview. Automatica, 1991, 27(6): 997−1009 54 Zhang Q, Zhang J F. Distributed parameter estimation over 31 M¨akil¨a P M, Partington J R, Gustafsson T K. Worst-case unreliable networks with Markovian switching topologies. control-relevant identification. Automatica, 1995, 31(12): IEEE Transactions on Automatic Control, 2012, 57(10): 1799−1819 2545−2560 32 Chen J, Gu G X. Control-Oriented System Identification: 55 Liu X H, Goldsmith A. Wireless communication tradeoffs An H∞ Approach. New York: Wiley-Interscience, 2000 in distributed control. In: Proceedings of the 42nd IEEE 33 Smith R S, Dahleh M A [Editor]. The modeling of uncer- Conference on Decision and Control. Maui, HI: IEEE, 2003. tainty in control systems. In: Proceedings of the 1992 Santa 688−694 Barbara Workshop. Berlin, Germany: Springer-Verlag, 1994 56 Yuksel S, Basar T. Optimal signaling policies for decen- 34 Helmicki A J, Jacobson C A, Nett C N. Control oriented tralized multicontroller stabilizability over communication system identification: a worst-case/deterministic approach channels. IEEE Transactions on Automatic Control, 2007, − in H∞. IEEE Transactions on Automatic Control, 1991, 52(10): 1969 1974 36(10): 1163−1176 57 Wang L Y, Zhang J F, Yin G. System identification using 35 Gu G X, Khargonekar P P. Linear and nonlinear algorithms binary sensors. IEEE Transactions on Automatic Control, − for identification in H∞. IEEE Transactions on Automatic 2003, 48(11): 1892 1907 Control, 1992, 37(7): 953−963 58 Wang L Y, Yin G, Zhang J F, Zhao Y L. System Identifica- 36 Gu G X, Khargonekar P P. A class of algorithms for identi- tion with Quantized Observations. Boston, MA: Birkh¨auser, fication in H∞. Automatica, 1992, 28(2): 299−312 2010 37 Chen J, Nett C N. The Caratheodory-Fejer problem and 59 Wang L Y, Li C Y, Yin G, Guo L, Xu C Z. State observability H∞/l1 identification: a approach. IEEE Trans- and observers of linear-time-invariant systems under irregu- actions on Automatic Control, 1995, 40(4): 729−735 lar sampling and sensor limitations. IEEE Transactions on − 38 Chen J, Nett C N, Fan M K H. Worst case system identifi- Automatic Control, 2011, 56(11): 2639 2654 cation in H∞: validation of a priori information, essentially 60 He T, Lu X L, Wu X Q, Lu J A, Zheng W X. Optimization- optimal algorithms, and error bounds. IEEE Transactions based structure identification of dynamical networks. Phys- on Automatic Control, 1995, 40(7): 1260−1265 ica A: Statistical Mechanics and Its Applications, 2013, − 39 Zhou T, Kimura H. Time domain identification for robust 392(4): 1038 1049 control. Systems and Control Letters, 1993, 20(3): 167−178 61 Zhou T, Wang Y L. Causal relationship inference for a large- − 40 Chen J. Frequency-domain tests for validation of linear frac- scale cellular network. , 2010, 26(16): 2020 tional uncertain models. IEEE Transactions on Automatic 2028 Control, 1997, 42(6): 748−760 62 Wang Y L, Zhou T. A relative variation-based method to 41 Poolla K, Khargonekar P, Tikku A, Krause J, Nagpal K. A unraveling gene regulatory networks. PLoS ONE, 2012, 7(2): time-domain approach to model validation. IEEE Transac- e31194 tions on Automatic Control, 1994, 39(5): 951−959 63 Bolouri H, Bower J M. Computational Modeling of Genetic 42 Smith R S, Doyle J C. Model validation: a connection be- and Biochemical Networks. Cambridge, Mass: MIT Press, tween robust control and identification. IEEE Transactions 2001 on Automatic Control, 1992, 37(7): 942−952 64 Billings S A. Identification of nonlinear systems — a survey. 43 Anderson T W. The Statistical Analysis of Time Series.New IEE Proceedings D: Control Theory and Applications, 1980, − York: Wiley, 1971 127(6): 272 285 44 Serfling R J. Approximation Theorems of Mathematical 65 Sj¨oberg J, Zhang Q H, Ljung L, Benveniste A, Delyon Statistics. New York: Wiley, 1980 B, Glorennec P Y, Hjalmarsson H, Juditskys A. Nonlin- ear black-box modeling in system identification: a unified 45 Bohlin T. On the maximum likelihood method of identifi- overview. Automatica, 1995, 31(12): 1691−1724 cation. IBM Journal of Research and Development, 1970, 14(1): 41−51 66 Verhaegen M, Westwick D. Identifying MIMO Wiener sys- tems using subspace model identification methods. Signal 46 Wang L Y. Uncertainty, information and complexity in iden- Processing, 1996, 52(2): 235−258 tification and control. International Journal of Robust and Nonlinear Control, 2000, 10(11−12): 857−874 67 Bai E W, Reyland J Jr. Towards identification of Wiener systems with the least amount of a priori information on the 47 Casini M, Garulli A, Vicino A. Time complexity and input nonlinearity. Automatica, 2008, 44(4): 910−919 design in worst-case identification using binary sensors. In: Proceedings of the 46th IEEE Conference on Decision and 68 Bai E W. An optimal two-stage identification algorithm for Control. New Orleans, LA, USA: IEEE, 2007. 5528−5533 Hammerstein-Wiener nonlinear system. Automatica, 1998, 34(3): 333−338 48 Milanese M, Vicino A. Information-based complexity and nonparametric worst-case system identification. Journal of 69 Ninness B, Gibson S. Quantifying the accuracy of Hammer- Complexity, 1993, 9(4): 427−446 stein model estimation. Automatica, 2002, 38(12): 2037− 2051 49 Wang L Y, Yin G G. Towards a harmonic blending of deter- ministic and stochastic frameworks in information process- 70 Zhu Y. Estimation of an N-L-N Hammerstein-Wiener model. ing. Robustness in Identification and Control, Lecture Notes Automatica, 2002, 38: 1607−1614 in Control and Information Sciences Volume 245. London: 71 Bai E W, Li K. Convergence of the iterative algorithm for Springer-Verlag, 1999. 102−116 a general Hammerstein system identification. Automatica, 50 Wang L Y, Yin G G. Persistent identification of sys- 2010, 46(11): 1891−1896 tems with unmodeled dynamics and exogenous disturbances. 72 Zhao W X, Chen H F. Adaptive tracking and recursive iden- IEEE Transactions on Automatic Control, 2000, 45(7): tification for Hammerstein systems. Automatica, 2009, 45: 1246−1256 2773−2783 942 ACTA AUTOMATICA SINICA Vol. 39

73 Zhao W X, Chen H F. Identification of Wiener, Hammer- 96 Cramer H. Mathematical Methods of Statistics. Princeton stein, and NARX systems as Markov chains with improved NJ: Princeton University Press, 1946 estimates for their nonlinearities. Systems and Control Let- 97 Frieden B R. Science from Fisher Information: A Unification. − ters, 2012, 61(12): 1175 1186 Cambridge: Cambridge University Press, 2004 74 Mu B Q, Chen H F. Recursive identification of Wiener- 98 Hannan E J, Deistler M. The of Linear Hammerstein systems. SIAM Journal on Control and Op- Systems. New York: John Wiley and Sons, 1988 timization, 2012, 50(5): 2621−2658 99 Cover T M, Thomas J A. Elements of Information Theory. 75 Roll J, Nazin A, Ljung L. Nonlinear system identification via New York: Wiley-Interscience, 1991 direct weight optimization. Automatica, 2005, 41(3): 475− 100 Gallager R G. Information Theory and Reliable Communi- 490 cation. New York: John Wiley and Sons, 1968 76 Zhao Y L, Wang L Y, Yin G G, Zhang J F. Identification 101 Gersho A, Gray R M. Vector Quantization and Signal Com- of Wiener systems with binary-valued output observations. pression. Norwell, MA: Kluwer Academic Publishers, 1991 Automatica, 2007, 43(10): 1752−1765 102 Sayood K. Introduction to Data Compression (2nd edition). 77 Wang L Y, Yin G G. Asymptotically efficient parameter es- San Francisco: Morgan Kaufmann, 2000 timation using quantized output observations. Automatica, 2007, 43(7): 1178−1191 103 Davis M D, Sigal R, Weyuker E J. Computability, Com- plexity, and Languages (2nd edition). San Diego: Academic 78 Katayama T. Subspace Methods for System Identification. Press, 1994 London: Springer, 2005 104 Zurek W H. Complexity, Entropy, and the Physics of Infor- 79 Van Overschee P, de Moor B. A unifying theorem for mation. Reading, MA: Addison-Wesley, 1990 three subspace system identification algorithms. Automat- 105 Gevers M. Towards a joint design of identification and con- ica, 1995, 31(12): 1853−1864 trol? Essays on Control: Perspectives in the Theory and Its 80 Vidal R. Recursive identification of switched ARX systems. Applications. Boston, MA: Birkh¨auser, 1993. 111−151 − Automatica, 2008, 44(9): 2274 2287 106 Ljung L. System Identification Toolbox — for Use with 81 Wang J, Chen T. Parameter estimation of periodically Matlab, Users Guide (5th edition). Natick, MA: The Math- switched linear systems. IET Control Theory and Applica- Works, Inc. Sherborn, Mass, 2000 1−350 − tions, 2012, 6(6): 768 775 107 Garnier H, Gilson M, Laurain V. The CONTSID toolbox for 82 Zames G, Lin L, Wang L Y. Fast identification n-widths and Matlab: Extensions and latest developments. In: Proceed- uncertainty principles for LTI and slowly varying systems. ings of the 15th IFAC Symposium on System Identification. IEEE Transactions on Automatic Control, 1994, 39(9): Saint-Malo, France, 2009. 735−740 − 1827 1838 108 Ljung L. Educational aspects of identification software user 83 Tan P N, Steinbach M, Kumar V. Introduction to Data Min- interfaces. In: Proceedings of the 13th IFAC Symposium on ing. Boston, MA: Addison-Wesley, 2006 System Identification. Rotterdam, The Netherlands: IFAC, 84 Sayeed A M. A signal modeling framework for integrated 2003. 1590−1594 design of sensor networks. In: Proceedings of the 2003 IEEE Workshop Statistical Signal Processing. St. Louis, MO, USA: 109 Rivera D E, Lee H, Braun M W, Mittelmann H D. Plant- IEEE, 2003. 7 friendly system identification: a challenge for the process 85 Vapnik V N. Statistical Learning Theory. New York: Wiley- industries. In: Proceedings of the 13th IFAC Symposium on Interscience, 1998 System Identification. Rotterdam, Netherlands: IFAC, 2003. 917−922 86 Vapnik V N. The Nature of Statistical Learning Theory (In- formation Science and Statistics for Engineering and Infor- mation Science) (2nd edition). New York: Springer Verlag, 1999 WA N G L e - Y i Received his Ph. D. de- gree in electrical engineering from McGill 87 Vidyasagar M. Learning and Generalization: With Applica- University, Montreal, Canada, in 1990. tions to Neural Networks (2nd edition). London: Springer, Since 1990, he has been with Wayne State 2003 University, Detroit, Michigan, where he is 88 Kolmogorov A N. On some asymptotic characteristics of currently a professor in the Department completely bounded spaces. Doklady Akademii Nauk SSSR, of Electrical and Computer Engineering. 1956, 108: 385−389 (in Russian) He is a fellow of IEEE. His research inter- 89 Pinkus A. N-Widths in Approximation Theory.NewYork: est covers complexity and information, sys- Springer-Verlag, 1985 tem identification, robust control, H∞ op- timization, time-varying systems, adaptive systems, hybrid and 90 Traub J F, Wasilkowski G W, Wozniakowski H. Information- nonlinear systems. Corresponding author of this paper. based Complexity. New York: Academic Press, 1988 E-mail: [email protected] 91 Wang L Y, Yin G G, Zhang J F, Zhao Y L. Space and time complexities and sensor threshold selection in quantized identification. Automatica, 2008, 44(12): 3014−3024 ZHAO Wen-Xiao Received the bach- elor degree from Shandong University, 92 Lin L, Wang L Y, Zames G. Time complexity and model China in 2003 and the Ph. D. degree in complexity of fast identification of continuous-time LTI operation research and cybernetics from systems. IEEE Transactions on Automatic Control, 1999, − the Institute of Systems Science (ISS), 44(10): 1814 1828 Academy of Mathematics and Systems Sci- 93 Poolla K, Tikku A. On the time complexity of worst-case ence (AMSS), Chinese Academy of Sci- system identification. IEEE Transactions on Automatic Con- ences (CAS), China in 2008. After gradua- trol, 1994, 39(5): 944−950 tion, he worked as a postdoctoral fellow in 94 Tse D C N, Dahleh M A, Tsitsiklis J N. Optimal asymptotic the Department of Automation, Tsinghua identification under bounded disturbances. IEEE Transac- University, and a visiting scholar at the School of Mathemat- tions on Automatic Control, 1993, 38(8): 1176−1190 ics, University of Western Sydney, Australia. He is currently an 95 Zames G. On the metric complexity of causal linear systems: associate professor with ISS, AMSS, CAS. His research interest ε-entropy and ε-dimension for continuous time. IEEE Trans- covers identification, adaptive control, and systems biology. actions on Automatic Control, 1979, 24(2): 222−230 E-mail: [email protected]