Contents - imtuoradea.ro
Transcription
Contents - imtuoradea.ro
Contents Overview and Introduction 1 Systems analysis 1 Hardware architecture 3 Stability theory 5 Control engineering 8 Signal processing 11 Feedback 14 Negative feedback 22 Positive feedback 24 Systems design 33 Systems design 33 Requirement 35 Systems architecture 39 Dynamic models 41 Function model 41 Function block diagram 48 Transfer function 49 Convolution 52 Dynamical system 62 Graph dynamical system 72 Dynamic response 75 First-order logic 75 List of first-order theories 94 Pole splitting 103 Damping ratio 108 Second-order logic 111 Impulse response 116 LTI system theory 118 Routh–Hurwitz stability criterion 129 Feedback properties 133 Control theory 133 Control system 142 Controller (control theory) 146 Process control 149 Three-term control 151 Proportionality (mathematics) 151 Integral 154 Derivative 176 Control methods 191 OLE for process control 191 Nonlinear control 193 Proportional control 196 Optimal control 197 Robust control 203 Adaptive control 205 Stochastic control 207 Neural network 208 Fuzzy logic 217 Expert system 223 The Root-Locus Design Method 231 Root locus 231 Compensation (engineering) 234 Frequency compensation 235 Error detection and correction 237 Linear encoder 244 Gray code 249 Rotary encoder 264 The Frequency-Response Design Method 271 Bode plot 271 Nyquist stability criterion 281 Instability 284 Frequency response 286 Nyquist plot 288 Nichols plot 289 Bandwidth (signal processing) 290 Lead–lag compensator 293 Static VAR compensator 295 PID controller 298 References Article Sources and Contributors 314 Image Sources, Licenses and Contributors 320 Article Licenses License 323 1 Overview and Introduction Systems analysis Systems analysis is the study of sets of interacting entities, including computer systems analysis. This field is closely related to requirements analysis or operations research. It is also "an explicit formal inquiry carried out to help someone (referred to as the decision maker) identify a better course of action and make a better decision than he might otherwise have made."[1] Overview The terms analysis and synthesis come from Greek where they mean respectively "to take apart" and "to put together". These terms are in scientific disciplines from mathematics and logic to economy and psychology to denote similar investigative procedures. Analysis is defined as the procedure by which we break down an intellectual or substantial whole into parts. Synthesis is defined as the procedure by which we combine separate elements or components in order to form a coherent whole.[2] Systems analysis researchers apply methodology to the analysis of systems involved to form an overall picture. System analysis is used in every field where there is a work of developing something. Analysis can also be defined as a series of components that perform organic function together. Information technology The development of a computer-based information system includes a systems analysis phase which produces or enhances the data model which itself is a precursor to creating or enhancing a database (see Christopher J. Date "An Introduction to Database Systems"). There are a number of different approaches to system analysis. When a computer-based information system is developed, systems analysis (according to the Waterfall model) would constitute the following steps: • The development of a feasibility study, involving determining whether a project is economically, socially, technologically and organizationally feasible. • Conducting fact-finding measures, designed to ascertain the requirements of the system's end-users. These typically span interviews, questionnaires, or visual observations of work on the existing system. • Gauging how the end-users would operate the system (in terms of general experience in using computer hardware or software), what the system would be used for etc. Another view outlines a phased approach to the process. This approach breaks systems analysis into 5 phases: • • • • • Scope definition Problem analysis Requirements analysis Logical design Decision analysis Use cases are a widely-used systems analysis modeling tool for identifying and expressing the functional requirements of a system. Each use case is a business scenario or event for which the system must provide a defined response. Use cases evolved out of object-oriented analysis; however, their use as a modeling tool has become common in many other methodologies for system analysis and design. Systems analysis Practitioners Practitioners of systems analysis are often called up to dissect systems that have grown haphazardly to determine the current components of the system. This was shown during the year 2000 re-engineering effort as business and manufacturing processes were examined as part of the Y2K automation upgrades. Employment utilizing systems analysis include systems analyst, business analyst, manufacturing engineer, enterprise architect, etc. While practitioners of systems analysis can be called upon to create new systems, they often modify, expand or document existing systems (processes, procedures and methods). A set of components interact with each other to accomplish some specific purpose. Systems are all around us. Our body is itself a system. A business is also a system. People, money, machine, market and material are the components of business system that work together that achieve the common goal of the organization. References [1] SYSTEMS ANALYSIS (http:/ / web. archive. org/ web/ 20070822025602/ http:/ / pespmc1. vub. ac. be/ ASC/ SYSTEM_ANALY. html) [2] Tom Ritchey, [http://www.swemorph.com/pdf/anaeng-r.pdf Analysis and . External links • Systems Analysis, Modelling and Prediction (SAMP), University of Oxford (http://www.eng.ox.ac.uk/samp) • Software Requirement Analysis using UML (http://www.slideshare.net/dhirajmusings/ software-requirement-analysis-using-uml) article by Dhiraj Shetty. • Introduction to Social Macrodynamics (http://urss.ru/cgi-bin/db.pl?cp=&page=Book&id=34250&lang=en& blang=en&list=Found) • A useful set of guides and a case study about the practical application of business and systems analysis methods (http://www.cilco.co.uk/briefing-studies/index.html) • Complete online tutorial for system analysis and design (http://www.systemsanalysis.co.nr) • A comprehensive description of the discipline of systems analysis from Simmons College, Boston, MA, USA (www.simmons.edu) (http://web.simmons.edu/~benoit/LIS486/SystemsAnalysis.html) 2 Hardware architecture 3 Hardware architecture In engineering, hardware architecture refers to the identification of a system's physical components and their interrelationships. This description, often called a hardware design model, allows hardware designers to understand how their components fit into a system architecture and provides software component designers important information needed for software development and integration. Clear definition of a hardware architecture allows the various traditional engineering disciplines (e.g., electrical and mechanical engineering) to work more effectively together to develop and manufacture new machines, devices and components.[1] An orthographically projected diagram of the F-117A Nighthawk. Hardware is also an expression used within the computer engineering industry to explicitly distinguish the (electronic computer) hardware from the software which runs in it. But hardware, within the automation and software engineering disciplines, need not simply be a computer of some sort. A modern automobile runs vastly more software than the Apollo spacecraft. Also, modern aircraft cannot function without running tens of millions of computer instructions embedded and distributed throughout the aircraft and resident in both standard computer hardware and in specialized hardward components An F-117 conducts a live exercise bombing run such as IC wired logic gates, analog and hybrid devices, and other using GBU-27 laser-guided bombs. digital components. The need to effectively model how separate physical components combine to form complex systems is important over a wide range of applications, including computers, personal digital assistants (PDAs), cell phones, surgical instrumentation, satellites, and submarines. Hardware architecture is therefore the representation of an engineered (or to be engineered) electronic or electromechanical hardware system, and the process and discipline for effectively implementing the design(s) for such a system. It is generally part of a larger integrated system encompassing information, software, and device prototyping.[2] It is a representation because it is used to convey information about the related elements comprising a hardware system, the relationships among those elements, and the rules governing those relationships. It is a process because a sequence of steps is prescribed to produce or change the architecture, and/or a design from that architecture, of a hardware system within a set of constraints. It is a discipline because a body of knowledge is used to inform practitioners as to the most effective way to design the system within a set of constraints. A hardware architecture is primarily concerned with the internal electrical (and, more rarely, the mechanical) interfaces among the system's components or subsystems, and the interface between the Electric multi-turn actuator with controls. system and its external environment, especially the devices operated by or the electronic displays viewed by a user. (This latter, special interface, is known as the computer human interface, AKA human computer interface, or HCI; formerly called the Hardware architecture 4 man-machine interface.)[3] Integrated circuit (IC) designers are driving current technologies into innovative approaches for new products. Hence, multiple layers of active devices are being proposed as single chip, opening up opportunities for disruptive microelectronic, optoelectronic, and new microelectromechanical hardware implementation.[4] [5] Background Prior to the advent of digital computers, the electronics and other engineering disciplines used the terms system and hardware as they are still commonly used today. However, with the arrival of digital computers on the scene and the development of software engineering as a separate discipline, it was often necessary to distinguish among engineered hardware artifacts, software artifacts, and the combined artifacts. A programmable hardware artifact, or machine, that lacks its software program is impotent; even as a software artifact, or program, is equally impotent unless it can be used to alter the sequential states of a suitable (hardware) machine. However, a hardware machine and its software program can be designed to perform an almost illimitable number of abstract and physical tasks. Within the computer and software engineering disciplines (and, often, other engineering disciplines, such as communications), then, the terms hardware, software, and system came to be used to distinguish among the hardware which could run a software program, the software program itself, and the hardware device complete with its program, respectively. A hardware architecture example, which is integrated as a handheld medical device for diabetes monitoring. The hardware engineer or architect deals (more or less) exclusively with the hardware device; the software engineer or architect deals (more or less) exclusively with the software U-Boat layout, with detailed equipment hardware specification and functionality. program; and the systems engineer or systems architect is responsible for seeing that the software program is capable of properly running within the hardware device, and that the system composed of the two entities is capable of properly interacting with its external environment, especially the user, and performing its intended function. A hardware architecture, then, is an abstract representation of an electronic and/or an electromechanical device which is capable of running a fixed or changeable program.[6] [7] A hardware architecture generally includes some form of analog, digital, or hybrid electronic computer, along with electronic and mechanical sensors and actuators. Hardware architecting may be viewed as a 'partitioning scheme,' or algorithm, which considers all of the system's present and foreseeable requirements and arranges the necessary hardware components into a workable set of cleanly bounded subsystems with no more parts than are required. That is, it is a partitioning scheme which is exclusive, inclusive, and exhaustive. A major purpose of the partitioning is to arrange the elements in the hardware subsystems so that there is a minimum of electrical connections and electronic communications needed among them. In both software and hardware, a good subsystem tends to be seen as a meaningful "object." Moreover, a clear allocation of user requirements to the architecture (hardware and software) provides an effective basis for validation tests of the user's requirements in the as-built system. Hardware architecture References [1] Rai, L. & Kang, S.J. (2008). "Rule-based modular software and hardware architecture for multi-shaped robots using real-time dynamic behavior identification and selection". Knowledge-Based Systems 21 (4): 273–283. doi:10.1016/j.knosys.2007.05.008. [2] Frampton, K.D., Martin, S.E. & Minor, K. (2003). "The scaling of acoustic streaming for application in micro-fluidic devices". Applied Acoustics 64 (7): 681–692. doi:10.1016/S0003-682X(03)00005-7. [3] Brunelli, C., Garzia, F. & Nurmi, J. (2008). "A coarse-grain reconfigurable architecture for multimedia applications featuring subword computation capabilities". Journal of Real-Time Image Processing 3 (1–2): 21–32. doi:10.1007/s11554-008-0071-3. [4] Cale, T.S., Lu, J.-Q. & Gutmann, R.J. (2008). "Three-dimensional integration in microelectronics: Motivation, processing, and thermomechanical modeling". Chemical Engineering Communications 195 (8): 847–888. doi:10.1080/00986440801930302. [5] Cavalcanti, A., Shirinzadeh, B., Zhang, M. & Kretly, L.C. (2008). "Nanorobot Hardware Architecture for Medical Defense". Sensors 8 (5): 2932–2958. doi:10.3390/s8052932. [6] Assif, D., Himel, R. & Grajower, Y. (1988). "A new electromechanical device to measure the accuracy of interocclusal records". Journal of Prosthetic Dentistry 59 (6): 672–676. doi:10.1016/0022-3913(88)90380-0. PMID 3165452. [7] Zimmermann, M., Volden, T., Kirstein, K.-U., Hafizovic, S., Lichtenberg, J., Brand, O. & Hierlemann, A. (2008). "A CMOS-based integrated-system architecture for a static cantilever array". Sensors and Actuators B: Chemical 131 (1): 254–264. doi:10.1016/j.snb.2007.11.016. Stability theory In mathematics, stability theory addresses the stability of solutions of differential equations and of trajectories of dynamical systems under small perturbations of initial conditions. The heat equation, for example, is a stable partial differential equation because small perturbations of initial data lead to small variations in temperature at a later time as a result of the maximum principle. More generally, a theorem is stable if small changes in the hypothesis lead to small variations in the conclusion. One must specify the metric used to measure the perturbations when claiming a theorem is stable. In partial differential equations one may measure the distances between functions using Lp norms or the sup norm, while in differential geometry one may measure the distance between spaces using the Gromov-Hausdorff distance. In Dynamical Systems, an orbit is called Lyapunov stable if the forward orbit of any point in a small enough neighborhood of it stays in a small (but perhaps, larger) neighborhood. Various criteria have been developed to prove stability or instability of an orbit. Under favorable circumstances, the question may be reduced to a well-studied problem involving eigenvalues of matrices. A more general method involves Lyapunov functions. Overview in Dynamical Systems Many parts of the qualitative theory of differential equations and dynamical systems deal with asymptotic properties of solutions and the trajectories: what happens with the system after a large time? The simplest kind of behavior is exhibited by equilibrium points, or fixed points, and by periodic orbits. If a particular orbit is well understood, it is natural to ask next whether a small change in the initial condition will lead to similar behavior. Stability theory addresses the following questions: will a nearby orbit indefinitely stay close to a given orbit? will it converge to the given orbit (this is a stronger property)? In the former case, the orbit is called stable and in the latter case, asymptotically stable, or attracting. Stability means that the trajectories do not change too much under small perturbations. The opposite situation, where a nearby orbit is getting repelled from the given orbit, is also of interest. In general, perturbing the initial state in some directions results in the trajectory asymptotically approaching the given one and in other directions to the trajectory getting away from it. There may also be directions for which the behavior of the perturbed orbit is more complicated (neither converging nor escaping completely), and then stability theory does not give sufficient information about the dynamics. One of the key ideas in stability theory is that the qualitative behavior of an orbit under perturbations can be analyzed using the linearization of the system near the orbit. In particular, at each equilibrium of a smooth dynamical system with an n-dimensional phase space, there is a certain n×n matrix A whose eigenvalues characterize the 5 Stability theory behavior of the nearby points (Hartman-Grobman theorem). More precisely, if all eigenvalues are negative real numbers or complex numbers with negative real parts then the point is a stable attracting fixed point, and the nearby points converge to it at an exponential rate, cf Lyapunov stability and exponential stability. If none of the eigenvalues is purely imaginary (or zero) then the attracting and repelling directions are related to the eigenspaces of the matrix A with eigenvalues whose real part is negative and, respectively, positive. Analogous statements are known for perturbations of more complicated orbits. Stability of fixed points The simplest kind of an orbit is a fixed point, or an equilibrium. If a mechanical system is in a stable equilibrium state then a small push will result in a localized motion, for example, small oscillations as in the case of a pendulum. In a system with damping, a stable equilibrium state is moreover asymptotically stable. On the other hand, for an unstable equilibrium, such as a ball resting on a top of a hill, certain small pushes will result in a motion with a large amplitude that may or may not converge to the original state. There are useful tests of stability for the case of a linear system. Stability of a nonlinear system can often be inferred from the stability of its linearization. Maps Let f: R → R be a continuously differentiable function with a fixed point a, f(a) = a. Consider the dynamical system obtained by iterating the function f: The fixed point a is stable if the absolute value of the derivative of f at a is strictly less than 1, and unstable if it is strictly greater than 1. This is because near the point a, the function f has a linear approximation with slope f′(a): Thus which means that the derivative measures the rate at which the successive iterates approach the fixed point a or diverge from it. If the derivative at a is exactly 1 or −1, then more information is needed in order to decide stability. There is an analogous criterion for a continuously differentiable map f: Rn → Rn with a fixed point a, expressed in terms of its Jacobian matrix at a, J = Ja(f). If all eigenvalues of J are real or complex numbers with absolute value strictly less than 1 then a is a stable fixed point; if at least one of them has absolute value strictly greater than 1 then a is unstable. Just as for n=1, the case of all eigenvalues having absolute value 1 needs to be investigated further — the Jacobian matrix test is inconclusive. The same criterion holds more generally for diffeomorphisms of a smooth manifold. 6 Stability theory Linear autonomous systems The stability of fixed points of a system of constant coefficient linear differential equations of first order can be analyzed using the eigenvalues of the corresponding matrix. An autonomous system where x(t)∈Rn and A is an n×n matrix with real entries, has a constant solution (In a different language, the origin 0∈Rn is an equilibrium point of the corresponding dynamical system.) This solution is asymptotically stable as t → ∞ ("in the future") if and only if for all eigenvalues λ of A, Re(λ) < 0. Similarly, it is asymptotically stable as t → −∞ ("in the past") if and only if for all eigenvalues λ of A, Re(λ) > 0. If there exists an eigenvalue λ of A with Re(λ) > 0 then the solution is unstable for t → ∞. Application of this result in practice, in order to decide the stability of the origin for a linear system, is facilitated by the Routh–Hurwitz stability criterion. The eigenvalues of a matrix are the roots of its characteristic polynomial. A polynomial in one variable with real coefficients is called a Hurwitz polynomial if the real parts of all roots are strictly negative. The Routh–Hurwitz theorem implies a characterization of Hurwitz polynomials by means of an algorithm that avoids computing the roots. Non-linear autonomous systems Asymptotic stability of fixed points of a non-linear system can often be established using the Hartman–Grobman theorem. Suppose that v is a C1-vector field in Rn which vanishes at a point p, v(p)=0. Then the corresponding autonomous system has a constant solution Let J = Jp(v) be the n×n Jacobian matrix of the vector field v at the point p. If all eigenvalues of J have strictly negative real part then the solution is asymptotically stable. This condition can be tested using the Routh–Hurwitz criterion. Lyapunov function for general dynamical systems A general way to establish Lyapunov stability or asymptotic stability of a dynamical system is by means of Lyapunov functions. References • Stability [1] at Scholarpedia, curated by Philip Holmes and Eric T. Shea-Brown. External links • Stable Equilibria [2] by Michael Schreiber, The Wolfram Demonstrations Project. References [1] http:/ / www. scholarpedia. org/ article/ Stability [2] http:/ / demonstrations. wolfram. com/ StableEquilibria/ 7 Control engineering 8 Control engineering Control engineering or Control systems engineering is the engineering discipline that applies control theory to design systems with predictable behaviors. The practice uses sensors to measure the output performance of the device being controlled (often a vehicle) and those measurements can be used to give feedback to the input actuators that can make corrections toward desired performance. When a device is designed to perform without the need of human inputs for correction it is called automatic control (such as cruise control for regulating a car's speed). Multi-disciplinary in nature, control systems engineering activities focus on implementation of control systems mainly derived by mathematical modeling of systems of a diverse range. Control systems play a critical role in space flight Overview Modern day control engineering (also called control systems engineering) is a relatively new field of study that gained a significant attention during 20th century with the advancement in technology. It can be broadly defined as practical application of control theory. Control engineering has an essential role in a wide range of control systems, from simple household washing machines to high-performance F-16 fighter aircraft. It seeks to understand physical systems, using mathematical modeling, in terms of inputs, outputs and various components with different behaviors; use control systems design tools to develop controllers for those systems; and implement controllers in physical systems employing available technology. A system can be mechanical, electrical, fluid, chemical, financial and even biological, and the mathematical modeling, analysis and controller design uses control theory in one or many of the time, frequency and complex-s domains, depending on the nature of the design problem. History Automatic control Systems were first developed over two thousand years ago. The first feedback control device on record is thought to be the ancient Ktesibios's water clock in Alexandria, Egypt around the third century B.C. It kept time by regulating the water level in a vessel and, therefore, the water flow from that vessel. This certainly was a successful device as water clocks of similar design were still being made in Baghdad when the Mongols captured the city in 1258 A.D. A variety of automatic devices have been used over the centuries to accomplish useful tasks or simply to just entertain. The latter includes the automata, popular in Europe in the 17th and 18th centuries, featuring dancing figures that would repeat the same task over and over again; these automata are examples of open-loop control. Milestones among feedback, or "closed-loop" automatic control devices, include the temperature regulator of a furnace attributed to Drebbel, circa 1620, and the centrifugal flyball governor used for regulating the speed of steam engines by James Watt in 1788. Control engineering In his 1868 paper "On Governors", J. C. Maxwell (who discovered the Maxwell electromagnetic field equations) was able to explain instabilities exhibited by the flyball governor using differential equations to describe the control system. This demonstrated the importance and usefulness of mathematical models and methods in understanding complex phenomena, and signaled the beginning of mathematical control and systems theory. Elements of control theory had appeared earlier but not as dramatically and convincingly as in Maxwell's analysis. Control theory made significant strides in the next 100 years. New mathematical techniques made it possible to control, more accurately, significantly more complex dynamical systems than the original flyball governor. These techniques include developments in optimal control in the 1950s and 1960s, followed by progress in stochastic, robust, adaptive and optimal control methods in the 1970s and 1980s. Applications of control methodology have helped make possible space travel and communication satellites, safer and more efficient aircraft, cleaner auto engines, cleaner and more efficient chemical processes, to mention but a few. Before it emerged as a unique discipline, control engineering was practiced as a part of mechanical engineering and control theory was studied as a part of electrical engineering, since electrical circuits can often be easily described using control theory techniques. In the very first control relationships, a current output was represented with a voltage control input. However, not having proper technology to implement electrical control systems, designers left with the option of less efficient and slow responding mechanical systems. A very effective mechanical controller that is still widely used in some hydro plants is the governor. Later on, previous to modern power electronics, process control systems for industrial applications were devised by mechanical engineers using pneumatic and hydraulic control devices, many of which are still in use today. Control theory There are two major divisions in control theory, namely, classical and modern, which have direct implications over the control engineering applications. The scope of classical control theory is limited to single-input and single-output (SISO) system design. The system analysis is carried out in time domain using differential equations, in complex-s domain with Laplace transform or in frequency domain by transforming from the complex-s domain. All systems are assumed to be second order and single variable, and higher-order system responses and multivariable effects are ignored. A controller designed using classical theory usually requires on-site tuning due to design approximations. Yet, due to easier physical implementation of classical controller designs as compared to systems designed using modern control theory, these controllers are preferred in most industrial applications. The most common controllers designed using classical control theory are PID controllers. In contrast, modern control theory is carried out in the state space, and can deal with multi-input and multi-output (MIMO) systems. This overcomes the limitations of classical control theory in more sophisticated design problems, such as fighter aircraft control. In modern design, a system is represented as a set of first order differential equations defined using state variables. Nonlinear, multivariable, adaptive and robust control theories come under this division. Being fairly new, modern control theory has many areas yet to be explored. Scholars like Rudolf E. Kalman and Aleksandr Lyapunov are well-known among the people who have shaped modern control theory. Control systems Control engineering is the engineering discipline that focuses on the modeling of a diverse range of dynamic systems (e.g. mechanical systems) and the design of controllers that will cause these systems to behave in the desired manner. Although such controllers need not be electrical many are and hence control engineering is often viewed as a subfield of electrical engineering. However, the falling price of microprocessors is making the actual implementation of a control system essentially trivial. As a result, focus is shifting back to the mechanical engineering discipline, as intimate knowledge of the physical system being controlled is often desired. 9 Control engineering Electrical circuits, digital signal processors and microcontrollers can all be used to implement Control systems. Control engineering has a wide range of applications from the flight and propulsion systems of commercial airliners to the cruise control present in many modern automobiles. In most of the cases, control engineers utilize feedback when designing control systems. This is often accomplished using a PID controller system. For example, in an automobile with cruise control the vehicle's speed is continuously monitored and fed back to the system, which adjusts the motor's torque accordingly. Where there is regular feedback, control theory can be used to determine how the system responds to such feedback. In practically all such systems stability is important and control theory can help ensure stability is achieved. Although feedback is an important aspect of control engineering, control engineers may also work on the control of systems without feedback. This is known as open loop control. A classic example of open loop control is a washing machine that runs through a pre-determined cycle without the use of sensors. Control engineering education At many universities, control engineering courses are taught in Electrical and Electronic Engineering, Mechatronics Engineering, Mechanical engineering, and Aerospace engineering; in others it is connected to computer science, as most control techniques today are implemented through computers, often as Embedded systems (as in the automotive field). The field of control within chemical engineering is often known as process control. It deals primarily with the control of variables in a chemical process in a plant. It is taught as part of the undergraduate curriculum of any chemical engineering program, and employs many of the same principles in control engineering. Other engineering disciplines also overlap with control engineering, as it can be applied to any system for which a suitable model can be derived. Control engineering has diversified applications that include science, finance management, and even human behavior. Students of control engineering may start with a linear control system course dealing with the time and complex-s domain, which requires a thorough background in elementary mathematics and Laplace transform (called classical control theory). In linear control, the student does frequency and time domain analysis. Digital control and nonlinear control courses require z transformation and algebra respectively, and could be said to complete a basic control education. From here onwards there are several sub branches. Recent advancement Originally, control engineering was all about continuous systems. Development of computer control tools posed a requirement of discrete control system engineering because the communications between the computer-based digital controller and the physical system are governed by a computer clock. The equivalent to Laplace transform in the discrete domain is the z-transform. Today many of the control systems are computer controlled and they consist of both digital and analog components. Therefore, at the design stage either digital components are mapped into the continuous domain and the design is carried out in the continuous domain, or analog components are mapped in to discrete domain and design is carried out there. The first of these two methods is more commonly encountered in practice because many industrial systems have many continuous systems components, including mechanical, fluid, biological and analog electrical components, with a few digital controllers. Similarly, the design technique has progressed from paper-and-ruler based manual design to computer-aided design, and now to computer-automated design (CAutoD), which has been made possible by evolutionary computation. CAutoD can be applied not just to tuning a predefined control scheme, but also to controller structure optimisation, system identification and invention of novel control systems, based purely upon a performance requirement, independent of any specific control scheme.[1] [2] 10 Control engineering Literature • Christopher Kilian (2005). Modern Control Technology. Thompson Delmar Learning. ISBN 1-4018-5806-6. • Arnold Zankl (2006). Milestones in Automation: From the Transistor to the Digital Factory. Wiley-VCH. ISBN 3-89578-259-9. References [1] Tan, K.C. and Li, Y. (2001) Performance-based control system design automation via evolutionary computing. Engineering Applications of Artificial Intelligence, 14 (4). pp. 473-486. ISSN 0952-1976, http:/ / eprints. gla. ac. uk/ 3807/ [2] Li, Y., et al. (2004). CAutoCSD - Evolutionary search and optimisation enabled computer automated control system design. International Journal of Automation and Computing, 1(1). pp.76-88. ISSN 1751-8520, http:/ / eprints. gla. ac. uk/ 3818/ • Bennett, Stuart (1986-06). A history of control engineering, 1800-1930. IET. ISBN 978-0-86341047-5. • Bennett, Stuart (1993). A history of control engineering, 1930-1955. IET. ISBN 978-0-86341299-8. External links • Control Labs Worldwide (http://www-control.eng.cam.ac.uk/extras/Virtual_Library/Control_VL.html) • The Michigan Chemical Engineering Process Dynamics and Controls Open Textbook (http://controls.engin. umich.edu/wiki) Signal processing Signal processing is an area of systems engineering, electrical engineering and applied mathematics that deals with operations on or analysis of signals, in either discrete or continuous time. Signals of interest can include sound, images, time-varying measurement values and sensor data, for example biological data such as electrocardiograms, control system signals, telecommunication transmission signals, and many others. Signals are analog or digital electrical representations of time-varying or spatial-varying physical quantities. In the context of signal processing, arbitrary binary data streams and on-off signalling are not considered as signals, but only analog and digital signals that are representations of analog physical quantities. Typical operations and applications Processing of signals includes the following operations and algorithms with application examples:[1] • Filtering (for example in tone controls and equalizers) • Smoothing, deblurring (for example in image enhancement) • Adaptive filtering (for example for echo-cancellation in a conference telephone, or denoising for aircraft identification by radar) • Spectrum analysis (for example in magnetic resonance imaging, tomographic reconstruction and OFDM modulation) • Digitization, reconstruction and compression (for example, image compression, sound coding and other source coding) • Storage (in digital delay lines and reverb) • Modulation (in modems and radio receivers and transmitters) • Wavetable synthesis (in modems and music synthesizers) • Feature extraction (for example speech-to-text conversion and optical character recognition) • Pattern recognition and correlation analysis (in spread spectrum receivers and computer vision) • Prediction 11 Signal processing • A variety of other operations In communication systems, signal processing may occur at OSI layer 1, the Physical Layer (modulation, equalization, multiplexing, etc.) in the seven layer OSI model, as well as at OSI layer 6, the Presentation Layer (source coding, including analog-to-digital conversion and data compression). History According to Alan V. Oppenheim and Ronald W. Schafer, the principles of signal processing can be found in the classical numerical analysis techniques of the 17th century. They further state that the "digitalization" or digital refinement of these techniques can be found in the digital control systems of the 1940s and 1950s.[2] Mathematical topics embraced by signal processing • • • • • Linear signals and systems, and transform theory System identification and classification Calculus Differential equations Vector spaces and Linear algebra • • • • • • • • Functional analysis Probability and stochastic processes Detection theory Estimation theory Optimization Programming Numerical methods Iterative methods Categories of signal processing Analog signal processing Analog signal processing is for signals that have not been digitized, as in legacy radio, telephone, radar, and television systems. This involves linear electronic circuits such as passive filters, active filters, additive mixers, integrators and delay lines. It also involves non-linear circuits such as compandors, multiplicators (frequency mixers and voltage-controlled amplifiers), voltage-controlled filters, voltage-controlled oscillators and phase-locked loops. Discrete time signal processing Discrete time signal processing is for sampled signals that are considered as defined only at discrete points in time, and as such are quantized in time, but not in magnitude. Analog discrete-time signal processing is a technology based on electronic devices such as sample and hold circuits, analog time-division multiplexers, analog delay lines and analog feedback shift registers. This technology was a predecessor of digital signal processing (see below), and is still used in advanced processing of gigahertz signals. The concept of discrete-time signal processing also refers to a theoretical discipline that establishes a mathematical basis for digital signal processing, without taking quantization error into consideration. 12 Signal processing Digital signal processing Digital signal processing is the processing of digitised discrete time sampled signals. Processing is done by general-purpose computers or by digital circuits such as ASICs, field-programmable gate arrays or specialized digital signal processors (DSP chips). Typical arithmetical operations include fixed-point and floating-point, real-valued and complex-valued, multiplication and addition. Other typical operations supported by the hardware are circular buffers and look-up tables. Examples of algorithms are the Fast Fourier transform (FFT), finite impulse response (FIR) filter, Infinite impulse response (IIR) filter, and adaptive filters such as the Wiener and Kalman filters. Fields of signal processing • Statistical signal processing – analyzing and extracting information from signals and noise based on their stochastic properties • Audio signal processing – for electrical signals representing sound, such as speech or music • Speech signal processing – for processing and interpreting spoken words • Image processing – in digital cameras, computers, and various imaging systems • Video processing – for interpreting moving pictures • Array processing – for processing signals from arrays of sensors • • • • Time-frequency analysis – for processing non-stationary signals[3] Filtering – used in many fields to process signals Seismic signal processing Data mining Notes and references [1] Mathematical Methods and Algorithms for Signal Processing, Todd K. Moon, Wynn C. Stirling, Prentice Hall, 2000, ISBN 0-201-36186-8, page 4. [2] Oppenheim, Alan V.; Schafer, Ronald W. (1975). Digital Signal Processing. Prentice Hall. p. 5. ISBN 0-13-2146355. [3] Boashash, B. (ed.), (2003) Time-Frequency Signal Analysis and Processing: A Comprehensive Reference, Elsevier Science, Oxford, 2003; ISBN 0080443354 External links • Signal Processing for Communications (http://www.sp4comm.org/) — free online textbook by Paolo Prandoni and Martin Vetterli (2008) • Scientists and Engineers Guide to Digital Signal Processing (http://www.dspguide.com) — free online textbook by Stephen Smith 13 Feedback Feedback Feedback describes the situation when output from (or information about the result of) an event or phenomenon in the past will influence an occurrence or occurrences of the same (i.e. same defined) event / phenomenon (or the continuation / development of the original phenomenon) in the present or future. When an event is part of a chain of cause-and-effect that forms a circuit or loop, then the event is said to "feed back" into itself. Feedback is also a synonym for: • Feedback signal - the information about the initial event that is the basis for subsequent modification of the event • Feedback loop - the causal path that leads from the initial generation of the feedback signal to the subsequent modification of the event • Audio feedback - the special kind of positive feedback that occurs when a loop exists between an audio input and output. Overview Feedback is a mechanism, process or signal that is looped back to control a system within itself. Such a loop is called a feedback loop. In systems containing an input and output, feeding back part of the output so as to increase the input is positive feedback (regeneration); feeding back part of the output in such a way as to partially oppose the input is negative feedback (degeneration). Generally, a control system has input from an external signal source and output to an external load; this defines a natural sense (or direction) or path of propagation of signal; the feedforward sense or path describes the signal propagation from input to output; feedback describes signal propagation in the reverse sense. When a sample of the output of the system is fed back, in the reverse sense, by a distinct feedback path into the interior of the system, to contribute to the input of one of its internal feedforward components, especially an active device or a substance that is consumed in an irreversible reaction, it is called the "feedback". The propagation of the signal around the feedback loop takes a finite time because it is causal. The natural sense of feedforward is defined chemically by some irreversible reaction, or electronically by an active circuit element that has access to an auxiliary power supply, so as to be able to provide power gain to amplify the signal as it propagates from input to output. For example, an amplifier can use power from its controlled power reservoir, such as its battery, to provide power gain to amplify the signal; but the reverse is not possible: the signal cannot provide power to re-charge the battery of the amplifier. Feedforward, feedback and regulation are self related. The feedforward carries the signal from source to load. Negative feedback helps to maintain stability in a system in spite of external changes. It is related to homeostasis. For example, in a population of foxes (predators) and rabbits (prey), an increase in the number of foxes will cause a reduction in the number of rabbits; the smaller rabbit population will sustain fewer foxes, and the fox population will fall back. In an electronic amplifier feeding back a negative copy of the output to the input will tend to cancel distortion, making the output a more accurate replica of the input signal.[1] Positive feedback amplifies possibilities of divergences (evolution, change of goals); it is the condition to change, evolution, growth; it gives the system the ability to access new points of equilibrium. For example, in an organism, most positive feedback provides for fast autoexcitation of elements of endocrine and nervous systems (in particular, in stress responses conditions) and are believed to play a key role in morphogenesis, growth, and development of organs, all processes that are, in essence, a rapid escape from the initial state. Homeostasis is especially visible in the nervous and endocrine systems when considered at organism level. Chemical potential energy for irreversible reactions or electrical potential energy for irreversible cell-membrane current powers the feedforward sense of the process. However, in the case of morphogenesis, feedback may only be enough to explain the increase in momentum of the system, and may not be sufficient in itself to account for the movement or 14 Feedback 15 direction of its parts. When a public-address system is used with a microphone to amplify speech, the output from a random sound at the microphone may produce sound at a loudspeaker that reaches the microphone such as to reinforce and amplify the original signal (positive feedback), building up to a howl (of frequency dependent upon the acoustics of the hall). A similar process is used deliberately to produce oscillating electrical signals. Feedback is distinctly different from reinforcement that occurs in learning, or in conditioned reflexes. Feedback combines immediately with the immediate input signal to drive the responsive power gain element, without changing the basic responsiveness of the system to future signals. Reinforcement changes the basic responsiveness of the system to future signals, without combining with the immediate input signal. Reinforcement is a permanent change in the responsiveness of the system to all future signals. Feedback is only transient, being limited by the duration of the immediate signal. Types of feedback When feedback acts in response to an event/phenomenon, it can influence the input signal in one of two ways: 1. An in-phase feedback signal, where a positive-going wave on the input leads to a positive-going change on the output, will amplify the input signal, leading to more modification. This is known as positive feedback. Figure 1: Ideal feedback model. The feedback is negative if B < 0 2. A feedback signal which is inverted, where a positive-going change on the input leads to a negative-going change on the output, will dampen the effect of the input signal, leading to less modification. This is known as negative feedback. • Note that an increase or decrease of the feedback signal here refers to the magnitude relative to the input signal's absolute value, without regard to the polarity or sign of the feedback signal. For example if the input signal changes by 100 then a change in feedback signal value from +5 to +10 is a positive feedback. If the input signal changes by -100 then a change in the feedback signal from -5 to -10 is also a positive feedback. Positive feedback tends to increase the event that caused it, such as in a nuclear chain-reaction. It is also known as a self-reinforcing loop.[2] An event influenced by positive feedback can increase or decrease its output/activation until it hits a limiting constraint. Such a constraint may be destructive, as in thermal runaway or a nuclear chain reaction. Self-reinforcing loops can be a smaller part of a larger balancing loop, especially in biological systems such as regulatory circuits. Negative feedback, which tends to reduce the input signal that caused it, is also known as a self-correcting or balancing loop.[2] Such loops tend to be goal-seeking, as in a thermostat, which compares actual temperature with desired temperature and seeks to reduce the difference. Balancing loops are sometimes prone to hunting: an oscillation caused by an excessive or delayed negative feedback signal, resulting in over-correction, wherein the signal becomes a positive feedback. The terms negative and positive feedback can be used loosely or colloquially to describe or imply criticism and praise, respectively. This may lead to confusion with the more technically accurate terms positive and negative reinforcement, which refer to something that changes the likelihood of a future behaviour. Moreover, when used technically, negative feedback leads to stability that is, in general, considered good, whereas positive feedback can lead to unstable and explosive situations that are considered bad. Thus, when used colloquially, these terms imply the opposite desirability to that when used technically. Feedback Negative feedback was applied by Harold Stephen Black to electrical amplifiers in 1927, but he could not get his idea patented until 1937.[3] Arturo Rosenblueth, a Mexican researcher and physician, co-authored a seminal 1943 paper Behavior, Purpose and Teleology[4] that, according to Norbert Wiener (another co-author of the paper), set the basis for the new science cybernetics. Rosenblueth proposed that behaviour controlled by negative feedback, whether in animal, human or machine, was a determinative, directive principle in nature and human creations. This kind of feedback is studied in cybernetics and control theory. Applications Biology In biological systems such as organisms, ecosystems, or the biosphere, most parameters must stay under control within a narrow range around a certain optimal level under certain environmental conditions. The deviation of the optimal value of the controlled parameter can result from the changes in internal and external environments. A change of some of the environmental conditions may also require change of that range to change for the system to function. The value of the parameter to maintain is recorded by a reception system and conveyed to a regulation module via an information channel. An example of this is Insulin oscillations. Biological systems contain many types of regulatory circuits, both positive and negative. As in other contexts, positive and negative do not imply consequences of the feedback have good or bad final effect. A negative feedback loop is one that tends to slow down a process, whereas the positive feedback loop tends to accelerate it. The mirror neurons are part of a social feedback system, when an observed action is "mirrored" by the brain - like a self-performed action. Feedback is also central to the operations of genes and gene regulatory networks. Repressor (see Lac repressor) and activator proteins are used to create genetic operons, which were identified by Francois Jacob and Jacques Monod in 1961 as feedback loops. These feedback loops may be positive (as in the case of the coupling between a sugar molecule and the proteins that import sugar into a bacterial cell), or negative (as is often the case in metabolic consumption). Any self-regulating natural process involves feedback and/or is prone to hunting. A well-known example in ecology is the oscillation of the population of snowshoe hares due to predation from lynxes. In zymology, feedback serves as regulation of activity of an enzyme by its direct product(s) or downstream metabolite(s) in the metabolic pathway (see Allosteric regulation). Hypothalamo-pituitary-adrenal and gonadal axis is largely controlled by positive and negative feedback, much of which is still unknown. In psychology, the body receives a stimulus from the environment or internally that causes the release of hormones. Release of hormones then may cause more of those hormones to be released, causing a positive feedback loop. This cycle is also found in certain behaviour. For example, "shame loops" occur in persons who blush easily. When they realize that they are blushing, they become even more embarrassed, which leads to further blushing, and so on.[5] 16 Feedback Climate science The climate system is characterized by strong positive and negative feedback loops between processes that affect the state of the atmosphere, ocean, and land. A simple example is the ice-albedo positive feedback loop whereby melting snow exposes more dark ground (of lower albedo), which in turn absorbs heat and causes more snow to melt. Control theory Feedback is extensively used in control theory, using a variety of methods including state space (controls), full state feedback (also known as pole placement), and so forth. Further information: PID controller The most common general-purpose controller using a control-loop feedback mechanism is a proportional-integral-derivative (PID) controller. Heuristically, the terms of a PID controller can be interpreted as corresponding to time: the proportional term depends on the present error, the integral term on the accumulation of past errors, and the derivative term is a prediction of future error, based on current rate of change.[6] Mechanical engineering In ancient times, the float valve was used to regulate the flow of water in Greek and Roman water clocks; similar float valves are used to regulate fuel in a carburettor and also used to regulate tank water level in the flush toilet. In 1745, the windmill was improved with by blacksmith Edmund Lee who added a fantail to keep the face of the windmill pointing into the wind. In 1787, Thomas Mead regulated the rotation speed of a windmill by using a centrifugal pendulum to adjust the distance between the bedstone and the runner stone (i.e., to adjust the load). The use of the centrifugal governor by James Watt in 1788 to regulate the speed of his steam engine was one factor leading to the Industrial Revolution. Steam engines also use float valves and pressure release valves as mechanical regulation devices. A mathematical analysis of Watt's governor was done by James Clerk Maxwell in 1868. The Great Eastern was one of the largest steamships of its time and employed a steam powered rudder with feedback mechanism designed in 1866 by J.McFarlane Gray. Joseph Farcot coined the word servo in 1873 to describe steam-powered steering systems. Hydraulic servos were later used to position guns. Elmer Ambrose Sperry of the Sperry Corporation designed the first autopilot in 1912. Nicolas Minorsky published a theoretical analysis of automatic ship steering in 1922 and described the PID controller. Internal combustion engines of the late 20th century employed mechanical feedback mechanisms such as the vacuum timing advance but mechanical feedback was replaced by electronic engine management systems once small, robust and powerful single-chip microcontrollers became affordable. Electronic engineering The main applications of feedback in electronics are in the designs of amplifiers, oscillators, and logic circuit elements. The processing and control of feedback is engineered into many electronic devices and may also be embedded in other technologies. If the signal is inverted on its way round the control loop, the system is said to have negative feedback; otherwise, the feedback is said to be positive. Negative feedback is often deliberately introduced to increase the stability and accuracy of a system by correcting unwanted changes. This scheme can fail if the input changes faster than the system can respond to it. When this happens, the lag in arrival of the correcting signal results in unintended positive feedback, causing the output to oscillate or "hunt".[7] Oscillation is usually an unwanted consequence of system behaviour. 17 Feedback 18 Harry Nyquist contributed the Nyquist plot for assessing the stability of feedback systems. An easier assessment, but less general, is based upon gain margin and phase margin using Bode plots (contributed by Hendrik Bode). Design to ensure stability often involves frequency compensation, one method of compensation being pole splitting. The high-pitched squeal that sometimes occurs in audio systems, PA systems, and rock music is known as audio feedback. If a microphone is in front of a speaker that it is connected to, the noise put into the microphone will come out of the speaker. Since the microphone is in front of the speaker, the original sound (now coming from the speaker) goes back into the microphone. This happens over and over, getting louder each time. This process produces the squeal. Electronic feedback loops They are used to control the output of electronic devices, such as amplifiers. A feedback loop is created when all or some portion of the output from an electronic device is fed-back to the input. A device is said to be operating open loop if no output feedback is being employed and closed loop if feedback is being used. Electronic feedback loops take two forms: negative feedback loops and positive feedback loops.[8] Negative feedback loops They exist when the fed-back output signal is out of phase with the input signal. This occurs when the fed-back signal is anywhere from 90° to 270° with respect to the input signal. Negative feedback is generally used to correct output errors or to lower device output gain to a pre-determined level. In feedback amplifiers, this correction is generally for waveform distortion reduction or to establish a specified gain level. A general expression for the gain of a negative feedback amplifier is the asymptotic gain model. Positive feedback loops They occur when the fed-back signal is in phase with the input signal. Under certain gain conditions, positive feedback reinforces the input signal to the point where the output of the device oscillates between its maximum and minimum possible states. Positive feedback may also introduce hysteresis into a circuit. This can cause the circuit to ignore small signals and respond only to large ones. It is sometimes used to eliminate noise from a digital signal. Under some circumstances, positive feedback may cause a device to latch, i.e., to reach a condition in which the output is locked to its maximum or minimum state. Software engineering and computing systems Feedback loops provide generic mechanisms for controlling the running, maintenance, and evolution of software and computing systems.[9] Feedback-loops are important models in the engineering of adaptive software, as they define the behaviour of the interactions among the control elements over the adaptation process, to guarantee system properties at run-time. Feedback loops and foundations of control theory has been successfully applied to computing systems.[10] In particular, they have been applied to the development of products such as IBM's Universal Database server and IBM Tivoli. From a software perspective, the autonomic (MAPE, Monitor Analyze Plan Execute) loop proposed by researchers of IBM is another valuable contribution to the application of feedback loops to the control of dynamic properties and the design and evolution of autonomic software systems.[11] [12] Social sciences A feedback loop to control human behaviour involves four distinct stages.[13] Firstly - Evidence. A behaviour must be measured, captured, and data stored. Secondly - Relevance. The information must be relayed to the individual, not in the raw-data form in which it was captured but in a context that makes it emotionally resonant. Thirdly Consequence. The information must illuminate one or more paths ahead. Fourthly - Action. There must be a clear moment when the individual can recalibrate a behavior, make a choice, and act. Then that action is measured, and the feedback loop can run once more, every action stimulating new behaviors that inch the individual closer to their goals. Feedback Reflexive feedback A sociological concept that states a feedback association is created within a certain relationship whereby the subject/object that delivers a stimulus to a second subject/object, will in response receive the stimulus back. This first impulse is reflected back and forth over and over again. Economics and finance A system prone to hunting (oscillating) is the stock market, which has both positive and negative feedback mechanisms. This is due to cognitive and emotional factors belonging to the field of behavioural finance. For example, • When stocks are rising (a bull market), the belief that further rises are probable gives investors an incentive to buy (positive feedback - reinforcing the rise, see also stock market bubble); but the increased price of the shares, and the knowledge that there must be a peak after which the market will fall, ends up deterring buyers (negative feedback - stabilizing the rise). • Once the market begins to fall regularly (a bear market), some investors may expect further losing days and refrain from buying (positive feedback - reinforcing the fall), but others may buy because stocks become more and more of a bargain (negative feedback - stabilizing the fall). George Soros used the word reflexivity, to describe feedback in the financial markets and developed an investment theory based on this principle. The conventional economic equilibrium model of supply and demand supports only ideal linear negative feedback and was heavily criticized by Paul Ormerod in his book "The Death of Economics", which, in turn, was criticized by traditional economists. This book was part of a change of perspective as economists started to recognise that Chaos Theory applied to nonlinear feedback systems including financial markets. World-system development The hyperbolic growth of the world population observed till the 1970s has recently been correlated to a non-linear second-order positive feedback between the demographic growth and technological development that can be spelled out as follows: technological growth - increase in the carrying capacity of land for people - demographic growth more people - more potential inventors - acceleration of technological growth - accelerating growth of the carrying capacity - the faster population growth - accelerating growth of the number of potential inventors - faster technological growth - hence, the faster growth of the Earth's carrying capacity for people, and so on.[14] Education Learners have different conceptions of learning activities, and preconceptions about hierarchy in education. Some may look up to instructors as experts in the field and take to heart most of the things instructors say. Thus, it is believed that spending a fair amount of time and effort thinking about how to respond to students may be a worthwhile time investment. The Educational research literature on formative feedback also referred to as formative assessment was reviewed recently by V. Shute. Some general types of reinforcement that can be used in many types of student assessment are: 19 Feedback 20 Confirmation Your answer was correct. Corrective Your answer was incorrect. The correct answer was Jefferson. Explanatory Your answer was incorrect because Carter was from Georgia; only Jefferson called Virginia home. Diagnostic Your answer was incorrect. Your choice of Carter suggests some extra instruction on the home states of past presidents might be helpful. Elaborative Your answer, Jefferson, was correct. The University of Virginia, a campus rich with Jeffersonian architecture and writings, is sometimes referred to as "Mr. Jefferson's University". (Adapted from Flemming and Levie.[15] ) A different application of feedback in education is the system for "continuous improvement" of engineering curricula monitored by the Accreditation Board for Engineering and Technology (ABET).[16] Government Examples of feedback in government are: • • • • • Elections Mass media Revolution Curfews Surveys Email administration A mechanism to alert the purported sender of an email with information about the email. In organizations As an organization seeks to improve its performance, feedback helps it to make required adjustments. Feedback serves as motivation for many people in the work place. When one receives either negative or positive feedback, they decide how they will apply it to his or her job. Joseph Folkman says that to find the greatest level of success in an organization, working with other people, a person should learn how to accept any kind of feedback, analyze it in the most positive manner possible, and use it to further impact future decision making.[17] Examples of feedback in organizations: • • • • • • • Financial audit Performance appraisal Shareholders' meeting Marketing research 360-degree feedback Walkout Lockout Feedback References [1] H.S. Black, "Stabilized feed-back amplifiers", Electrical Engineering, vol. 53, pp. 114-120, Jan. 1934. [2] Peter M. Senge (1990). The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday. pp. 424. ISBN 0-385-260-946. [3] Richard R Spencer & Ghausi MS (2003). Introduction to electronic circuit design (http:/ / worldcat. org/ isbn/ 0-201-36183-3). Upper Saddle River NJ: Prentice Hall/Pearson Education. pp. 661. ISBN 0-201-36183-3. . [4] Rosenblueth A, Wiener N & Bigelow J: Behavior, Purpose and Teleology (http:/ / links. jstor. org/ sici?sici=0031-8248(194301)10:1<18:BPAT>2. 0. CO;2-8) [5] http:/ / www. psychologytoday. com/ blog/ lets-connect/ 200909/ the-emotionalrelational-world [6] Araki, M., PID Control (http:/ / www. eolss. net/ ebooks/ Sample Chapters/ C18/ E6-43-03-03. pdf), [7] With mechanical devices, hunting can be severe enough to destroy the device. [8] P. Horowitz & W. Hill, The Art of Electronics, Cambridge University Press (1980), Chapter 3, relating to operational amplifiers. [9] H. Giese, Y. Brun, J. D. M. Serugendo, C. Gacek, H. Kienle, H. Müller, M. Pezzè, and M. Shaw (2009). "Engineering self-adaptive and self-managing systems". Springer-Verlag. [10] J. L. Hellerstein, Y. Diao, S. Parekh, and D. M. Tilbury (2004). Feedback Control of Computing Systems. John Wiley & Sons. [11] J. O. Kephart and D. M. Chess (2003). "The vision of autonomic computing". [12] H. A. Müller, H. M. Kienle, and U. Stege, (2009). "Autonomic computing: Now you see it, now you don’t—design and evolution of autonomic software systems". [13] Thomas, Goetz. "Feedback Loops Are Changing What People Do" (http:/ / www. wired. com/ magazine/ 2011/ 06/ ff_feedbackloop/ ). Wired Magazine. Wired Magazine. . Retrieved 20 June 2011. [14] Introduction to Social Macrodynamics (http:/ / cliodynamics. ru/ index. php?option=com_content& task=view& id=172& Itemid=70) by Andrey Korotayev et al. [15] Fleming, M., & Levie, W.H. (1993). Instructional message design: principles from the behavioral and cognitive sciences (http:/ / books. google. com/ books?id=vLxPAAAAMAAJ& dq=intitle:Instructional+ intitle:message+ intitle:design& pgis=1) (Second Edition ed.). Englewood Cliffs NJ: Educational Technology Publications. ISBN 0877782539. . [16] Accreditation provides you with a structured mechanism to assess and improve the quality of your program: The two-loop feedback diagram (http:/ / www. vanth. org/ curriculum/ twoloop. html) [17] Folkman, Joseph R. The Power of Feedback. 1st ed. New Jersey: John Wiley & Sons Inc., 2006. Print. Further reading • Katie Salen and Eric Zimmerman. Rules of Play. MIT Press. 2004. ISBN 0-262-24045-9. Chapter 18: Games as Cybernetic Systems. • Korotayev A., Malkov A., Khaltourina D. Introduction to Social Macrodynamics: Secular Cycles and Millennial Trends. (http://cliodynamics.ru/index.php?option=com_content&task=view&id=172&Itemid=70) Moscow: URSS, 2006. ISBN 5-484-00559-0 • Dijk, E., Cremer, D.D., Mulder, L.B., and Stouten, J. "How Do We React to Feedback in Social Dilemmas?" In Biel, Eek, Garling & Gustafsson, (eds.), New Issues and Paradigms in Research on Social Dilemmas, New York: Springer, 2008. 21 Negative feedback 22 Negative feedback Negative feedback occurs when the output of a system acts to oppose changes to the input of the system, with the result that the changes are attenuated. If the overall feedback of the system is negative, then the system will tend to be stable. Overview Figure 1: Ideal feedback model. The feedback is negative if B < 0 In many physical and biological systems, qualitatively different influences can oppose each other. For example, in biochemistry, one set of chemicals drives the system in a given direction, whereas another set of chemicals drives it in an opposing direction. If one, or both of these opposing influences are non-linear, equilibrium point(s) result. In biology, this process (generally biochemical) is often referred to as homeostasis; whereas in mechanics, the more common term is equilibrium. In engineering, mathematics and the physical and biological sciences, common terms for the points around which the system gravitates include: attractors, stable states, eigenstates/eigenfunctions, equilibrium points, and setpoints. Negative refers to the sign of the multiplier in mathematical models for feedback. In delta notation, output is added to or mixed into the input. In multivariate systems, vectors help to illustrate how several influences can both partially complement and partially oppose each other. In contrast, positive feedback is feedback in which the system responds so as to increase the magnitude of any particular perturbation, resulting in amplification of the original signal instead of stabilization. Any system where there is a net positive feedback will result in a runaway situation. Both positive and negative feedback require a feedback loop to operate. Negative feedback is used to describe the act of reversing any discrepancy between desired and actual output. Examples Mechanical engineering Negative feedback was first implemented in the 16th Century with the invention of the centrifugal governor. Its operation is most easily seen in its use by James Watt to control the speed of his steam engine. Two heavy balls on an upright frame rotate at the same speed as the engine. As their speed increases they swing up and outwards due to the centrifugal force. This causes them to lift a mechanism which closes the steam inlet valve and the engine slows. When the speed of the engine falls too far, the balls will fall by gravity and open the steam valve. Control systems Examples of the use of negative feedback to control its system are: thermostat control, phase-locked loop, hormonal regulation (see diagram above), and temperature regulation in animals. A simple and practical example is a thermostat. When the temperature in a heated room reaches a certain upper limit the room heating is switched off so that the temperature begins to fall. When the temperature drops to a lower limit, the heating is switched on again. Provided the limits are close to each other, a steady room temperature is maintained. Similar control mechanisms are used in cooling systems, such as an air conditioner, a refrigerator, or a freezer. Negative feedback 23 Biology and chemistry Some biological systems exhibit negative feedback such as the baroreflex in blood pressure regulation and erythropoiesis. Many biological process (e.g., in the human anatomy) use negative feedback. Examples of this are numerous, from the regulating of body temperature, to the regulating of blood glucose levels. The disruption of feedback loops can lead to undesirable results: in the case of blood glucose levels, if negative feedback fails, the glucose levels in the blood may begin to rise dramatically, thus resulting in diabetes. For hormone secretion regulated by the negative feedback loop: when gland X releases hormone X, this stimulates target cells to release hormone Y. When there is an excess of hormone Y, gland X "senses" this and inhibits its release of hormone X. Economics In economics, automatic stabilisers are government programs which work as negative feedback to dampen fluctuations in real GDP. Electronic amplifiers Most endocrine hormones are controlled by a physiologic negative feedback inhibition loop, such as the glucocorticoids secreted by the adrenal cortex. The hypothalamus secretes corticotropin-releasing hormone (CRH), which directs the anterior pituitary gland to secrete adrenocorticotropic hormone (ACTH). In turn, ACTH directs the adrenal cortex to secrete glucocorticoids, such as cortisol. Glucocorticoids not only perform their respective functions throughout the body but also negatively affect the release of further stimulating secretions of both the hypothalamus and the pituitary gland, effectively reducing the output of glucocorticoids [1] once a sufficient amount has been released. The negative feedback amplifier was invented by Harold Stephen Black at Bell Laboratories in 1927, and patented by him in 1934. Fundamentally, all electronic devices (e.g. vacuum tubes, bipolar transistors, MOS transistors) exhibit some nonlinear behavior. Negative feedback corrects this by trading unused gain for higher linearity (lower distortion). An amplifier with too large an open-loop gain, possibly in a specific frequency range, will additionally produce too large a feedback signal in that same range. This feedback signal, when subtracted from the original input, will act to reduce the original input, also by "too large" an amount. This "too small" input will be amplified again by the "too large" open-loop gain, creating a signal that is "just right". The net result is a flattening of the amplifier's gain over all frequencies (desensitising). Though much more accurate, amplifiers with negative feedback can become unstable if not designed correctly, causing them to oscillate. Harry Nyquist of Bell Laboratories managed to work out a theory about how to make this behaviour stable. Negative feedback is used in this way in many types of amplification systems to stabilize and improve their operating characteristics (see e.g., operational amplifiers). References [1] Raven, PH; Johnson, GB. Biology, Fifth Edition, Boston: Hill Companies, Inc. 1999. page 1058. Negative feedback 24 External links • http://www.biology-online.org/4/1_physiological_homeostasis.htm Positive feedback Positive feedback is a process in which the effects of a small disturbance on (a perturbation of) a system include an increase in the magnitude of the perturbation. That is, A produces more of B which in turn produces more of A.[1] In contrast, a system that responds to a perturbation in a way that reduces its effect is said to exhibit negative feedback. These concepts were first recognized as broadly applicable by Norbert Wiener in his 1948 work on cybernetics.[2] Positive feedback tends to cause system instability, divergence from equilibrium, oscillation and often exponential growth. When there is more positive feedback than there are stabilizing tendencies, systems will typically accelerate towards a non-linear region, which may stabilise the system or (depending on the system) even damage or destroy it. Positive feedback may end with the system 'latched' into a new stable state. Positive feedback is controlled by signals in the system being filtered, damped or limited, or it can cancelled or reduced by adding negative feedback. Figure 1: Ideal feedback model. The feedback is positive if B > 0 Alarm or panic can spread by positive feedback among a herd of animals to cause a stampede. Positive feedback is used in digital electronics to force voltages away from intermediate voltages into '0' and '1' states. On the other hand, thermal runaway is a positive feedback that can destroy semiconductor junctions. Positive feedback in chemical reactions can increase the rate of reactions, and in some cases can lead to explosions. Positive feedback in mechanical design causes tipping-point, or 'over-centre', mechanisms to snap into position, for example in switches and locking pliers. Out of control, it can cause bridges to collapse. Positive feedback in economic systems can cause boom-then-bust cycles. In sociology a network effect can quickly create In the term "positive feedback", "positive" refers to the mathematical the positive feedback of a bank run. The above photo is of the UK Northern Rock 2007 bank run. sign of the direction of change rather than the desirability of the See also viral video. outcome. To avoid this confusion, it is sometimes better to use other terms such as self-reinforcing feedback.[3] In social and financial systems, positive feedback effects may also be referred to as 'virtuous' or 'vicious' circles. Overview The key feature of positive feedback is that small disturbances are amplified. When positive feedback is present, there is some causal loop where a small change creates an effect that causes an even bigger change, like a ball rolling down an increasingly steep hill. When a change in a variable occurs in a system which exhibits positive feedback, the system responds by changing that variable even more in the same direction. Positive feedback 25 The end result of positive feedback is to amplify so that small perturbations may result in big changes. For example, if a chemical reaction causes the release of heat, and the reaction itself happens faster at higher temperatures, then there is a high likelihood of positive feedback. If the heat produced is not removed from the reactants fast enough, thermal runaway can occur and very quickly lead to a chemical explosion. Equally, if a PA system microphone picks up sounds from its own loudspeakers and these sounds are re-amplified enough, the effect can be loud squealing or howling noises from the loudspeakers. Formally, a system in equilibrium in which there is positive feedback to any change from its current state is said to be in an unstable equilibrium. The magnitude of the forces which act to move such a system away from its set point are an increasing function of the "distance" from the set point. In the real world, positive feedback loops do not cause ever-increasing growth, but are modified by limiting effects of some sort. Acoustic feedback causes a PA system to reach its maximum volume and so it cannot amplify any further; a flask may crack and the chemical reactants spray into the air or spill onto the floor, where they spread out and cool. Negative feedback effects within the same system can also modulate the effect of positive feedback so that it may add to the responsiveness of the system but does not necessarily lead to a runaway process. Chaotic systems exhibit positive feedback at the small scale, but non linearities prevent the systems from expanding infinitely, but do so without reaching equilibrium. Examples and Applications In biology ] In physiology A number of examples of positive feedback systems may be found in physiology, some of which are well described in works such as Arthur Guyton's 'Textbook of Medical Physiology'.[4] • One example is the onset of contractions in childbirth, known as the Ferguson reflex. When a contraction occurs, the hormone oxytocin is released into the body, which stimulates further contractions. This results in contractions increasing in amplitude and frequency.[] Positive feedback is a mechanism by which an output is enhanced, such as protein levels. However, in order to avoid any fluctuation in the protein level, the mechanism is inhibited stochastically (I), therefore when the concentration of the activated protein (A) is past the threshold ([I]), the loop mechanism is activated and the concentration of A increases exponentially if d[A]=k [A • Another example is the process of blood clotting. The loop is initiated when injured tissue releases signal chemicals that activate platelets in the blood. An activated platelet releases chemicals to activate more platelets, causing a rapid cascade and the formation of a blood clot.[] • Lactation also involves positive feedback in that the more the baby sucks, the more milk is produced, via a surge in prolactin secretion.[5] • Estrogen that functions during the follicular phase of menstruation is also an example of positive feedback.[6] • The generation of nerve signals is another example, in which the membrane of a nerve fibre causes slight leakage of sodium ions through sodium channels, resulting in a change in the membrane potential, which in turn causes more opening of channels, and so on. So a slight initial leakage results in an explosion of sodium leakage which creates the nerve action potential.[7] In most cases, such feedback loops culminate in counter-signals being released that suppress or breaks the loop. Childbirth contractions stop when the baby is out of the mother's body. Chemicals break down the blood clot. Lactation stops when the baby no longer nurses.[4] Positive feedback In evolutionary biology Positive feedback loops have been used to describe aspects of the dynamics of change in biological evolution. For example, beginning at the macro level, Alfred J. Lotka (1945) argued that the evolution of the species was most essentially a matter of selection that fed back energy flows to capture more and more energy for use by living systems.[8] At the human level, Richard Alexander (1989) proposed that social competition between and within human groups fed back to the selection of intelligence thus constantly producing more and more refined human intelligence.[9] Crespi (2004) discussed several other examples of positive feedback loops in evolution.[10] The analogy of Evolutionary arms races provide further examples of positive feedback in biological systems.[11] It have been shown that changes in biodiversity through the Phanerozoic correlate much better with hyperbolic model (widely used in demography and macrosociology) than with exponential and logistic models (traditionally used in population biology and extensively applied to fossil biodiversity as well). The latter models imply that changes in diversity are guided by a first-order positive feedback (more ancestors, more descendants) and/or a negative feedback arising from resource During the Phanerozoic the biodiversity shows a steady but not monotonic increase from near zero to several thousands of genera. limitation. Hyperbolic model implies a second-order positive feedback. The hyperbolic pattern of the world population growth has been demonstrated (see below) to arise from a second-order positive feedback between the population size and the rate of technological growth. The hyperbolic character of biodiversity growth can be similarly accounted for by a positive feedback between the diversity and community structure complexity. It has been suggested that the similarity between the curves of biodiversity and human population probably comes from the fact that both are derived from the interference of the hyperbolic trend (produced by the positive feedback) with cyclical and stochastic dynamics.[12] In psychology Winner (1996) described gifted children as driven by positive feedback loops involving setting their own learning course, this feeding back satisfaction, thus further setting their learning goals to higher levels and so on.[13] Winner termed this positive feedback loop as a “rage to master.” Vandervert (2009a, 2009b) proposed that the child prodigy can be explained in terms of a positive feedback loop between the output of thinking/performing in working memory, which then is fed to the cerebellum where it is streamlined, and then fed back to working memory thus steadily increasing the quantitative and qualitative output of working memory.[14] [15] Vandervert also argued that this working memory/cerebellar positive feedback loop was responsible for language evolution in working memory. 26 Positive feedback 27 In electronics Electronic amplification devices may have positive feedback signal paths intentionally added, or such paths may come into being inadvertently. In addition, thermal positive feedback may take place in electronic circuits. This is called thermal runaway and can be destructive. In networking, a form of positive feedback known as a broadcast storm can result when multiple switches are connected in such a way that they form a loop. Say for example, you have two switches, each with 4 ports. By accident, ports 1 and 2 are connected to the other switches ports 1 and 2. A single multi-cast packet is sent, switch one receives it, and sends it out through every port besides the one it came in on. Switch 2 receives 2 multi-casts, and sends each of them out on every port besides the ones they came in on. Switch 1 then receives 2 again, and the process repeats. This begins flooding the network with packets being rapidly bounced back and forth until the entire network is crippled. Regenerative circuits were invented and patented in 1914[16] for the amplification and reception of very weak radio signals. Carefully controlled positive feedback around a single transistor amplifier can multiply its gain by 1,000 or more.[17] Therefore a signal can be amplified 20,000 or even 100,000 times in one stage, that would normally have a gain of only 20 to 50. The problem with regenerative amplifiers working at these very high gains is that they easily become unstable and start to oscillate. The radio operator has to be prepared to tweak the amount of feedback fairly continuously for good reception. Modern radio receivers use the superheterodyne design, with many more amplification stages, but much more stable operation and no positive feedback. A vintage style regenerative radio receiver. Due to the controlled use of positive feedback, sufficient amplification can be derived from a single vacuum tube or valve (centre). The oscillation that can break out in a regenerative radio circuit can be put to good use in the design of electronic oscillators. By the use of tuned circuits or a piezoelectric crystal (commonly quartz), the signal that is amplified by the positive feedback remains linear and sinusoidal. There are several designs for such harmonic oscillators, including the Armstrong oscillator, hartley oscillator, colpitts oscillator, and the wien bridge oscillator. They all use positive feedback to maintain the oscillations.[18] Many electronic circuits, especially amplifiers, incorporate negative feedback. This reduces their gain, but improves their input impedance, output impedance, and bandwidth, and stabilises all of these parameters, including the closed-loop gain. These parameters also become less dependent on the details of the amplifying device itself, and more dependent on the feedback components, which are less likely to vary with manufacturing tolerance, age and temperature. The difference between positive and negative feedback for AC signals is one of phase: if the signal is fed back out of phase, the feedback is negative and if it is in-phase the feedback is positive. One problem for amplifier designers who use negative feedback is that some of the components of the circuit will introduce phase shift in the feedback path. If there is a frequency (usually a high frequency) where the phase shift reaches 180°, then the designer must ensure that the amplifier gain at that frequency is very low (usually by low-pass filtering). If the loop gain (the product of the amplifier gain and the extent of the positive feedback) at any frequency is greater than one, then the amplifier will oscillate at that frequency (Barkhausen stability criterion). Such oscillations are sometimes called parasitic oscillations. An amplifier that is stable in one set of conditions can break into parasitic oscillation in another. This may be due to changes in temperature, supply voltage, adjustment of front-panel controls, or even the proximity of a person or other conductive item. Amplifiers may oscillate gently in ways that are hard to detect without an oscilloscope, or the oscillations may be so extensive that only a very distorted or no required signal at all gets through, or that damage occurs. Low frequency parasitic oscillations have been called 'motorboating' due to the similarity to the sound of a low-revving exhaust note.[19] Positive feedback Digital electronic circuits are sometimes designed to benefit from positive feedback. Normal logic gates usually rely simply on gain to push digital signal voltages away from intermediate values to the values that are meant to represent boolean '0' and '1'. When an input voltage is expected to vary in an analogue way, but sharp thresholds are required for later digital processing, the Schmitt trigger circuit uses positive feedback to ensure that if the input voltage creeps gently above the threshold, the output is forced smartly and rapidly from one logic state to the other. One of the corollaries of the Schmitt trigger's use of positive feedback is that, should the input voltage move gently down again past the same threshold, the positive feedback will hold the The effect of using a Schmitt trigger (B) instead output in the same state with no change. This effect is called hysteresis: of a comparator (A) the input voltage has to drop past a different, lower threshold to 'un-latch' the output and reset it to its original digital value. By reducing the extent of the positive feedback, the hysteresis-width can be reduced, but it can not entirely be eradicated. The Schmitt trigger is, to some extent, a latching circuit.[20] An electronic latch is a circuit that has the effect of memory. For example, a Schmitt trigger circuit in a stable state, with an input voltage between its two threshold levels, in the middle of its hysteresis band, can tell the observer whether the last input swing it saw was higher or lower than the outer edges of its hysteresis band. This is the basis of one bit of electronic memory. The actual memory circuits used in most digital electronics are based on the 'flip-flop' or 'bistable multivibrator'. A bistable multivibrator uses logic gates connected to each other so that positive feedback maintains the state of the circuit Illustration of an R-S ('reset-set') flip-flop made after the input signal has been removed, until a suitable alternative from two digital nor gates with positive feedback. signal is applied to change the state.[21] This is usually a form of Red and black mean logical '1' and '0', respectively. volatile memory, in the sense that removing power from the flip-flop circuit will usually cause it to lose state. Computer random access memory (RAM) can be made in this way, with one latching circuit for each bit of memory, eight to the byte.[22] Thermal runaway occurs in electronic systems because some aspect of a circuit is allowed to pass more current when it gets hotter, then the hotter it gets, the more current it passes, which, directly or indirectly, makes it pass yet more current. The effects are usually catastrophic for the device in question. If devices have to be used near to their maximum power-handling capacity, and thermal runaway is possible or likely under certain conditions, improvements can usually be achieved by careful design.[23] 28 Positive feedback 29 Audio and video systems can easily be made to demonstrate positive feedback. If a microphone picks up the amplified sound loudspeakers in the same circuit, then howling and screeching sounds of audio feedback (at up to the maximum power capacity of the amplifier) will be heard, as random noise is re-amplified by positive feedback and filtered by the characteristics of the audio system and the room. Microphones are not the only transducers subject to this effect. Phonograph turntables are also prone to picking up acoustic feedback, usually in the low frequency range < 100 Hz, manifesting as a low A phonograph turntable is prone to acoustic rumble. Usually moving the placement or orientation (due to the feedback directional nature of all transducers) of the mic, turntable, or speakers is the cure for this, although using EQ can help if the system tends to be resonant mostly in certain bands. This resonance is due to acoustical characteristics of the room and nature of electrical circuits. Similarly, if a video camera is pointed at a monitor screen that is displaying the camera's own signal, then weird repeating patterns can be formed on the screen by positive feedback. This video feedback effect was used in the opening sequences to early series of the television programme Dr Who. Jimi Hendrix helped to develop the controlled and musical use of audio feedback in electric guitar playing,[24] and later Brian May was a famous proponent of the technique.[25] Video feedback. In economics In the World System development The exponential growth of the world population observed until the 1970s has recently been correlated to a non-linear second order positive feedback between the demographic growth and technological development that can be spelled out as follows: technological growth - increase in the carrying capacity of land for people - demographic growth more people - more potential inventors - acceleration of technological growth - accelerating growth of the carrying capacity - the faster population growth - accelerating growth of the number of potential inventors - faster technological growth - hence, the faster growth of the Earth's carrying capacity for people, and so on (see, e.g., Introduction to Social Macrodynamics [26] by Andrey Korotayev et al.). Systemic risk Systemic risk is the risk that an amplification or leverage or positive feedback process is built into a system, this is usually unknown, and under certain conditions this process can amplify exponentially and rapidly lead to destructive or chaotic behavior. A Ponzi scheme is a good example of a positive-feedback system, because its output (profit) is fed back to the input (new investors), causing rapid growth toward collapse. W. Brian Arthur has also studied and written on positive feedback in the economy (e.g. W. Brian Arthur, 1990)[27] Simple systems that clearly separate the inputs from the outputs are not prone to systemic risk. This risk is more likely as the complexity of the system increases, because it becomes more difficult to see or analyze all the possible combinations of variables in the system even under careful stress testing conditions. The more efficient a complex Positive feedback system is, the more likely it is to be prone to systemic risks, because it takes only a small amount of deviation to disrupt the system. Therefore well-designed complex systems generally have built-in features to avoid this condition, such as a small amount of friction, or resistance, or inertia, or time delay to decouple the outputs from the inputs within the system. These factors amount to an inefficiency, but they are necessary to avoid instabilities. Population and agriculture Agriculture and human population can be considered to be in a positive feedback mode,[28] which means that one drives the other with increasing intensity. It is suggested that this positive feedback system will end sometime with a catastrophe, as modern agriculture is using up all of the easily available phosphate and is resorting to highly-efficient monocultures which are more susceptible to systemic risk. Prejudice, social institutions and poverty Gunnar Myrdal described a vicious circle of increasing inequalities, and poverty, which is known as "Circular Cumulative Causation (CCC)".[29] In climatology Within climate a positive feedback subsystem never acts in isolation, but is always embedded within the overall climate system, which itself is always subject to a very powerful negative feedback, the Stefan–Boltzmann law: that emitted radiation rises with the fourth power of temperature. Hence, on earth the gain of the overall system is always less than one, stopping the system from suffering runaway effects. While there may have been periods of time such as the exit from an ice age where the gain was greater than one, this has not lasted long enough for extreme effects such as the evaporation of the oceans as is believed to have happened on Venus. Examples of positive feedback subsystems in climatology include: • A warmer atmosphere will melt ice and this changes the albedo which further warms the atmosphere. • Methane hydrates can be unstable so that a warming ocean could release more methane, which is also a greenhouse gas. In sociology A self-fulfilling prophecy is a social positive feedback loop between beliefs and behavior: if enough people believe that something is true, their behavior can make it true, and observations of their behavior may turn increase belief. A classic example is a bank run. Another sociological example of positive feedback is the network effect. When more people are encouraged to join a network this increases the reach of the network therefore the network expands ever more quickly. A viral video is an example of the network effect in which links to a popular video are shared and redistributed, ensuring that more people see the video and then re-publish the links. This is the basis for many social phenomena, including Ponzi schemes and chain letters. In many cases population size is the limiting factor to the feedback effect. 30 Positive feedback References [1] Keesing, R.M. (1981). Cultural anthropology: A contemporary perspective (2nd ed.) p.149. Sydney: Holt, Rinehard & Winston, Inc. [2] Norbert Wiener (1948), Cybernetics or Control and Communication in the Animal and the Machine, Paris, Hermann et Cie - MIT Press, Cambridge, MA. [3] Peter M. Senge (1990). The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday. pp. 424. ISBN 0-385-260-946. [4] Guyton, Arthur C. (1991) Textbook of Medical Physiology. (8th ed). Philadelphia: W.B. Saunders. ISBN 0-7216-3994-1 [5] Guyton, Arthur C. (1991), p.926. [6] Guyton, Arthur C. (1991), p.907. [7] Guyton, Arthur C. (1991), p.59. [8] Lotka, A. (1945). The law of evolution as a maximal principle. Human Biology, 17, 168-194. [9] Alexander, R. (1989). Evolution of the human psyche. In P. Millar & C. Stringer (Eds.), The human revolution: Behavioral and biological perspectives on the origins of modern humans (pp. 455-513). Princeton: Princeton University Press. [10] Crespi B. J. (2004) Vicious circles: positive feedback in major evolutionary and ecological transitions. Trends in Ecology and Evolution, 19, 627-633. [11] Dawkins, R. 1991. The Blind Watchmaker London: Penguin. Note: W.W. Norton also published this book, and some citations may refer to that publication. However, the text is identical, so it depends on which book is at hand [12] Markov A., Korotayev A. Phanerozoic marine biodiversity follows a hyperbolic trend // Palaeoworld. Volume 16, Issue 4, December 2007, Pages 311-318 (http:/ / www. sciencedirect. com/ science?_ob=ArticleURL& _udi=B83WC-4N0HJMK-2& _user=10& _rdoc=1& _fmt=& _orig=search& _sort=d& _docanchor=& view=c& _acct=C000050221& _version=1& _urlVersion=0& _userid=10& md5=74a80d7c55ff987c9fc8d9c7963feab9); Markov A., Korotayev A. Hyperbolic growth of marine and continental biodiversity through the Phanerozoic and community evolution // Journal of General Biology. Volume 69, 2008. N 3, pp. 175–194 (http:/ / elementy. ru/ genbio/ abstracts?artid=177). [13] Winner, E. (1996). Gifted children: Myths and Realities. New York: Basic Books. [14] Vandervert, L. (2009a). Working memory, the cognitive functions of the cerebellum and the child prodigy. In L.V. Shavinina (Ed.), International handbook on giftedness (pp. 295-316). The Netherlands: Springer Science. [15] Vandervert, L. (2009b). The emergence of the child prodigy 10,000 years ago: An evolutionary and developmental explanation. The Journal of Mind and Behavior, 30, 15-32. [16] Armstrong, E. H., U.S. Patent 1113149 (http:/ / www. google. com/ patents?vid=1113149), Wireless receiving system, 1914. [17] Kitchin, Charles. "A SHORT WAVE REGENERATIVE RECEIVER PROJECT" (http:/ / www. electronics-tutorials. com/ receivers/ regen-radio-receiver. htm). . Retrieved 23 September 2010. [18] "Sinewave oscillators" (http:/ / www. educypedia. be/ electronics/ analogosciltypes. htm). EDUCYPEDIA - electronics. . Retrieved 23 September 2010. [19] Self, Douglas (2009). Audio Power Amplifier Design Handbook (http:/ / books. google. com/ books?id=Qpmi4ia2nhcC& pg=PA254& lpg=PA254#v=onepage& q& f=false). Focal Press. pp. 254–255. ISBN 978-0240521626. . [20] "CMOS Schmitt Trigger—A Uniquely Versatile Design Component" (http:/ / www. fairchildsemi. com/ an/ AN/ AN-140. pdf). Fairchild Semiconductor Application Note 140. Fairchild Semiconductors. 1975. . Retrieved 29 September 2010. [21] Strandh, Robert. "Latches and flip-flops" (http:/ / www. labri. fr/ perso/ strandh/ Teaching/ AMP/ Common/ Strandh-Tutorial/ flip-flops. html). Laboratoire Bordelais de Recherche en Informatique. . Retrieved 4 November 2010. [22] Wayne, Storr. "Sequential Logic Basics: SR Flip-Flop" (http:/ / www. electronics-tutorials. ws/ sequential/ seq_1. html). Electronics-Tutorials.ws. . Retrieved 29 September 2010. [23] Sharma, Bijay Kumar (2009). "Analog Electronics Lecture 4 Part C RC coupled Amplifier Design Procedure" (http:/ / cnx. org/ content/ m31058/ latest/ ). . Retrieved 29 September 2010. [24] Shadwick, Keith (2003). Jimi Hendrix, Musician. Backbeat Books. p. 92. ISBN 0879307641. [25] May, Brian. "Burns Brian May Tri-Sonic Pickups" (http:/ / www. brianmayguitars. co. uk/ accessories/ 19). House Music & Duck Productions. . Retrieved 2 February 2011. [26] http:/ / urss. ru/ cgi-bin/ db. pl?cp=& page=Book& id=37484& lang=en& blang=en& list=14 [27] W. Brian Arthur (February 1990). "Positive Feedbacks in the Economy". Scientific American, Vol 262. No.2, p.80 [28] Brown, A. Duncan. (2003) (http:/ / www. amazon. com/ dp/ 905727048X) Feed or Feedback. Publisher: International Books. [29] Berger, Sebastian. "Circular Cumulative Causation (CCC) à la Myrdal and Kapp — Political Institutionalism for Minimizing Social Costs" (http:/ / www. kwilliam-kapp. de/ pdf/ Circular Cumulative Causation a la Myrdal & Kapp. pdf). . Retrieved 26 November 2011. 31 Positive feedback Further reading • Norbert Wiener (1948), Cybernetics or Control and Communication in the Animal and the Machine, Paris, Hermann et Cie - MIT Press, Cambridge, MA. • Katie Salen and Eric Zimmerman. Rules of Play. MIT Press. 2004. ISBN 0-262-24045-9. Chapter 18: Games as Cybernetic Systems. 32 33 Systems design Systems design Systems design is the process of defining the architecture, components, modules, interfaces, and data for a system to satisfy specified requirements. One could see it as the application of systems theory to product development. There is some overlap with the disciplines of systems analysis, systems architecture and systems engineering.[1] [2] If the broader topic of product development "blends the perspective of marketing, design, and manufacturing into a single approach to product development,"[3] then design is the act of taking the marketing information and creating the design of the product to be manufactured. Systems design is therefore the process of defining and developing systems to satisfy specified requirements of the user. Until the 1990s systems design had a crucial and respected role in the data processing industry. In the 1990s standardization of hardware and software resulted in the ability to build modular systems. The increasing importance of software running on generic platforms has enhanced the discipline of software engineering. Object-oriented analysis and design methods are becoming the most widely used methods for computer systems design. The UML has become the standard language in object-oriented analysis and design. It is widely used for modeling software systems and is increasingly used for high designing non-software systems and organizations. Logical design The logical design of a system pertains to an abstract representation of the data flows, inputs and outputs of the system. This is often conducted via modelling, using an over-abstract (and sometimes graphical) model of the actual system. In the context of systems design are included. Physical design The physical design relates to the actual input and output processes of the system. This is laid down in terms of how data is input into a system, how it is verified/authenticated, how it is processed, and how it is displayed as output. Physical design, in this context, does not refer to the tangible physical design of an information system. To use an analogy, a personal computer's physical design involves input via a keyboard, processing within the CPU, and output via a monitor, printer, etc. It would not concern the actual layout of the tangible hardware, which for a PC would be a monitor, CPU, motherboard, hard drive, modems, video/graphics cards, USB slots, etc. it involve detail design of user and a product database structure processing and control processor .The H/S personal specification a develop for the proposed system . Systems design Alternative design methodologies Rapid application development (RAD) Rapid application development (RAD) is a methodology in which a systems designer produces prototypes for an end-user. The end-user reviews the prototype, and offers feedback on its suitability. This process is repeated until the end-user is satisfied with the final system. Joint application design (JAD) Joint application design (JAD) is a methodology which evolved from RAD, in which a systems designer consults with a group consisting of the following parties: • Executive sponsor • Systems Designer • Managers of the system JAD involves a number of stages, in which the group collectively develops an agreed pattern for the design and implementation of the system. Systems design: Topics • Requirements analysis - analyzes the needs of the end users or customers • Benchmarking — is an effort to evaluate how current systems perform • Systems architecture - creates a blueprint for the design with the necessary specifications for the hardware, software, people and data resources. In many cases, multiple architectures are evaluated before one is selected. • Design — designers will produce one or more 'models' of what they see a system eventually looking like, with ideas from the analysis section either used or discarded. A document will be produced with a description of the system, but nothing is specific — they might say 'touchscreen' or 'GUI operating system', but not mention any specific brands; • Computer programming and debugging in the software world, or detailed design in the consumer, enterprise or commercial world - specifies the final system components. • System testing - evaluates the system's actual functionality in relation to expected or intended functionality, including all integration aspects. References [1] This article incorporates public domain material from the General Services Administration document "Federal Standard 1037C" (http:/ / www. its. bldrdoc. gov/ fs-1037/ fs-1037c. htm). [2] This article incorporates public domain material from the United States Department of Defense document "Dictionary of Military and Associated Terms". [3] Ulrich & Eppinger (2000). Product Design & Development. Irwin McGraw-Hill. ISBN 0-07-229647-X. Further reading • C. West Churchman The Design of Inquiring Systems: Basic Concepts of Systems and Organization. Basic Books, New York, 1971, SBN 465-01608-1 • Levin, Mark Sh., Composite Systems Decisions, Springer, New York, 2006 • Maier, Mark W., and Rechtin, Eberhardt, The Art of Systems Architecting, Second Edition, CRC Press, Boca Raton, 2000 • Saltzer, J.H, et al., End-to-End arguments in Systems Design in: ACM Transactions in Computer Systems Vol. 2, nr 4 (Nov 1984), pp 277–288. 34 Systems design • Ulrich, Karl T. and Eppinger, Steven D., Product Design and Development, Second Edition, Irwin McGraw-Hill, Boston, 2000 External links • (http://www.google.com/search?hl=en&defl=en&q=define:Systems+design&sa=X& oi=glossary_definition&ct=title) Internet definitions • Interactive Systems Design (http://www.dcs.gla.ac.uk/~johnson/teaching/isd/course.html) by Chris Johnson, 1993 • Department of Systems Design Engineering (http://www.systems.uwaterloo.ca/), Faculty of Engineering at the University of Waterloo, Ontario, Canada • System Design and Management (http://sdm.mit.edu/), Systems Design and Management at the Massachusetts Institute of Technology, Cambridge, MA, USA • Course on System Design (http://www.mslevin.iitp.ru/) by Mark Sh. Levin Requirement In engineering, a requirement is a singular documented physical and functional need that a particular product or service must be or perform. It is most commonly used in a formal sense in systems engineering, software engineering, or enterprise engineering. It is a statement that identifies a necessary attribute, capability, characteristic, or quality of a system for it to have value and utility to a user. In the classical engineering approach, sets of requirements are used as inputs into the design stages of product development. Requirements are also an important input into the verification process, since tests should trace back to specific requirements. Requirements show what elements and functions are necessary for the particular project. This is reflected in the waterfall model of the software life-cycle. However, when iterative methods of software development or agile methods are used, the system requirements are incrementally developed in parallel with design and implementation. Requirements engineering is the set of activities that lead to the derivation of the system or software requirements. Requirements engineering may involve a feasibility study, or a conceptual analysis phase of the project and requirements elicitation (gathering, understanding, reviewing, and articulating the needs of the stakeholders) and requirements analysis,[1] analysis (checking for consistency and completeness), specification (documenting the requirements) and validation (making sure the specified requirements are correct).[2] [3] Product versus process requirements Projects are subject to three sorts of requirements: • Business requirements describe in business terms what must be delivered or accomplished to provide value. • Product requirements describe properties of a system or product (which could be one of several ways to accomplish a set of business requirements.) • Process requirements describe activities performed by the developing organization. For instance, process requirements could specify specific methodologies that must be followed, and constraints that the organization must obey. Product and process requirements are closely linked. Process requirements often specify the activities that will be performed to satisfy a product requirement. For example, a maximum development cost requirement (a process requirement) may be imposed to help achieve a maximum sales price requirement (a product requirement); a requirement that the product be maintainable (a Product requirement) often is addressed by imposing requirements to 35 Requirement follow particular development styles (e.g., object-oriented programming), style-guides, or a review/inspection process (process requirements). Requirements in systems and software engineering In systems engineering, a requirement can be a description of what a system must do, referred to as a Functional Requirement. This type of requirement specifies something that the delivered system must be able to do. Another type of requirement specifies something about the system itself, and how well it performs its functions. Such requirements are often called Non-functional requirements, or 'performance requirements' or 'quality of service requirements.' Examples of such requirements include usability, availability, reliability, supportability, testability and maintainability. A collection of requirements define the characteristics or features of the desired system. A 'good' list of requirements as far as possible avoids saying how the system should implement the requirements, leaving such decisions to the system designer. Specifying how the system should be implemented is called "implementation bias" or "solution engineering". However, implementation constraints on the solution may validly be expressed by the future owner, for example for required interfaces to external systems; for interoperability with other systems; and for commonality (e.g. of user interfaces) with other owned products. In software engineering, the same meanings of requirements apply, except that the focus of interest is the software itself. Product requirements Types of product requirements Requirements are typically placed into these categories: • Architectural requirements describe what must be done by identifying the necessary system architecture of a system, • Functional requirements describe the functionality that the system is to execute; for example, formatting some text or modulating a signal. They are sometimes known as capabilities. • Non-functional requirements describe characteristics of the system that the user cannot affect or (immediately) perceive. Nonfunctional requirements are sometimes known as quality requirements or ilities. • Constraint requirements impose limits upon the design alternatives or project/process operations. No matter how the problem is solved the constraint requirements must be adhered to. Non-functional requirements can be further classified according to whether they are usability requirements, look and feel requirements, humanity requirements, performance requirements, maintainability requirements, operational requirements, safety requirements, reliability requirements, or one of many other types of requirements. In software engineering this categorization is useful because only functional requirements can be directly implemented in software. The non-functional requirements are controlled by other aspects of the system. For example, in a computer system reliability is related to hardware failure rates, and performance is controlled by CPU and memory. Non-functional requirements can in some cases be decomposed into functional requirements for software. For example, a system level non-functional safety requirement can be decomposed into one or more functional requirements. See FURPS. In addition, a non-functional requirement may be converted into a process requirement when the requirement is not easily measurable. For example, a system level maintainability requirement may be decomposed into restrictions on software constructs or limits on lines or code. 36 Requirement 37 Characteristics of good requirements The characteristics of good requirements are variously stated by different writers, with each writer generally emphasizing the characteristics most appropriate to their general discussion or the specific technology domain being addressed. However, the following characteristics are generally acknowledged.[4] Characteristic Explanation Unitary (Cohesive) The requirement addresses one and only one thing. Complete The requirement is fully stated in one place with no missing information. Consistent The requirement does not contradict any other requirement and is fully consistent with all authoritative external documentation. Non-Conjugated (Atomic) The requirement is atomic, i.e., it does not contain conjunctions. E.g., "The postal code field must validate American and Canadian postal codes" should be written as two separate requirements: (1) "The postal code field must validate American postal codes" and (2) "The postal code field must validate Canadian postal codes". Traceable The requirement meets all or part of a business need as stated by stakeholders and authoritatively documented. Current The requirement has not been made obsolete by the passage of time. Feasible The requirement can be implemented within the constraints of the project. Unambiguous The requirement is concisely stated without recourse to technical jargon, acronyms (unless defined elsewhere in the Requirements document), or other esoteric verbiage. It expresses objective facts, not subjective opinions. It is subject to one and only one interpretation. Vague subjects, adjectives, prepositions, verbs and subjective phrases are avoided. Negative statements and compound statements are prohibited. Mandatory The requirement represents a stakeholder-defined characteristic the absence of which will result in a deficiency that cannot be ameliorated. An optional requirement is a contradiction in terms. Verifiable The implementation of the requirement can be determined through one of four possible methods: inspection, demonstration, test or analysis. To the above some add Externally Observable, that is, the requirement specifies a characteristic of the product that is externally observable or experienced by the user. Such advocates argue that requirements that specify internal architecture, design, implementation, or testing decisions are probably constraints, and should be clearly articulated in the Constraints section of the Requirements document. The contrasting view is that this perspective fails on two points. First, the perspective does not recognize that the user experience may be supported by requirements not perceivable by the user. For example, a requirement to present geocoded information to the user may be supported by a requirement for an interface with an external third party business partner. The interface will be imperceptible to the user, though the presentation of information obtained through the interface certainly would not. Second, a constraint limits design alternatives, whereas a requirement specifies design characteristics. To continue the example, a requirement selecting a web service interface is different from a constraint limiting design alternatives to methods compatible with a Single Sign-On architecture. Verification All requirements should be verifiable. The most common method is by test. If this is not the case, another verification method should be used instead (e.g. analysis, demonstration or inspection or review of design). Certain requirements, by their very structure, are not verifiable. These include requirements that say the system must never or always exhibit a particular property. Proper testing of these requirements would require an infinite testing cycle. Such requirements must be rewritten to be verifiable. As stated above all requirements must be verifiable. Non-functional requirements, which are unverifiable at the software level, must still be kept as a documentation of customer intent. However, they may be traced to process requirements that are determined to be a practical way of meeting them. For example, a non-functional requirement to be free from backdoors may be satisfied by replacing it with a process requirement to use pair programming. Other non-functional requirements will trace to other system Requirement components and be verified at that level. For example system reliability is often verified by analysis at the system level. Avionics software with its complicated safety requirements must follow the DO-178B development process. Requirements analysis or requirements engineering Requirements are prone to issues of ambiguity, incompleteness, and inconsistency. Techniques such as rigorous inspection have been shown to help deal with these issues. Ambiguities, incompleteness, and inconsistencies that can be resolved in the requirements phase typically cost orders of magnitude less to correct than when these same issues are found in later stages of product development. Requirements analysis strives to address these issues. There is an engineering trade off to consider between requirements which are too vague, and those which are so detailed that they 1. take a long time to produce - sometimes to the point of being obsolete once completed 2. limit the implementation options available 3. are costly to produce Documenting requirements Requirements are usually written as a means for communication between the different stakeholders. This means that the requirements should be easy to understand both for normal users and for developers. One common way to document a requirement is stating what the system must do. Example: 'The contractor must deliver the product no later than xyz date.' Other ways include use cases and user stories. Changes in requirements Requirements generally change with time. Once defined and approved, requirements should fall under change control. For many projects, requirements are altered before the system is complete. This is partly due to the complexity of computer software and the fact that users don't know what they want before they see it. This characteristic of requirements has led to requirements management studies and practices. Disputes regarding the necessity of rigour in software requirements Most agile software development methodologies question the need for rigorously describing software requirements upfront, which they consider a moving target. Instead, extreme programming for example describes requirements informally using user stories (short summaries fitting on an index card explaining one aspect of what the system should do), and considers it the developer's duty to directly ask the customer for clarification. Agile methodologies also typically capture requirements in a series of automated acceptance tests. References [1] Stellman, Andrew; Greene, Jennifer (2005). Applied Software Project Management (http:/ / www. stellman-greene. com/ aspm/ ). O'Reilly Media. p. 98. ISBN 978-0-596-00948-9. . [2] Wiegers, Karl E. (2003). Software Requirements, Second Edition. Microsoft Press. ISBN 0-7356-1879-8. [3] Young, Ralph R. (2001). Effective Requirements Practices. Addison-Wesley. ISBN 978-0201709124. [4] Davis, Alan M. (1993). Software Requirements: Objects, Functions, and States, Second Edition. Prentice Hall. ISBN 0-13-805763-X. External links • Discovering System Requirements (http://www.sie.arizona.edu/sysengr/publishedPapers/ReqsSandiaReport. pdf) • The International Institute for Business Analysis and The IIBA's Business Analysis Body of Knowledge (http:// theiiba.org) 38 Requirement • Writing Good Requirements (http://www.reqid.com/requirements-management/ writing-good-requirements-part1/) • Gottesdiener, Ellen (2009). The Software Requirements Memory Jogger: A Desktop Guide to Help Business and Technical Teams Develop and Manage Requirements (http://www.ebgconsulting.com/Pubs/srmj.php). Addison-Wesley. ISBN 157681114X. Systems architecture A system architecture or systems architecture is the conceptual model that defines the structure, behavior, and more views of a system. An architecture description is a formal description and representation of a system, organized in a way that supports reasoning about the structure of the system which comprises system components, the externally visible properties of those components, the relationships (e.g. the behavior) between them, and provides a plan from which products can be procured, and systems developed, that will work together to implement the overall system. The language for architecture description is called the architecture description language (ADL). History It is important to keep in mind that the modern systems architecture did not appear out of nowhere. Systems architecture depends heavily on practices and techniques which were developed over thousands of years in many other fields most importantly being, perhaps, civil architecture. Prior to the advent of digital computers, the electronics and other engineering disciplines used the term system as it is still commonly used today. However, with the arrival of digital computers and the development of software engineering as a separate discipline, it was often necessary to distinguish among engineered hardware artifacts, software artifacts, and the combined artifacts. A programmable hardware artifact, or computing machine, that lacks its software program is impotent; even as a software artifact, or program, is equally impotent unless it can be used to alter the sequential states of a suitable (hardware) machine. However, a hardware machine and its software program can be designed to perform an almost illimitable number of abstract and physical tasks. Within the computer and software engineering disciplines (and, often, other engineering disciplines, such as communications), then, the term system came to be defined as containing all of the elements necessary (which generally includes both hardware and software) to perform a useful function. Consequently, within these engineering disciplines, a system generally refers to a programmable hardware machine and its included program. And a systems engineer is defined as one concerned with the complete device, both hardware and software and, more particularly, all of the interfaces of the device, including that between hardware and software, and especially between the complete device and its user (the CHI). The hardware engineer deals (more or less) exclusively with the hardware device; the software engineer deals (more or less) exclusively with the software program; and the systems engineer is responsible for seeing that the software program is capable of properly running within the hardware device, and that the system composed of the two entities is capable of properly interacting with its external environment, especially the user, and performing its intended function. By analogy, then, a systems architecture makes use of elements of both software and hardware and is used to enable design of such a composite system. A good architecture may be viewed as a 'partitioning scheme,' or algorithm, which partitions all of the system's present and foreseeable requirements into a workable set of cleanly bounded subsystems with nothing left over. That is, it is a partitioning scheme which is exclusive, inclusive, and exhaustive. A major purpose of the partitioning is to arrange the elements in the sub systems so that there is a minimum of communications needed among them. In both software and hardware, a good sub system tends to be seen to be a meaningful "object". Moreover, a good architecture provides for an easy mapping to the user's requirements and the 39 Systems architecture validation tests of the user's requirements. Ideally, a mapping also exists from every least element to every requirement and test. Types of systems architectures Several types of systems architectures (underlain by the same fundamental principles[1] ) have been identified as follows[2] : • • • • • • • • Collaborative Systems (such as the Internet, intelligent transportation systems, and joint air defense systems) Manufacturing Systems Social Systems Software and Information Technology Systems Strategic Systems Architecture[3] Hardware architecture Software architecture Enterprise architecture References [1] fundamental principles of Systems Architecture (http:/ / www. lix. polytechnique. fr/ ~golden/ systems_architecture. html#principles''The), by Boris Golden [2] The Art of Systems Architecture, Mark Maier and Eberhardt Rechtin, 2nd ed 2002 [3] Choosing A Strategic Systems Architecture (http:/ / www. forrester. com/ Research/ Document/ Excerpt/ 0,7211,39702,00. html), by Brad Day External links • • • • Principles of system architecture (http://sysarch.pbwiki.com) What is Systems Architecture ? (http://www.lix.polytechnique.fr/~golden/systems_architecture.html) INCOSE Systems Architecture Working Group (http://www.incose.org/practice/techactivities/wg/sysarch/) Journal of Systems Architecture (http://www.elsevier.com/wps/find/journaldescription.cws_home/505616/ description#description) 40 41 Dynamic models Function model A function model or functional model in systems engineering and software engineering is a structured representation of the functions (activities, actions, processes, operations) within the modeled system or subject area.[1] A function model, also called an activity model or process model, is a graphical representation of an enterprise's function within a defined scope. The purposes of the function model are to describe the functions and processes, assist with discovery of information needs, help identify opportunities, and establish a basis for determining product and service costs.[2] History The model originates in the 1950s, after in the first half of the 20th Example of a function model of the process of "Maintain Reparable Spares" in IDEF0 century other types of management notation. diagrams had already been developed. The first known Gantt Chart was developed in 1896 by Karol Adamiecki, who called it a harmonogram. Because Adamiecki did not publish his chart until 1931 - and in any case his works were published in either Polish or Russian, languages not popular in the West - the chart now bears the name of Henry Gantt (1861–1919), who designed his chart around the years 1910-1915 and popularized it in the West.[3] [4] [5] The first structured method for documenting process flow, the flow process chart, was introduced by Frank Gilbreth to members of American Society of Mechanical Engineers (ASME) in 1921 as the presentation “Process Charts—First Steps in Finding the One Best Way”.[6] Gilbreth's tools quickly found their way into industrial engineering curricula. One of the first well defined function models, was the Functional Flow Block Diagram (FFBD) developed by the defense-related TRW Incorporated in the 1950s.[7] In the 1960s it was exploited by the NASA to visualize the time sequence of events in a space systems and flight missions.[8] It is further widely used in classical systems engineering to show the order of execution of system functions.[9] Function model 42 Functional modeling topics Functional perspective In systems engineering and software engineering a function model is created with a functional modeling perspective. The functional perspective is one of the perspectives possible in business process modelling, other perspecifives are for example behavioural, organisational or informational.[10] A functional modeling perspective concentrates on describing the dynamic process. The main concept in this modeling perspective is the process, this could be a function, transformation, activity, action, task etc. A well-known example of a modeling language employing this perspective is data flow diagrams. The perspective uses four symbols to describe a process, these being: • • • • Process: Illustrates transformation from input to output. Store: Data-collection or some sort of material. Flow: Movement of data or material in the process. External Entity: External to the modeled system, but interacts with it. Now, with these symbols, a process can be represented as a network of these symbols. This decomposed process is a DFD, data flow diagram. In Dynamic Enterprise Modeling a division is made in the Control model, Function Model, Process model and Organizational model. Functional decomposition Functional decomposition refers broadly to the process of resolving a functional relationship into its constituent parts in such a way that the original function can be reconstructed from those parts by function composition. In general, this process of decomposition is undertaken either for the purpose of gaining insight into the identity of the constituent components, or for the purpose of obtaining a compressed representation of the global function, a task which is feasible only when the constituent processes possess a certain level of modularity. Functional decomposition has a prominent role in computer programming, where a major goal is to modularize processes to the greatest extent possible. For example, a library management system may be broken up into an inventory module, a patron information module, and a fee assessment module. In the early decades of computer programming, this was manifested as the "art of subroutining," as it was called by some prominent practitioners. Example of functional decomposition in a Functional decomposition of engineering systems is a method for systems analysis. analyzing engineered systems. The basic idea is to try to divide a system in such a way that each block of the block diagram can be described without an "and" or "or" in the description. This exercise forces each part of the system to have a pure function. When a system is composed of pure functions, they can be reused, or replaced. A usual side effect is that the interfaces between blocks become simple and generic. Since the interfaces usually become simple, it is easier to replace a pure function with a related, similar function. Function model 43 Functional modeling methods The functional approach is extended in multiple diagrammic techniques and modeling notations. This section gives an overview of the important techniques in chronological order. Functional Flow Block Diagram The Functional flow block diagram (FFBD) is a multi-tier, time-sequenced, step-by-step flow diagram of the system’s functional flow.[12] The diagram is developed in the 1950s and widely used in classical systems engineering. The Functional Flow Block Diagram is also referred to as Functional Flow Diagram, functional block diagram, and functional flow.[13] Functional Flow Block Diagrams (FFBD) usually define the detailed, step-by-step operational and support [11] Functional Flow Block Diagram Format. sequences for systems, but they are also used effectively to define processes in developing and producing systems. The software development processes also use FFBDs extensively. In the system context, the functional flow steps may include combinations of hardware, software, personnel, facilities, and/or procedures. In the FFBD method, the functions are organized and depicted by their logical order of execution. Each function is shown with respect to its logical relationship to the execution and completion of other functions. A node labeled with the function name depicts each function. Arrows from left to right show the order of execution of the functions. Logic symbols represent sequential or parallel execution of functions.[14] HIPO and IPO HIPO for hierarchical input process output is a popular 1970s systems analysis design aid and documentation technique[15] for representing the modules of a system as a hierarchy and for documenting each module.[16] It was used to develop requirements, construct the design, and support implementation of an expert system to demonstrate automated rendezvous. Verification was then conducted systematically because of the method of design and implementation.[17] The overall design of the system is documented using HIPO charts or structure charts. The structure chart is similar in appearance to an organizational chart, but has been modified to show additional detail. Structure charts can be usedto display several types of information, but are used most commonly to diagram either data structures or code structures.[16] An expanded IPO Model. Function model 44 N2 Chart The N2 Chart is a diagram in the shape of a matrix, representing functional or physical interfaces between system elements. It is used to systematically identify, define, tabulate, design, and analyze functional and physical interfaces. It applies to system interfaces and hardware and/or software interfaces.[12] The N2 diagram has been used extensively to develop data interfaces, primarily in the software areas. However, it can also be used to develop hardware interfaces. The basic N2 chart is shown in Figure 2. The system functions are placed on the diagonal; the remainder of the squares in the N x N matrix represent the interface inputs and outputs. [18] [18] Figure 2. N2 chart definition. Structured Analysis and Design Technique SADT basis element. Structured Analysis and Design Technique (SADT) is a software engineering methodology for describing systems as a hierarchy of functions, a diagrammatic notation for constructing a sketch for a software application. It offers building blocks to represent entities and activities, and a variety of arrows to relate boxes. These boxes and arrows have an associated informal semantics.[19] SADT can be used as a functional analysis tool of a given process, using successive levels of details. The SADT method allows to define user needs for IT developments, which is used in industrial Information Systems, but also to explain and to present an activity’s manufacturing processes, procedures.[20] The SADT supplies a specific functional view of any enterprise by describing the functions and their relationships in a company. These functions fulfill the objectives of a company, such as sales, order planning, product design, part manufacturing, and human resource management. The SADT can depict simple functional relationships and can reflect data and control flow relationships between different functions. The IDEF0 formalism is based on SADT, developed by Douglas T. Ross in 1985.[21] Function model IDEF0 IDEF0 is a function modeling methodology for describing manufacturing functions, which offers a functional modeling language for the analysis, development, re-engineering, and integration of information systems; business processes; or software engineering analysis.[22] It is part of the IDEF family of modeling languages in the field of software engineering, and is built on the functional modeling language building SADT. The IDEF0 Functional Modeling method is designed to model the decisions, actions, and activities of an organization or system.[23] It was IDEF0 Diagram Example derived from the established graphic modeling language Structured Analysis and Design Technique (SADT) developed by Douglas T. Ross and SofTech, Inc.. In its original form, IDEF0 includes both a definition of a graphical modeling language (syntax and semantics) and a description of a comprehensive methodology for developing models.[1] The US Air Force commissioned the SADT developers to develop a function model method for analyzing and communicating the functional perspective of a system. IDEF0 should assist in organizing system analysis and promote effective communication between the analyst and the customer through simplified graphical devices.[23] Business Process Model and Notation Business Process Model and Notation (BPMN) is a graphical representation for specifying business processes in a workflow. BPMN was developed by Business Process Management Initiative (BPMI), and is currently maintained by the Object Management Group since the two organizations merged in 2005. The current version of BPMN is 2.0.[24] The Business Process Model and Business Process Modeling Notation Example. Notation (BPMN) specification provides a graphical notation for specifying business processes in a Business Process Diagram (BPD).[25] The objective of BPMN is to support business process management for both technical users and business users by providing a notation that is intuitive to business users yet able to represent complex process semantics. The BPMN specification also provides a mapping between the graphics of the notation to the underlying constructs of execution languages, particularly BPEL4WS.[26] 45 Function model Axiomatic Design Axiomatic design is a top down hierarchical functional decomposition process used as a solution synthesis framework for the analysis, development, re-engineering, and integration of products, information systems, business processes or software engineering solutions.[27] Its structure is suited mathematically to analyze coupling between functions in order to optimize the architectural robustness of potential functional solution models. Other types of function models In the field of systems and software engineering numerous specific function and functional models have been defined. Here only a few general types will be explained. Business function model A Business Function Model (BFM) is a general description or category of operations performed routinely to carry out an organization's mission. It can show the critical business processes in the context of the business area functions. The processes in the business function model must be consistent with the processes in the value chain models. Processes are a group of related business activities performed to produce an end product or to provide a service. Unlike business functions that are performed on a continual basis, processes are characterized by the fact that they have a specific beginning and an end point marked by the delivery of a desired output. The figure on the right depicts the relationship between the business processes, business functions, and the business area’s business reference model.[28] Business reference model A Business reference model is a reference model, concentrating on the functional and organizational aspects of the core business of an enterprise, service organization or government agency. In enterprise engineering a business reference model is part of an Enterprise Architecture Framework or Architecture Framework, which defines how to organize the structure and views associated with an Enterprise Architecture. This FEA Business reference model depicts the relationship between the business A reference model in general is a model processes, business functions, and the business area’s business reference model. of something that embodies the basic goal or idea of something and can then be looked at as a reference for various purposes. A business reference model is a means to describe the business operations of an organization, independent of the organizational structure that perform them. Other types of business reference model can also depict the relationship between the business processes, business functions, and the business area’s business reference model. These reference model can be constructed in layers, and offer a foundation for the analysis of service components, technology, data, and performance. 46 Function model Operator function model The Operator Function Model (OFM) is proposed as an alternative to traditional task analysis techniques used by human factors engineers. An operator function model attempts to represent in mathematical form how an operator might decompose a complex system into simpler parts and coordinate control actions and system configurations so that acceptable overall system performance is achieved. The model represents basic issues of knowledge representation, information flow, and decision making in complex systems. Miller (1985) suggests that the network structure can be thought of as a possible representation of an operator's internal model of the system plus a control structure which specifies how the model is used to solve the decision problems that comprise operator control functions.[29] References This article incorporates public domain material from websites or documents of the National Institute of Standards and Technology [30]. [1] FIPS Publication 183 (http:/ / www. itl. nist. gov/ fipspubs/ idef02. doc) released of IDEFØ December 1993 by the Computer Systems Laboratory of the National Institute of Standards and Technology (NIST). [2] Reader's Guide to IDEF0 Function Models (http:/ / www. archives. gov/ era/ pdf/ rmsc-19951006-dod-rm-function-and-information-models. pdf). Accessed 27 Nov 2008. [3] H.L. Gantt, Work, Wages and Profit, published by The Engineering Magazine, New York, 1910; republished as Work, Wages and Profits, Easton, Pennsylvania, Hive Publishing Company, 1974, ISBN 0879600489. [4] Gerard Blokdijk, Project Management 100 Success Secrets, Lulu.com, 2007, ISBN 0980459907, Google Print, p.76 (http:/ / books. google. com/ books?id=dgB-QWHlnrUC& pg=PA76& dq=Adamiecki+ Gantt& as_brr=3& sig=Jp-mgVODNRJpxqBRM1PYJbs7mOU) [5] Peter W. G. Morris, The Management of Projects, Thomas Telford, 1994, ISBN 0727725939, Google Print, p.18 (http:/ / books. google. com/ books?id=5ekyoWaeZ1UC& pg=PA18-IA7& dq=Adamiecki+ Gantt& as_brr=3& sig=xe_RAipoqlvhnu0xLkIsxx-8OAQ) [6] Ben B. Graham (2002). Detail Process Charting. p.2. [7] Tim Weilkiens (2008). Systems Engineering with SysML/UML: Modeling, Analysis, Design. Page 287. [8] Harold Chestnut (1967). Systems Engineering Methods. Page 254. [9] Thomas Dufresne & James Martin (2003). "Process Modeling for E-Business" (http:/ / mason. gmu. edu/ ~tdufresn/ paper. doc). INFS 770 Methods for Information Systems Engineering: Knowledge Management and E-Business. Spring 2003 [10] Process perspectives (http:/ / users. jyu. fi/ ~jpt/ ME2000/ Me14/ sld004. htm). In: Metamodeling and method engineering, Minna Koskinen, 2000. [11] Systems Engineering Fundamentals. (http:/ / www. dau. mil/ pubs/ pdf/ SEFGuide 01-01. pdf) Defense Acquisition University Press, 2001 [12] The first version of this article is completely based on the NAS SYSTEM ENGINEERING MANUAL SECTION (http:/ / www. faa. gov/ about/ office_org/ headquarters_offices/ ato/ service_units/ operations/ sysengsaf/ seman/ SEM3. 1/ Section 4. 4. pdf) 4.4 VERSION 3.1 06/06/06. [13] Task Analysis Tools Used Throughout Development (http:/ / www. hf. faa. gov/ webtraining/ Usability/ u17_tools2. htm). FAA 2008. Retrieved 25 Sept 2008. [14] FAA (2006). NAS SYSTEM ENGINEERING MANUAL SECTION (http:/ / www. faa. gov/ about/ office_org/ headquarters_offices/ ato/ service_units/ operations/ sysengsaf/ seman/ SEM3. 1/ Section 4. 4. pdf) 4.4 VERSION 3.1 06/06/06. [15] IBM Corporation (1974).HIPO—A Design Aid and Documentation Technique, Publication Number GC20-1851, IBM Corporation, White Plains, NY, 1974. [16] Sandia National Laboratories (1992). Sandia Software Guidelines Volume 5 Tools, Techniques,and Methodologies (http:/ / www. prod. sandia. gov/ cgi-bin/ techlib/ access-control. pl/ 1985/ 852348. pdf) SANDIA REPORTS 85–2348qUC–32 [17] Mary Ann Goodwin and Charles C. Robertson (1986). EXPERT SYSTEM VERIFICATION CONCERNS IN AN OPERATIONS ENVIRONMENT (http:/ / ntrs. nasa. gov/ archive/ nasa/ casi. ntrs. nasa. gov/ 19880007850_1988007850. pdf). NASA paper N88-17234. [18] NASA (1995). "Techniques of Functional Analysis". In: NASA Systems Engineering Handbook (http:/ / human. space. edu/ old/ docs/ Systems_Eng_Handbook. pdf) June 1995. p.142. [19] John Mylopoulos (2004). Conceptual Modelling III. Structured Analysis and Design Technique (SADT) (http:/ / www. cs. toronto. edu/ ~jm/ 2507S/ Notes04/ SADT. pdf). Retrieved 21 Sep 2008. [20] SADT (http:/ / www. free-logistics. com/ index. php/ Download-document/ 22-SADT_eng. html) at Free-logisitcs.com. Retrieved 21 Sep 2008. [21] Gavriel Salvendy (2001). Handbook of Industrial Engineering: Technology and Operations Management.. p.508. [22] Systems Engineering Fundamentals. (http:/ / www. dau. mil/ pubs/ pdf/ SEFGuide 01-01. pdf) Defense Acquisition University Press, 2001. [23] Varun Grover, William J. Kettinger (2000). Process Think: Winning Perspectives for Business Change in the Information Age. p.168. [24] "BPMN Information" (http:/ / www. bpmn. org/ Documents/ FAQ. htm). . Retrieved 2008-11-02. 47 Function model 48 [25] Richard C. Simpson (2004). An XML Representation for Crew Procedures (http:/ / ntrs. nasa. gov/ archive/ nasa/ casi. ntrs. nasa. gov/ 20050202022_2005202152. pdf). Final Report NASA Faculty Fellowship Program - 2004. Johnson Space Center. [26] S.A. White, "Business Process Modeling Notation (BPMN)," In: Business Process Management Initiative (BPMI) 3 May 2004. [27] Suh (2001). Axiomatic Design: Advances and Applications, Oxford University Press, 2001, ISBN 0-19-513466-4 [28] Analyze the Business and Define the Target Business Environment (http:/ / www. doi. gov/ ocio/ architecture/ mbt/ step3. htm). Accessed 27 Nov 2008. [29] Operator Function Model (OFM) (http:/ / www2. hf. faa. gov/ workbenchtools/ default. aspx?rPage=Tooldetails& toolID=192). Accessed 27 Nov 2008. [30] http:/ / www. nist. gov Function block diagram A function block diagram (FBD) is a block diagram that describes a function between input variables and output variables. A function is described as a set of elementary blocks. Input and output variables are connected to blocks by connection lines. An output of a block may also be connected to an input of another block: Inputs and outputs of the blocks are wired together with connection lines, or links. Single lines may be used to connect two logical points of the diagram: • An input variable and an input of a block • An output of a block and an input of another block • An output of a block and an output variable Functional block diagram of the attitude control and maneuvering electronics system. June 1962. The connection is oriented, meaning that the line carries associated data from the left end to the right end. The left and right ends of the connection line must be of the same type. Multiple right connection, also called divergence can be used to broadcast information from its left end to each of its right ends. All ends of the connection must be of the same type. Function block diagram is one of five languages for logic or control configuration supported by standard IEC 61131-3 for a control system such as a Programmable Logic Controller (PLC) or a Distributed Control System (DCS). The other supported languages are ladder logic, sequential function chart, structured text, and instruction list. Transfer function 49 Transfer function A transfer function (also known as the system function[1] or network function) is a mathematical representation, in terms of spatial or temporal frequency, of the relation between the input and output of a linear time-invariant system. With optical imaging devices, for example, it is the Fourier transform of the point spread function (hence a function of spatial frequency) i.e. the intensity distribution caused by a point object in the field of view. Explanation Transfer functions are commonly used in the analysis of systems such as single-input single-output filters, typically within the fields of signal processing, communication theory, and control theory. The term is often used exclusively to refer to linear, time-invariant systems (LTI), as covered in this article. Most real systems have non-linear input/output characteristics, but many systems, when operated within nominal parameters (not "over-driven") have behavior that is close enough to linear that LTI system theory is an acceptable representation of the input/output behavior. In its simplest form for continuous-time input signal mapping of the Laplace transform of the input, and output , to the output , the transfer function is the linear : or where is the transfer function of the LTI system. In discrete-time systems, the function is similarly written as (see Z transform) and is often referred to as the pulse-transfer function. Direct derivation from differential equations Consider a linear differential equation with constant coefficients where u and r are suitably smooth functions of t, and L is the operator defined on the relevant function space, that transforms u into r. That kind of equation can be used to constrain the output function u in terms of the forcing function r. The transfer function, written as an operator , is the right inverse of L, since . Solutions of the homogeneous equation can be found by trying . That substitution yields the characteristic polynomial The inhomogeneous case can be easily solved if the input function r is also of the form substituting one finds that . In that case, by if and only if Taking that as the definition of the transfer function[2] requires careful disambiguation between complex vs. real values, which is traditionally influenced by the interpretation of abs(H(s)) as the gain and -atan(H(s)) as the phase lag. Transfer function 50 Signal processing Let be the input to a general linear time-invariant system, and transform of and be the output, and the bilateral Laplace be . Then the output is related to the input by the transfer function as and the transfer function itself is therefore . In particular, if a complex harmonic signal with a sinusoidal component with amplitude , angular frequency and phase where is input to a linear time-invariant system, then the corresponding component in the output is: and Note that, in a linear time-invariant system, the input frequency has not changed, only the amplitude and the phase angle of the sinusoid has been changed by the system. The frequency response describes this change for every frequency in terms of gain: and phase shift: . The phase delay (i.e., the frequency-dependent amount of delay introduced to the sinusoid by the transfer function) is: . The group delay (i.e., the frequency-dependent amount of delay introduced to the envelope of the sinusoid by the transfer function) is found by computing the derivative of the phase shift with respect to angular frequency , . The transfer function can also be shown using the Fourier transform which is only a special case of the bilateral Laplace transform for the case where . Transfer function Common transfer function families While any LTI system can be described by some transfer function or another, there are certain "families" of special transfer functions that are commonly used. Typical infinite impulse response filters are designed to implement one of these special transfer functions. Some common transfer function families and their particular characteristics are: • • • • • • • • • Butterworth filter – maximally flat in passband and stopband for the given order Chebyshev filter (Type I) – maximally flat in stopband, sharper cutoff than Butterworth of same order Chebyshev filter (Type II) – maximally flat in passband, sharper cutoff than Butterworth of same order Bessel filter – best pulse response for a given order because it has no group delay ripple Elliptic filter – sharpest cutoff (narrowest transition between pass band and stop band) for the given order Optimum "L" filter Gaussian filter – minimum group delay; gives no overshoot to a step function. Hourglass filter Raised-cosine filter Control engineering In control engineering and control theory the transfer function is derived using the Laplace transform. The transfer function was the primary tool used in classical control engineering. However, it has proven to be unwieldy for the analysis of multiple-input multiple-output (MIMO) systems, and has been largely supplanted by state space representations for such systems. In spite of this, a transfer matrix can be always obtained for any linear system, in order to analyze its dynamics and other properties: each element of a transfer matrix is a transfer function relating a particular input variable to an output variable. Optics In optics, modulation transfer function indicates the capability of optical contrast transmission. For example, when observing a series of black-white-light fringes drawn with a specific spatial frequency, the image quality may decay. White fringes fade while black ones turn brighter. The modulation transfer function in a specific spatial frequency is defined by: Where modulation (M) is computed from the following image or light brightness: 51 Transfer function 52 References [1] Bernd Girod, Rudolf Rabenstein, Alexander Stenger, Signals and systems, 2nd ed., Wiley, 2001, ISBN 0471988006 p. 50 [2] The transfer function is defined by in, e.g., Birkhoff, Garrett; Rota, Gian-Carlo (1978). Ordinary differential equations. New York: John Wiley & Sons. ISBN 0-471-05224-8. External links • Transfer function (http://planetmath.org/?op=getobj&from=objects&id=5394) on PlanetMath • ECE 209: Review of Circuits as LTI Systems (http://www.tedpavlic.com/teaching/osu/ece209/support/ circuits_sys_review.pdf) — Short primer on the mathematical analysis of (electrical) LTI systems. • ECE 209: Sources of Phase Shift (http://www.tedpavlic.com/teaching/osu/ece209/lab3_opamp_FO/ lab3_opamp_FO_phase_shift.pdf) — Gives an intuitive explanation of the source of phase shift in two simple LTI systems. Also verifies simple transfer functions by using trigonometric identities. Convolution In mathematics and, in particular, functional analysis, convolution is a mathematical operation on two functions f and g, producing a third function that is typically viewed as a modified version of one of the original functions. Convolution is similar to cross-correlation. It has applications that include probability, statistics, computer vision, image and signal processing, electrical engineering, and differential equations. Convolution of two square pulses: the resulting waveform is a triangular pulse. One of the functions (in this case g) is first reflected about and then offset by t, making it . The area under the resulting product gives the convolution at t. The horizontal axis is for f and g, and t for . The convolution can be defined for functions on groups other than Euclidean space. In particular, the circular convolution can be defined for periodic functions (that is, functions on the circle), and the discrete Convolution of a square pulse (as input signal) with the impulse response of an RC circuit convolution can be defined for to obtain the output signal waveform. The integral of their product is the area of the yellow region. In both animations the function g is symmetric, and so is unchanged under functions on the set of integers. These reflection. generalizations of the convolution have applications in the field of numerical analysis and numerical linear algebra, and in the design and implementation of finite impulse response filters in signal processing. Computing the inverse of the convolution operation is known as deconvolution. History The operation Convolution 53 is a particular case of composition products considered by the Italian mathematician Vito Volterra in 1913.[1] Convolution is also sometimes called "Faltung" (which means folding in German); both Faltung and convolution were used as early as 1903, though the definition is rather unfamiliar in older uses.[2] [3] The term Faltung was sometimes used in English through the 1940s, before the notion of convolution became widely used, along with other terms such as composition product, superposition integral, and Carson's integral.[4] Definition The convolution of ƒ and g is written ƒ∗g, using an asterisk or star. It is defined as the integral of the product of the two functions after one is reversed and shifted. As such, it is a particular kind of integral transform: (commutativity) While the symbol t is used above, it need not represent the time domain. But in that context, the convolution formula can be described as a weighted average of the function ƒ(τ) at the moment t where the weighting is given by g(−τ) simply shifted by amount t. As t changes, the weighting function emphasizes different parts of the input function. More generally, if f and g are complex-valued functions on Rd, then their convolution may be defined as the integral: Visual explanation of convolution. 1. Express each function in terms of a dummy variable 2. Reflect one of the functions: → 3. Add a time-offset, t, which allows to slide along the -axis. 4. Start t at -∞ and slide it all the way to +∞. Wherever the two functions intersect, find the integral of their product. In other words, compute a sliding, weighted-average of function , where the weighting function is The resulting waveform (not shown here) is the convolution of functions f and g. If f(t) is a unit impulse, the result of this process is simply g(t), which is therefore called the impulse response. Convolution 54 Circular convolution When a function gT is periodic, with period T, then for functions, ƒ, such that ƒ∗gT exists, the convolution is also periodic and identical to: where to is an arbitrary choice. The summation is called a periodic summation of the function ƒ. If gT is a periodic summation of another function, g, then ƒ∗gT is known as a circular, cyclic, or periodic convolution of ƒ and g. Discrete convolution For complex-valued functions f, g defined on the set Z of integers, the discrete convolution of f and g is given by: (commutativity) When multiplying two polynomials, the coefficients of the product are given by the convolution of the original coefficient sequences, extended with zeros where necessary to avoid undefined terms; this is known as the Cauchy product of the coefficients of the two polynomials. Circular discrete convolution When a function gN is periodic, with period N, then for functions, f, such that f∗gN exists, the convolution is also periodic and identical to: The summation on k is called a periodic summation of the function f. If gN is a periodic summation of another function, g, then f∗gN is known as a circular convolution of f and g. When the non-zero durations of both f and g are limited to the interval [0, N − 1], f∗gN reduces to these common forms: (Eq.1) The notation for cyclic convolution denotes convolution over the cyclic group of integers modulo N. Circular convolution is frequently used to characterized systems analyzed through the lens of the Discrete Fourier Transform. Convolution Fast convolution algorithms In many situations, discrete convolutions can be converted to circular convolutions so that fast transforms with a convolution property can be used to implement the computation. For example, convolution of digit sequences is the kernel operation in multiplication of multi-digit numbers, which can therefore be efficiently implemented with transform techniques (Knuth 1997, §4.3.3.C; von zur Gathen & Gerhard 2003, §8.2). Eq.1 requires N arithmetic operations per output value and N2 operations for N outputs. That can be significantly reduced with any of several fast algorithms. Digital signal processing and other applications typically use fast convolution algorithms to reduce the cost of the convolution to O(N log N) complexity. The most common fast convolution algorithms use fast Fourier transform (FFT) algorithms via the circular convolution theorem. Specifically, the circular convolution of two finite-length sequences is found by taking an FFT of each sequence, multiplying pointwise, and then performing an inverse FFT. Convolutions of the type defined above are then efficiently implemented using that technique in conjunction with zero-extension and/or discarding portions of the output. Other fast convolution algorithms, such as the Schönhage–Strassen algorithm, use fast Fourier transforms in other rings. Domain of definition The convolution of two complex-valued functions on Rd is well-defined only if ƒ and g decay sufficiently rapidly at infinity in order for the integral to exist. Conditions for the existence of the convolution may be tricky, since a blow-up in g at infinity can be easily offset by sufficiently rapid decay in ƒ. The question of existence thus may involve different conditions on ƒ and g. Compactly supported functions If ƒ and g are compactly supported continuous functions, then their convolution exists, and is also compactly supported and continuous (Hörmander). More generally, if either function (say ƒ) is compactly supported and the other is locally integrable, then the convolution ƒ∗g is well-defined and continuous. Integrable functions The convolution of ƒ and g exists if ƒ and g are both Lebesgue integrable functions (in L1(Rd)), and in this case ƒ∗g is also integrable (Stein & Weiss 1971, Theorem 1.3). This is a consequence of Tonelli's theorem. Likewise, if ƒ ∈ L1(Rd) and g ∈ Lp(Rd) where 1 ≤ p ≤ ∞, then ƒ∗g ∈ Lp(Rd) and In the particular case p= 1, this shows that L1 is a Banach algebra under the convolution (and equality of the two sides holds if f and g are non-negative almost everywhere). More generally, Young's inequality implies that the convolution is a continuous bilinear map between suitable Lp spaces. Specifically, if 1 ≤ p,q,r ≤ ∞ satisfy then so that the convolution is a continuous bilinear mapping from Lp×Lq to Lr. 55 Convolution Functions of rapid decay In addition to compactly supported functions and integrable functions, functions that have sufficiently rapid decay at infinity can also be convolved. An important feature of the convolution is that if ƒ and g both decay rapidly, then ƒ∗g also decays rapidly. In particular, if ƒ and g are rapidly decreasing functions, then so is the convolution ƒ∗g. Combined with the fact that convolution commutes with differentiation (see Properties), it follows that the class of Schwartz functions is closed under convolution. Distributions Under some circumstances, it is possible to define the convolution of a function with a distribution, or of two distributions. If ƒ is a compactly supported function and g is a distribution, then ƒ∗g is a smooth function defined by a distributional formula analogous to More generally, it is possible to extend the definition of the convolution in a unique way so that the associative law remains valid in the case where ƒ is a distribution, and g a compactly supported distribution (Hörmander 1983, §4.2). Measures The convolution of any two Borel measures μ and ν of bounded variation is the measure λ defined by This agrees with the convolution defined above when μ and ν are regarded as distributions, as well as the convolution of L1 functions when μ and ν are absolutely continuous with respect to the Lebesgue measure. The convolution of measures also satisfies the following version of Young's inequality where the norm is the total variation of a measure. Because the space of measures of bounded variation is a Banach space, convolution of measures can be treated with standard methods of functional analysis that may not apply for the convolution of distributions. Properties Algebraic properties The convolution defines a product on the linear space of integrable functions. This product satisfies the following algebraic properties, which formally mean that the space of integrable functions with the product given by convolution is a commutative algebra without identity (Strichartz 1994, §3.3). Other linear spaces of functions, such as the space of continuous functions of compact support, are closed under the convolution, and so also form commutative algebras. Commutativity Associativity Distributivity 56 Convolution 57 Associativity with scalar multiplication for any real (or complex) number . Multiplicative identity No algebra of functions possesses an identity for the convolution. The lack of identity is typically not a major inconvenience, since most collections of functions on which the convolution is performed can be convolved with a delta distribution or, at the very least (as is the case of L1) admit approximations to the identity. The linear space of compactly supported distributions does, however, admit an identity under the convolution. Specifically, where δ is the delta distribution. Inverse element Some distributions have an inverse element for the convolution, S(−1), which is defined by The set of invertible distributions forms an abelian group under the convolution. Complex conjugation Integration If ƒ and g are integrable functions, then the integral of their convolution on the whole space is simply obtained as the product of their integrals: This follows from Fubini's theorem. The same result holds if ƒ and g are only assumed to be nonnegative measurable functions, by Tonelli's theorem. Differentiation In the one-variable case, where d/dx is the derivative. More generally, in the case of functions of several variables, an analogous formula holds with the partial derivative: A particular consequence of this is that the convolution can be viewed as a "smoothing" operation: the convolution of ƒ and g is differentiable as many times as ƒ and g are together. These identities hold under the precise condition that ƒ and g are absolutely integrable and at least one of them has an absolutely integrable (L1) weak derivative, as a consequence of Young's inequality. For instance, when ƒ is continuously differentiable with compact support, and g is an arbitrary locally integrable function, These identities also hold much more broadly in the sense of tempered distributions if one of ƒ or g is a compactly supported distribution or a Schwartz function and the other is a tempered distribution. On the other hand, two positive integrable and infinitely differentiable functions may have a nowhere continuous convolution. Convolution 58 In the discrete case, the difference operator D ƒ(n) = ƒ(n + 1) − ƒ(n) satisfies an analogous relationship: Convolution theorem The convolution theorem states that where denotes the Fourier transform of , and is a constant that depends on the specific normalization of the Fourier transform (see “Properties of the Fourier transform”). Versions of this theorem also hold for the Laplace transform, two-sided Laplace transform, Z-transform and Mellin transform. See also the less trivial Titchmarsh convolution theorem. Translation invariance The convolution commutes with translations, meaning that where τxƒ is the translation of the function ƒ by x defined by If ƒ is a Schwartz function, then τxƒ is the convolution with a translated Dirac delta function τxƒ = ƒ∗τx δ. So translation invariance of the convolution of Schwartz functions is a consequence of the associativity of convolution. Furthermore, under certain conditions, convolution is the most general translation invariant operation. Informally speaking, the following holds • Suppose that S is a linear operator acting on functions which commutes with translations: S(τxƒ) = τx(Sƒ) for all x. Then S is given as convolution with a function (or distribution) gS; that is Sƒ = gS∗ƒ. Thus any translation invariant operation can be represented as a convolution. Convolutions play an important role in the study of time-invariant systems, and especially LTI system theory. The representing function gS is the impulse response of the transformation S. A more precise version of the theorem quoted above requires specifying the class of functions on which the convolution is defined, and also requires assuming in addition that S must be a continuous linear operator with respect to the appropriate topology. It is known, for instance, that every continuous translation invariant continuous linear operator on L1 is the convolution with a finite Borel measure. More generally, every continuous translation invariant continuous linear operator on Lp for 1 ≤ p < ∞ is the convolution with a tempered distribution whose Fourier transform is bounded. To wit, they are all given by bounded Fourier multipliers. Convolutions on groups If G is a suitable group endowed with a measure λ, and if f and g are real or complex valued integrable functions on G, then we can define their convolution by In typical cases of interest G is a locally compact Hausdorff topological group and λ is a (left-) Haar measure. In that case, unless G is unimodular, the convolution defined in this way is not the same as . The preference of one over the other is made so that convolution with a fixed function g commutes with left translation in the group: Convolution Furthermore, the convention is also required for consistency with the definition of the convolution of measures given below. However, with a right instead of a left Haar measure, the latter integral is preferred over the former. On locally compact abelian groups, a version of the convolution theorem holds: the Fourier transform of a convolution is the pointwise product of the Fourier transforms. The circle group T with the Lebesgue measure is an immediate example. For a fixed g in L1(T), we have the following familiar operator acting on the Hilbert space L2(T): The operator T is compact. A direct calculation shows that its adjoint T* is convolution with By the commutativity property cited above, T is normal: T*T = TT*. Also, T commutes with the translation operators. Consider the family S of operators consisting of all such convolutions and the translation operators. Then S is a commuting family of normal operators. According to spectral theory, there exists an orthonormal basis {hk} that simultaneously diagonalizes S. This characterizes convolutions on the circle. Specifically, we have which are precisely the characters of T. Each convolution is a compact multiplication operator in this basis. This can be viewed as a version of the convolution theorem discussed above. A discrete example is a finite cyclic group of order n. Convolution operators are here represented by circulant matrices, and can be diagonalized by the discrete Fourier transform. A similar result holds for compact groups (not necessarily abelian): the matrix coefficients of finite-dimensional unitary representations form an orthonormal basis in L2 by the Peter–Weyl theorem, and an analog of the convolution theorem continues to hold, along with many other aspects of harmonic analysis that depend on the Fourier transform. Convolution of measures Let G be a topological group. If μ and ν are finite Borel measures on a group G, then their convolution μ∗ν is defined by for each measurable subset E of G. The convolution is also a finite measure, whose total variation satisfies In the case when G is locally compact with (left-)Haar measure λ, and μ and ν are absolutely continuous with respect to a λ, so that each has a density function, then the convolution μ∗ν is also absolutely continuous, and its density function is just the convolution of the two separate density functions. If μ and ν are probability measures, then the convolution μ∗ν is the probability distribution of the sum X + Y of two independent random variables X and Y whose respective distributions are μ and ν. 59 Convolution Bialgebras Let (X, Δ, ∇, ε, η) be a bialgebra with comultiplication Δ, multiplication ∇, unit η, and counit ε. The convolution is a product defined on the endomorphism algebra End(X) as follows. Let φ, ψ ∈ End(X), that is, φ,ψ : X → X are functions that respect all algebraic structure of X, then the convolution φ∗ψ is defined as the composition The convolution appears notably in the definition of Hopf algebras (Kassel 1995, §III.3). A bialgebra is a Hopf algebra if and only if it has an antipode: an endomorphism S such that Applications Convolution and related operations are found in many applications of engineering and mathematics. • In electrical engineering, the convolution of one function (the input signal) with a second function (the impulse response) gives the output of a linear time-invariant system (LTI). At any given moment, the output is an accumulated effect of all the prior values of the input function, with the most recent values typically having the most influence (expressed as a multiplicative factor). The impulse response function provides that factor as a function of the elapsed time since each input value occurred. • • • • • • • • • • • In digital signal processing and image processing applications, the entire input function is often available for computing every sample of the output function. In that case, the constraint that each output is the effect of only prior inputs can be relaxed. • Convolution amplifies or attenuates each frequency component of the input independently of the other components. In statistics, as noted above, a weighted moving average is a convolution. In probability theory, the probability distribution of the sum of two independent random variables is the convolution of their individual distributions. In optics, many kinds of "blur" are described by convolutions. A shadow (e.g., the shadow on the table when you hold your hand between the table and a light source) is the convolution of the shape of the light source that is casting the shadow and the object whose shadow is being cast. An out-of-focus photograph is the convolution of the sharp image with the shape of the iris diaphragm. The photographic term for this is bokeh. Similarly, in digital image processing, convolutional filtering plays an important role in many important algorithms in edge detection and related processes. In linear acoustics, an echo is the convolution of the original sound with a function representing the various objects that are reflecting it. In artificial reverberation (digital signal processing, pro audio), convolution is used to map the impulse response of a real room on a digital audio signal (see previous and next point for additional information). In time-resolved fluorescence spectroscopy, the excitation signal can be treated as a chain of delta pulses, and the measured fluorescence is a sum of exponential decays from each delta pulse. In radiotherapy treatment planning systems, most part of all modern codes of calculation applies a convolution-superposition algorithm. In physics, wherever there is a linear system with a "superposition principle", a convolution operation makes an appearance. In kernel density estimation, a distribution is estimated from sample points by convolution with a kernel, such as an isotropic Gaussian. (Diggle 1995). • In computational fluid dynamics, the large eddy simulation (LES) turbulence model uses the convolution operation to lower the range of length scales necessary in computation thereby reducing computational cost. 60 Convolution Notes [1] According to [Lothar von Wolfersdorf (2000), "Einige Klassen quadratischer Integralgleichungen", Sitzungsberichte der Sächsischen Akademie der Wissenschaften zu Leipzig, Mathematisch-naturwissenschaftliche Klasse, volume 128, number 2, 6–7], the source is Volterra, Vito (1913), "Leçons sur les fonctions de linges". Gauthier-Villars, Paris 1913. [2] John Hilton Grace and Alfred Young (1903). The algebra of invariants (http:/ / books. google. com/ books?id=NIe4AAAAIAAJ& pg=PA40). Cambridge University Press. p. 40. . [3] Leonard Eugene Dickson (1914). Algebraic invariants (http:/ / books. google. com/ books?id=LRGoAAAAIAAJ& pg=PA85). J. Wiley. p. 85. . [4] R. N. Bracewell (2005). "Early work on imaging theory in radio astronomy" (http:/ / books. google. com/ books?id=v2SqL0zCrwcC& pg=PA172). In W. T. Sullivan. The Early Years of Radio Astronomy: Reflections Fifty Years After Jansky's Discovery. Cambridge University Press. p. 172. ISBN 9780521616027. . References • Bracewell, R. (1986), The Fourier Transform and Its Applications (2nd ed.), McGraw–Hill, ISBN 0071160434. • Hewitt, Edwin; Ross, Kenneth A. (1979), Abstract harmonic analysis. Vol. I, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], 115 (2nd ed.), Berlin, New York: Springer-Verlag, ISBN 978-3-540-09434-0, MR551496. • Hewitt, Edwin; Ross, Kenneth A. (1970), Abstract harmonic analysis. Vol. II: Structure and analysis for compact groups. Analysis on locally compact Abelian groups, Die Grundlehren der mathematischen Wissenschaften, Band 152, Berlin, New York: Springer-Verlag, MR0262773. • Hörmander, L. (1983), The analysis of linear partial differential operators I, Grundl. Math. Wissenschaft., 256, Springer, ISBN 3-540-12104-8, MR0717035. • Kassel, Christian (1995), Quantum groups, Graduate Texts in Mathematics, 155, Berlin, New York: Springer-Verlag, ISBN 978-0-387-94370-1, MR1321145. • Knuth, Donald (1997), Seminumerical Algorithms (3rd. ed.), Reading, Massachusetts: Addison–Wesley, ISBN 0-201-89684-2. • Rudin, Walter (1962), Fourier analysis on groups, Interscience Tracts in Pure and Applied Mathematics, No. 12, Interscience Publishers (a division of John Wiley and Sons), New York–London, ISBN 047152364X, MR0152834. • Sobolev, V.I. (2001), "Convolution of functions" (http://www.encyclopediaofmath.org/index.php?title=C/ c026430), in Hazewinkel, Michiel, Encyclopedia of Mathematics, Springer, ISBN 978-1556080104. • Stein, Elias; Weiss, Guido (1971), Introduction to Fourier Analysis on Euclidean Spaces, Princeton University Press, ISBN 0-691-08078-X. • Strichartz, R. (1994), A Guide to Distribution Theory and Fourier Transforms, CRC Press, ISBN 0849382734. • Titchmarsh, E (1948), Introduction to the theory of Fourier integrals (2nd ed.), New York, N.Y.: Chelsea Pub. Co. (published 1986), ISBN 978-0828403245. • Uludag, A. M. (1998), "On possible deterioration of smoothness under the operation of convolution", J. Math. Anal. Appl. 227 no. 2, 335–358 • Treves, François (1967), Topological Vector Spaces, Distributions and Kernels, Academic Press, ISBN 0486453529. • von zur Gathen, J.; Gerhard, J. (2003), Modern Computer Algebra, Cambridge University Press, ISBN 0-521-82646-2. • Diggle, P. J. (1995), "A kernel method for smoothing point process data", Journal of the Royal Statistical Society, Series C) 34 (1985) 138–147 61 Convolution 62 External links • Earliest Uses: The entry on Convolution has some historical information. (http://jeff560.tripod.com/c.html) • Convolution (http://rkb.home.cern.ch/rkb/AN16pp/node38.html#SECTION000380000000000000000), on The Data Analysis BriefBook (http://rkb.home.cern.ch/rkb/titleA.html) • http://www.jhu.edu/~signals/convolve/index.html Visual convolution Java Applet. • http://www.jhu.edu/~signals/discreteconv2/index.html Visual convolution Java Applet for discrete-time functions. • Lectures on Image Processing: A collection of 18 lectures in pdf format from Vanderbilt University. Lecture 7 is on 2-D convolution. (http://www.archive.org/details/Lectures_on_Image_Processing), by Alan Peters. • • • • • http://www.vuse.vanderbilt.edu/~rap2/EECE253/EECE253_01_Intro.pdf Convolution Kernel Mask Operation Interactive tutorial (http://micro.magnet.fsu.edu/primer/java/ digitalimaging/processing/kernelmaskoperation/) Convolution (http://mathworld.wolfram.com/Convolution.html) at MathWorld Freeverb3 Impulse Response Processor (http://freeverb3.sourceforge.net/): Opensource zero latency impulse response processor with VST plugins Stanford University CS 178 interactive Flash demo (http://graphics.stanford.edu/courses/cs178/applets/ convolution.html) showing how spatial convolution works. Dynamical system A dynamical system is a concept in mathematics where a fixed rule describes the time dependence of a point in a geometrical space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water in a pipe, and the number of fish each springtime in a lake. At any given time a dynamical system has a state given by a set of real numbers (a vector) that can be represented by a point in an appropriate state space (a geometrical manifold). Small changes in the state of the system create small changes in the numbers. The evolution rule of the dynamical system is a fixed rule that describes what future states follow from the current state. The rule is deterministic; in other words, for a given time interval only one future state follows from the current state. Overview The Lorenz attractor is an example of a non-linear dynamical system. Studying this system helped give rise to Chaos theory. The concept of a dynamical system has its origins in Newtonian mechanics. There, as in other natural sciences and engineering disciplines, the evolution rule of dynamical systems is given implicitly by a relation that gives the state of the system only a short time into the future. (The relation is either a differential equation, difference equation or other time scale.) To determine the state for all future times requires iterating the relation many times—each advancing time a small step. The iteration procedure is referred to as solving the system or integrating the system. Once the system can be solved, given an initial point it is possible to determine all its future positions, a collection of points known as a trajectory or orbit. Before the advent of fast computing machines, solving a dynamical system required sophisticated mathematical techniques and could be accomplished only for a small class of dynamical systems. Numerical methods implemented Dynamical system on electronic computing machines have simplified the task of determining the orbits of a dynamical system. For simple dynamical systems, knowing the trajectory is often sufficient, but most dynamical systems are too complicated to be understood in terms of individual trajectories. The difficulties arise because: • The systems studied may only be known approximately—the parameters of the system may not be known precisely or terms may be missing from the equations. The approximations used bring into question the validity or relevance of numerical solutions. To address these questions several notions of stability have been introduced in the study of dynamical systems, such as Lyapunov stability or structural stability. The stability of the dynamical system implies that there is a class of models or initial conditions for which the trajectories would be equivalent. The operation for comparing orbits to establish their equivalence changes with the different notions of stability. • The type of trajectory may be more important than one particular trajectory. Some trajectories may be periodic, whereas others may wander through many different states of the system. Applications often require enumerating these classes or maintaining the system within one class. Classifying all possible trajectories has led to the qualitative study of dynamical systems, that is, properties that do not change under coordinate changes. Linear dynamical systems and systems that have two numbers describing a state are examples of dynamical systems where the possible classes of orbits are understood. • The behavior of trajectories as a function of a parameter may be what is needed for an application. As a parameter is varied, the dynamical systems may have bifurcation points where the qualitative behavior of the dynamical system changes. For example, it may go from having only periodic motions to apparently erratic behavior, as in the transition to turbulence of a fluid. • The trajectories of the system may appear erratic, as if random. In these cases it may be necessary to compute averages using one very long trajectory or many different trajectories. The averages are well defined for ergodic systems and a more detailed understanding has been worked out for hyperbolic systems. Understanding the probabilistic aspects of dynamical systems has helped establish the foundations of statistical mechanics and of chaos. It was in the work of Poincaré that these dynamical systems themes developed. Basic definitions A dynamical system is a manifold M called the phase (or state) space endowed with a family of smooth evolution functions Φ t that for any element of , the time, map a point of the phase space back into the phase space. The notion of smoothness changes with applications and the type of manifold. There are several choices for the set T. When T is taken to be the reals, the dynamical system is called a flow; and if T is restricted to the non-negative reals, then the dynamical system is a semi-flow. When T is taken to be the integers, it is a cascade or a map; and the restriction to the non-negative integers is a semi-cascade. Examples The evolution function Φ t is often the solution of a differential equation of motion The equation gives the time derivative, represented by the dot, of a trajectory x(t) on the phase space starting at some point x0. The vector field v(x) is a smooth function that at every point of the phase space M provides the velocity vector of the dynamical system at that point. (These vectors are not vectors in the phase space M, but in the tangent space TxM of the point x.) Given a smooth Φ t, an autonomous vector field can be derived from it. There is no need for higher order derivatives in the equation, nor for time dependence in v(x) because these can be eliminated by considering systems of higher dimensions. Other types of differential equations can be used to define the evolution rule: 63 Dynamical system is an example of an equation that arises from the modeling of mechanical systems with complicated constraints. The differential equations determining the evolution function Φ t are often ordinary differential equations: in this case the phase space M is a finite dimensional manifold. Many of the concepts in dynamical systems can be extended to infinite-dimensional manifolds—those that are locally Banach spaces—in which case the differential equations are partial differential equations. In the late 20th century the dynamical system perspective to partial differential equations started gaining popularity. Further examples • • • • • • • • • Logistic map Dyadic transformation Tent map Double pendulum Arnold's cat map Horseshoe map Baker's map is an example of a chaotic piecewise linear map Billiards and outer billiards Hénon map • • • • • • • • Lorenz system Circle map Rössler map Kaplan-Yorke map List of chaotic maps Swinging Atwood's machine Quadratic map simulation system Bouncing ball dynamics Linear dynamical systems Linear dynamical systems can be solved in terms of simple functions and the behavior of all orbits classified. In a linear system the phase space is the N-dimensional Euclidean space, so any point in phase space can be represented by a vector with N numbers. The analysis of linear systems is possible because they satisfy a superposition principle: if u(t) and w(t) satisfy the differential equation for the vector field (but not necessarily the initial condition), then so will u(t) + w(t). Flows For a flow, the vector field Φ(x) is a linear function of the position in the phase space, that is, with A a matrix, b a vector of numbers and x the position vector. The solution to this system can be found by using the superposition principle (linearity). The case b ≠ 0 with A = 0 is just a straight line in the direction of b: When b is zero and A ≠ 0 the origin is an equilibrium (or singular) point of the flow, that is, if x0 = 0, then the orbit remains there. For other initial conditions, the equation of motion is given by the exponential of a matrix: for an initial point x0, When b = 0, the eigenvalues of A determine the structure of the phase space. From the eigenvalues and the eigenvectors of A it is possible to determine if an initial point will converge or diverge to the equilibrium point at the 64 Dynamical system 65 origin. The distance between two different initial conditions in the case A ≠ 0 will change exponentially in most cases, either converging exponentially fast towards a point, or diverging exponentially fast. Linear systems display sensitive dependence on initial conditions in the case of divergence. For nonlinear systems this is one of the (necessary but not sufficient) conditions for chaotic behavior. Linear vector fields and a few trajectories. Maps A discrete-time, affine dynamical system has the form with A a matrix and b a vector. As in the continuous case, the change of coordinates x → x + (1 − A) –1b removes the term b from the equation. In the new coordinate system, the origin is a fixed point of the map and the solutions are of the linear system A nx0. The solutions for the map are no longer curves, but points that hop in the phase space. The orbits are organized in curves, or fibers, which are collections of points that map into themselves under the action of the map. As in the continuous case, the eigenvalues and eigenvectors of A determine the structure of phase space. For example, if u1 is an eigenvector of A, with a real eigenvalue smaller than one, then the straight lines given by the points along α u1, with α ∈ R, is an invariant curve of the map. Points in this straight line run into the fixed point. There are also many other discrete dynamical systems. Local dynamics The qualitative properties of dynamical systems do not change under a smooth change of coordinates (this is sometimes taken as a definition of qualitative): a singular point of the vector field (a point where v(x) = 0) will remain a singular point under smooth transformations; a periodic orbit is a loop in phase space and smooth deformations of the phase space cannot alter it being a loop. It is in the neighborhood of singular points and periodic orbits that the structure of a phase space of a dynamical system can be well understood. In the qualitative study of dynamical systems, the approach is to show that there is a change of coordinates (usually unspecified, but computable) that makes the dynamical system as simple as possible. Dynamical system Rectification A flow in most small patches of the phase space can be made very simple. If y is a point where the vector field v(y) ≠ 0, then there is a change of coordinates for a region around y where the vector field becomes a series of parallel vectors of the same magnitude. This is known as the rectification theorem. The rectification theorem says that away from singular points the dynamics of a point in a small patch is a straight line. The patch can sometimes be enlarged by stitching several patches together, and when this works out in the whole phase space M the dynamical system is integrable. In most cases the patch cannot be extended to the entire phase space. There may be singular points in the vector field (where v(x) = 0); or the patches may become smaller and smaller as some point is approached. The more subtle reason is a global constraint, where the trajectory starts out in a patch, and after visiting a series of other patches comes back to the original one. If the next time the orbit loops around phase space in a different way, then it is impossible to rectify the vector field in the whole series of patches. Near periodic orbits In general, in the neighborhood of a periodic orbit the rectification theorem cannot be used. Poincaré developed an approach that transforms the analysis near a periodic orbit to the analysis of a map. Pick a point x0 in the orbit γ and consider the points in phase space in that neighborhood that are perpendicular to v(x0). These points are a Poincaré section S(γ, x0), of the orbit. The flow now defines a map, the Poincaré map F : S → S, for points starting in S and returning to S. Not all these points will take the same amount of time to come back, but the times will be close to the time it takes x0. The intersection of the periodic orbit with the Poincaré section is a fixed point of the Poincaré map F. By a translation, the point can be assumed to be at x = 0. The Taylor series of the map is F(x) = J · x + O(x2), so a change of coordinates h can only be expected to simplify F to its linear part This is known as the conjugation equation. Finding conditions for this equation to hold has been one of the major tasks of research in dynamical systems. Poincaré first approached it assuming all functions to be analytic and in the process discovered the non-resonant condition. If λ1, ..., λν are the eigenvalues of J they will be resonant if one eigenvalue is an integer linear combination of two or more of the others. As terms of the form λi – ∑ (multiples of other eigenvalues) occurs in the denominator of the terms for the function h, the non-resonant condition is also known as the small divisor problem. Conjugation results The results on the existence of a solution to the conjugation equation depend on the eigenvalues of J and the degree of smoothness required from h. As J does not need to have any special symmetries, its eigenvalues will typically be complex numbers. When the eigenvalues of J are not in the unit circle, the dynamics near the fixed point x0 of F is called hyperbolic and when the eigenvalues are on the unit circle and complex, the dynamics is called elliptic. In the hyperbolic case the Hartman–Grobman theorem gives the conditions for the existence of a continuous function that maps the neighborhood of the fixed point of the map to the linear map J · x. The hyperbolic case is also structurally stable. Small changes in the vector field will only produce small changes in the Poincaré map and these small changes will reflect in small changes in the position of the eigenvalues of J in the complex plane, implying that the map is still hyperbolic. The Kolmogorov–Arnold–Moser (KAM) theorem gives the behavior near an elliptic point. 66 Dynamical system Bifurcation theory When the evolution map Φt (or the vector field it is derived from) depends on a parameter μ, the structure of the phase space will also depend on this parameter. Small changes may produce no qualitative changes in the phase space until a special value μ0 is reached. At this point the phase space changes qualitatively and the dynamical system is said to have gone through a bifurcation. Bifurcation theory considers a structure in phase space (typically a fixed point, a periodic orbit, or an invariant torus) and studies its behavior as a function of the parameter μ. At the bifurcation point the structure may change its stability, split into new structures, or merge with other structures. By using Taylor series approximations of the maps and an understanding of the differences that may be eliminated by a change of coordinates, it is possible to catalog the bifurcations of dynamical systems. The bifurcations of a hyperbolic fixed point x0 of a system family Fμ can be characterized by the eigenvalues of the first derivative of the system DFμ(x0) computed at the bifurcation point. For a map, the bifurcation will occur when there are eigenvalues of DFμ on the unit circle. For a flow, it will occur when there are eigenvalues on the imaginary axis. For more information, see the main article on Bifurcation theory. Some bifurcations can lead to very complicated structures in phase space. For example, the Ruelle–Takens scenario describes how a periodic orbit bifurcates into a torus and the torus into a strange attractor. In another example, Feigenbaum period-doubling describes how a stable periodic orbit goes through a series of period-doubling bifurcations. Ergodic systems In many dynamical systems it is possible to choose the coordinates of the system so that the volume (really a ν-dimensional volume) in phase space is invariant. This happens for mechanical systems derived from Newton's laws as long as the coordinates are the position and the momentum and the volume is measured in units of (position) × (momentum). The flow takes points of a subset A into the points Φ t(A) and invariance of the phase space means that In the Hamiltonian formalism, given a coordinate it is possible to derive the appropriate (generalized) momentum such that the associated volume is preserved by the flow. The volume is said to be computed by the Liouville measure. In a Hamiltonian system not all possible configurations of position and momentum can be reached from an initial condition. Because of energy conservation, only the states with the same energy as the initial condition are accessible. The states with the same energy form an energy shell Ω, a sub-manifold of the phase space. The volume of the energy shell, computed using the Liouville measure, is preserved under evolution. For systems where the volume is preserved by the flow, Poincaré discovered the recurrence theorem: Assume the phase space has a finite Liouville volume and let F be a phase space volume-preserving map and A a subset of the phase space. Then almost every point of A returns to A infinitely often. The Poincaré recurrence theorem was used by Zermelo to object to Boltzmann's derivation of the increase in entropy in a dynamical system of colliding atoms. One of the questions raised by Boltzmann's work was the possible equality between time averages and space averages, what he called the ergodic hypothesis. The hypothesis states that the length of time a typical trajectory spends in a region A is vol(A)/vol(Ω). The ergodic hypothesis turned out not to be the essential property needed for the development of statistical mechanics and a series of other ergodic-like properties were introduced to capture the relevant aspects of physical systems. Koopman approached the study of ergodic systems by the use of functional analysis. An observable a is a function that to each point of the phase space associates a number (say instantaneous pressure, or average height). 67 Dynamical system 68 The value of an observable can be computed at another time by using the evolution function φ t. This introduces an operator U t, the transfer operator, By studying the spectral properties of the linear operator U it becomes possible to classify the ergodic properties of Φ t. In using the Koopman approach of considering the action of the flow on an observable function, the finite-dimensional nonlinear problem involving Φ t gets mapped into an infinite-dimensional linear problem involving U. The Liouville measure restricted to the energy surface Ω is the basis for the averages computed in equilibrium statistical mechanics. An average in time along a trajectory is equivalent to an average in space computed with the Boltzmann factor exp(−βH). This idea has been generalized by Sinai, Bowen, and Ruelle (SRB) to a larger class of dynamical systems that includes dissipative systems. SRB measures replace the Boltzmann factor and they are defined on attractors of chaotic systems. Nonlinear dynamical systems and chaos Simple nonlinear dynamical systems and even piecewise linear systems can exhibit a completely unpredictable behavior, which might seem to be random, despite the fact that they are fundamentally deterministic. This seemingly unpredictable behavior has been called chaos. Hyperbolic systems are precisely defined dynamical systems that exhibit the properties ascribed to chaotic systems. In hyperbolic systems the tangent space perpendicular to a trajectory can be well separated into two parts: one with the points that converge towards the orbit (the stable manifold) and another of the points that diverge from the orbit (the unstable manifold). This branch of mathematics deals with the long-term qualitative behavior of dynamical systems. Here, the focus is not on finding precise solutions to the equations defining the dynamical system (which is often hopeless), but rather to answer questions like "Will the system settle down to a steady state in the long term, and if so, what are the possible attractors?" or "Does the long-term behavior of the system depend on its initial condition?" Note that the chaotic behavior of complex systems is not the issue. Meteorology has been known for years to involve complex—even chaotic—behavior. Chaos theory has been so surprising because chaos can be found within almost trivial systems. The logistic map is only a second-degree polynomial; the horseshoe map is piecewise linear. Geometrical definition A dynamical system is the tuple , with a manifold (locally a Banach space or Euclidean space), the domain for time (non-negative reals, the integers, ...) and an evolution rule t → f t (with t f is a diffeomorphism of the manifold to itself. So, f is a mapping of the time-domain ) such that into the space of diffeomorphisms of the manifold to itself. In other terms, f(t) is a diffeomorphism, for every time t in the domain . Measure theoretical definition See main article Measure-preserving dynamical system. A dynamical system may be defined formally, as a measure-preserving transformation of a sigma-algebra, the quadruplet . Here, X is a set, and Σ is a sigma-algebra on X, so that the pair is a measurable space. μ is a finite measure on the sigma-algebra, so that the triplet is said to be Σ-measurable if and only if, for every to preserve the measure if and only if, for every map , one has is a probability space. A map , one has . A map is said . Combining the above, a is said to be a measure-preserving transformation of X , if it is a map from X to itself, it is Σ-measurable, and is measure-preserving. The quadruple , for such a , is then defined to be a dynamical system. The map embodies the time evolution of the dynamical system. Thus, for discrete dynamical systems the iterates for integer n are studied. For continuous dynamical systems, the map is understood to be Dynamical system a finite time evolution map and the construction is more complicated. Examples of dynamical systems Internal links • • • • • • • • • • • • Arnold's cat map Baker's map is an example of a chaotic piecewise linear map Circle map Double pendulum Billiards and Outer Billiards Hénon map Horseshoe map Irrational rotation List of chaotic maps Logistic map Lorenz system Rossler map External links • Interactive applet for the Standard and Henon Maps [1] by A. Luhn Multidimensional generalization Dynamical systems are defined over a single independent variable, usually thought of as time. A more general class of systems are defined over multiple independent variables and are therefore called multidimensional systems. Such systems are useful for modeling, for example, image processing. References [1] http:/ / complexity. xozzox. de/ nonlinmappings. html Further reading Works providing a broad coverage: • Ralph Abraham and Jerrold E. Marsden (1978). Foundations of mechanics. Benjamin–Cummings. ISBN 0-8053-0102-X. (available as a reprint: ISBN 0-201-40840-6) • Encyclopaedia of Mathematical Sciences (ISSN 0938-0396) has a sub-series on dynamical systems with reviews of current research. • Anatole Katok and Boris Hasselblatt (1996). Introduction to the modern theory of dynamical systems. Cambridge. ISBN 0-521-57557-5. • Christian Bonatti, Lorenzo J. Díaz, Marcelo Viana (2005). Dynamics Beyond Uniform Hyperbolicity: A Global Geometric and Probabilistic Perspective. Springer. ISBN 3-540-22066-6. • Stephen Smale (1967). "Differential dynamical systems". Bulletin of the American Mathematical Society 73 (6): 747–817. doi:10.1090/S0002-9904-1967-11798-1. Introductory texts with a unique perspective: • V. I. Arnold (1982). Mathematical methods of classical mechanics. Springer-Verlag. ISBN 0-387-96890-3. • Jacob Palis and Wellington de Melo (1982). Geometric theory of dynamical systems: an introduction. Springer-Verlag. ISBN 0-387-90668-1. 69 Dynamical system • David Ruelle (1989). Elements of Differentiable Dynamics and Bifurcation Theory. Academic Press. ISBN 0-12-601710-7. • Tim Bedford, Michael Keane and Caroline Series, eds. (1991). Ergodic theory, symbolic dynamics and hyperbolic spaces. Oxford University Press. ISBN 0-19-853390-X. • Ralph H. Abraham and Christopher D. Shaw (1992). Dynamics—the geometry of behavior, 2nd edition. Addison-Wesley. ISBN 0-201-56716-4. Textbooks • James Meiss (2007). Differential Dynamical Systems. SIAM. ISBN 0898716357. • Steven H. Strogatz (1994). Nonlinear dynamics and chaos: with applications to physics, biology chemistry and engineering. Addison Wesley. ISBN 0-201-54344-3. • Kathleen T. Alligood, Tim D. Sauer and James A. Yorke (2000). Chaos. An introduction to dynamical systems. Springer Verlag. ISBN 0-387-94677-2. • Morris W. Hirsch, Stephen Smale and Robert Devaney (2003). Differential Equations, dynamical systems, and an introduction to chaos. Academic Press. ISBN 0-12-349703-5. • Julien Clinton Sprott (2003). Chaos and time-series analysis. Oxford University Press. ISBN 0-19-850839-5. • Oded Galor (2011). Discrete Dynamical Systems. Springer. ISBN 978-3-642-07185-0. Popularizations: • Florin Diacu and Philip Holmes (1996). Celestial Encounters. Princeton. ISBN 0-691-02743-9. • James Gleick (1988). Chaos: Making a New Science. Penguin. ISBN 0-14-009250-1. • Ivar Ekeland (1990). Mathematics and the Unexpected (Paperback). University Of Chicago Press. ISBN 0-226-19990-8. • Ian Stewart (1997). Does God Play Dice? The New Mathematics of Chaos. Penguin. ISBN 0140256024. External links • A collection of dynamic and non-linear system models and demo applets (http://vlab.infotech.monash.edu.au/ simulations/non-linear/) (in Monash University's Virtual Lab) • Arxiv preprint server (http://www.arxiv.org/list/math.DS/recent) has daily submissions of (non-refereed) manuscripts in dynamical systems. • DSWeb (http://www.dynamicalsystems.org/) provides up-to-date information on dynamical systems and its applications. • Encyclopedia of dynamical systems (http://www.scholarpedia.org/article/ Encyclopedia_of_Dynamical_Systems) A part of Scholarpedia — peer reviewed and written by invited experts. • Nonlinear Dynamics (http://www.egwald.ca/nonlineardynamics/index.php). Models of bifurcation and chaos by Elmer G. Wiens • Oliver Knill (http://www.dynamical-systems.org) has a series of examples of dynamical systems with explanations and interactive controls. • Sci.Nonlinear FAQ 2.0 (Sept 2003) (http://amath.colorado.edu/faculty/jdm/faq-Contents.html) provides definitions, explanations and resources related to nonlinear science Online books or lecture notes: • Geometrical theory of dynamical systems (http://arxiv.org/pdf/math.HO/0111177). Nils Berglund's lecture notes for a course at ETH at the advanced undergraduate level. • Dynamical systems (http://www.ams.org/online_bks/coll9/). George D. Birkhoff's 1927 book already takes a modern approach to dynamical systems. • Chaos: classical and quantum (http://chaosbook.org). An introduction to dynamical systems from the periodic orbit point of view. 70 Dynamical system • Modeling Dynamic Systems (http://www.embedded.com/2000/0008/0008feat2.htm). An introduction to the development of mathematical models of dynamic systems. • Learning Dynamical Systems (http://www.cs.brown.edu/research/ai/dynamics/tutorial/home.html). Tutorial on learning dynamical systems. • Ordinary Differential Equations and Dynamical Systems (http://www.mat.univie.ac.at/~gerald/ftp/book-ode/ ). Lecture notes by Gerald Teschl Research groups: • Dynamical Systems Group Groningen (http://www.math.rug.nl/~broer/), IWI, University of Groningen. • Chaos @ UMD (http://www-chaos.umd.edu/). Concentrates on the applications of dynamical systems. • Dynamical Systems (http://www.math.sunysb.edu/dynamics/), SUNY Stony Brook. Lists of conferences, researchers, and some open problems. • Center for Dynamics and Geometry (http://www.math.psu.edu/dynsys/), Penn State. • Control and Dynamical Systems (http://www.cds.caltech.edu/), Caltech. • Laboratory of Nonlinear Systems (http://lanoswww.epfl.ch/), Ecole Polytechnique Fédérale de Lausanne (EPFL). • Center for Dynamical Systems (http://www.math.uni-bremen.de/ids.html/), University of Bremen • Systems Analysis, Modelling and Prediction Group (http://www.eng.ox.ac.uk/samp/), University of Oxford • Non-Linear Dynamics Group (http://sd.ist.utl.pt/), Instituto Superior Técnico, Technical University of Lisbon • Dynamical Systems (http://www.impa.br/), IMPA, Instituto Nacional de Matemática Pura e Applicada. • Nonlinear Dynamics Workgroup (http://ndw.cs.cas.cz/), Institute of Computer Science, Czech Academy of Sciences. Simulation software based on Dynamical Systems approach: • FyDiK (http://fydik.kitnarf.cz/) • iDMC (http://idmc.googlecode.com), simulation and dynamical analysis of nonlinear models 71 Graph dynamical system 72 Graph dynamical system In mathematics, the concept of graph dynamical systems can be used to capture a wide range of processes taking place on graphs or networks. A major theme in the mathematical and computational analysis of GDSs is to relate their structural properties (e.g. the network connectivity) and the global dynamics that result. The work on GDSs considers finite graphs and finite state spaces. As such, the research typically involves techniques from, e.g., graph theory, combinatorics, algebra, and dynamical systems rather than differential geometry. In principle, one could define and study GDSs over an infinite graph (e.g. cellular automata over or interacting particle systems), as well as GDSs with infinite state space (e.g. as in coupled map lattices); see, e.g., Wu.[1] In the following everything is implicitly assumed to be finite unless stated otherwise. Formal definition A graph dynamical system is constructed from the following components: • A finite graph Y with vertex set v[Y] = {1,2, ... , n}. Depending on the context the graph can be directed or undirected. • A state xv for each vertex v of Y taken from a finite set K. The system state is the n-tuple x = (x1, x2, ... , xn), and x[v] is the tuple consisting of the states associated to the vertices in the 1-neighborhood of v in Y (in some fixed order). • A vertex function fv for each vertex v. The vertex function maps the state of vertex v at time t to the vertex state at time t + 1 based on the states associated to the 1-neighborhood of v in Y. • An update scheme specifying the mechanism by which the mapping of individual vertex states is carried out so as to induce a discrete dynamical system with map F: Kn → Kn. The phase space associated to a dynamical system with map F: Kn → Kn is the finite directed graph with vertex set Kn and directed edges (x, F(x)). The structure of the phase space is governed by the properties of the graph Y, the vertex functions (fi)i, and the update scheme. The research in this area seeks to infer phase space properties based on the structure of the system constituents. The analysis has a local-to-global character. Generalized cellular automata (GCA) If, for example, the update scheme consists of applying the vertex functions synchronously one obtains the class of generalized cellular automata (CA). In this case, the global map F: Kn → Kn is given by This class is referred to as generalized cellular automata since the classical or standard cellular automata are typically defined and studied over regular graphs or grids, and the vertex functions are typically assumed to be identical. Example: Let Y be the circle graph on vertices {1,2,3,4} with edges {1,2}, {2,3}, {3,4} and {1,4}, denoted Circ4. Let K = {0,1} be the state space for each vertex and use the function nor3 : K3 → K defined by nor3(x,y,z) = (1 + x)(1 + y)(1 + z) with arithmetic modulo 2 for all vertex functions. Then for example the system state (0,1,0,0) is mapped to (0, 0, 0, 1) using a synchronous update. All the transitions are shown in the phase space below. Graph dynamical system Sequential dynamical systems (SDS) If the vertex functions are applied asynchronously in the sequence specified by a word w = (w1, w2, ... , wm) or permutation = ( , ) of v[Y] one obtains the class of Sequential dynamical systems (SDS).[2] In this case it is convenient to introduce the Y-local maps Fi constructed from the vertex functions by The SDS map F = [FY , w] : Kn → Kn is the function composition If the update sequence is a permutation one frequently speaks of a permutation SDS to emphasize this point. Example: Let Y be the circle graph on vertices {1,2,3,4} with edges {1,2}, {2,3}, {3,4} and {1,4}, denoted Circ4. Let K={0,1} be the state space for each vertex and use the function nor3 : K3 → K defined by nor3(x, y, z) = (1 + x)(1 + y)(1 + z) with arithmetic modulo 2 for all vertex functions. Using the update sequence (1,2,3,4) then the system state (0, 1, 0, 0) is mapped to (0, 0, 1, 0). All the system state transitions for this sequential dynamical system are shown in the phase space below. Stochastic graph dynamical systems From, e.g., the point of view of applications it is interesting to consider the case where one or more of the components of a GDS contains stochastic elements. Motivating applications could include processes that are not fully understood (e.g. dynamics within a cell) and where certain aspects for all practical purposes seem to behave according to some probability distribution. There are also applications governed by deterministic principles whose description is so complex or unwieldy that it makes sense to consider probabilistic approximations. Every element of a graph dynamical system can be made stochastic in several ways. For example, in a sequential dynamical system the update sequence can be made stochastic. At each iteration step one may choose the update sequence w at random from a given distribution of update sequences with corresponding probabilities. The matching probability space of update sequences induces a probability space of SDS maps. A natural object to study in this regard is the Markov chain on state space induced by this collection of SDS maps. This case is referred to as update 73 Graph dynamical system sequence stochastic GDS and is motivated by, e.g., processes where "events" occur at random according to certain rates (e.g. chemical reactions), synchronization in parallel computation/discrete event simulations, and in computational paradigms described later. This specific example with stochastic update sequence illustrates two general facts for such systems: when passing to a stochastic graph dynamical system one is generally led to (1) a study of Markov chains (with specific structure governed by the constituents of the GDS), and (2) the resulting Markov chains tend to be large having an exponential number of states. A central goal in the study of stochastic GDS is to be able to derive reduced models. One may also consider the case where the vertex functions are stochastic, i.e., function stochastic GDS. For example, Random Boolean networks are examples of function stochastic GDS using a synchronous update scheme and where the state space is K = {0, 1}. Finite probabilistic cellular automata (PCA) is another example of function stochastic GDS. In principle the class of Interacting particle systems (IPS) covers finite and infinite PCA, but in practice the work on IPS is largely concerned with the infinite case since this allows one to introduce more interesting topologies on state space. Applications Graph dynamical systems constitute a natural framework for capturing distributed systems such as biological networks and epidemics over social networks, many of which are frequently referred to as complex systems. References [1] Wu, Chai Wah (2005). "Synchronization in networks of nonlinear dynamical systems coupled via a directed graph". Nonlinearity 18 (3): 1057–1064. doi:10.1088/0951-7715/18/3/007. [2] Mortveit, Henning S.; Reidys, Christian M. (2007). An introduction to sequential dynamical systems. Universitext. New York: Springer Verlag. ISBN 978-0-387-30654-4. External links • Graph Dynamical Systems – A Mathematical Framework for Interaction-Based Systems, Their Analysis and Simulations by Henning Mortveit (http://legacy.samsi.info/200809/algebraic/presentations/discrete/friday/ samsi-05-dec-08.pdf) Further reading • Macauley, Matthew; Mortveit, Henning S. (2009). "Cycle equivalence of graph dynamical systems". Nonlinearity 22 (2): 421–436. doi:10.1088/0951-7715/22/2/010. • Golubitsky, Martin; Stewart, Ian (2003). The Symmetry Perspective. Basel: Birkhauser. ISBN 0817621717. 74 75 Dynamic response First-order logic First-order logic is a formal system used in mathematics, philosophy, linguistics, and computer science. Over the past 100-odd years, what is now called first-order logic has gone by many names, including: first-order predicate calculus, the lower predicate calculus, quantification theory, and predicate logic (a less precise term). First-order logic is distinguished from propositional logic by its use of quantified variables. First-order logic with a specified domain of discourse over which the quantified variables range, one or more interpreted predicate letters, and proper axioms involving the interpreted predicate letters, is a first-order theory. The adjective "first-order" distinguishes first-order logic from higher-order logic in which there are predicates having predicates or functions as arguments, or in which one or both of predicate quantifiers or function quantifiers are permitted.[1] In first-order theories, predicates are often associated with sets. In interpreted higher-order theories, predicates may be interpreted as sets of sets. There are many deductive systems for first-order logic that are sound (all provable statements are true) and complete (all true statements are provable). Although the logical consequence relation is only semidecidable, much progress has been made in automated theorem proving in first-order logic. First-order logic also satisfies several metalogical theorems that make it amenable to analysis in proof theory, such as the Löwenheim–Skolem theorem and the compactness theorem. First-order logic is of great importance to the foundations of mathematics, because it is the standard formal logic for axiomatic systems. Many common axiomatic systems, such as first-order Peano arithmetic and axiomatic set theory, including the canonical Zermelo–Fraenkel set theory (ZF), can be formalized as first-order theories. No first-order theory, however, has the strength to describe fully and categorically structures with an infinite domain, such as the natural numbers or the real line. Categorical axiom systems for these structures can be obtained in stronger logics such as second-order logic. For a history of first-order logic and how it came to be the dominant formal logic, see Ferreirós (2001). Introduction While propositional logic deals with simple declarative propositions, first-order logic additionally covers predicates and quantification. A predicate resembles a function that returns either True or False. Consider the following sentences: "Socrates is a philosopher", "Plato is a philosopher". In propositional logic these are treated as two unrelated propositions, denoted for example by p and q. In first-order logic, however, the sentences can be expressed in a more parallel manner using the predicate Phil(a), which asserts that the object represented by a is a philosopher. Thus if a represents Socrates then Phil(a) asserts the first proposition, p; if a instead represents Plato then Phil(a) asserts the second proposition, q. A key aspect of first-order logic is visible here: the string "Phil" is a syntactic entity which is given semantic meaning by declaring that Phil(a) holds exactly when a is a philosopher. An assignment of semantic meaning is called an interpretation. First-order logic allows reasoning about properties that are shared by many objects, through the use of variables. For example, let Phil(a) assert that a is a philosopher and let Schol(a) assert that a is a scholar. Then the formula First-order logic 76 asserts that if a is a philosopher then a is a scholar. The symbol is used to denote a conditional (if/then) statement. The hypothesis lies to the left of the arrow and the conclusion to the right. The truth of this formula depends on which object is denoted by a, and on the interpretations of "Phil" and "Schol". Assertions of the form "for every a, if a is a philosopher then a is a scholar" require both the use of variables and the use of a quantifier. Again, let Phil(a) assert a is a philosopher and let Schol(a) assert that a is a scholar. Then the first-order sentence asserts that no matter what a represents, if a is a philosopher then a is scholar. Here , the universal quantifier, expresses the idea that the claim in parentheses holds for all choices of a. To show that the claim "If a is a philosopher then a is a scholar" is false, one would show there is some philosopher who is not a scholar. This counterclaim can be expressed with the existential quantifier : Here: • is the negation operator: • not a scholar. is the conjunction operator: is true if and only if is false, in other words if and only if a is asserts that a is a philosopher and also not a scholar. The predicates Phil(a) and Schol(a) take only one parameter each. First-order logic can also express predicates with more than one parameter. For example, "there is someone who can be fooled every time" can be expressed as: Here Person(x) is interpreted to mean x is a person, Time(y) to mean that y is a moment of time, and Canfool(x,y) to mean that (person) x can be fooled at (time) y. For clarity, this statement asserts that there is at least one person who can be fooled at all times, which is stronger than asserting that at all times at least one person exists who can be fooled. Asserting the latter (that there is always at least one foolable person) does not signify whether this foolable person is always the same for all moments of time. The range of the quantifiers is the set of objects that can be used to satisfy them. (In the informal examples in this section, the range of the quantifiers was left unspecified.) In addition to specifying the meaning of predicate symbols such as Person and Time, an interpretation must specify a nonempty set, known as the domain of discourse or universe, as a range for the quantifiers. Thus a statement of the form is said to be true, under a particular interpretation, if there is some object in the domain of discourse of that interpretation that satisfies the predicate that the interpretation uses to assign meaning to the symbol Phil. Syntax There are two key parts of first order logic. The syntax determines which collections of symbols are legal expressions in first-order logic, while the semantics determine the meanings behind these expressions. Alphabet Unlike natural languages, such as English, the language of first-order logic is completely formal, so that it can be mechanically determined whether a given expression is legal. There are two key types of legal expressions: terms, which intuitively represent objects, and formulas, which intuitively express predicates that can be true or false. The terms and formulas of first-order logic are strings of symbols which together form the alphabet of the language. As with all formal languages, the nature of the symbols themselves is outside the scope of formal logic; they are often regarded simply as letters and punctuation symbols. It is common to divide the symbols of the alphabet into logical symbols, which always have the same meaning, and non-logical symbols, whose meaning varies by interpretation. For example, the logical symbol always represents First-order logic 77 "and"; it is never interpreted as "or". On the other hand, a non-logical predicate symbol such as Phil(x) could be interpreted to mean "x is a philosopher", "x is a man named Philip", or any other unary predicate, depending on the interpretation at hand. Logical symbols There are several logical symbols in the alphabet, which vary by author but usually include: • The quantifier symbols and • The logical connectives: for conjunction, for disjunction, for implication, for biconditional, for negation. Occasionally other logical connective symbols are included. Some authors use , or Cpq, instead of , and , or Epq, instead of , especially in contexts where is used for other purposes. Moreover, the horseshoe may replace ; the triple-bar may replace , and a tilde (~), Np, or Fpq, may replace ; ||, or Apq may replace ; and &, or Kpq, may replace , especially if these symbols are not available for technical reasons. • Parentheses, brackets, and other punctuation symbols. The choice of such symbols varies depending on context. • An infinite set of variables, often denoted by lowercase letters at the end of the alphabet x, y, z, … . Subscripts are often used to distinguish variables: x0, x1, x2, … . • An equality symbol (sometimes, identity symbol) =; see the section on equality below. It should be noted that not all of these symbols are required - only one of the quantifiers, negation and conjunction, variables, brackets and equality suffice. There are numerous minor variations that may define additional logical symbols: • Sometimes the truth constants T, Vpq, or , for "true" and F, Opq, or , for "false" are included. Without any such logical operators of valence 0, these two constants can only be expressed using quantifiers. • Sometimes additional logical connectives are included, such as the Sheffer stroke, Dpq (NAND), and exclusive or, Jpq. Non-logical symbols The non-logical symbols represent predicates (relations), functions and constants on the domain of discourse. It used to be standard practice to use a fixed, infinite set of non-logical symbols for all purposes. A more recent practice is to use different non-logical symbols according to the application one has in mind. Therefore it has become necessary to name the set of all non-logical symbols used in a particular application. This choice is made via a signature.[2] The traditional approach is to have only one, infinite, set of non-logical symbols (one signature) for all applications. Consequently, under the traditional approach there is only one language of first-order logic.[3] This approach is still common, especially in philosophically oriented books. 1. For every integer n ≥ 0 there is a collection of n-ary, or n-place, predicate symbols. Because they represent relations between n elements, they are also called relation symbols. For each arity n we have an infinite supply of them: Pn0, Pn1, Pn2, Pn3, … 2. For every integer n ≥ 0 there are infinitely many n-ary function symbols: f n0, f n1, f n2, f n3, … In contemporary mathematical logic, the signature varies by application. Typical signatures in mathematics are {1, ×} or just {×} for groups, or {0, 1, +, ×, <} for ordered fields. There are no restrictions on the number of non-logical symbols. The signature can be empty, finite, or infinite, even uncountable. Uncountable signatures occur for example in modern proofs of the Löwenheim-Skolem theorem. In this approach, every non-logical symbol is of one of the following types. First-order logic 78 1. A predicate symbol (or relation symbol) with some valence (or arity, number of arguments) greater than or equal to 0. These which are often denoted by uppercase letters P, Q, R,... . • Relations of valence 0 can be identified with propositional variables. For example, P, which can stand for any statement. • For example, P(x) is a predicate variable of valence 1. One possible interpretation is "x is a man". • Q(x,y) is a predicate variable of valence 2. Possible interpretations include "x is greater than y" and "x is the father of y". 2. A function symbol, with some valence greater than or equal to 0. These are often denoted by lowercase letters f, g, h,... . • Examples: f(x) may be interpreted as for "the father of x". In arithmetic, it may stand for "-x". In set theory, it may stand for "the power set of x". In arithmetic, g(x,y) may stand for "x+y". In set theory, it may stand for "the union of x and y". • Function symbols of valence 0 are called constant symbols, and are often denoted by lowercase letters at the beginning of the alphabet a, b, c,... . The symbol a may stand for Socrates. In arithmetic, it may stand for 0. In set theory, such a constant may stand for the empty set. The traditional approach can be recovered in the modern approach by simply specifying the "custom" signature to consist of the traditional sequences of non-logical symbols. Formation rules The formation rules define the terms and formulas of first order logic. When terms and formulas are represented as strings of symbols, these rules can be used to write a formal grammar for terms and formulas. These rules are generally context-free (each production has a single symbol on the left side), except that the set of symbols may be allowed to be infinite and there may be many start symbols, for example the variables in the case of terms. Terms The set of terms is inductively defined by the following rules: 1. Variables. Any variable is a term. 2. Functions. Any expression f(t1,...,tn) of n arguments (where each argument ti is a term and f is a function symbol of valence n) is a term. In particular, symbols denoting individual constants are 0-ary function symbols, and are thus terms. Only expressions which can be obtained by finitely many applications of rules 1 and 2 are terms. For example, no expression involving a predicate symbol is a term. Formulas The set of formulas (also called well-formed formulas[4] or wffs) is inductively defined by the following rules: 1. 2. 3. 4. Predicate symbols. If P is an n-ary predicate symbol and t1, ..., tn are terms then P(t1,...,tn) is a formula. Equality. If the equality symbol is considered part of logic, and t1 and t2 are terms, then t1 = t2 is a formula. Negation. If φ is a formula, then φ is a formula. Binary connectives. If φ and ψ are formulas, then (φ ψ) is a formula. Similar rules apply to other binary logical connectives. 5. Quantifiers. If φ is a formula and x is a variable, then and are formulas. Only expressions which can be obtained by finitely many applications of rules 1–5 are formulas. The formulas obtained from the first two rules are said to be atomic formulas. For example, First-order logic 79 is a formula, if f is a unary function symbol, P a unary predicate symbol, and Q a ternary predicate symbol. On the other hand, is not a formula, although it is a string of symbols from the alphabet. The role of the parentheses in the definition is to ensure that any formula can only be obtained in one way by following the inductive definition (in other words, there is a unique parse tree for each formula). This property is known as unique readability of formulas. There are many conventions for where parentheses are used in formulas. For example, some authors use colons or full stops instead of parentheses, or change the places in which parentheses are inserted. Each author's particular definition must be accompanied by a proof of unique readability. This definition of a formula does not support defining an if-then-else function ite(c,a,b) where "c" is a condition expressed as a formula, that would return "a" if c is true, and "b" if it is false. This is because both predicates and functions can only accept terms as parameters, but the first parameter is a formula. Some languages built on first-order logic, such as SMT-LIB 2.0, add this. [5] Notational conventions For convenience, conventions have been developed about the precedence of the logical operators, to avoid the need to write parentheses in some cases. These rules are similar to the order of operations in arithmetic. A common convention is: • is evaluated first • and are evaluated next • Quantifiers are evaluated next • is evaluated last. Moreover, extra punctuation not required by the definition may be inserted to make formulas easier to read. Thus the formula might be written as In some fields, it is common to use infix notation for binary relations and functions, instead of the prefix notation defined above. For example, in arithmetic, one typically writes "2 + 2 = 4" instead of "=(+(2,2),4)". It is common to regard formulas in infix notation as abbreviations for the corresponding formulas in prefix notation. The definitions above use infix notation for binary connectives such as . A less common convention is Polish notation, in which one writes , , and so on in front of their arguments rather than between them. This convention allows all punctuation symbols to be discarded. Polish notation is compact and elegant, but rarely used in practice because it is hard for humans to read it. In Polish notation, the formula becomes "∀x∀y→Pfx¬→ PxQfyxz". Example In mathematics the language of ordered abelian groups has one constant symbol 0, one unary function symbol −, one binary function symbol +, and one binary relation symbol ≤. Then: • The expressions +(x, y) and +(x, +(y, −(z))) are terms. These are usually written as x + y and x + y − z. • The expressions +(x, y) = 0 and ≤(+(x, +(y, −(z))), +(x, y)) are atomic formulas. These are usually written as x + y = 0 and x + y − z ≤ x + y. • The expression is a formula, which is usually written as First-order logic 80 Free and bound variables In a formula, a variable may occur free or bound. Intuitively, a variable is free in a formula if it is not quantified: in , variable x is free while y is bound. The free and bound variables of a formula are defined inductively as follows. 1. Atomic formulas. If φ is an atomic formula then x is free in φ if and only if x occurs in φ. Moreover, there are no bound variables in any atomic formula. 2. Negation. x is free in φ if and only if x is free in φ. x is bound in φ if and only if x is bound in φ. 3. Binary connectives. x is free in (φ ψ) if and only if x is free in either φ or ψ. x is bound in (φ ψ) if and only if x is bound in either φ or ψ. The same rule applies to any other binary connective in place of . 4. Quantifiers. x is free in y φ if and only if x is free in φ and x is a different symbol from y. Also, x is bound in y φ if and only if x is y or x is bound in φ. The same rule holds with For example, in x y (P(x) in place of . Q(x,f(x),z)), x and y are bound variables, z is a free variable, and w is neither because it does not occur in the formula. Freeness and boundness can be also specialized to specific occurrences of variables in a formula. For example, in , the first occurrence of x is free while the second is bound. In other words, the x in is free while the in is bound. A formula in first-order logic with no free variables is called a first-order sentence. These are the formulas that will have well-defined truth values under an interpretation. For example, whether a formula such as Phil(x) is true must depend on what x represents. But the sentence will be either true or false in a given interpretation. Semantics An interpretation of a first-order language assigns a denotation to all non-logical constants in that language. It also determines a domain of discourse that specifies the range of the quantifiers. The result is that each term is assigned an object that it represents, and each sentence is assigned a truth value. In this way, an interpretation provides semantic meaning to the terms and formulas of the language. The study of the interpretations of formal languages is called formal semantics. The domain of discourse D is a nonempty set of "objects" of some kind. Intuitively, a first-order formula is a statement about these objects; for example, states the existence of an object x such that the predicate P is true where referred to it. The domain of discourse is the set of considered objects. For example, one can take to be the set of integer numbers. The interpretation of a function symbol is a function. For example, if the domain of discourse consists of integers, a function symbol f of arity 2 can be interpreted as the function that gives the sum of its arguments. In other words, the symbol f is associated with the function I(f) which, in this interpretation, is addition. The interpretation of a constant symbol is a function from the one-element set D0 to D, which can be simply identified with an object in D. For example, an interpretation may assign the value to the constant symbol . The interpretation of an n-ary predicate symbol is a set of n-tuples of elements of the domain of discourse. This means that, given an interpretation, a predicate symbol, and n elements of the domain of discourse, one can tell whether the predicate is true of those elements according to the given interpretation. For example, an interpretation I(P) of a binary predicate symbol P may be the set of pairs of integers such that the first one is less than the second. According to this interpretation, the predicate P would be true if its first argument is less than the second. First-order logic 81 First-order structures The most common way of specifying an interpretation (especially in mathematics) is to specify a structure (also called a model; see below). The structure consists of a nonempty set D that forms the domain of discourse and an interpretation I of the non-logical terms of the signature. This interpretation is itself a function: • Each function symbol f of arity n is assigned a function I(f) from to . In particular, each constant symbol of the signature is assigned an individual in the domain of discourse. • Each predicate symbol P of arity n is assigned a relation I(P) over or, equivalently, a function from to . Thus each predicate symbol is interpreted by a Boolean-valued function on D. Evaluation of truth values A formula evaluates to true or false given an interpretation, and a variable assignment μ that associates an element of the domain of discourse with each variable. The reason that a variable assignment is required is to give meanings to formulas with free variables, such as . The truth value of this formula changes depending on whether x and y denote the same individual. First, the variable assignment μ can be extended to all terms of the language, with the result that each term maps to a single element of the domain of discourse. The following rules are used to make this assignment: 1. Variables. Each variable x evaluates to μ(x) 2. Functions. Given terms that have been evaluated to elements discourse, and a n-ary function symbol f, the term of the domain of evaluates to . Next, each formula is assigned a truth value. The inductive definition used to make this assignment is called the T-schema. 1. Atomic formulas (1). A formula is associated the value true or false depending on whether , where are the evaluation of the terms interpretation of , which by assumption is a subset of . 2. Atomic formulas (2). A formula is assigned true if and of discourse (see the section on equality below). 3. Logical connectives. A formula in the form , is the evaluate to the same object of the domain , etc. is evaluated according to the truth table for the connective in question, as in propositional logic. 4. Existential quantifiers. A formula is true according to M and variables that only differs from and if there exists an evaluation of the regarding the evaluation of x and such that φ is true according to the interpretation M and the variable assignment . This formal definition captures the idea that is true if and only if there is a way to choose a value for x such that φ(x) is satisfied. 5. Universal quantifiers. A formula is true according to M and if φ(x) is true for every pair composed by the interpretation M and some variable assignment that differs from only on the value of x. This captures the idea that is true if every possible choice of a value for x causes φ(x) to be true. If a formula does not contain free variables, and so is a sentence, then the initial variable assignment does not affect its truth value. In other words, a sentence is true according to M and if and only if is true according to M and every other variable assignment . There is a second common approach to defining truth values that does not rely on variable assignment functions. Instead, given an interpretation M, one first adds to the signature a collection of constant symbols, one for each element of the domain of discourse in M; say that for each d in the domain the constant symbol cd is fixed. The interpretation is extended so that each new constant symbol is assigned to its corresponding element of the domain. One now defines truth for quantified formulas syntactically, as follows: 1. Existential quantifiers (alternate). A formula of discourse such that holds. Here is true according to M if there is some d in the domain is the result of substituting cd for every free occurrence of x in First-order logic 82 φ. 2. Universal quantifiers (alternate). A formula is true according to M if, for every d in the domain of discourse, is true according to M. This alternate approach gives exactly the same truth values to all sentences as the approach via variable assignments. Validity, satisfiability, and logical consequence If a sentence φ evaluates to True under a given interpretation M, one says that M satisfies φ; this is denoted . A sentence is satisfiable if there is some interpretation under which it is true. Satisfiability of formulas with free variables is more complicated, because an interpretation on its own does not determine the truth value of such a formula. The most common convention is that a formula with free variables is said to be satisfied by an interpretation if the formula remains true regardless which individuals from the domain of discourse are assigned to its free variables. This has the same effect as saying that a formula is satisfied if and only if its universal closure is satisfied. A formula is logically valid (or simply valid) if it is true in every interpretation. These formulas play a role similar to tautologies in propositional logic. A formula φ is a logical consequence of a formula ψ if every interpretation that makes ψ true also makes φ true. In this case one says that φ is logically implied by ψ. Algebraizations An alternate approach to the semantics of first-order logic proceeds via abstract algebra. This approach generalizes the Lindenbaum–Tarski algebras of propositional logic. There are three ways of eliminating quantified variables from first-order logic, that do not involve replacing quantifiers with other variable binding term operators: • Cylindric algebra, by Alfred Tarski and his coworkers; • Polyadic algebra, by Paul Halmos; • Predicate functor logic, mainly due to Willard Quine. These algebras are all lattices that properly extend the two-element Boolean algebra. Tarski and Givant (1987) showed that the fragment of first-order logic that has no atomic sentence lying in the scope of more than three quantifiers, has the same expressive power as relation algebra. This fragment is of great interest because it suffices for Peano arithmetic and most axiomatic set theory, including the canonical ZFC. They also prove that first-order logic with a primitive ordered pair is equivalent to a relation algebra with two ordered pair projection functions. First-order theories, models, and elementary classes Further information: List of first-order theories A first-order theory consists of a set of axioms in a particular first-order signature. The set of axioms is often finite or recursively enumerable, in which case the theory is called effective. Some authors require theories to also include all logical consequences of the axioms. A first-order structure that satisfies all sentences in a given theory is said to be a model of the theory. An elementary class is the set of all structures satisfying a particular theory. These classes are a main subject of study in model theory. Many theories have an intended interpretation, a certain model that is kept in mind when studying the theory. For example, the intended interpretation of Peano arithmetic consists of the usual natural numbers with their usual operations. However, the Löwenheim–Skolem theorem shows that most first-order theories will also have other, nonstandard models. First-order logic A theory is consistent if it is not possible to prove a contradiction from the axioms of the theory. A theory is complete if, for every formula in its signature, either that formula or its negation is a logical consequence of the axioms of the theory. Gödel's incompleteness theorem shows that effective first-order theories that include a sufficient portion of the theory of the natural numbers can never be both consistent and complete. Empty domains The definition above requires that the domain of discourse of any interpretation must be a nonempty set. There are settings, such as inclusive logic, where empty domains are permitted. Moreover, if a class of algebraic structures includes an empty structure (for example, there is an empty poset), that class can only be an elementary class in first-order logic if empty domains are permitted or the empty structure is removed from the class. There are several difficulties with empty domains, however: • Many common rules of inference are only valid when the domain of discourse is required to be nonempty. One example is the rule stating that implies when x is not a free variable in φ. This rule, which is used to put formulas into prenex normal form, is sound in nonempty domains, but unsound if the empty domain is permitted. • The definition of truth in an interpretation that uses a variable assignment function cannot work with empty domains, because there are no variable assignment functions whose range is empty. (Similarly, one cannot assign interpretations to constant symbols.) This truth definition requires that one must select a variable assignment function (μ above) before truth values for even atomic formulas can be defined. Then the truth value of a sentence is defined to be its truth value under any variable assignment, and it is proved that this truth value does not depend on which assignment is chosen. This technique does not work if there are no assignment functions at all; it must be changed to accommodate empty domains. Thus, when the empty domain is permitted, it must often be treated as a special case. Most authors, however, simply exclude the empty domain by definition. Deductive systems A deductive system is used to demonstrate, on a purely syntactic basis, that one formula is a logical consequence of another formula. There are many such systems for first-order logic, including Hilbert-style deductive systems, natural deduction, the sequent calculus, the tableaux method, and resolution. These share the common property that a deduction is a finite syntactic object; the format of this object, and the way it is constructed, vary widely. These finite deductions themselves are often called derivations in proof theory. They are also often called proofs, but are completely formalized unlike natural-language mathematical proofs. A deductive system is sound if any formula that can be derived in the system is logically valid. Conversely, a deductive system is complete if every logically valid formula is derivable. All of the systems discussed in this article are both sound and complete. They also share the property that it is possible to effectively verify that a purportedly valid deduction is actually a deduction; such deduction systems are called effective. A key property of deductive systems is that they are purely syntactic, so that derivations can be verified without considering any interpretation. Thus a sound argument is correct in every possible interpretation of the language, regardless whether that interpretation is about mathematics, economics, or some other area. In general, logical consequence in first-order logic is only semidecidable: if a sentence A logically implies a sentence B then this can be discovered (for example, by searching for a proof until one is found, using some effective, sound, complete proof system). However, if A does not logically imply B, this does not mean that A logically implies the negation of B. There is no effective procedure that, given formulas A and B, always correctly decides whether A logically implies B. 83 First-order logic 84 Rules of inference Further information: List of rules of inference A rule of inference states that, given a particular formula (or set of formulas) with a certain property as a hypothesis, another specific formula (or set of formulas) can be derived as a conclusion. The rule is sound (or truth-preserving) if it preserves validity in the sense that whenever any interpretation satisfies the hypothesis, that interpretation also satisfies the conclusion. For example, one common rule of inference is the rule of substitution. If t is a term and φ is a formula possibly containing the variable x, then φ[t/x] (often denoted φ[x/t]) is the result of replacing all free instances of x by t in φ. The substitution rule states that for any φ and any term t, one can conclude φ[t/x] from φ provided that no free variable of t becomes bound during the substitution process. (If some free variable of t becomes bound, then to substitute t for x it is first necessary to change the bound variables of φ to differ from the free variables of t.) To see why the restriction on bound variables is necessary, consider the logically valid formula φ given by , in the signature of (0,1,+,×,=) of arithmetic. If t is the term "x + 1", the formula φ[t/y] is , which will be false in many interpretations. The problem is that the free variable x of t became bound during the substitution. The intended replacement can be obtained by renaming the bound variable x of φ to something else, say z, so that the formula after substitution is , which is again logically valid. The substitution rule demonstrates several common aspects of rules of inference. It is entirely syntactical; one can tell whether it was correctly applied without appeal to any interpretation. It has (syntactically-defined) limitations on when it can be applied, which must be respected to preserve the correctness of derivations. Moreover, as is often the case, these limitations are necessary because of interactions between free and bound variables that occur during syntactic manipulations of the formulas involved in the inference rule. Hilbert-style systems and natural deduction A deduction in a Hilbert-style deductive system is a list of formulas, each of which is a logical axiom, a hypothesis that has been assumed for the derivation at hand, or follows from previous formulas via a rule of inference. The logical axioms consist of several axiom schemes of logically valid formulas; these encompass a significant amount of propositional logic. The rules of inference enable the manipulation of quantifiers. Typical Hilbert-style systems have a small number of rules of inference, along with several infinite schemes of logical axioms. It is common to have only modus ponens and universal generalization as rules of inference. Natural deduction systems resemble Hilbert-style systems in that a deduction is a finite list of formulas. However, natural deduction systems have no logical axioms; they compensate by adding additional rules of inference that can be used to manipulate the logical connectives in formulas in the proof. Sequent calculus Further information: Sequent calculus The sequent calculus was developed to study the properties of natural deduction systems. Instead of working with one formula at a time, it uses sequents, which are expressions of the form where A1, ..., An, B1, ..., Bk are formulas and the turnstile symbol halves. Intuitively, a sequent expresses the idea that is used as punctuation to separate the two implies . First-order logic 85 Tableaux method Further information: Method of analytic tableaux Unlike the methods just described, the derivations in the tableaux method are not lists of formulas. Instead, a derivation is a tree of formulas. To show that a formula A is provable, the tableaux method attempts to demonstrate that the negation of A is unsatisfiable. The tree of the derivation has at its root; the tree branches in a way that reflects the structure of the formula. For example, to show that is unsatisfiable requires showing that C and D are each unsatisfiable; the corresponds to a branching point in the tree with parent and children C and D. Resolution The resolution rule is a single rule of inference that, together with unification, is sound and complete for first-order logic. As with the tableaux method, a formula is proved by showing that the negation of the formula is unsatisfiable. Resolution is commonly used in automated theorem proving. A tableau proof for the propositional formula ((a ∨ ~b) & b) → a. The resolution method works only with formulas that are disjunctions of atomic formulas; arbitrary formulas must first be converted to this form through Skolemization. The resolution rule states that from the hypotheses and , the conclusion can be obtained. Provable identities The following sentences can be called "identities" because the main connective in each is the biconditional. (where must not occur free in ) (where must not occur free in ) Equality and its axioms There are several different conventions for using equality (or identity) in first-order logic. The most common convention, known as first-order logic with equality, includes the equality symbol as a primitive logical symbol which is always interpreted as the real equality relation between members of the domain of discourse, such that the "two" given members are the same member. This approach also adds certain axioms about equality to the deductive system employed. These equality axioms are: 1. Reflexivity. For each variable x, x = x. First-order logic 86 2. Substitution for functions. For all variables x and y, and any function symbol f, x = y → f(...,x,...) = f(...,y,...). 3. Substitution for formulas. For any variables x and y and any formula φ(x), if φ' is obtained by replacing any number of free occurrences of x in φ with y, such that these remain free occurrences of y, then x = y → (φ → φ'). These are axiom schemes, each of which specifies an infinite set of axioms. The third scheme is known as Leibniz's law, "the principle of substitutivity", "the indiscernibility of identicals", or "the replacement property". The second scheme, involving the function symbol f, is (equivalent to) a special case of the third scheme, using the formula x = y → (f(...,x,...) = z → f(...,y,...) = z). Many other properties of equality are consequences of the axioms above, for example: 1. Symmetry. If x = y then y = x. 2. Transitivity. If x = y and y = z then x = z. First-order logic without equality An alternate approach considers the equality relation to be a non-logical symbol. This convention is known as first-order logic without equality. If an equality relation is included in the signature, the axioms of equality must now be added to the theories under consideration, if desired, instead of being considered rules of logic. The main difference between this method and first-order logic with equality is that an interpretation may now interpret two distinct individuals as "equal" (although, by Leibniz's law, these will satisfy exactly the same formulas under any interpretation). That is, the equality relation may now be interpreted by an arbitrary equivalence relation on the domain of discourse that is congruent with respect to the functions and relations of the interpretation. When this second convention is followed, the term normal model is used to refer to an interpretation where no distinct individuals a and b satisfy a = b. In first-order logic with equality, only normal models are considered, and so there is no term for a model other than a normal model. When first-order logic without equality is studied, it is necessary to amend the statements of results such as the Löwenheim–Skolem theorem so that only normal models are considered. First-order logic without equality is often employed in the context of second-order arithmetic and other higher-order theories of arithmetic, where the equality relation between sets of natural numbers is usually omitted. Defining equality within a theory If a theory has a binary formula A(x,y) which satisfies reflexivity and Leibniz's law, the theory is said to have equality, or to be a theory with equality. The theory may not have all instances of the above schemes as axioms, but rather as derivable theorems. For example, in theories with no function symbols and a finite number of relations, it is possible to define equality in terms of the relations, by defining the two terms s and t to be equal if any relation is unchanged by changing s to t in any argument. Some theories allow other ad hoc definitions of equality: • In the theory of partial orders with one relation symbol ≤, one could define s = t to be an abbreviation for s ≤ t t ≤ s. • In set theory with one relation , one may define s = t to be an abbreviation for x (s x t x) x (x s x t). This definition of equality then automatically satisfies the axioms for equality. In this case, one should replace the usual axiom of extensionality, , by , i.e. if x and y have the same elements, then they belong to the same sets. First-order logic Metalogical properties One motivation for the use of first-order logic, rather than higher-order logic, is that first-order logic has many metalogical properties that stronger logics do not have. These results concern general properties of first-order logic itself, rather than properties of individual theories. They provide fundamental tools for the construction of models of first-order theories. Completeness and undecidability Gödel's completeness theorem, proved by Kurt Gödel in 1929, establishes that there are sound, complete, effective deductive systems for first-order logic, and thus the first-order logical consequence relation is captured by finite provability. Naively, the statement that a formula φ logically implies a formula ψ depends on every model of φ; these models will in general be of arbitrarily large cardinality, and so logical consequence cannot be effectively verified by checking every model. However, it is possible to enumerate all finite derivations and search for a derivation of ψ from φ. If ψ is logically implied by φ, such a derivation will eventually be found. Thus first-order logical consequence is semidecidable: it is possible to make an effective enumeration of all pairs of sentences (φ,ψ) such that ψ is a logical consequence of φ. Unlike propositional logic, first-order logic is undecidable (although semidecidable), provided that the language has at least one predicate of arity at least 2 (other than equality). This means that there is no decision procedure that determines whether arbitrary formulas are logically valid. This result was established independently by Alonzo Church and Alan Turing in 1936 and 1937, respectively, giving a negative answer to the Entscheidungsproblem posed by David Hilbert in 1928. Their proofs demonstrate a connection between the unsolvability of the decision problem for first-order logic and the unsolvability of the halting problem. There are systems weaker than full first-order logic for which the logical consequence relation is decidable. These include propositional logic and monadic predicate logic, which is first-order logic restricted to unary predicate symbols and no function symbols. The Bernays–Schönfinkel class of first-order formulas is also decidable. The Löwenheim–Skolem theorem The Löwenheim–Skolem theorem shows that if a first-order theory of cardinality λ has any infinite model then it has models of every infinite cardinality greater than or equal to λ. One of the earliest results in model theory, it implies that it is not possible to characterize countability or uncountability in a first-order language. That is, there is no first-order formula φ(x) such that an arbitrary structure M satisfies φ if and only if the domain of discourse of M is countable (or, in the second case, uncountable). The Löwenheim–Skolem theorem implies that infinite structures cannot be categorically axiomatized in first-order logic. For example, there is no first-order theory whose only model is the real line: any first-order theory with an infinite model also has a model of cardinality larger than the continuum. Since the real line is infinite, any theory satisfied by the real line is also satisfied by some nonstandard models. When the Löwenheim–Skolem theorem is applied to first-order set theories, the nonintuitive consequences are known as Skolem's paradox. The compactness theorem The compactness theorem states that a set of first-order sentences has a model if and only if every finite subset of it has a model. This implies that if a formula is a logical consequence of an infinite set of first-order axioms, then it is a logical consequence of some finite number of those axioms. This theorem was proved first by Kurt Gödel as a consequence of the completeness theorem, but many additional proofs have been obtained over time. It is a central tool in model theory, providing a fundamental method for constructing models. The compactness theorem has a limiting effect on which collections of first-order structures are elementary classes. For example, the compactness theorem implies that any theory that has arbitrarily large finite models has an infinite 87 First-order logic model. Thus the class of all finite graphs is not an elementary class (the same holds for many other algebraic structures). There are also more subtle limitations of first-order logic that are implied by the compactness theorem. For example, in computer science, many situations can be modeled as a directed graph of states (nodes) and connections (directed edges). Validating such a system may require showing that no "bad" state can be reached from any "good" state. Thus one seeks to determine if the good and bad states are in different connected components of the graph. However, the compactness theorem can be used to show that connected graphs are not an elementary class in first-order logic, and there is no formula φ(x,y) of first-order logic, in the signature of graphs, that expresses the idea that there is a path from x to y. Connectedness can be expressed in second-order logic, however, but not with only existential set quantifiers, as also enjoys compactness. Lindström's theorem Per Lindström showed that the metalogical properties just discussed actually characterize first-order logic in the sense that no stronger logic can also have those properties (Ebbinghaus and Flum 1994, Chapter XIII). Lindström defined a class of abstract logical systems, and a rigorous definition of the relative strength of a member of this class. He established two theorems for systems of this type: • A logical system satisfying Lindström's definition that contains first-order logic and satisfies both the Löwenheim–Skolem theorem and the compactness theorem must be equivalent to first-order logic. • A logical system satisfying Lindström's definition that has a semidecidable logical consequence relation and satisfies the Löwenheim–Skolem theorem must be equivalent to first-order logic. Limitations Although first-order logic is sufficient for formalizing much of mathematics, and is commonly used in computer science and other fields, it has certain limitations. These include limitations on its expressiveness and limitations of the fragments of natural languages that it can describe. Expressiveness The Löwenheim–Skolem theorem shows that if a first-order theory has any infinite model, then it has infinite models of every cardinality. In particular, no first-order theory with an infinite model can be categorical. Thus there is no first-order theory whose only model has the set of natural numbers as its domain, or whose only model has the set of real numbers as its domain. Many extensions of first-order logic, including infinitary logics and higher-order logics, are more expressive in the sense that they do permit categorical axiomatizations of the natural numbers or real numbers. This expressiveness comes at a metalogical cost, however: by Lindström's theorem, the compactness theorem and the downward Löwenheim–Skolem theorem cannot hold in any logic stronger than first-order. Formalizing natural languages First-order logic is able to formalize many simple quantifier constructions in natural language, such as "every person who lives in Perth lives in Australia". But there are many more complicated features of natural language that cannot be expressed in (single-sorted) first-order logic. "Any logical system which is appropriate as an instrument for the analysis of natural language needs a much richer structure than first-order predicate logic" (Gamut 1991, p. 75). 88 First-order logic 89 Type Example Comment Quantification over properties If John is self-satisfied, then there is at least one thing he has in common with Peter Requires a quantifier over predicates, which cannot be implemented in single-sorted first-order logic: Zj→ ∃X(Xj∧Xp) Quantification over properties Santa Claus has all the attributes of a sadist Requires quantifiers over predicates, which cannot be implemented in single-sorted first-order logic: ∀X(∀x(Sx → Xx)→Xs) Predicate adverbial John is walking quickly Cannot be analysed as Wj ∧ Qj; predicate adverbials are not the same kind of thing as second-order predicates such as colour Relative adjective Jumbo is a small elephant Cannot be analysed as Sj ∧ Ej; predicate adjectives are not the same kind of thing as second-order predicates such as colour Predicate adverbial modifier John is walking very quickly - Relative adjective modifier Jumbo is terribly small An expression such as "terribly", when applied to a relative adjective such as "small", results in a new composite relative adjective "terribly small" Prepositions Mary is sitting next to John The preposition "next to" when applied to "John" results in the predicate adverbial "next to John" Restrictions, extensions and variations There are many variations of first-order logic. Some of these are inessential in the sense that they merely change notation without affecting the semantics. Others change the expressive power more significantly, by extending the semantics through additional quantifiers or other new logical symbols. For example, infinitary logics permit formulas of infinite size, and modal logics add symbols for possibility and necessity. Restricted languages First-order logic can be studied in languages with fewer logical symbols than were described above. • Because can be expressed as the two quantifiers and can be dropped. • Since can be expressed as or , and can be expressed as and , either of can be expressed as can be dropped. In other words, it is sufficient to have and , or and , either , as the only logical connectives. • Similarly, it is sufficient to have only and as logical connectives, or to have only the Sheffer stroke (NAND) or the Peirce arrow (NOR) operator. • It is possible to entirely avoid function symbols and constant symbols, rewriting them via predicate symbols in an appropriate way. For example, instead of using a constant symbol one may use a predicate (interpreted as ), and replace every predicate such as with will similarly be replaced by a predicate . A function such as interpreted as . This change requires adding additional axioms to the theory at hand, so that interpretations of the predicate symbols used have the correct semantics. Restrictions such as these are useful as a technique to reduce the number of inference rules or axiom schemes in deductive systems, which leads to shorter proofs of metalogical results. The cost of the restrictions is that it becomes more difficult to express natural-language statements in the formal system at hand, because the logical connectives used in the natural language statements must be replaced by their (longer) definitions in terms of the restricted collection of logical connectives. Similarly, derivations in the limited systems may be longer than derivations in systems that include additional connectives. There is thus a trade-off between the ease of working within the formal system and the ease of proving results about the formal system. First-order logic 90 It is also possible to restrict the arities of function symbols and predicate symbols, in sufficiently expressive theories. One can in principle dispense entirely with functions of arity greater than 2 and predicates of arity greater than 1 in theories that include a pairing function. This is a function of arity 2 that takes pairs of elements of the domain and returns an ordered pair containing them. It is also sufficient to have two predicate symbols of arity 2 that define projection functions from an ordered pair to its components. In either case it is necessary that the natural axioms for a pairing function and its projections are satisfied. Many-sorted logic Ordinary first-order interpretations have a single domain of discourse over which all quantifiers range. Many-sorted first-order logic allows variables to have different sorts, which have different domains. This is also called typed first-order logic, and the sorts called types (as in data type), but it is not the same as first-order type theory. Many-sorted first-order logic is often used in the study of second-order arithmetic. When there are only finitely many sorts in a theory, many-sorted first-order logic can be reduced to single-sorted first-order logic. One introduces into the single-sorted theory a unary predicate symbol for each sort in the many-sorted theory, and adds an axiom saying that these unary predicates partition the domain of discourse. For example, if there are two sorts, one adds predicate symbols and and the axiom . Then the elements satisfying are thought of as elements of the first sort, and elements satisfying as elements of the second sort. One can quantify over each sort by using the corresponding predicate symbol to limit the range of quantification. For example, to say there is an element of the first sort satisfying formula φ(x), one writes . Additional quantifiers Additional quantifiers can be added to first-order logic. • Sometimes it is useful to say that "P(x) holds for exactly one x", which can be expressed as notation, called uniqueness quantification, may be taken to abbreviate a formula such as x P(x). This x (P(x) y (P(y) (x = y))). • First-order logic with extra quantifiers has new quantifiers Qx,..., with meanings such as "there are many x such that ...". Also see branching quantifiers and the plural quantifiers of George Boolos and others. • Bounded quantifiers are often used in the study of set theory or arithmetic. Infinitary logics Infinitary logic allows infinitely long sentences. For example, one may allow a conjunction or disjunction of infinitely many formulas, or quantification over infinitely many variables. Infinitely long sentences arise in areas of mathematics including topology and model theory. Infinitary logic generalizes first-order logic to allow formulas of infinite length. The most common way in which formulas can become infinite is through infinite conjunctions and disjunctions. However, it is also possible to admit generalized signatures in which function and relation symbols are allowed to have infinite arities, or in which quantifiers can bind infinitely many variables. Because an infinite formula cannot be represented by a finite string, it is necessary to choose some other representation of formulas; the usual representation in this context is a tree. Thus formulas are, essentially, identified with their parse trees, rather than with the strings being parsed. The most commonly studied infinitary logics are denoted Lαβ, where α and β are each either cardinal numbers or the symbol ∞. In this notation, ordinary first-order logic is Lωω. In the logic L∞ω, arbitrary conjunctions or disjunctions are allowed when building formulas, and there is an unlimited supply of variables. More generally, the logic that permits conjunctions or disjunctions with less than κ constituents is known as Lκω. For example, Lω1ω permits First-order logic countable conjunctions and disjunctions. The set of free variables in a formula of Lκω can have any cardinality strictly less than κ, yet only finitely many of them can be in the scope of any quantifier when a formula appears as a subformula of another.[6] In other infinitary logics, a subformula may be in the scope of infinitely many quantifiers. For example, in Lκ∞, a single universal or existential quantifier may bind arbitrarily many variables simultaneously. Similarly, the logic Lκλ permits simultaneous quantification over fewer than λ variables, as well as conjunctions and disjunctions of size less than κ. Non-classical and modal logics • Intuitionistic first-order logic uses intuitionistic rather than classical propositional calculus; for example, ¬¬φ need not be equivalent to φ. • First-order modal logic allows one to describe other possible worlds as well as this contingently true world which we inhabit. In some versions, the set of possible worlds varies depending on which possible world one inhabits. Modal logic has extra modal operators with meanings which can be characterized informally as, for example "it is necessary that φ" (true in all possible worlds) and "it is possible that φ" (true in some possible world). With standard first-order logic we have a single domain and each predicate is assigned one extension. With first-order modal logic we have a domain function that assigns each possible world its own domain, so that each predicate gets an extension only relative to these possible worlds. This allows us to model cases where, for example, Alex is a Philosopher, but might have been a Mathematician, and might not have existed at all. In the first possible world P(a) is true, in the second P(a) is false, and in the third possible world there is no a in the domain at all. • first-order fuzzy logics are first-order extensions of propositional fuzzy logics rather than classical propositional calculus. Higher-order logics The characteristic feature of first-order logic is that individuals can be quantified, but not predicates. Thus is a legal first-order formula, but is not, in most formalizations of first-order logic. Second-order logic extends first-order logic by adding the latter type of quantification. Other higher-order logics allow quantification over even higher types than second-order logic permits. These higher types include relations between relations, functions from relations to relations between relations, and other higher-type objects. Thus the "first" in first-order logic describes the type of objects that can be quantified. Unlike first-order logic, for which only one semantics is studied, there are several possible semantics for second-order logic. The most commonly employed semantics for second-order and higher-order logic is known as full semantics. The combination of additional quantifiers and the full semantics for these quantifiers makes higher-order logic stronger than first-order logic. In particular, the (semantic) logical consequence relation for second-order and higher-order logic is not semidecidable; there is no effective deduction system for second-order logic that is sound and complete under full semantics. Second-order logic with full semantics is more expressive than first-order logic. For example, it is possible to create axiom systems in second-order logic that uniquely characterize the natural numbers and the real line. The cost of this expressiveness is that second-order and higher-order logics have fewer attractive metalogical properties than first-order logic. For example, the Löwenheim–Skolem theorem and compactness theorem of first-order logic become false when generalized to higher-order logics with full semantics. 91 First-order logic Automated theorem proving and formal methods Further information: First-order theorem proving Automated theorem proving refers to the development of computer programs that search and find derivations (formal proofs) of mathematical theorems. Finding derivations is a difficult task because the search space can be very large; an exhaustive search of every possible derivation is theoretically possible but computationally infeasible for many systems of interest in mathematics. Thus complicated heuristic functions are developed to attempt to find a derivation in less time than a blind search. The related area of automated proof verification uses computer programs to check that human-created proofs are correct. Unlike complicated automated theorem provers, verification systems may be small enough that their correctness can be checked both by hand and through automated software verification. This validation of the proof verifier is needed to give confidence that any derivation labeled as "correct" is actually correct. Some proof verifiers, such as Metamath, insist on having a complete derivation as input. Others, such as Mizar and Isabelle, take a well-formatted proof sketch (which may still be very long and detailed) and fill in the missing pieces by doing simple proof searches or applying known decision procedures: the resulting derivation is then verified by a small, core "kernel". Many such systems are primarily intended for interactive use by human mathematicians: these are known as proof assistants. They may also use formal logics that are stronger than first-order logic, such as type theory. Because a full derivation of any nontrivial result in a first-order deductive system will be extremely long for a human to write,[7] results are often formalized as a series of lemmas, for which derivations can be constructed separately. Automated theorem provers are also used to implement formal verification in computer science. In this setting, theorem provers are used to verify the correctness of programs and of hardware such as processors with respect to a formal specification. Because such analysis is time-consuming and thus expensive, it is usually reserved for projects in which a malfunction would have grave human or financial consequences. Notes [1] Mendelson, Elliott (1964). Introduction to Mathematical Logic. Van Nostrand Reinhold. pp. 56. [2] The word language is sometimes used as a synonym for signature, but this can be confusing because "language" can also refer to the set of formulas. [3] More precisely, there is only one language of each variant of one-sorted first-order logic: with or without equality, with or without functions, with or without propositional variables, …. [4] Some authors who use the term "well-formed formula" use "formula" to mean any string of symbols from the alphabet. However, most authors in mathematical logic use "formula" to mean "well-formed formula" and have no term for non-well-formed formulas. In every context, it is only the well-formed formulas that are of interest. [5] The SMT-LIB Standard: Version 2.0, by Clark Barrett, Aaron Stump, and Cesare Tinelli. http:/ / goedel. cs. uiowa. edu/ smtlib/ [6] Some authors only admit formulas with finitely many free variables in Lκω, and more generally only formulas with < λ free variables in Lκλ. [7] Avigad et al. (2007) discuss the process of formally verifying a proof of the prime number theorem. The formalized proof required approximately 30,000 lines of input to the Isabelle proof verifier. 92 First-order logic References • Peter Andrews. 2002. An Introduction to Mathematical Logic and Type Theory: To Truth Through Proof. 2nd ed. Berlin: Kluwer Academic Publishers, available from Springer. • Jeremy Avigad, Kevin Donnelly, David Gray, Paul Raff, 2007. "A formally verified proof of the prime number theorem", ACM Transactions on Computational Logic, v. 9 n. 1. doi:10.1145/1297658.1297660 • Jon Barwise, 1977. "An introduction to first-order logic", in Barwise, Jon, ed (1982). Handbook of Mathematical Logic. Studies in Logic and the Foundations of Mathematics. Amsterdam: North-Holland. ISBN 978-0-444-86388-1 • Jon Barwise and John Etchemendy, 2000. Language Proof and Logic. Stanford, CA: CSLI Publications (Distributed by the University of Chicago Press). • Józef Maria Bocheński, 2007. A Précis of Mathematical Logic. Translated from the French and German editions by Otto Bird. Dordrecht, South Holland: D. Reidel. • José Ferreirós. The Road to Modern Logic—An Interpretation. (http://jstor.org/stable/2687794) Bulletin of Symbolic Logic, Volume 7, Issue 4, 2001, pp. 441–484. DOI 10.2307/2687794. JStor (http://links.jstor.org/ sici?sici=1079-8986(200112)7:4<441:TRTMLI>2.0.CO;2-O) • L. T. F. Gamut, 1991. Logic, Language, and Meaning, Volume 2: Introduction to Logic. Chicago: University Of Chicago Press. ISBN 0-226-28088-8. • David Hilbert and Wilhelm Ackermann 1950. Principles of Mathematical Logic (English translation). Chelsea. The 1928 first German edition was titled Grundzüge der theoretischen Logik. • Wilfrid Hodges, 2001, "Classical Logic I: First Order Logic," in Lou Goble, ed., The Blackwell Guide to Philosophical Logic. Blackwell. • Heinz-Dieter Ebbinghaus, Jörg Flum, and Wolfgang Thomas. 1994. Mathematical Logic. Berlin, New York: Springer-Verlag. Second Edition. Undergraduate Texts in Mathematics. ISBN 978-0-387-94258-2. External links • Stanford Encyclopedia of Philosophy: " Classical Logic (http://plato.stanford.edu/entries/logic-classical/) -by Stewart Shapiro. Covers syntax, model theory, and metatheory for first-order logic in the natural deduction style. • forall x: an introduction to formal logic (http://www.fecundity.com/logic/), by P.D. Magnus, covers formal semantics and proof theory for first-order logic. • Metamath (http://us.metamath.org/index.html): an ongoing online project to reconstruct mathematics as a huge first-order theory, using first-order logic and the axiomatic set theory ZFC. Principia Mathematica modernized. • Podnieks, Karl. Introduction to mathematical logic. (http://www.ltn.lv/~podnieks/) • Cambridge Mathematics Tripos Notes (http://john.fremlin.de/schoolwork/logic/index.html) (typeset by John Fremlin). These notes cover part of a past Cambridge Mathematics Tripos course taught to undergraduates students (usually) within their third year. The course is entitled "Logic, Computation and Set Theory" and covers Ordinals and cardinals, Posets and zorn’s Lemma, Propositional logic, Predicate logic, Set theory and Consistency issues related to ZFC and other set theories. 93 List of first-order theories List of first-order theories In mathematical logic, a first-order theory is given by a set of axioms in some language. This entry lists some of the more common examples used in model theory and some of their properties. Preliminaries For every natural mathematical structure there is a signature σ listing the constants, functions, and relations of the theory together with their valences, so that the object is naturally a σ-structure. Given a signature σ there is a unique first-order language Lσ that can be used to capture the first-order expressible facts about the σ-structure. There are two common ways to specify theories: 1. List or describe a set of sentences in the language Lσ, called the axioms of the theory. 2. Give a set of σ-structures, and define a theory to be the set of sentences in Lσ holding in all these models. For example, the "theory of finite fields" consists of all sentences in the language of fields that are true in all finite fields. An Lσ theory may: • be consistent: no proof of contradiction exists; • be satisfiable: there exists a σ-structure for which the sentences of the theory are all true (by the completeness theorem, satisfiability is equivalent to consistency); • be complete: for any statement, either it or its negation is provable; • have quantifier elimination; • eliminate imaginaries; • be finitely axiomatizable; • be decidable: There is an algorithm to decide which statements are provable; • be recursively axiomatizable; • be Model complete or sub-model complete; • be κ-categorical: All models of cardinality κ are isomorphic; • be Stable or unstable. • be ω-stable (same as totally transcendental for countable theories). • be superstable • have an atomic model • have a prime model • have a saturated model Pure identity theories The signature of the pure identity theory is empty, with no functions, constants, or relations. Pure identity theory has no (non-logical) axioms. It is decidable. One of the few interesting properties that can be stated in the language of pure identity theory is that of being infinite. This is given by an infinite set of axioms stating there are at least 2 elements, there are at least 3 elements, and so on: • ∃x1 ∃x2 ¬x1 = x2, ∃x1 ∃x2 ∃x3 ¬x1 = x2 ∧ ¬x1 = x3 ∧ ¬x2 = x3,... These axioms define the theory of an infinite set. The opposite property of being finite cannot be stated in first-order logic for any theory that has arbitrarily large finite models: in fact any such theory has infinite models by the compactness theorem. In general if a property can be stated by a finite number of sentences of first-order logic then the opposite property can also be stated in first-order 94 List of first-order theories logic, but if a property needs an infinite number of sentences then its opposite property cannot be stated in first-order logic. Any statement of pure identity theory is equivalent to either σ(N) or to ¬σ(N) for some finite subset N of the non-negative integers, where σ(N) is the statement that the number of elements is in N. It is even possible to describe all possible theories in this language as follows. Any theory is either the theory of all sets of cardinality in N for some finite subset N of the non-negative integers, or the theory of all sets whose cardinality is not in N, for some finite or infinite subset N of the non-negative integers. (There are no theories whose models are exactly sets of cardinality N if N is an infinite subset of the integers.) The complete theories are the theories of sets of cardinality n for some finite n, and the theory of infinite sets. One special case of this is the inconsistent theory defined by the axiom ∃x ¬x = x. It is a perfectly good theory with many good properties: it is complete, decidable, finitely axiomatizable, and so on. The only problem is that it has no models at all. By Gödel's completeness theorem, it is the only theory (for any given language) with no models. Unary relations A set of unary relations Pi for i in some set I is called independent if for every two disjoint finite subsets A and B of I there is some element x such that Pi(x) is true for i in A and false for i in B. Independence can be expressed by a set of first-order statements. The theory of a countable number of independent unary relations is complete, but has no atomic models. It is also an example of a theory that is superstable but not totally transcendental. Equivalence relations The signature of equivalence relations has one binary infix relation symbol ~, no constants, and no functions. Equivalence relations satisfy the axioms: • Reflexivity ∀x x~x; • Symmetry ∀x ∀y x~y → y~x; • Transitivity: ∀x ∀y ∀z (x~y ∧ y~z) → x~z. Some first order properties of equivalence relations are: • • • • ~ has an infinite number of equivalence classes; ~ has exactly n equivalence classes (for any fixed positive integer n); All equivalence classes are infinite; All equivalence classes have size exactly n (for any fixed positive integer n). The theory of an equivalence relation with exactly 2 infinite equivalence classes is an easy example of a theory which is ω-categorical but not categorical for any larger cardinal. The equivalence relation ~ should not be confused with the identity symbol '=': if x=y then x~y, but the converse is not necessarily true. Theories of equivalence relations are not all that difficult or interesting, but often give easy examples or counterexamples for various statements. The following constructions are sometimes used to produce examples of theories with certain spectra; in fact by applying them to a small number of explicit theories T one gets examples of complete countable theories with all possible uncountable spectra. If T is a theory in some language, we define a new theory 2T by adding a new binary relation to the language, and adding axioms stating that it is an equivalence relation, such that there are an infinite number of equivalence classes all of which are models of T. It is possible to iterate this construction transfinitely: given an ordinal α, define a new theory by adding an equivalence relation Eβ for each β<α, together with axioms stating that whenever β<γ then each Eγ equivalence class is the union of infinitely many Eβ equivalence classes, and each E0 equivalence class is a model of T. Informally, one can visualize models of this theory as infinitely branching 95 List of first-order theories 96 trees of height α with models of T attached to all leaves. Orders The signature of orders has no constants or functions, and one binary relation symbols ≤. (It is of course possible to use ≥, < or > instead as the basic relation, with the obvious minor changes to the axioms.) We define x≥y, x<y, x>y as abbreviations for y≤x, x≤y ∧¬y≤x, y<x, Some first-order properties of orders: • • • • • • • • • Transitive: ∀x ∀y ∀z x ≤ y∧y ≤ z → x ≤ z Reflexive: ∀x x ≤ x Antisymmetric: ∀x ∀y x ≤ y ∧ y ≤ x → x = y Partial: Transitive∧Reflexive∧Antisymmetric; Linear (or total): Partial ∧ ∀x ∀y x≤y ∨ y≤x Dense ∀x ∀z x<z → ∃y x<y ∧ y<z ("Between any 2 distinct elements there is another element") There is a smallest element: ∃x ∀y x≤y There is a largest element: ∃x ∀y y≤x Every element has an immediate successor: ∀x ∃y ∀z x<z ↔ y≤z The theory DLO of dense linear orders with no endpoints (i.e. no smallest or largest element) is complete, ω-categorical, but not categorical for any uncountable cardinal. There are 3 other very similar theories: the theory of dense linear orders with a: • Smallest but no largest element; • Largest but no smallest element; • Largest and smallest element. Being well ordered ("any non-empty subset has a minimal element") is not a first-order property; the usual definition involves quantifying over all subsets. Lattices Lattices can be considered either as special sorts of partially ordered sets, with a signature consisting of one binary relation symbol ≤, or as algebraic structures with a signature consisting of two binary operations ∧ and ∨. The two approaches can be related by defining a≤ b to mean a∧b=a. For two binary operations the axioms for a lattice are: Commutative laws: Associative laws: Absorption laws: For one relation ≤ the axioms are: • Axioms stating ≤ is a partial order, as above. • • (existence of c=a∧b) (existence of c=a∨b) First order properties include: • • Completeness is not a first order property of lattice. (distributive lattices) (modular lattices) List of first-order theories Graphs The signature of graphs has no constants or functions, and one binary relation symbol R, where R(x,y) is read as "there is an edge from x to y". The axioms for the theory of graphs are • Symmetric: ∀x ∀y R(x,y)→ R(y,x) • Anti-reflexive: ∀x ¬R(x,x) ("no loops") The theory of random graphs has the following extra axioms for each positive integer n: • For any two disjoint finite sets of size n, there is a point joined to all points of the first set and to no points of the second set. (For each fixed n, it is easy to write this statement in the language of graphs.) The theory of random graphs is ω categorical, complete, and decidable, and its countable model is called the Rado graph. A statement in the language of graphs is true in this theory if and only if it is true with probability 1 for a random graph on a countable number of points. Boolean algebras There are several different signatures and conventions used for Boolean algebras: 1. The signature has 2 constants, 0 and 1, and two binary functions ∧ and ∨ ("and" and "or"), and one unary function ¬ ("not"). This is a little confusing as the functions use the same symbols as the propositional functions of first-order logic. 2. In set theory, a common convention is that the language has 2 constants, 0 and 1, and two binary functions · and +, and one unary function −. The three functions have the same interpretation as the functions in the first convention. Unfortunately, this convention clashes badly with the next convention: 3. In algebra, the usual convention is that the language has 2 constants, 0 and 1, and two binary functions · and +. The function · has the same meaning as ∧, but a+b means a∨b∧¬(a∧b). The reason for this is that the axioms for a Boolean algebra are then just the axioms for a ring with 1 plus ∀x x2 = x. Unfortunately this clashes with the standard convention in set theory given above. The axioms are: • The axioms for a distributive lattice (see above) • ∀a∀b a∧¬a = 0, ∀a∀b a∨¬a = 1 (properties of negation) • Some authors add the extra axiom ¬0=1, to exclude the trivial algebra with one element. Tarski proved that the theory of Boolean algebras is decidable. We write x ≤ y as an abbreviation for x ∧ y = x, and atom(x) as an abbreviation for ¬x = 0 ∧ ∀y y≤x → y = 0 ∨ y = x, read as "x is an atom", in other words a non-zero element with nothing between it and 0. Here are some first-order properties of Boolean algebras: • Atomic: ∀x x=0 ∨ ∃y y≤x ∧ atom(y) • Atomless: ∀x ¬atom(x) The theory of atomless Boolean algebras is ω-categorical and complete. For any Boolean algebra B, there are several invariants defined as follows. • the ideal I(B) consists of elements that are the sum of an atomic and an atomless element. • The quotient algebras Bi of B are defined inductively by B0=B, Bk+1 = Bk/I(Bk). • The invariant m(B) is the smallest integer such that Bm+1 is trivial, or ∞ if no such integer exists. • If m(B) is finite, the invariant n(B) is the number of atoms of Bm(B) if this number is finite, or ∞ if this number is infinite. • The invariant l(B) is 0 if Bm(B) is atomic or if m(B) is ∞, and 1 otherwise. 97 List of first-order theories Then two Boolean algebras are elementarily equivalent if and only if their invariants l, m, and n are the same. In other words, the values of these invariants classify the possible completions of the theory of Boolean algebras. So the possible complete theories are: • The trivial algebra (if this is allowed; sometimes 0≠1 is included as an axiom.) • The theory with m=∞ • The theories with m a natural number, n a natural number or ∞, and l = 0 or 1 (with l = 0 if n=0). Groups The signature of group theory has one constant 1 (the identity), one function of arity 1 (the inverse) whose value on t is denoted by t−1, and one function of arity 2, which is usually omitted from terms. For any integer n. tn is an abbreviation for the obvious term for the nth power of t. Groups are defined by the axioms • Identity: ∀x 1x = x ∧ x1 = x • Inverse: ∀x x−1x = 1 ∧ xx−1 = 1 • Associative: ∀x∀y∀z (xy)z = x(yz) Some properties of groups that can be defined in the first-order language of groups are: • • • • • • • Abelian ∀x ∀y xy = yx. Torsion free ∀x x2 = 1→x = 1, ∀x x3 = 1 → x = 1, ∀x x4 = 1 → x = 1, ... Divisible ∀x ∃y y2 = x, ∀x ∃y y3 = x, ∀x ∃y y4 = x, ... Infinite (as in identity theory) Exponent n (for any fixed positive integer n) ∀x xn = 1 Nilpotent of class n (for any fixed positive integer n) Solvable of class n (for any fixed positive integer n) The theory of Abelian groups is decidable. The theory of Infinite divisible torsion-free abelian groups is complete, as is the theory of Infinite abelian groups of exponent p (for p prime). The theory of finite groups is the set of first-order statements in the language of groups that are true in all finite groups (there are plenty of infinite models of this theory). It is not completely trivial to find any such statement that is not true for all groups: one example is "given two elements of order 2, either they are conjugate or there is a non-trivial element commuting with both of them". The properties of being finite, or free, or simple, or torsion are not first-order. More precisely, the first-order theory of all groups with one of these properties has models that do not have this property. Rings and fields The signature of (unital) rings has 2 constants 0 and 1, two binary functions + and ×, and, optionally, one unary inverse functions − −1. Rings Axioms: Addition makes the ring into an abelian group, multiplication is associative and has an identity 1, and multiplication is left and right distributive. Commutative rings The axioms for rings plus ∀x ∀y xy=yx. Fields The axioms for commutative rings plus ∀x ∃y xy=1 and ¬ 1=0. Many of the examples given here have only universal, or algebraic axioms. The class of structures satisfying such a theory has the property of being closed under substructure. For example, a subset of a group closed under the group actions of multiplication and inverse is again a group. Since the signature of fields does not usually include multiplicative and additive inverse, the axioms for inverses are not universal, and therefore a substructure of a field closed under addition and multiplication is not always a field. This can be remedied by adding unary inverse functions to the language. 98 List of first-order theories 99 For any positive integer n the property that all equations of degree n have a root can be expressed by a single first-order sentence: • ∀ a1 ∀ a2... ∀ an ∃x (...((x+a1)x +a2)x+...)x+an = 0 Perfect fields The axioms for fields, plus axioms for each prime number p stating that if p 1 = 0 (i.e. the field has characteristic p), then every field element has a pth root. Algebraically closed fields of characteristic p The axioms for fields, plus for every positive n the axiom that all polynomials of degree n have a root, plus axioms fixing the characteristic. The classical examples of complete theories. Categorical in all uncountable cardinals. The theory ACFp has a universal domain property, in the sense that every structure N satisfying the universal axioms of ACFp is a substructure of a sufficiently large algebraically closed field , and additionally any two such embeddings N → M induce an automorphism of M. Finite fields. The theory of finite fields is the set of all first-order statements that are true in all finite fields. Significant examples of such statements can, for example, be given by applying the Chevalley–Warning theorem, over the prime fields. The name is a little misleading as the theory has plenty of infinite models. Ax proved that the theory is decidable. Formally real fields These are fields with the axiom • For every positive n, the axiom ∀ a1 ∀ a2... ∀ an a1a1+a2a2+ ...+anan=0 → a1=0∨a2=0∨ ... ∨an=0 (0 is not a non-trivial sum of squares). Real closed fields Axioms: • ∀x ∃y x=yy ∨ x+yy=0. • For every odd positive n, the axiom stating that every polynomial of degree n has a root. • For every positive n, the axiom ∀ a1 ∀ a2... ∀ an a1a1+a2a2+ ...+anan=0 → a1=0∨a2=0∨ ... ∨an=0 (0 is not a non-trivial sum of squares). The theory of real closed fields is decidable (Tarski) and therefore complete. p-adic fields: Ax & Kochen (1965) showed that the theory of p-adic fields is decidable and gave a set of axioms for it. Geometry Axioms for various systems of geometry usually use a typed language, with the different types corresponding to different geometric objects such as points, lines, circles, planes, and so on. The signature will often consist of binary incidence relations between objects of different types; for example, the relation that a point lies on a line. The signature may have more complicated relations; for example ordered geometry might have a ternary "betweenness" relation for 3 points, which says whether one lies between two others, or a "congruence" relation between 2 pairs of points. Some examples of axiomatized systems of geometry include ordered geometry, absolute geometry, affine geometry, Euclidean geometry, projective geometry, and hyperbolic geometry. For each of these geometries there are many different and inequivalent systems of axioms for various dimensions. Some of these axiom systems include "completeness" axioms that are not first order. As a typical example, the axioms for projective geometry use 2 types, points and lines, and a binary incidence relation between points and lines. If point and line variables are indicated by small and capital letter, and a incident to A is written as aA, then one set of axioms is • • (There is a line through any 2 distinct points a,b ...) (... which is unique) • (Veblen's axiom: if ab and cd lie on intersecting lines, then so do ac and bd.) List of first-order theories • 100 (Every line has at least 3 points) Euclid did not state all the axioms for Euclidean geometry explicitly, and the first complete list was given by Hilbert in Hilbert's axioms. This is not a first order axiomatization as one of Hilbert's axioms is a second order completeness axiom. Tarski's axioms are a first order axiomatization of Euclidean geometry. Tarski showed this axiom system is complete and decidable by relating it to the complete and decidable theory of real closed fields. Differential algebra • The theory DF of differential fields. The signature is that of fields (0, 1, +, -, ×) together with a unary function ∂, the derivation. The axioms are those for fields together with For this theory one can add the condition that the characteristic is p, a prime or zero, to get the theory DFp of differential fields of characteristic p (and similarly with the other theories below). If K is a differential field then the field of constants The theory of differentially perfect fields is the theory of differential fields together with the condition that the field of constants is perfect; in other words for each prime p it has the axiom: (There is little point in demanding that the whole field should be perfect field, because in non-zero characteristic this implies the differential is 0.) For technical reasons to do with quantifier elimination it is sometimes more convenient to force the constant field to be perfect by adding a new symbol r to the signature with the axioms • The theory of DCF differentially closed fields is the theory of differentially perfect fields with axioms saying that such that if f and g are differential polynomials and the separant of f is nonzero and g≠0 and f has order greater than that of g, then there is some x in the field with f(x)=0 and g(x)≠0. Addition The theory of the natural numbers with a successor function has signature consisting of a constant 0 and a unary function S ("successor": S(x) is interpreted as x+1), and has axioms: 1. ∀x ¬ Sx = 0 2. ∀x∀y Sx = Sy → x = y 3. Let P(x) be a first-order formula with a single free variable x. Then the following formula is an axiom: (P(0) ∧ ∀x(P(x)→P(Sx))) → ∀y P(y). The last axiom (induction) can be replaced by the axioms • For each integer n>0, the axiom ∀x SSS...Sx ≠ x (with n copies of S) • ∀x ¬ x = 0 → ∃y Sy = x The theory of the natural numbers with a successor function is complete and decidable, and is κ-categorical for uncountable κ but not for countable κ. Presburger arithmetic is the theory of the natural numbers under addition, with signature consisting of a constant 0, a unary function S, and a binary function +. It is complete and decidable. The axioms are 1. ∀x ¬ Sx = 0 2. ∀x∀y Sx = Sy → x = y List of first-order theories 101 3. ∀x x + 0 = x 4. ∀x∀y x + Sy = S(x + y) 5. Let P(x) be a first-order formula with a single free variable x. Then the following formula is an axiom: (P(0) ∧ ∀x(P(x)→P(Sx))) → ∀y P(y). Arithmetic Many of the first order theories described above can be extended to complete recursively enumerable consistent theories. This is no longer true for most of the following theories; they can usually encode both multiplication and addition of natural numbers, and this gives them enough power to encode themselves, which implies that Gödel's incompleteness theorem applies and the theories can no longer be both complete and recursively enumerable (unless they are inconsistent). The signature of a theory of arithmetic has: • The constant 0; • The unary function, the successor function, here denoted by prefix S, or by prefix σ or postfix ′ elsewhere; • Two binary functions, denoted by infix + and ×, called "addition" and "multiplication." Some authors take the signature to contain a constant 1 instead of the function S, then define S in the obvious way as St = 1 + t. Robinson arithmetic (also called Q). Axioms (1) and (2) govern the distinguished element 0. (3) assures that S is an injection. Axioms (4) and (5) are the standard recursive definition of addition; (6) and (7) do the same for multiplication. Robinson arithmetic can be thought of as Peano arithmetic without induction. Q is a weak theory for which Gödel's incompleteness theorem holds. Axioms: 1. 2. 3. 4. 5. 6. 7. ∀x ¬ Sx = 0 ∀x ¬ x = 0 → ∃y Sy = x ∀x∀y Sx = Sy → x = y ∀x x + 0 = x ∀x∀y x + Sy = S(x + y) ∀x x × 0 = 0 ∀x∀y x × Sy = (x × y) + x. IΣn is first order Peano arithmetic with induction restricted to Σn formulas (for n = 0, 1, 2, ...). The theory IΣ0 is often denoted by IΔ0. This is a series of more and more powerful fragments of Peano arithmetic. The case n = 1 has about the same strength as primitive recursive arithmetic (PRA). Exponential function arithmetic (EFA) is IΣ0 with an axiom stating that xy exists for all x and y (with the usual properties). First order Peano arithmetic, PA. The "standard" theory of arithmetic. The axioms are the axioms of Robinson arithmetic above, together with the axiom scheme of induction: • for any formula φ in the language of PA. φ may contain free variables other than x. Kurt Gödel's 1931 paper proved that PA is incomplete, and has no consistent recursively enumerable completions. Complete arithmetic (also known as true arithmetic) is the theory of the standard model of arithmetic, the natural numbers N. It is complete but does not have a recursively enumerable set of axioms. List of first-order theories Second order arithmetic Second-order arithmetic can refer to a first order theory (in spite of the name) with two types of variables, thought of as varying over integers and subsets of the integers. (There is also a theory of arithmetic in second order logic that is called second order arithmetic. It has only one model, unlike the corresponding theory in first order logic, which is incomplete.) The signature will typically be the signature 0, S, +, × of arithmetic, together with a membership relation ∈ between integers and subsets (though there are numerous minor variations). The axioms are those of Robinson arithmetic, together with axiom schemes of induction and comprehension. There are many different subtheories of second order arithmetic that differ in which formulas are allowed in the induction and comprehension schemes. In order of increasing strength, five of the most common systems are • • • • • , Recursive Comprehension , Weak König's lemma , Arithmetical comprehension , Arithmetical Transfinite Recursion , comprehension These are defined in detail in the articles on second order arithmetic and reverse mathematics. Set theories The usual signature of set theory has one binary relation ∈, no constants, and no functions. Some of the theories below are "class theories" which have two sorts of object, sets and classes. There are three common ways of handling this in first-order logic: 1. Use first-order logic with two types. 2. Use ordinary first-order logic, but add a new unary predicate "Set", where "Set(t)" means informally "t is a set". 3. Use ordinary first-order logic, and instead of adding a new predicate to the language, treat "Set(t)" as an abbreviation for "∃y t∈y" Some first order set theories include: • Weak theories lacking powersets: • • • • • • • • • • S' (Tarski, Mostowksi, and Robinson, 1953); (finitely axiomatizable) • General set theory; • Kripke-Platek set theory; Zermelo set theory; Ackermann set theory Zermelo-Fraenkel set theory; Von Neumann-Bernays-Gödel set theory; (finitely axiomatizable) Morse–Kelley set theory; Tarski–Grothendieck set theory; New Foundations; (finitely axiomatizable) Scott-Potter set theory Positive set theory Some extra first order axioms that can be added to one of these (usually ZF) include: • axiom of choice, axiom of dependent choice • Generalized continuum hypothesis • Martin's axiom (usually together with the negation of the continuum hypothesis), Martin's maximum • ◊ and ♣ • Axiom of constructibility (V=L) 102 List of first-order theories • proper forcing axiom • analytic determinacy, projective determinacy, Axiom of determinacy • Many large cardinal axioms References • Ax, James; Kochen, Simon (1965), "Diophantine problems over local fields. II. A complete set of axioms for p-adic number theory.", Amer. J. Math. (The Johns Hopkins University Press) 87 (3): 631–648, doi:10.2307/2373066, JSTOR 2373066, MR0184931 • Chang, C.C.; Keisler, H. Jerome (1989), Model Theory (3 ed.), Elsevier, ISBN 0-7204-0692-7 • Hodges, Wilfrid (1997), A shorter model theory, Cambridge University Press, ISBN 0-521-58713-1 • Marker, David (2002), Model Theory: An Introduction, Graduate Texts in Mathematics, 217, Springer, ISBN 0-387-98760-6 Pole splitting Pole splitting is a phenomenon exploited in some forms of frequency compensation used in an electronic amplifier. When a capacitor is introduced between the input and output sides of the amplifier with the intention of moving the pole lowest in frequency (usually an input pole) to lower frequencies, pole splitting causes the pole next in frequency (usually an output pole) to move to a higher frequency. This pole movement increases the stability of the amplifier and improves its step response at the cost of decreased speed.[1] [2] [3] [4] Example of pole splitting This example shows that introduction of the capacitor referred to as CC in the amplifier of Figure 1 has two results: first it causes the lowest frequency pole of the amplifier to move still lower in frequency and second, it causes the higher pole to move higher in frequency.[5] The amplifier of Figure 1 has a low frequency pole due to the added input resistance Ri and capacitance Ci, with the time constant Ci ( RA // Ri ). This pole is moved Figure 1: Operational amplifier with compensation capacitor CC down in frequency by the Miller effect. The amplifier is between input and output; notice the amplifier has both input given a high frequency output pole by addition of the impedance Ri and output impedance Ro. load resistance RL and capacitance CL, with the time constant CL ( Ro // RL ). The upward movement of the high-frequency pole occurs because the Miller-amplified compensation capacitor CC alters the frequency dependence of the output voltage divider. The first objective, to show the lowest pole moves down in frequency, is established using the same approach as the Miller's theorem article. Following the procedure described in the article on Miller's theorem, the circuit of Figure 1 is transformed to that of Figure 103 Pole splitting 104 2, which is electrically equivalent to Figure 1. Application of Kirchhoff's current law to the input side of Figure 2 determines the input voltage to the ideal op amp as a function of the applied signal voltage , namely, Figure 2: Operational amplifier with compensation capacitor transformed using Miller's theorem to replace the compensation capacitor with a Miller capacitor at the input and a frequency-dependent current source at the output. which exhibits a roll-off with frequency beginning at f1 where which introduces notation for the time constant of the lowest pole. This frequency is lower than the initial low frequency of the amplifier, which for CC = 0 F is . Turning to the second objective, showing the higher pole moves still higher in frequency, it is necessary to look at the output side of the circuit, which contributes a second factor to the overall gain, and additional frequency dependence. The voltage is determined by the gain of the ideal op amp inside the amplifier as Using this relation and applying Kirchhoff's current law to the output side of the circuit determines the load voltage as a function of the voltage at the input to the ideal op amp as: This expression is combined with the gain factor found earlier for the input side of the circuit to obtain the overall gain as This gain formula appears to show a simple two-pole response with two time constants. (It also exhibits a zero in the numerator but, assuming the amplifier gain Av is large, this zero is important only at frequencies too high to matter in this discussion , so the numerator can be approximated as unity.) However, although the amplifier does have a two-pole behavior, the two time-constants are more complicated than the above expression suggests because the Miller capacitance contains a buried frequency dependence that has no importance at low frequencies, but has considerable effect at high frequencies. That is, assuming the output R-C product, CL ( Ro // RL ), corresponds to a frequency well above the low frequency pole, the accurate form of the Miller capacitance must be used, rather than Pole splitting 105 the Miller approximation. According to the article on Miller effect, the Miller capacitance is given by (For a positive Miller capacitance, Av is negative.) Upon substitution of this result into the gain expression and collecting terms, the gain is rewritten as: with Dω given by a quadratic in ω, namely: Every quadratic has two factors, and this expression looks simpler if it is rewritten as where and are combinations of the capacitances and resistances in the formula for Dω.[6] They correspond to the time constants of the two poles of the amplifier. One or the other time constant is the longest; suppose is the longest time constant, corresponding to the lowest pole, and suppose >> . (Good step response requires >> . See Selection of CC below.) At low frequencies near the lowest pole of this amplifier, ordinarily the linear term in ω is more important than the quadratic term, so the low frequency behavior of Dω is: where now CM is redefined using the Miller approximation as which is simply the previous Miller capacitance evaluated at low frequencies. On this basis is determined, provided >> . Because CM is large, the time constant is much larger than its original value of Ci ( RA // Ri ).[7] At high frequencies the quadratic term becomes important. Assuming the above result for is valid, the second time constant, the position of the high frequency pole, is found from the quadratic term in Dω as Substituting in this expression the quadratic coefficient corresponding to the product for , an estimate for the position of the second pole is found: along with the estimate and because CM is large, it seems is reduced in size from its original value CL ( Ro // RL ); that is, the higher pole has moved still higher in frequency because of CC.[8] In short, introduction of capacitor CC moved the low pole lower and the high pole higher, so the term pole splitting seems a good description. Pole splitting 106 Selection of CC What value is a good choice for CC? For general purpose use, traditional design (often called dominant-pole or single-pole compensation) requires the amplifier gain to drop at 20 dB/decade from the corner frequency down to 0 dB gain, or even lower.[9] [10] With this design the amplifier is stable and has near-optimal step response even as a unity gain voltage buffer. A more aggressive technique is two-pole compensation.[11] [12] Figure 3: Idealized Bode plot for a two pole amplifier design. Gain drops from first pole at f1 at 20 dB / decade down to second pole at f2 where the slope increases to 40 dB / decade. The way to position f2 to obtain the design is shown in Figure 3. At the lowest pole f1, the Bode gain plot breaks slope to fall at 20 dB/decade. The aim is to maintain the 20 dB/decade slope all the way down to zero dB, and taking the ratio of the desired drop in gain (in dB) of 20 log10 Av to the required change in frequency (on a log frequency scale[13] ) of ( log10 f2 − log10 f1 ) = log10 ( f2 / f1 ) the slope of the segment between f1 and f2 is: Slope per decade of frequency which is 20 dB/decade provided f2 = Av f1 . If f2 is not this large, the second break in the Bode plot that occurs at the second pole interrupts the plot before the gain drops to 0 dB with consequent lower stability and degraded step response. Figure 3 shows that to obtain the correct gain dependence on frequency, the second pole is at least a factor Av higher in frequency than the first pole. The gain is reduced a bit by the voltage dividers at the input and output of the amplifier, so with corrections to Av for the voltage dividers at input and output the pole-ratio condition for good step response becomes: Using the approximations for the time constants developed above, Figure 4: Miller capacitance at low frequencies CM (top) and compensation capacitor CC (bottom) as a function of gain using Excel. Capacitance units are pF. or Pole splitting 107 which provides a quadratic equation to determine an appropriate value for CC. Figure 4 shows an example using this equation. At low values of gain this example amplifier satisfies the pole-ratio condition without compensation (that is, in Figure 4 the compensation capacitor CC is small at low gain), but as gain increases, a compensation capacitance rapidly becomes necessary (that is, in Figure 4 the compensation capacitor CC increases rapidly with gain) because the necessary pole ratio increases with gain. For still larger gain, the necessary CC drops with increasing gain because the Miller amplification of CC, which increases with gain (see the Miller equation ), allows a smaller value for CC. To provide more safety margin for design uncertainties, often Av is increased to two or three times Av on the right side of this equation.[14] See Sansen[4] or Huijsing[10] and article on step response. Slew rate The above is a small-signal analysis. However, when large signals are used, the need to charge and discharge the compensation capacitor adversely affects the amplifier slew rate; in particular, the response to an input ramp signal is limited by the need to charge CC. References and notes [1] That is, the rise time is selected to be the fastest possible consistent with low overshoot and ringing. [2] C. Toumazu, Moschytz GS & Gilbert B (Editors) (2007). Trade-offs in analog circuit design: the designer's companion (http:/ / books. google. com/ books?id=VoBIOvirkiMC& pg=PA272& lpg=PA272& dq="pole+ splitting"& source=web& ots=MC083mOWhv& sig=duZQKaGECaAH80qDj-YNMdRd8nA). New York/Berlin/Dordrecht: Springer. pp. 272–275. ISBN 1402070373. . [3] Marc T. Thompson (2006). Intuitive analog circuit design: a problem-solving approach using design case studies (http:/ / books. google. com/ books?id=1Tyzjmf0DI8C& pg=PA200& dq=pole+ splitting+ analog+ amplifier& lr=& as_brr=0& sig=gmvG9dtlK48hcqpvf3NwwqcF2Hk). Amsterdam: Elsevier Newnes. pp. 200. ISBN 0750677864. . [4] Wally M. C. Sansen (2006). Analog design essentials (http:/ / worldcat. org/ isbn/ 0-387-25746-2). New York; Berlin: Springer. pp. §097, p. 266 et seq. ISBN 0-387-25746-2. . [5] Although this example appears very specific, the associated mathematical analysis is very much used in circuit design. [6] The sum of the time constants is the coefficient of the term linear in jω and the product of the time constants is the coefficient of the quadratic term in (jω)2. [7] The expression for differs a little from ( CM+Ci ) ( RA // Ri ) as found initially for f1, but the difference is minor assuming the load capacitance is not so large that it controls the low frequency response instead of the Miller capacitance. [8] As an aside, the higher the high-frequency pole is made in frequency, the more likely it becomes for a real amplifier that other poles (not considered in this analysis) play a part. [9] A.S. Sedra and K.C. Smith (2004). Microelectronic circuits (http:/ / worldcat. org/ isbn/ 0-19-514251-9) (Fifth Edition ed.). New York: Oxford University Press. pp. 849 and Example 8.6, p. 853. ISBN 0-19-514251-9. . [10] Huijsing, Johan H. (2001). Operational amplifiers: theory and design (http:/ / books. google. com/ books?id=tiuV_agzk_EC& pg=PA102& dq=isbn:0792372840& sig=d-oEw_n992coA6bU0h6gkoJzoUo#PPA206,M1). Boston, MA: Kluwer Academic. pp. §6.2, pp.205–206 and Figure 6.2.1. ISBN 0-7923-7284-0. . [11] Feucht, Dennis: Two-pole compensation (http:/ / www. analogzone. com/ col_0719. pdf) [12] Self, Douglas (2006). Audio power amplifier design handbook (http:/ / books. google. com/ books?id=BRQZppvawWwC& pg=PA191& lpg=PA191& dq="two+ pole+ compensation"& source=web& ots=qsxRG-z1Xl& sig=41uVzeYZW3vi3BndJORUNHNZqPY#PPA191,M1). Oxford: Newnes. pp. 191–193. ISBN 0750680725. . [13] That is, the frequency is plotted in powers of ten, as 1, 10, 102 etc. [14] A factor of two results in the maximally flat or Butterworth design for a two-pole amplifier. However, real amplifiers have more than two poles, and a factor greater than two often is necessary. Pole splitting 108 External links • Bode Plots (http://en.wikibooks.org/wiki/Circuit_Theory/Bode_Plots) in the Circuit Theory Wikibook • Bode Plots (http://en.wikibooks.org/wiki/Control_Systems/Bode_Plots) in the Control Systems Wikibook Damping ratio In engineering, the damping ratio is a dimensionless measure describing how oscillations in a system decay after a disturbance. Many systems exhibit oscillatory behavior when they are disturbed from their position of static equilibrium. A mass suspended from a spring, for example, might, if pulled and released, bounce up and down. On each bounce, the system is "trying" to return to its equilibrium position, but overshoots it. Sometimes losses (eg frictional) damp the system and can cause the oscillations to gradually decay in amplitude towards zero. The damping ratio is a measure of describing how rapidly the oscillations decay from one bounce to the next. The behaviour of oscillating systems is often of interest in a diverse range of disciplines that include control engineering, mechanical engineering, structural engineering and electrical engineering. The physical quantity that is oscillating varies greatly, and could be the swaying of a tall building in the wind, or the speed of an electric motor, but a normalised, or non-dimensionalised approach can be convenient in describing common aspects of behavior. Oscillation modes • Where the spring–mass system is completely lossless, the mass would oscillate indefinitely, with each bounce of equal height to the last. This hypothetical case is called undamped. • If the system contained high losses, for example if the spring–mass experiment were conducted in a viscous fluid, the mass could slowly return to its rest position without ever overshooting. This case is called overdamped. • Commonly, the mass tends to overshoot its starting position, and then return, overshooting again. With each overshoot, some energy in the system is dissipated, and the oscillations die towards zero. This case is called underdamped. Underdamped spring–mass system with ζ<1 • Between the overdamped and underdamped cases, there exists a certain level of damping at which the system will just fail to overshoot and will not make a single oscillation. The key difference between critical damping and overdamping is that, in critical damping, the system returns to equilibrium in the minimum amount of time. This case is called critical damping. Damping ratio 109 Definition The damping ratio is a parameter, usually denoted by ζ (zeta),[1] that characterizes the frequency response of a second order ordinary differential equation. It is particularly important in the study of control theory. It is also important in the harmonic oscillator. The damping ratio provides a mathematical means of expressing the level of damping in a system relative to critical damping. For a damped harmonic oscillator with mass m, damping coefficient c, and spring constant k, it can be defined as the ratio of the damping coefficient in the system's differential equation to the critical damping coefficient: The effect of varying damping ratio on a second-order system. where the system differential equation is and the corresponding critical damping coefficient is The damping ratio is dimensionless, being the ratio of two coefficients of identical units. Derivation Using the natural frequency of the simple harmonic oscillator and the definition of the damping ratio above, we can rewrite this as: This equation can be solved with the ansatz where C and s are both complex constants. That ansatz assumes a solution that is oscillatory and/or decaying exponentially. Using it in the ODE gives a condition on the frequency of the damped oscillations, • Overdamped:If s is a real number, then the solution is simply a decaying exponential with no oscillation. This case occurs for , and is referred to as overdamped. • Underdamped:If s is a complex number, then the solution is a decaying exponential combined with an oscillatory portion that looks like . This case occurs for , and is referred to as underdamped. (The case where case the solution looks like corresponds to the undamped simple harmonic oscillator, and in that , as expected.) Damping ratio • Critically damped:The case where 110 is the border between the overdamped and underdamped cases, and is referred to as critically damped. This turns out to be a desirable outcome in many cases where engineering design of a damped oscillator is required (e.g., a door closing mechanism). Q factor and decay rate The factors Q, damping ratio ζ, and exponential decay rate α are related such that[2] When a second-order system has (that is, when the system is underdamped), it has two complex conjugate poles that each have a real part of ; that is, the decay rate parameter represents the rate of exponential decay of the oscillations. A lower damping ratio implies a lower decay rate, and so very underdamped systems oscillate for long times.[3] For example, a high quality tuning fork, which has a very low damping ratio, has an oscillation that lasts a long time, decaying very slowly after being struck by a hammer. Logarithmic decrement The damping ratio is also related to the logarithmic decrement for underdamped vibrations via the relation This relation is only meaningful for underdamped systems because the logarithmic decrement is defined as the natural log of the ratio of any two successive amplitudes, and only underdamped systems exhibit oscillation. References [1] Alciatore, David G. (2007). Introduction to Mechatronics and Measurement Systems (3rd ed.). McGraw Hill. ISBN 978-0-07-296305-2. [2] William McC. Siebert. Circuits, Signals, and Systems. MIT Press. [3] Ming Rao and Haiming Qiu (1993). Process control engineering: a textbook for chemical, mechanical and electrical engineers (http:/ / books. google. com/ books?id=NOpmEHNRH98C& pg=PA96). CRC Press. p. 96. ISBN 9782881246289. . Second-order logic Second-order logic In logic and mathematics second-order logic is an extension of first-order logic, which itself is an extension of propositional logic.[1] Second-order logic is in turn extended by higher-order logic and type theory. First-order logic uses only variables that range over individuals (elements of the domain of discourse); second-order logic has these variables as well as additional variables that range over sets of individuals. For example, the second-order sentence says that for every set P of individuals and every individual x, either x is in P or it is not (this is the principle of bivalence). Second-order logic also includes variables quantifying over functions, and other variables as explained in the section Syntax below. Both first-order and second-order logic use the idea of a domain of discourse (often called simply the "domain" or the "universe"). The domain is a set of individual elements which can be quantified over. Expressive power Second-order logic is more expressive than first-order logic. For example, if the domain is the set of all real numbers, one can assert in first-order logic the existence of an additive inverse of each real number by writing ∀x ∃y (x + y = 0) but one needs second-order logic to assert the least-upper-bound property for sets of real numbers, which states that every bounded, nonempty set of real numbers has a supremum. If the domain is the set of all real numbers, the following second-order sentence expresses the least upper bound property: In second-order logic, it is possible to write formal sentences which say "the domain is finite" or "the domain is of countable cardinality." To say that the domain is finite, use the sentence that says that every surjective function from the domain to itself is injective. To say that the domain has countable cardinality, use the sentence that says that there is a bijection between every two infinite subsets of the domain. It follows from the compactness theorem and the upward Löwenheim–Skolem theorem that it is not possible to characterize finiteness or countability, respectively, in first-order logic. Syntax The syntax of second-order logic tells which expressions are well formed formulas. In addition to the syntax of first-order logic, second-order logic includes many new sorts (sometimes called types) of variables. These are: • A sort of variables that range over sets of individuals. If S is a variable of this sort and t is a first-order term then the expression t ∈ S (also written S(t) or St) is an atomic formula. Sets of individuals can also be viewed as unary relations on the domain. • For each natural number k there is a sort of variable that ranges over all k-ary relations on the individuals. If R is such a k-ary relation variable and t1,..., tk are first-order terms then the expression R(t1,...,tk) is an atomic formula. • For each natural number k there is a sort of variable that ranges over functions that take k elements of the domain and return a single element of the domain. If f is such a k-ary function symbol and t1,...,tk are first-order terms then the expression f(t1,...,tk) is a first-order term. For each of the sorts of variable just defined, it is permissible to build up formulas by using universal and/or existential quantifiers. Thus there are many sorts of quantifiers, two for each sort of variable. A sentence in second-order logic, as in first-order logic, is a well-formed formula with no free variables (of any sort). In monadic second-order logic (MSOL), only variables for subsets of the domain are added. The second-order logic with all the sorts of variables just described is sometimes called full second-order logic to distinguish it from the monadic version. 111 Second-order logic Just as in first-order logic, second-order logic may include non-logical symbols in a particular second-order language. These are restricted, however, in that all terms that they form must be either first-order terms (which can be substituted for a first-order variable) or second-order terms (which can be substituted for a second-order variable of an appropriate sort). Semantics The semantics of second-order logic establish the meaning of each sentence. Unlike first-order logic, which has only one standard semantics, there are two different semantics that are commonly used for second-order logic: standard semantics and Henkin semantics. In each of these semantics, the interpretations of the first-order quantifiers and the logical connectives are the same as in first-order logic. Only the ranges of quantifiers over second-order variables differ in the two types of semantics. In standard semantics, also called full semantics, the quantifiers range over all sets or functions of the appropriate sort. Thus once the domain of the first-order variables is established, the meaning of the remaining quantifiers is fixed. It is these semantics that give second-order logic its expressive power, and they will be assumed for the remainder of this article. In Henkin semantics, each sort of second-order variable has a particular domain of its own to range over, which may be a proper subset of all sets or functions of that sort. Leon Henkin (1950) defined these semantics and proved that Gödel's completeness theorem and compactness theorem, which hold for first-order logic, carry over to second-order logic with Henkin semantics. This is because Henkin semantics are almost identical to many-sorted first-order semantics, where additional sorts of variables are added to simulate the new variables of second-order logic. Second-order logic with Henkin semantics is not more expressive than first-order logic. Henkin semantics are commonly used in the study of second-order arithmetic. Deductive systems A deductive system for a logic is a set of inference rules and logical axioms that determine which sequences of formulas constitute valid proofs. Several deductive systems can be used for second-order logic, although none can be complete for the standard semantics (see below). Each of these systems is sound, which means any sentence they can be used to prove is logically valid in the appropriate semantics. The weakest deductive system that can be used consists of a standard deductive system for first-order logic (such as natural deduction) augmented with substitution rules for second-order terms.[2] This deductive system is commonly used in the study of second-order arithmetic. The deductive systems considered by Shapiro (1991) and Henkin (1950) add to the augmented first-order deductive scheme both comprehension axioms and choice axioms. These axioms are sound for standard second-order semantics. They are sound for Henkin semantics if only Henkin models that satisfy the comprehension and choice axioms are considered.[3] Non-reducibility to first-order logic One might attempt to reduce the second-order theory of the real numbers, with full second-order semantics, to the first-order theory in the following way. First expand the domain from the set of all real numbers to a two-sorted domain, with the second sort containing all sets of real numbers. Add a new binary predicate to the language: the membership relation. Then sentences that were second-order become first-order, with the formerly second-order quantifiers ranging over the second sort instead. This reduction can be attempted in a one-sorted theory by adding unary predicates that tell whether an element is a number or a set, and taking the domain to be the union of the set of real numbers and the power set of the real numbers. 112 Second-order logic But notice that the domain was asserted to include all sets of real numbers. That requirement cannot be reduced to a first-order sentence, as the Löwenheim-Skolem theorem shows. That theorem implies that there is some countably infinite subset of the real numbers, whose members we will call internal numbers, and some countably infinite collection of sets of internal numbers, whose members we will call "internal sets", such that the domain consisting of internal numbers and internal sets satisfies exactly the same first-order sentences satisfied as the domain of real-numbers-and-sets-of-real-numbers. In particular, it satisfies a sort of least-upper-bound axiom that says, in effect: Every nonempty internal set that has an internal upper bound has a least internal upper bound. Countability of the set of all internal numbers (in conjunction with the fact that those form a densely ordered set) implies that that set does not satisfy the full least-upper-bound axiom. Countability of the set of all internal sets implies that it is not the set of all subsets of the set of all internal numbers (since Cantor's theorem implies that the set of all subsets of a countably infinite set is an uncountably infinite set). This construction is closely related to Skolem's paradox. Thus the first-order theory of real numbers and sets of real numbers has many models, some of which are countable. The second-order theory of the real numbers has only one model, however. This follows from the classical theorem that there is only one Archimedean complete ordered field, along with the fact that all the axioms of an Archimedean complete ordered field are expressible in second-order logic. This shows that the second-order theory of the real numbers cannot be reduced to a first-order theory, in the sense that the second-order theory of the real numbers has only one model but the corresponding first-order theory has many models. There are more extreme examples showing that second-order logic with standard semantics is more expressive than first-order logic. There is a finite second-order theory whose only model is the real numbers if the continuum hypothesis holds and which has no model if the continuum hypothesis does not hold (cf. Shapiro 2000 p. 105). This theory consists of a finite theory characterizing the real numbers as a complete Archimedean ordered field plus an axiom saying that the domain is of the first uncountable cardinality. This example illustrates that the question of whether a sentence in second-order logic is consistent is extremely subtle. Additional limitations of second order logic are described in the next section. Metalogical results It is a corollary of Gödel's incompleteness theorem that there is no deductive system (that is, no notion of provability) for second-order formulas that simultaneously satisfies these three desired attributes:[4] • (Soundness) Every provable second-order sentence is universally valid, i.e., true in all domains under standard semantics. • (Completeness) Every universally valid second-order formula, under standard semantics, is provable. • (Effectiveness) There is a proof-checking algorithm that can correctly decide whether a given sequence of symbols is a valid proof or not. This corollary is sometimes expressed by saying that second-order logic does not admit a complete proof theory. In this respect second-order logic with standard semantics differs from first-order logic; Quine (1970, pp. 90–91) pointed to the lack of a complete proof system as a reason for thinking of second-order logic as not logic, properly speaking. As mentioned above, Henkin proved that the standard deductive system for first-order logic is sound, complete, and effective for second-order logic with Henkin semantics, and the deductive system with comprehension and choice principles is sound, complete, and effective for Henkin semantics using only models that satisfy these principles. 113 Second-order logic History and disputed value Predicate logic was primarily introduced to the mathematical community by C. S. Peirce, who coined the term second-order logic and whose notation is most similar to the modern form (Putnam 1982). However, today most students of logic are more familiar with the works of Frege, who actually published his work several years prior to Peirce but whose works remained in obscurity until Bertrand Russell and Alfred North Whitehead made them famous. Frege used different variables to distinguish quantification over objects from quantification over properties and sets; but he did not see himself as doing two different kinds of logic. After the discovery of Russell's paradox it was realized that something was wrong with his system. Eventually logicians found that restricting Frege's logic in various ways—to what is now called first-order logic—eliminated this problem: sets and properties cannot be quantified over in first-order-logic alone. The now-standard hierarchy of orders of logics dates from this time. It was found that set theory could be formulated as an axiomatized system within the apparatus of first-order logic (at the cost of several kinds of completeness, but nothing so bad as Russell's paradox), and this was done (see Zermelo-Fraenkel set theory), as sets are vital for mathematics. Arithmetic, mereology, and a variety of other powerful logical theories could be formulated axiomatically without appeal to any more logical apparatus than first-order quantification, and this, along with Gödel and Skolem's adherence to first-order logic, led to a general decline in work in second (or any higher) order logic. This rejection was actively advanced by some logicians, most notably W. V. Quine. Quine advanced the view that in predicate-language sentences like Fx the "x" is to be thought of as a variable or name denoting an object and hence can be quantified over, as in "For all things, it is the case that . . ." but the "F" is to be thought of as an abbreviation for an incomplete sentence, not the name of an object (not even of an abstract object like a property). For example, it might mean " . . . is a dog." But it makes no sense to think we can quantify over something like this. (Such a position is quite consistent with Frege's own arguments on the concept-object distinction). So to use a predicate as a variable is to have it occupy the place of a name which only individual variables should occupy. This reasoning has been rejected by Boolos. In recent years second-order logic has made something of a recovery, buoyed by George Boolos' interpretation of second-order quantification as plural quantification over the same domain of objects as first-order quantification (Boolos 1984). Boolos furthermore points to the claimed nonfirstorderizability of sentences such as "Some critics admire only each other" and "Some of Fianchetto's men went into the warehouse unaccompanied by anyone else" which he argues can only be expressed by the full force of second-order quantification. However, generalized quantification and partially ordered, or branching, quantification may suffice to express a certain class of purportedly nonfirstorderizable sentences as well and it does not appeal to second-order quantification. Applications to complexity The expressive power of various forms of second-order logic on finite structures is intimately tied to computational complexity theory. The field of descriptive complexity studies which computational complexity classes can be characterized by the power of the logic needed to express languages (sets of finite strings) in them. A string w = w1···wn in a finite alphabet A can be represented by a finite structure with domain D = {1,...,n}, unary predicates Pa for each a ∈ A, satisfied by those indices i such that wi = a, and additional predicates which serve to uniquely identify which index is which (typically, one takes the graph of the successor function on D or the order relation <, possibly with other arithmetic predicates). Conversely, the table of any finite structure can be encoded by a finite string. With this identification, we have the following characterizations of variants of second-order logic over finite structures: • NP is the set of languages definable by existential, second-order formulas (Fagin's theorem, 1974). • co-NP is the set of languages definable by universal, second-order formulas. 114 Second-order logic • PH is the set of languages definable by second-order formulas. • PSPACE is the set of languages definable by second-order formulas with an added transitive closure operator. • EXPTIME is the set of languages definable by second-order formulas with an added least fixed point operator. Relationships among these classes directly impact the relative expressiveness of the logics over finite structures; for example, if PH = PSPACE, then adding a transitive closure operator to second-order logic would not make it any more expressive over finite structures. Notes [1] [2] [3] [4] Shapiro (1991) and Hinman (2005) give complete introductions to the subject, with full definitions. Such a system is used without comment by Hinman (2005). These are the models originally studied by Henkin (1950). The proof of this corollary is that a sound, complete, and effective deduction system for standard semantics could be used to produce a recursively enumerable completion of Peano arithmetic, which Gödel's theorem shows cannot exist. References • Andrews, Peter (2002). An Introduction to Mathematical Logic and Type Theory: To Truth Through Proof (2nd ed.). Kluwer Academic Publishers. • Boolos, George (1984). "To Be Is To Be a Value of a Variable (or to Be Some Values of Some Variables)". Journal of Philosophy 81 (8): 430–50. doi:10.2307/2026308. JSTOR 2026308.. Reprinted in Boolos, Logic, Logic and Logic, 1998. • Henkin, L. (1950). "Completeness in the theory of types". Journal of Symbolic Logic 15 (2): 81–91. doi:10.2307/2266967. JSTOR 2266967. • Hinman, P. (2005). Fundamentals of Mathematical Logic. A K Peters. ISBN 1-56881-262-0. • Putnam, Hilary (1982). "Peirce the Logician". Historia Mathematica 9 (3): 290–301. doi:10.1016/0315-0860(82)90123-9.. Reprinted in Putnam, Hilary (1990), Realism with a Human Face, Harvard University Press, pp. 252–260. • W.V. Quine (1970). Philosophy of Logic. Prentice-Hall. • Rossberg, M. (2004). "First-Order Logic, Second-Order Logic, and Completeness" (http://homepages.uconn. edu/~mar08022/papers/RossbergCompleteness.pdf). In V. Hendricks et al., eds.. First-order logic revisited. Berlin: Logos-Verlag. • Shapiro, S. (2000). Foundations without Foundationalism: A Case for Second-order Logic. Oxford University Press. ISBN 0-19-825029-0. • Vaananen, J. (2001). "Second-Order Logic and Foundations of Mathematics" (http://www.math.ucla.edu/~asl/ bsl/0704/0704-003.ps). Bulletin of Symbolic Logic 7 (4): 504–520. doi:10.2307/2687796. JSTOR 2687796. 115 Impulse response Impulse response In signal processing, the impulse response, or impulse response function (IRF), of a dynamic system is its output when presented with a brief input signal, called an impulse. More generally, an impulse response refers to the reaction of any dynamic system in response to some external change. In both cases, the impulse response describes the reaction of the system as a function of time (or possibly as a function of some other independent variable that parameterizes the dynamic behavior of the system). The Impulse response from a simple audio system. Showing the original impulse, For example, the dynamic system might be a the response after high frequency boosting, and the response after low frequency planetary system in orbit around a star; the boosting. external influence in this case might be another massive object arriving from elsewhere in the galaxy; the impulse response is the change in the motion of the planetary system caused by interaction with the new object. In all these cases, the 'dynamic system' and its 'impulse response' may refer to actual physical objects, or to a mathematical system of equations describing these objects. Mathematical considerations Mathematically, how the impulse is described depends on whether the system is modeled in discrete or continuous time. The impulse can be modeled as a Dirac delta function for continuous-time systems, or as the Kronecker delta for discrete-time systems. The Dirac delta represents the limiting case of a pulse made very short in time while maintaining its area or integral (thus giving an infinitely high peak). While this is impossible in any real system, it is a useful idealisation. In Fourier analysis theory, such an impulse comprises equal portions of all possible excitation frequencies, which makes it a convenient test probe. Any system in a large class known as linear, time-invariant (LTI) is completely characterized by its impulse response. That is, for any input function, the output function can be calculated in terms of the input and the impulse response. (See LTI system theory.) The impulse response of a linear transformation is the image of Dirac's delta function under the transformation, analogous to the fundamental solution of a partial differential operator. The Laplace transform of the impulse response function is known as the transfer function. It is usually easier to analyze systems using transfer functions as opposed to impulse response functions. The Laplace transform of a system's output may be determined by the multiplication of the transfer function with the input function in the complex plane, also known as the frequency domain. An inverse Laplace transform of this result will yield the output function in the time domain. To determine an output function directly in the time domain requires the convolution of the input function with the impulse response function. This requires the use of integrals, and is usually more difficult than simply multiplying two functions in the frequency domain. The impulse response, considered as a Green's function, can be thought of as an "influence function:" how a point of input influences output. 116 Impulse response Practical applications In practical systems, it is not possible to produce a perfect impulse to serve as input for testing; therefore, a brief pulse is sometimes used as an approximation of an impulse. Provided that the pulse is short enough compared to the impulse response, the result will be close to the true, theoretical, impulse response. In many systems, however, driving with a very short strong pulse may drive the system into a nonlinear regime, so instead the system is driven with a pseudo-random sequence, and the impulse response is computed from the input and output signals.[1] Loudspeakers An application that demonstrates this idea was the development of impulse response loudspeaker testing in the 1970s. Loudspeakers suffer from phase inaccuracy, a defect unlike other measured properties such as frequency response. Phase inaccuracy is caused by (slightly) delayed frequencies/octaves that are mainly the result of passive cross overs (especially higher order filters) but are also caused by resonance, energy storage in the cone, the internal volume, or the enclosure panels vibrating. Measuring the impulse response, which is a direct plot of this "time-smearing," provided a tool for use in reducing resonances by the use of improved materials for cones and enclosures, as well as changes to the speaker crossover. The need to limit input amplitude to maintain the linearity of the system led to the use of inputs such as pseudo-random maximum length sequences, and to the use of computer processing to derive the impulse response.[2] Digital filtering Impulse response is a very important concept in the design of digital filters for audio processing, because digital filters can differ from 'real' filters in often having a pre-echo, which the ear is not accustomed to. Electronic processing Impulse response analysis is a major facet of radar, ultrasound imaging, and many areas of digital signal processing. An interesting example would be broadband internet connections. DSL/Broadband services use adaptive equalisation techniques to help compensate for signal distortion and interference introduced by the copper phone lines used to deliver the service. Control systems In control theory the impulse response is the response of a system to a Dirac delta input. This proves useful in the analysis of dynamic systems: the Laplace transform of the delta function is 1, so the impulse response is equivalent to the inverse Laplace transform of the system's transfer function. Acoustic and audio applications In acoustic and audio applications, impulse responses enable the acoustic characteristics of a location, such as a concert hall, to be captured. Various commercial packages are available containing impulse responses from specific locations, ranging from small rooms to large concert halls. These impulse responses can then be utilized in convolution reverb applications to enable the acoustic characteristics of a particular location to be applied to target audio.[3] 117 Impulse response 118 Economics In economics, and especially in contemporary macroeconomic modeling, impulse response functions describe how the economy reacts over time to exogenous impulses, which economists usually call 'shocks', and are often modeled in the context of a vector autoregression. Impulses that are often treated as exogenous from a macroeconomic point of view include changes in government spending, tax rates, and other fiscal policy parameters; changes in the monetary base or other monetary policy parameters; changes in productivity or other technological parameters; and changes in preferences, such as the degree of impatience. Impulse response functions describe the reaction of endogenous macroeconomic variables such as output, consumption, investment, and employment at the time of the shock and over subsequent points in time.[4] [5] References [1] F. Alton Everest (2000). Master Handbook of Acoustics (http:/ / books. google. com/ books?id=sgwg1Vwm9VUC& pg=RA1-PA510& dq="impulse+ response"+ loudspeaker+ testing) (Fourth ed. ed.). McGraw-Hill Professional. ISBN 0071360972. . [2] Monitor: Proceedings of the Institution of Radio and Electronics Engineers Australia (1976) (http:/ / books. google. com/ books?id=q6w7AAAAMAAJ& q="impulse+ response"+ "loudspeaker+ testing"& dq="impulse+ response"+ "loudspeaker+ testing"& pgis=1) [3] http:/ / www. acoustics. hut. fi/ projects/ poririrs/ the Concert Hall Impulse Responses from Pori, Finland [4] Helmut Lütkepohl (2008), 'Impulse response function'. The New Palgrave Dictionary of Economics, 2nd. ed. [5] James Hamilton (1994), Time Series Analysis, Chapter 1, page 5. Princeton University Press. LTI system theory Linear time-invariant system theory, commonly known as LTI system theory, comes from applied mathematics and has direct applications in NMR spectroscopy, seismology, circuits, signal processing, control theory, and other technical areas. It investigates the response of a linear and time-invariant system to an arbitrary input signal. Trajectories of these systems are commonly measured and tracked as they move through time (e.g., an acoustic waveform), but in applications like image processing and field theory, the LTI systems also have trajectories in spatial dimensions. Thus these systems are also called linear translation-invariant to give the theory the most general reach. In the case of generic discrete-time (i.e., sampled) systems, linear shift-invariant is the corresponding term. A good example of LTI systems are electrical circuits that can be made up of resistors, capacitors and inductors.[1] Overview The defining properties of any LTI system are linearity and time invariance. • Linearity means that the relationship between the input and the output of the system is a linear map: If input produces response and input produces response then the scaled and summed input produces the scaled and summed response where and scalars. It follows that this can be extended to an arbitrary number of terms, and so for real numbers , Input produces output In particular, Input produces output (Eq.1) are real LTI system theory 119 where and are scalars and inputs that vary over a continuum indexed by . Thus if an input function can be represented by a continuum of input functions, combined "linearly", as shown, then the corresponding output function can be represented by the corresponding continuum of output functions, scaled and summed in the same way. • Time invariance means that whether we apply an input to the system now or T seconds from now, the output will be identical except for a time delay of the T seconds. That is, if the output due to input is , then the output due to input is . Hence, the system is time invariant because the output does not depend on the particular time the input is applied. The fundamental result in LTI system theory is that any LTI system can be characterized entirely by a single function called the system's impulse response. The output of the system is simply the convolution of the input to the system with the system's impulse response. This method of analysis is often called the time domain point-of-view. The same result is true of discrete-time linear shift-invariant systems in which signals are discrete-time samples, and convolution is defined on sequences. Equivalently, any LTI system can be characterized in the frequency domain by the system's transfer function, which is the Laplace transform of the system's impulse response (or Z transform in the case of discrete-time systems). As a result of the properties of these transforms, the output of the system in the frequency domain is the product of the transfer function and the transform of the input. In other words, convolution in the time domain is equivalent to multiplication in the frequency domain. For all LTI systems, the eigenfunctions, Relationship between the time domain and the frequency domain and the basis functions of the transforms, are complex exponentials. This is, if the input to a system is the complex waveform for some complex amplitude and complex frequency , the output will be some complex constant times the input, say . The ratio is the transfer function at frequency for some new complex amplitude . Because sinusoids are a sum of complex exponentials with complex-conjugate frequencies, if the input to the system is a sinusoid, then the output of the system will also be a sinusoid, perhaps with a different amplitude and a different phase, but always with the same frequency. LTI systems cannot produce frequency components that are not in the input. LTI system theory is good at describing many important systems. Most LTI systems are considered "easy" to analyze, at least compared to the time-varying and/or nonlinear case. Any system that can be modeled as a linear homogeneous differential equation with constant coefficients is an LTI system. Examples of such systems are electrical circuits made up of resistors, inductors, and capacitors (RLC circuits). Ideal spring–mass–damper systems are also LTI systems, and are mathematically equivalent to RLC circuits. Most LTI system concepts are similar between the continuous-time and discrete-time (linear shift-invariant) cases. In image processing, the time variable is replaced with two space variables, and the notion of time invariance is replaced by two-dimensional shift invariance. When analyzing filter banks and MIMO systems, it is often useful to LTI system theory 120 consider vectors of signals. A linear system that is not time-invariant can be solved using other approaches such as the Green function method. The same method must be used when the initial conditions of the problem are not null. Continuous-time systems Impulse response and convolution The behavior of a linear, continuous-time, time-invariant system with input signal x(t) and output signal y(t) is described by the convolution integral,[2] : (using commutativity) where is the system's response to an impulse: of the input function The weighting function is is therefore proportional to a weighted average simply shifted by amount weighting function emphasizes different parts of the input function. When depends only on values of prior to time As is zero for all negative with variable and the system is said to be causal. To understand why the convolution produces the output of an LTI system, let the notation function changes, the and constant And let the shorter notation continuous-time system transforms an input function, into an output function, represent represent the Then a And in general, every value of the output can depend on every value of the input. This concept is represented by: where is the transformation operator for time that occurred near time system is uninteresting. For a linear system, In a typical system, Unless the transform itself changes with depends most heavily on the values of the output function is just constant, and the must satisfy Eq.1: (Eq.2) And the time-invariance requirement is: (Eq.3) In this notation, we can write the impulse response as Similarly: (using Eq.3) Substituting this result into the convolution integral: LTI system theory 121 which has the form of the right side of Eq.2 for the case and Eq.2 then allows this continuation: In summary, the input function, can be represented by a continuum of time-shifted impulse functions, combined "linearly", as shown at Eq.1. The system's linearity property allows the system's response to be represented by the corresponding continuum of impulse responses, combined in the same way. And the time-invariance property allows that combination to be represented by the convolution integral. The mathematical operations above have a simple graphical simulation.[3] Exponentials as eigenfunctions An eigenfunction is a function for which the output of the operator is a scaled version of the same function. That is, , where f is the eigenfunction and The exponential functions is the eigenvalue, a constant. , where , are eigenfunctions of a linear, time-invariant operator. A simple proof illustrates this concept. Suppose the input is response . The output of the system with impulse is then which, by the commutative property of convolution, is equivalent to where the scalar is dependent only on the parameter s. So the system's response is a scaled version of the input. In particular, for any product of the input and the constant corresponding eigenvalue is . . Hence, , the system output is the is an eigenfunction of an LTI system, and the LTI system theory 122 Fourier and Laplace transforms The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The Laplace transform is exactly the way to get the eigenvalues from the impulse response. Of particular interest are pure sinusoids (i.e., exponential functions of the form where and ). These are generally called complex exponentials even though the argument is purely imaginary. The Fourier transform gives the eigenvalues for pure complex sinusoids. Both of and are called the system function, system response, or transfer function. The Laplace transform is usually used in the context of one-sided signals, i.e. signals that are zero for all values of t less than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality, with the transform integral being taken from zero to infinity (the transform shown above with lower limit of integration of negative infinity is formally known as the bilateral Laplace transform). The Fourier transform is used for analyzing systems that process signals that are infinite in extent, such as modulated sinusoids, even though it cannot be directly applied to input and output signals that are not square integrable. The Laplace transform actually works directly for these signals if they are zero before a start time, even if they are not square integrable, for stable systems. The Fourier transform is often applied to spectra of infinite signals via the Wiener–Khinchin theorem even when Fourier transforms of the signals do not exist. Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain, given signals for which the transforms exist Not only is it often easier to do the transforms, multiplication, and inverse transform than the original convolution, but one can also gain insight into the behavior of the system from the system response. One can look at the modulus of the system function |H(s)| to see whether the input is passed (let through) the system or rejected or attenuated by the system (not let through). Examples • A simple example of an LTI operator is the derivative. • • (i.e., it is linear) (i.e., it is time invariant) When the Laplace transform of the derivative is taken, it transforms to a simple multiplication by the Laplace variable s. That the derivative has such a simple Laplace transform partly explains the utility of the transform. • Another simple LTI operator is an averaging operator By the linearity of integration, LTI system theory 123 it is linear. Additionally, because it is time invariant. In fact, can be written as a convolution with the boxcar function . That is, where the boxcar function Important system properties Some of the most important properties of a system are causality and stability. Causality is a necessity if the independent variable is time, but not all systems have time as an independent variable. For example, a system that processes still images does not need to be causal. Non-causal systems can be built and can be useful in many circumstances. Even non-real systems can be built and are very useful in many contexts. Causality A system is causal if the output depends only on present and past inputs. A necessary and sufficient condition for causality is where is the impulse response. It is not possible in general to determine causality from the Laplace transform, because the inverse transform is not unique. When a region of convergence is specified, then causality can be determined. Stability A system is bounded-input, bounded-output stable (BIBO stable) if, for every bounded input, the output is finite. Mathematically, if every input satisfying leads to an output satisfying (that is, a finite maximum absolute value of implies a finite maximum absolute value of system is stable. A necessary and sufficient condition is that norm): ), then the 1 , the impulse response, is in L (has a finite L1 LTI system theory 124 In the frequency domain, the region of convergence must contain the imaginary axis . As an example, the ideal low-pass filter with impulse response equal to a sinc function is not BIBO stable, because the sinc function does not have a finite L1 norm. Thus, for some bounded input, the output of the ideal low-pass filter is unbounded. In particular, if the input is zero for and equal to a sinusoid at the cut-off frequency for , then the output will be unbounded for all times other than the zero crossings. Discrete-time systems Almost everything in continuous-time systems has a counterpart in discrete-time systems. Discrete-time systems from continuous-time systems In many contexts, a discrete time (DT) system is really part of a larger continuous time (CT) system. For example, a digital recording system takes an analog sound, digitizes it, possibly processes the digital signals, and plays back an analog sound for people to listen to. Formally, the DT signals studied are almost always uniformly sampled versions of CT signals. If is a CT signal, then an analog to digital converter will transform it to the DT signal: where T is the sampling period. It is very important to limit the range of frequencies in the input signal for faithful representation in the DT signal, since then the sampling theorem guarantees that no information about the CT signal is lost. A DT signal can only contain a frequency range of ; other frequencies are aliased to the same range. Impulse response and convolution Let represent the sequence And let the shorter notation . represent A discrete system transforms an input sequence, into an output sequence, In general, every element of the output can depend on every element of the input. Representing the transformation operator by , we can write: Note that unless the transform itself changes with n, the output sequence is just constant, and the system is uninteresting. (Thus the subscript, n.) In a typical system, y[n] depends most heavily on the elements of x whose indices are near n. For the special case of the Kronecker delta function, For a linear system, the output sequence is the impulse response: must satisfy: (Eq.4) And the time-invariance requirement is: (Eq.5) In such a system, the impulse response, characterizes the system completely. I.e., for any input sequence, the output sequence can be calculated in terms of the input and the impulse response. To see how that is done, consider the identity: LTI system theory 125 which expresses in terms of a sum of weighted delta functions. Therefore: where we have invoked Eq.4 for the case and And because of Eq.5, we may write: Therefore: (commutativity) which is the familiar discrete convolution formula. The operator can therefore be interpreted as proportional to a weighted average of the function x[k]. The weighting function is h[-k], simply shifted by amount n. As n changes, the weighting function emphasizes different parts of the input function. Equivalently, the system's response to an impulse at n=0 is a "time" reversed copy of the unshifted weighting function. When h[k] is zero for all negative k, the system is said to be causal. Exponentials as eigenfunctions An eigenfunction is a function for which the output of the operator is the same function, just scaled by some amount. In symbols, , where f is the eigenfunction and is the eigenvalue, a constant. The exponential functions , where is the sampling interval, and Suppose the input is , are eigenfunctions of a linear, time-invariant operator. . A simple proof illustrates this concept. . The output of the system with impulse response which is equivalent to the following by the commutative property of convolution where is dependent only on the parameter z. is then LTI system theory So 126 is an eigenfunction of an LTI system because the system response is the same as the input times the constant . Z and discrete-time Fourier transforms The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The Z transform is exactly the way to get the eigenvalues from the impulse response. Of particular interest are pure sinusoids, i.e. exponentials of the form , where . These can also be written as with . These are generally called complex exponentials even though the argument is purely imaginary. The Discrete-time Fourier transform (DTFT) gives the eigenvalues of pure sinusoids. Both of and are called the system function, system response, or transfer function'. The Z transform is usually used in the context of one-sided signals, i.e. signals that are zero for all values of t less than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality. The Fourier transform is used for analyzing signals that are infinite in extent. Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain. That is, Just as with the Laplace transform transfer function in continuous-time system analysis, the Z transform makes it easier to analyze systems and gain insight into their behavior. One can look at the modulus of the system function |H(z)| to see whether the input is passed (let through) by the system, or rejected or attenuated by the system (not let through). Examples • A simple example of an LTI operator is the delay operator • • . (i.e., it is linear) (i.e., it is time invariant) The Z transform of the delay operator is a simple multiplication by z-1. That is, • Another simple LTI operator is the averaging operator . Because of the linearity of sums, and so it is linear. Because, LTI system theory 127 it is also time invariant. Important system properties The input-output characteristics of discrete-time LTI system are completely described by its impulse response . Some of the most important properties of a system are causality and stability. Unlike CT systems, non-causal DT systems can be realized. It is trivial to make an acausal FIR system causal by adding delays. It is even possible to make acausal IIR systems.[4] Non-stable systems can be built and can be useful in many circumstances. Even non-real systems can be built and are very useful in many contexts. Causality A discrete-time LTI system is causal if the current value of the output depends on only the current value and past values of the input.,[5] A necessary and sufficient condition for causality is where is the impulse response. It is not possible in general to determine causality from the Z transform, because the inverse transform is not unique. When a region of convergence is specified, then causality can be determined. Stability A system is bounded input, bounded output stable (BIBO stable) if, for every bounded input, the output is finite. Mathematically, if implies that (that is, if bounded input implies bounded output, in the sense that the maximum absolute values of are finite), then the system is stable. A necessary and sufficient condition is that and , the impulse response, satisfies In the frequency domain, the region of convergence must contain the unit circle (i.e., the locus satisfying for complex z). LTI system theory Notes [1] [2] [3] [4] [5] Hespanha 2009, p. 78. Crutchfield[web] Crutchfield Vaidyanathan,1995 Phillips 2007, p. 508. References • Phillips, C.l., Parr, J.M., & Riskin, E.A (2007). Signals, systems and Transforms. Prentice Hall. ISBN 0-13-041207-4. • Hespanha,J.P. (2009). Linear System Theory. Princeton university press. ISBN 0691140219. • Crutchfield, Steve (October 12, 2010), "The Joy of Convolution" (http://www.jhu.edu/signals/convolve/ index.html), Johns Hopkins University, retrieved November 21, 2010 • Vaidyanathan, P. P.; Chen, T. (May 1995). "Role of anticausal inverses in multirate filter banks — Part I: system theoretic fundamentals". IEEE Trans. Signal Proc. 43 (5): 1090. doi:10.1109/78.382395. Further reading • Porat, Boaz (1997). A Course in Digital Signal Processing. New York: John Wiley. ISBN 9780471149613. • Vaidyanathan, P. P.; Chen, T. (May 1995). "Role of anticausal inverses in multirate filter banks — Part I: system theoretic fundamentals". IEEE Trans. Signal Proc. 43 (5): 1090. doi:10.1109/78.382395. External links • ECE 209: Review of Circuits as LTI Systems (http://www.tedpavlic.com/teaching/osu/ece209/support/ circuits_sys_review.pdf) – Short primer on the mathematical analysis of (electrical) LTI systems. • ECE 209: Sources of Phase Shift (http://www.tedpavlic.com/teaching/osu/ece209/lab3_opamp_FO/ lab3_opamp_FO_phase_shift.pdf) – Gives an intuitive explanation of the source of phase shift in two common electrical LTI systems. • JHU 520.214 Signals and Systems course notes (http://www.ece.jhu.edu/~cooper/courses/214/ signalsandsystemsnotes.pdf). An encapsulated course on LTI system theory. Adequate for self teaching. 128 RouthHurwitz stability criterion 129 Routh–Hurwitz stability criterion The Routh–Hurwitz stability criterion is a necessary and sufficient method to establish the stability of a single-input, single-output (SISO), linear time invariant (LTI) control system. More generally, given a polynomial, some calculations using only the coefficients of that polynomial can lead to the conclusion that it is not stable. For the discrete case, see the Jury test equivalent. The criterion establishes a systematic way to show that the linearized equations of motion of a system have only stable solutions exp(pt), that is where all p have negative real parts. It can be performed using either polynomial divisions or determinant calculus. The criterion is derived through the use of the Euclidean algorithm and Sturm's theorem in evaluating Cauchy indices. Using Euclid's algorithm The criterion is related to Routh–Hurwitz theorem. Indeed, from the statement of that theorem, we have where: • p is the number of roots of the polynomial ƒ(z) with negative Real Part; • q is the number of roots of the polynomial ƒ(z) with positive Real Part (let us remind ourselves that ƒ is supposed to have no roots lying on the imaginary line); • w(x) is the number of variations of the generalized Sturm chain obtained from and (by successive Euclidean divisions) where for a real y. By the fundamental theorem of algebra, each polynomial of degree n must have n roots in the complex plane (i.e., for an ƒ with no roots on the imaginary line, p + q = n). Thus, we have the condition that ƒ is a (Hurwitz) stable polynomial if and only if p − q = n (the proof is given below). Using the Routh–Hurwitz theorem, we can replace the condition on p and q by a condition on the generalized Sturm chain, which will give in turn a condition on the coefficients of ƒ. Using matrices Let f(z) be a complex polynomial. The process is as follows: 1. 2. 3. 4. 5. Compute the polynomials and such that where y is a real number. Compute the Sylvester matrix associated to and . Rearrange each row in such a way that an odd row and the following one have the same number of leading zeros. Compute each principal minor of that matrix. If at least one of the minors is negative (or zero), then the polynomial f is not stable. Example • Let (for the sake of simplicity we take real coefficients) where (to avoid a root in zero so that we can use the Routh–Hurwitz theorem). First, we have to calculate the real polynomials and : Next, we divide those polynomials to obtain the generalized Sturm chain: • • yields yields and the Euclidean division stops. Notice that we had to suppose b different from zero in the first division. The generalized Sturm chain is in this case . Putting , the sign of is the opposite sign of RouthHurwitz stability criterion 130 a and the sign of by is the sign of b. When we put , the sign of the first element of the chain is again the opposite sign of a and the sign of by is the opposite sign of b. Finally, -c has always the opposite sign of c. Suppose now that f is Hurwitz-stable. This means that properties of the function w, this is the same as (the degree of f). By the and . Thus, a, b and c must have the same sign. We have thus found the necessary condition of stability for polynomials of degree 2. Routh–Hurwitz criterion for second, third, and fourth-order polynomials • For a second-order polynomial, , all the roots are in the left half plane (and the system with characteristic equation • For a third-order polynomial is stable) if all the coefficients satisfy . , all the coefficients must satisfy , and • For a fourth-order polynomial satisfy , and , all the coefficients must and Higher-order example A tabular method can be used to determine the stability when the roots of a higher order characteristic polynomial are difficult to obtain. For an nth-degree polynomial • the table has n + 1 rows and the following structure: where the elements and can be computed as follows: • • When completed, the number of sign changes in the first column will be the number of non-negative poles. Consider a system with a characteristic polynomial • We have the following table: RouthHurwitz stability criterion 131 1 2 3 0 4 5 6 0 0.75 1.5 0 0 −3 6 3 0 6 0 0 In the first column, there are two sign changes (0.75 → −3, and −3 → 3), thus there are two non-negative roots where the system is unstable. " Sometimes the presence of poles on the imaginary axis creates a situation of marginal stability. In that case the coefficients of the "Routh Array" become zero and thus further solution of the polynomial for finding changes in sign is not possible. Then another approach comes into play. The row of polynomial which is just above the row containing the zeroes is called "Auxiliary Polynomial". • We have the following table: 1 8 20 16 2 12 16 0 2 12 16 0 0 0 In such a case the Auxiliary polynomial is 0 0 which is again equal to zero. The next step is to differentiate the above equation which yields the following polynomial. . The coefficients of the row containing zero now become "8" and "24". The process of Routh array is proceeded using these values which yield two points on the imaginary axis. These two points on the imaginary axis are the prime cause of marginal stability.[1] References [1] Saeed, Syed Hasan (2008). Automatic Control Systems. Delhi: Katson Publishers. pp. 206, 207. ISBN 978-81-906919-2-5. • Hurwitz, A. (1964). "‘On the conditions under which an equation has only roots with negative real parts". Selected Papers on Mathematical Trends in Control Theory. • Routh, E. J. (1877). A Treatise on the Stability of a Given State of Motion: Particularly Steady Motion. Macmillan and co.. • Gantmacher, F. R. (1959). "Applications of the Theory of Matrices". Interscience, New York 641 (9): 1–8. • Pippard, A. B.; Dicke, R. H. (1986). "Response and Stability, An Introduction to the Physical Theory" (http:// link.aip.org/link/?AJPIAS/54/1052/1). American Journal of Physics 54: 1052. doi:10.1119/1.14826. Retrieved 2008-05-07. • Weisstein, Eric W. "Routh-Hurwitz Theorem." (http://mathworld.wolfram.com/Routh-HurwitzTheorem. html). MathWorld--A Wolfram Web Resource. • Richard C. Dorf, Robert H. Bishop (2001). Modern Control Systems, 9th Edition. Prentice Hall. ISBN 0-13-030660-6. RouthHurwitz stability criterion External links • A MATLAB script implementing the Routh-Hurwitz test (http://www.mathworks.com/matlabcentral/ fileexchange/25956-routh-hurwitz-stability-test) 132 133 Feedback properties Control theory Control theory is an interdisciplinary branch of engineering and mathematics that deals with the behavior of dynamical systems. The external input of a system is called the reference. When one or more output variables of The concept of the feedback loop to control the dynamic behavior of the system: this is a system need to follow a certain negative feedback, because the sensed value is subtracted from the desired value to create reference over time, a controller the error signal, which is amplified by the controller. manipulates the inputs to a system to obtain the desired effect on the output of the system. The usual objective of control theory is to calculate solutions for the proper corrective action from the controller that result in system stability, that is, the system will hold the set point and not oscillate around it. The input and ouput of the system are related to each other by what is known as a transfer function (also known as the system function or network function). The transfer function is a mathematical representation, in terms of spatial or temporal frequency, of the relation between the input and output of a linear time-invariant system. Extensive use is usually made of a diagrammatic style known as the block diagram. Overview Control theory is • a theory that deals with influencing the behavior of dynamical systems • an interdisciplinary subfield of science, which originated in engineering and mathematics, and evolved into use by the social sciences, like psychology, sociology, criminology and in financial system. Control systems can be thought of as having four functions; Measure, Compare, Compute, and Correct. These four functions are completed by five elements; Detector, Transducer, Transmitter, Controller, and Final Control Element. The measuring function is completed by the detector, transducer and transmitter. In practical applications these three elements are typically contained in one unit. A standard example is a Resistance thermometer. The compare and compute functions are completed within the controller which may be completed electronically through a Proportional Control, PI Controller, PID Controller, Bistable, Hysteretic control or Programmable logic controller. The correct function is completed with a final control element. The final control element changes an input or output in the control system which affect the manipulated or controlled variable. Control theory 134 An example Consider a car's cruise control, which is a device designed to maintain vehicle speed at a constant desired or reference speed provided by the driver. The controller is the cruise control, the plant is the car, and the system is the car and the cruise control. The system output is the car's speed, and the control itself is the engine's throttle position which determines how much power the engine generates. A primitive way to implement cruise control is simply to lock the throttle position when the driver engages cruise control. However, if the cruise control is engaged on a stretch of flat road, then the car will travel slower going uphill and faster when going downhill. This type of controller is called an open-loop controller because no measurement of the system output (the car's speed) is used to alter the control (the throttle position.) As a result, the controller can not compensate for changes acting on the car, like a change in the slope of the road. In a closed-loop control system, a sensor monitors the system output (the car's speed) and feeds the data to a controller which adjusts the control (the throttle position) as necessary to maintain the desired system output (match the car's speed to the reference speed.) Now when the car goes uphill the decrease in speed is measured, and the throttle position changed to increase engine power, speeding the vehicle. Feedback from measuring the car's speed has allowed the controller to dynamically compensate for changes to the car's speed. It is from this feedback that the paradigm of the control loop arises: the control affects the system output, which in turn is measured and looped back to alter the control. History Although control systems of various types date back to antiquity, a more formal analysis of the field began with a dynamics analysis of the centrifugal governor, conducted by the physicist James Clerk Maxwell in 1868 entitled On Governors.[1] This described and analyzed the phenomenon of "hunting", in which lags in the system can lead to overcompensation and unstable behavior. This generated a flurry of interest in the topic, during which Maxwell's classmate Edward John Routh generalized the results of Maxwell for the general class of linear systems.[2] Independently, Adolf Hurwitz analyzed system stability using differential equations in 1877, resulting in what is now known as the Routh-Hurwitz theorem.[3] [4] A notable application of dynamic control was in the area of manned flight. The Wright brothers made their first successful test flights on December 17, 1903 and were distinguished by their ability to control their flights for substantial periods (more so than the ability to produce lift from an airfoil, which was known). Control of the airplane was necessary for safe flight. Centrifugal governor in a Boulton & Watt engine of 1788 By World War II, control theory was an important part of fire-control systems, guidance systems and electronics. Sometimes mechanical methods are used to improve the stability of systems. For example, ship stabilizers are fins mounted beneath the waterline and emerging laterally. In contemporary vessels, they may be gyroscopically controlled active fins, which have the capacity to change their angle of attack to counteract roll caused by wind or waves acting on the ship. The Sidewinder missile uses small control surfaces placed at the rear of the missile with spinning disks on their outer surface; these are known as rollerons. Airflow over the disk spins them to a high speed. If the missile starts to roll, the gyroscopic force of the disk drives the control surface into the airflow, cancelling the motion. Thus the Control theory Sidewinder team replaced a potentially complex control system with a simple mechanical solution. The Space Race also depended on accurate spacecraft control. However, control theory also saw an increasing use in fields such as economics. People in systems and control Many active and historical figures made significant contribution to control theory, including, for example: • Alexander Lyapunov (1857–1918) in the 1890s marks the beginning of stability theory. • Harold S. Black (1898–1983), invented the concept of negative feedback amplifiers in 1927. He managed to develop stable negative feedback amplifiers in the 1930s. • Harry Nyquist (1889–1976), developed the Nyquist stability criterion for feedback systems in the 1930s. • Richard Bellman (1920–1984), developed dynamic programming since the 1940s. • Andrey Kolmogorov (1903–1987) co-developed the Wiener-Kolmogorov filter (1941). • Norbert Wiener (1894–1964) co-developed the Wiener-Kolmogorov filter and coined the term cybernetics in the 1940s. • John R. Ragazzini (1912–1988) introduced digital control and the z-transform in the 1950s. • Lev Pontryagin (1908–1988) introduced the maximum principle and the bang-bang principle. Classical control theory To avoid the problems of the open-loop controller, control theory introduces feedback. A closed-loop controller uses feedback to control states or outputs of a dynamical system. Its name comes from the information path in the system: process inputs (e.g., voltage applied to an electric motor) have an effect on the process outputs (e.g., speed or torque of the motor), which is measured with sensors and processed by the controller; the result (the control signal) is used as input to the process, closing the loop. Closed-loop controllers have the following advantages over open-loop controllers: • disturbance rejection (such as unmeasured friction in a motor) • guaranteed performance even with model uncertainties, when the model structure does not match perfectly the real process and the model parameters are not exact • unstable processes can be stabilized • reduced sensitivity to parameter variations • improved reference tracking performance In some systems, closed-loop and open-loop control are used simultaneously. In such systems, the open-loop control is termed feedforward and serves to further improve reference tracking performance. A common closed-loop controller architecture is the PID controller. Closed-loop transfer function The output of the system y(t) is fed back through a sensor measurement F to the reference value r(t). The controller C then takes the error e (difference) between the reference and the output to change the inputs u to the system under control P. This is shown in the figure. This kind of controller is a closed-loop controller or feedback controller. This is called a single-input-single-output (SISO) control system; MIMO (i.e., Multi-Input-Multi-Output) systems, with more than one input/output, are common. In such cases variables are represented through vectors instead of simple scalar values. For some distributed parameter systems the vectors may be infinite-dimensional (typically functions). 135 Control theory 136 If we assume the controller C, the plant P, and the sensor F are linear and time-invariant (i.e., elements of their transfer function C(s), P(s), and F(s) do not depend on time), the systems above can be analysed using the Laplace transform on the variables. This gives the following relations: Solving for Y(s) in terms of R(s) gives: The expression is referred to as the closed-loop transfer function of the system. The numerator is the forward (open-loop) gain from r to y, and the denominator is one plus the gain in going around the feedback loop, the so-called loop gain. If , i.e., it has a large norm with each value of s, and if , then Y(s) is approximately equal to R(s) and the output closely tracks the reference input. PID controller The PID controller is probably the most-used feedback control design. PID is an acronym for Proportional-Integral-Derivative, referring to the three terms operating on the error signal to produce a control signal. If u(t) is the control signal sent to the system, y(t) is the measured output and r(t) is the desired output, and tracking error , a PID controller has the general form The desired closed loop dynamics is obtained by adjusting the three parameters , and , often iteratively by "tuning" and without specific knowledge of a plant model. Stability can often be ensured using only the proportional term. The integral term permits the rejection of a step disturbance (often a striking specification in process control). The derivative term is used to provide damping or shaping of the response. PID controllers are the most well established class of control systems: however, they cannot be used in several more complicated cases, especially if MIMO systems are considered. Applying Laplace transformation results in the transformed PID controller equation with the PID controller transfer function Control theory Modern control theory In contrast to the frequency domain analysis of the classical control theory, modern control theory utilizes the time-domain state space representation, a mathematical model of a physical system as a set of input, output and state variables related by first-order differential equations. To abstract from the number of inputs, outputs and states, the variables are expressed as vectors and the differential and algebraic equations are written in matrix form (the latter only being possible when the dynamical system is linear). The state space representation (also known as the "time-domain approach") provides a convenient and compact way to model and analyze systems with multiple inputs and outputs. With inputs and outputs, we would otherwise have to write down Laplace transforms to encode all the information about a system. Unlike the frequency domain approach, the use of the state space representation is not limited to systems with linear components and zero initial conditions. "State space" refers to the space whose axes are the state variables. The state of the system can be represented as a vector within that space.[5] Topics in control theory Stability The stability of a general dynamical system with no input can be described with Lyapunov stability criteria. A linear system that takes an input is called bounded-input bounded-output (BIBO) stable if its output will stay bounded for any bounded input. Stability for nonlinear systems that take an input is input-to-state stability (ISS), which combines Lyapunov stability and a notion similar to BIBO stability. For simplicity, the following descriptions focus on continuous-time and discrete-time linear systems. Mathematically, this means that for a causal linear system to be stable all of the poles of its transfer function must satisfy some criteria depending on whether a continuous or discrete time analysis is used: • In continuous time, the Laplace transform is used to obtain the transfer function. A system is stable if the poles of this transfer function lie strictly in the open left half of the complex plane (i.e. the real part of all the poles is less than zero). • In discrete time the Z-transform is used. A system is stable if the poles of this transfer function lie strictly inside the unit circle. i.e. the magnitude of the poles is less than one). When the appropriate conditions above are satisfied a system is said to be asymptotically stable: the variables of an asymptotically stable control system always decrease from their initial value and do not show permanent oscillations. Permanent oscillations occur when a pole has a real part exactly equal to zero (in the continuous time case) or a modulus equal to one (in the discrete time case). If a simply stable system response neither decays nor grows over time, and has no oscillations, it is marginally stable: in this case the system transfer function has non-repeated poles at complex plane origin (i.e. their real and complex component is zero in the continuous time case). Oscillations are present when poles with real part equal to zero have an imaginary part not equal to zero. Differences between the two cases are not a contradiction. The Laplace transform is in Cartesian coordinates and the Z-transform is in circular coordinates, and it can be shown that: • the negative-real part in the Laplace domain can map onto the interior of the unit circle • the positive-real part in the Laplace domain can map onto the exterior of the unit circle If a system in question has an impulse response of then the Z-transform (see this example), is given by 137 Control theory which has a pole in 138 (zero imaginary part). This system is BIBO (asymptotically) stable since the pole is inside the unit circle. However, if the impulse response was then the Z-transform is which has a pole at and is not BIBO stable since the pole has a modulus strictly greater than one. Numerous tools exist for the analysis of the poles of a system. These include graphical systems like the root locus, Bode plots or the Nyquist plots. Mechanical changes can make equipment (and control systems) more stable. Sailors add ballast to improve the stability of ships. Cruise ships use antiroll fins that extend transversely from the side of the ship for perhaps 30 feet (10 m) and are continuously rotated about their axes to develop forces that oppose the roll. Controllability and observability Controllability and observability are main issues in the analysis of a system before deciding the best control strategy to be applied, or whether it is even possible to control or stabilize the system. Controllability is related to the possibility of forcing the system into a particular state by using an appropriate control signal. If a state is not controllable, then no signal will ever be able to control the state. If a state is not controllable, but its dynamics are stable, then the state is termed Stabilizable. Observability instead is related to the possibility of "observing", through output measurements, the state of a system. If a state is not observable, the controller will never be able to determine the behaviour of an unobservable state and hence cannot use it to stabilize the system. However, similar to the stabilizability condition above, if a state cannot be observed it might still be detectable. From a geometrical point of view, looking at the states of each variable of the system to be controlled, every "bad" state of these variables must be controllable and observable to ensure a good behaviour in the closed-loop system. That is, if one of the eigenvalues of the system is not both controllable and observable, this part of the dynamics will remain untouched in the closed-loop system. If such an eigenvalue is not stable, the dynamics of this eigenvalue will be present in the closed-loop system which therefore will be unstable. Unobservable poles are not present in the transfer function realization of a state-space representation, which is why sometimes the latter is preferred in dynamical systems analysis. Solutions to problems of uncontrollable or unobservable system include adding actuators and sensors. Control specification Several different control strategies have been devised in the past years. These vary from extremely general ones (PID controller), to others devoted to very particular classes of systems (especially robotics or aircraft cruise control). A control problem can have several specifications. Stability, of course, is always present: the controller must ensure that the closed-loop system is stable, regardless of the open-loop stability. A poor choice of controller can even worsen the stability of the open-loop system, which must normally be avoided. Sometimes it would be desired to obtain particular dynamics in the closed loop: i.e. that the poles have , where is a fixed value strictly greater than zero, instead of simply asking that . Another typical specification is the rejection of a step disturbance; including an integrator in the open-loop chain (i.e. directly before the system under control) easily achieves this. Other classes of disturbances need different types of sub-systems to be included. Control theory Other "classical" control theory specifications regard the time-response of the closed-loop system: these include the rise time (the time needed by the control system to reach the desired value after a perturbation), peak overshoot (the highest value reached by the response before reaching the desired value) and others (settling time, quarter-decay). Frequency domain specifications are usually related to robustness (see after). Modern performance assessments use some variation of integrated tracking error (IAE,ISA,CQI). Model identification and robustness A control system must always have some robustness property. A robust controller is such that its properties do not change much if applied to a system slightly different from the mathematical one used for its synthesis. This specification is important: no real physical system truly behaves like the series of differential equations used to represent it mathematically. Typically a simpler mathematical model is chosen in order to simplify calculations, otherwise the true system dynamics can be so complicated that a complete model is impossible. System identification The process of determining the equations that govern the model's dynamics is called system identification. This can be done off-line: for example, executing a series of measures from which to calculate an approximated mathematical model, typically its transfer function or matrix. Such identification from the output, however, cannot take account of unobservable dynamics. Sometimes the model is built directly starting from known physical equations: for example, in the case of a mass-spring-damper system we know that . Even assuming that a "complete" model is used in designing the controller, all the parameters included in these equations (called "nominal parameters") are never known with absolute precision; the control system will have to behave correctly even when connected to physical system with true parameter values away from nominal. Some advanced control techniques include an "on-line" identification process (see later). The parameters of the model are calculated ("identified") while the controller itself is running: in this way, if a drastic variation of the parameters ensues (for example, if the robot's arm releases a weight), the controller will adjust itself consequently in order to ensure the correct performance. Analysis Analysis of the robustness of a SISO (single input single output) control system can be performed in the frequency domain, considering the system's transfer function and using Nyquist and Bode diagrams. Topics include gain and phase margin and amplitude margin. For MIMO (multi input multi output) and, in general, more complicated control systems one must consider the theoretical results devised for each control technique (see next section): i.e., if particular robustness qualities are needed, the engineer must shift his attention to a control technique by including them in its properties. Constraints A particular robustness issue is the requirement for a control system to perform properly in the presence of input and state constraints. In the physical world every signal is limited. It could happen that a controller will send control signals that cannot be followed by the physical system: for example, trying to rotate a valve at excessive speed. This can produce undesired behavior of the closed-loop system, or even damage or break actuators or other subsystems. Specific control techniques are available to solve the problem: model predictive control (see later), and anti-wind up systems. The latter consists of an additional control block that ensures that the control signal never exceeds a given threshold. 139 Control theory System classifications Linear systems control For MIMO systems, pole placement can be performed mathematically using a state space representation of the open-loop system and calculating a feedback matrix assigning poles in the desired positions. In complicated systems this can require computer-assisted calculation capabilities, and cannot always ensure robustness. Furthermore, all system states are not in general measured and so observers must be included and incorporated in pole placement design. Nonlinear systems control Processes in industries like robotics and the aerospace industry typically have strong nonlinear dynamics. In control theory it is sometimes possible to linearize such classes of systems and apply linear techniques, but in many cases it can be necessary to devise from scratch theories permitting control of nonlinear systems. These, e.g., feedback linearization, backstepping, sliding mode control, trajectory linearization control normally take advantage of results based on Lyapunov's theory. Differential geometry has been widely used as a tool for generalizing well-known linear control concepts to the non-linear case, as well as showing the subtleties that make it a more challenging problem. Decentralized systems When the system is controlled by multiple controllers, the problem is one of decentralized control. Decentralization is helpful in many ways, for instance, it helps control systems operate over a larger geographical area. The agents in decentralized control systems can interact using communication channels and coordinate their actions. Main control strategies Every control system must guarantee first the stability of the closed-loop behavior. For linear systems, this can be obtained by directly placing the poles. Non-linear control systems use specific theories (normally based on Aleksandr Lyapunov's Theory) to ensure stability without regard to the inner dynamics of the system. The possibility to fulfill different specifications varies from the model considered and the control strategy chosen. Here a summary list of the main control techniques is shown: Adaptive control Adaptive control uses on-line identification of the process parameters, or modification of controller gains, thereby obtaining strong robustness properties. Adaptive controls were applied for the first time in the aerospace industry in the 1950s, and have found particular success in that field. Hierarchical control A Hierarchical control system is a type of Control System in which a set of devices and governing software is arranged in a hierarchical tree. When the links in the tree are implemented by a computer network, then that hierarchical control system is also a form of Networked control system. Intelligent control Intelligent control uses various AI computing approaches like neural networks, Bayesian probability, fuzzy logic, machine learning, evolutionary computation and genetic algorithms to control a dynamic system. Optimal control Optimal control is a particular control technique in which the control signal optimizes a certain "cost index": for example, in the case of a satellite, the jet thrusts needed to bring it to desired trajectory that consume the least amount of fuel. Two optimal control design methods have been widely used in industrial applications, as it has been shown they can guarantee closed-loop stability. These are Model Predictive Control (MPC) and 140 Control theory Linear-Quadratic-Gaussian control (LQG). The first can more explicitly take into account constraints on the signals in the system, which is an important feature in many industrial processes. However, the "optimal control" structure in MPC is only a means to achieve such a result, as it does not optimize a true performance index of the closed-loop control system. Together with PID controllers, MPC systems are the most widely used control technique in process control. Robust control Robust control deals explicitly with uncertainty in its approach to controller design. Controllers designed using robust control methods tend to be able to cope with small differences between the true system and the nominal model used for design. The early methods of Bode and others were fairly robust; the state-space methods invented in the 1960s and 1970s were sometimes found to lack robustness. A modern example of a robust control technique is H-infinity loop-shaping developed by Duncan McFarlane and Keith Glover of Cambridge University, United Kingdom. Robust methods aim to achieve robust performance and/or stability in the presence of small modeling errors. Stochastic control Stochastic control deals with control design with uncertainty in the model. In typical stochastic control problems, it is assumed that there exist random noise and disturbances in the model and the controller, and the control design must take into account these random deviations. References [1] Maxwell, J.C. (1867). "On Governors". Proceedings of the Royal Society of London 16: 270–283. doi:10.1098/rspl.1867.0055. JSTOR 112510. [2] Routh, E.J.; Fuller, A.T. (1975). Stability of motion. Taylor & Francis. [3] Routh, E.J. (1877). A Treatise on the Stability of a Given State of Motion, Particularly Steady Motion: Particularly Steady Motion. Macmillan and co.. [4] Hurwitz, A. (1964). "On The Conditions Under Which An Equation Has Only Roots With Negative Real Parts". Selected Papers on Mathematical Trends in Control Theory. [5] Donald M Wiberg. State space & linear systems. Schaum's outline series. McGraw Hill. ISBN 0070700966. Further reading • Levine, William S., ed (1996). The Control Handbook. New York: CRC Press. ISBN 978-0-849-38570-4. • Karl J. Åström and Richard M. Murray (2008). Feedback Systems: An Introduction for Scientists and Engineers. (http://www.cds.caltech.edu/~murray/books/AM08/pdf/am08-complete_22Feb09.pdf). Princeton University Press. ISBN 0691135762. • Christopher Kilian (2005). Modern Control Technology. Thompson Delmar Learning. ISBN 1-4018-5806-6. • Vannevar Bush (1929). Operational Circuit Analysis. John Wiley and Sons, Inc.. • Robert F. Stengel (1994). Optimal Control and Estimation. Dover Publications. ISBN 0-486-68200-5, ISBN 978-0-486-68200-6. • Franklin et al. (2002). Feedback Control of Dynamic Systems (4 ed.). New Jersey: Prentice Hall. ISBN 0-13-032393-4. • Joseph L. Hellerstein, Dawn M. Tilbury, and Sujay Parekh (2004). Feedback Control of Computing Systems. John Wiley and Sons. ISBN 0-47-126637-X, ISBN 978-0-471-26637-2. • Diederich Hinrichsen and Anthony J. Pritchard (2005). Mathematical Systems Theory I - Modelling, State Space Analysis, Stability and Robustness. Springer. ISBN 0-978-3-540-44125-0. • Andrei, Neculai (2005). Modern Control Theory - A historical Perspective (http://camo.ici.ro/neculai/history. pdf). Retrieved 2007-10-10. • Sontag, Eduardo (1998). Mathematical Control Theory: Deterministic Finite Dimensional Systems. Second Edition (http://www.math.rutgers.edu/~sontag/FTP_DIR/sontag_mathematical_control_theory_springer98. 141 Control theory pdf). Springer. ISBN 0-387-984895. • Goodwin, Graham (2001). Control System Design. Prentice Hall. ISBN 0-13-958653-9. For Chemical Engineering • Luyben, William (1989). Process Modeling, Simulation, and Control for Chemical Engineers. Mc Graw Hill. ISBN 0-07-039159-9. External links • Control Tutorials for Matlab (http://www.engin.umich.edu/class/ctms/) - A set of worked through control examples solved by several different methods. • Control Tuning and Best Practices (http://www.controlguru.com) Control system A control system is a device, or set of devices to manage, command, direct or regulate the behavior of other devices or system. There are two common classes of control systems, with many variations and combinations: logic or sequential controls, and feedback or linear controls. There is also fuzzy logic, which attempts to combine some of the design simplicity of logic with the utility of linear control. Some devices or systems are inherently not controllable. Overview The term "control system" may be applied to the essentially manual controls that allow an operator, for example, to close and open a hydraulic press, perhaps including logic so that it cannot be moved unless safety guards are in place. An automatic sequential control system may trigger a series of mechanical actuators in the correct sequence to perform a task. For example various electric and pneumatic transducers may fold and glue a cardboard box, fill it with product and then seal it in an automatic packaging machine. In the case of linear feedback systems, a control loop, including sensors, control algorithms and actuators, is arranged in such a fashion as to try to regulate a variable at a setpoint or reference value. An example of this may increase the fuel supply to a furnace when a measured temperature drops. PID controllers are common and effective in cases such as this. Control systems that include some sensing of the results they are trying to achieve are making use of feedback and so can, to some extent, adapt to varying circumstances. Open-loop control systems do not make use of feedback, and run only in pre-arranged ways. Logic control Logic control systems for industrial and commercial machinery were historically implemented at mains voltage using interconnected relays, designed using ladder logic. Today, most such systems are constructed with programmable logic controllers (PLCs) or microcontrollers. The notation of ladder logic is still in use as a programming idiom for PLCs.[1] Logic controllers may respond to switches, light sensors, pressure switches, etc., and can cause the machinery to start and stop various operations. Logic systems are used to sequence mechanical operations in many applications. PLC software can be written in many different ways – ladder diagrams, SFC – sequential function charts or in language terms known as statement lists.[2] Examples include elevators, washing machines and other systems with interrelated stop-go operations. 142 Control system Logic systems are quite easy to design, and can handle very complex operations. Some aspects of logic system design make use of Boolean logic. On–off control For example, a thermostat is a simple negative-feedback control: when the temperature (the "process variable" or PV) goes below a set point (SP), the heater is switched on. Another example could be a pressure switch on an air compressor: when the pressure (PV) drops below the threshold (SP), the pump is powered. Refrigerators and vacuum pumps contain similar mechanisms operating in reverse, but still providing negative feedback to correct errors. Simple on–off feedback control systems like these are cheap and effective. In some cases, like the simple compressor example, they may represent a good design choice. In most applications of on–off feedback control, some consideration needs to be given to other costs, such as wear and tear of control valves and maybe other start-up costs when power is reapplied each time the PV drops. Therefore, practical on–off control systems are designed to include hysteresis, usually in the form of a deadband, a region around the setpoint value in which no control action occurs. The width of deadband may be adjustable or programmable. Linear control Linear control systems use linear negative feedback to produce a control signal mathematically based on other variables, with a view to maintaining the controlled process within an acceptable operating range. The output from a linear control system into the controlled process may be in the form of a directly variable signal, such as a valve that may be 0 or 100% open or anywhere in between. Sometimes this is not feasible and so, after calculating the current required corrective signal, a linear control system may repeatedly switch an actuator, such as a pump, motor or heater, fully on and then fully off again, regulating the duty cycle using pulse-width modulation. Proportional control When controlling the temperature of an industrial furnace, it is usually better to control the opening of the fuel valve in proportion to the current needs of the furnace. This helps avoid thermal shocks and applies heat more effectively. Proportional negative-feedback systems are based on the difference between the required set point (SP) and process value (PV). This difference is called the error. Power is applied in direct proportion to the current measured error, in the correct sense so as to tend to reduce the error (and so avoid positive feedback). The amount of corrective action that is applied for a given error is set by the gain or sensitivity of the control system. At low gains, only a small corrective action is applied when errors are detected: the system may be safe and stable, but may be sluggish in response to changing conditions; errors will remain uncorrected for relatively long periods of time: it is over-damped. If the proportional gain is increased, such systems become more responsive and errors are dealt with more quickly. There is an optimal value for the gain setting when the overall system is said to be critically damped. Increases in loop gain beyond this point will lead to oscillations in the PV; such a system is under-damped. Under-damped furnace example In the furnace example, suppose the temperature is increasing towards a set point at which, say, 50% of the available power will be required for steady-state. At low temperatures, 100% of available power is applied. When the PV is within, say 10° of the SP the heat input begins to be reduced by the proportional controller. (Note that this implies a 20° "proportional band" (PB) from full to no power input, evenly spread around the setpoint value). At the setpoint the controller will be applying 50% power as required, but stray stored heat within the heater sub-system and in the walls of the furnace will keep the measured temperature rising beyond what is required. At 10° above SP, we reach the top of the proportional band (PB) and no power is applied, but the temperature may continue to rise even further 143 Control system before beginning to fall back. Eventually as the PV falls back into the PB, heat is applied again, but now the heater and the furnace walls are too cool and the temperature falls too low before its fall is arrested, so that the oscillations continue. Over-damped furnace example The temperature oscillations that an under-damped furnace control system produces are unacceptable for many reasons, including the waste of fuel and time (each oscillation cycle may take many minutes), as well as the likelihood of seriously overheating both the furnace and its contents. Suppose that the gain of the control system is reduced drastically and it is restarted. As the temperature approaches, say 30° below SP (60° proportional band or PB now), the heat input begins to be reduced, the rate of heating of the furnace has time to slow and, as the heat is still further reduced, it eventually is brought up to set point, just as 50% power input is reached and the furnace is operating as required. There was some wasted time while the furnace crept to its final temperature using only 52% then 51% of available power, but at least no harm was done. By carefully increasing the gain (i.e. reducing the width of the PB) this over-damped and sluggish behavior can be improved until the system is critically damped for this SP temperature. Doing this is known as 'tuning' the control system. A well-tuned proportional furnace temperature control system will usually be more effective than on-off control, but will still respond more slowly than the furnace could under skillful manual control. PID control Further information: PID controller Apart from sluggish performance to avoid oscillations, another problem with proportional-only control is that power application is always in direct proportion to the error. In the example above we assumed that the set temperature could be maintained with 50% power. What happens if the furnace is required in a different application where a higher set temperature will require 80% power to maintain it? If the gain was finally set to a 50° PB, then 80% power will not be applied unless the furnace is 15° below setpoint, so for this other application the operators will have to remember always to set the setpoint temperature 15° higher than actually needed. This 15° figure is not completely constant either: it will depend on the surrounding ambient temperature, as well as other factors that affect heat loss from or absorption within the furnace. To resolve these two problems, many feedback control schemes include mathematical extensions to improve performance. The most common extensions lead to proportional-integral-derivative control, or PID control (pronounced pee-eye-dee). Derivative action The derivative part is concerned with the rate-of-change of the error with time: If the measured variable approaches the setpoint rapidly, then the actuator is backed off early to allow it to coast to the required level; conversely if the measured value begins to move rapidly away from the setpoint, extra effort is applied—in proportion to that rapidity—to try to maintain it. Derivative action makes a control system behave much more intelligently. On systems like the temperature of a furnace, or perhaps the motion-control of a heavy item like a gun or camera on a moving vehicle, the derivative action of a well-tuned PID controller can allow it to reach and maintain a setpoint better than most skilled human operators could. If derivative action is over-applied, it can lead to oscillations too. An example would be a PV that increased rapidly towards SP, then halted early and seemed to "shy away" from the setpoint before rising towards it again. 144 Control system Integral action The integral term magnifies the effect of long-term steady-state errors, applying ever-increasing effort until they reduce to zero. In the example of the furnace above working at various temperatures, if the heat being applied does not bring the furnace up to setpoint, for whatever reason, integral action increasingly moves the proportional band relative to the setpoint until the PV error is reduced to zero and the setpoint is achieved. Other techniques It is possible to filter the PV or error signal. Doing so can reduce the response of the system to undesirable frequencies, to help reduce instability or oscillations. Some feedback systems will oscillate at just one frequency. By filtering out that frequency, more "stiff" feedback can be applied, making the system more responsive without shaking itself apart. Feedback systems can be combined. In cascade control, one control loop applies control algorithms to a measured variable against a setpoint, but then provides a varying setpoint to another control loop rather than affecting process variables directly. If a system has several different measured variables to be controlled, separate control systems will be present for each of them. Control engineering in many applications produces control systems that are more complex than PID control. Examples of such fields include fly-by-wire aircraft control systems, chemical plants, and oil refineries. Model predictive control systems are designed using specialized computer-aided-design software and empirical mathematical models of the system to be controlled. Fuzzy logic Further information: Fuzzy logic Fuzzy logic is an attempt to get the easy design of logic controllers and yet control continuously-varying systems. Basically, a measurement in a fuzzy logic system can be partly true, that is if yes is 1 and no is 0, a fuzzy measurement can be between 0 and 1. The rules of the system are written in natural language and translated into fuzzy logic. For example, the design for a furnace would start with: "If the temperature is too high, reduce the fuel to the furnace. If the temperature is too low, increase the fuel to the furnace." Measurements from the real world (such as the temperature of a furnace) are converted to values between 0 and 1 by seeing where they fall on a triangle. Usually the tip of the triangle is the maximum possible value which translates to "1." Fuzzy logic, then, modifies Boolean logic to be arithmetical. Usually the "not" operation is "output = 1 - input," the "and" operation is "output = input.1 multiplied by input.2," and "or" is "output = 1 - ((1 - input.1) multiplied by (1 input.2))". This reduces to Boolean arithmetic if values are restricted to 0 and 1, instead of allowed to range in the unit interval [0,1]. The last step is to "defuzzify" an output. Basically, the fuzzy calculations make a value between zero and one. That number is used to select a value on a line whose slope and height converts the fuzzy value to a real-world output number. The number then controls real machinery. If the triangles are defined correctly and rules are right the result can be a good control system. When a robust fuzzy design is reduced into a single, quick calculation, it begins to resemble a conventional feedback loop solution and it might appear that the fuzzy design was unnecessary. However, the fuzzy logic paradigm may provide scalability for large control systems where conventional methods become unwieldy or costly to derive. Fuzzy electronics is an electronic technology that uses fuzzy logic instead of the two-value logic more commonly used in digital electronics. 145 Control system Physical implementations Since modern small microprocessors are so cheap (often less than $1 US), it's very common to implement control systems, including feedback loops, with computers, often in an embedded system. The feedback controls are simulated by having the computer make periodic measurements and then calculating from this stream of measurements (see digital signal processing, sampled data systems). Computers emulate logic devices by making measurements of switch inputs, calculating a logic function from these measurements and then sending the results out to electronically-controlled switches. Logic systems and feedback controllers are usually implemented with programmable logic controllers which are devices available from electrical supply houses. They include a little computer and a simplified system for programming. Most often they are programmed with personal computers. Logic controllers have also been constructed from relays, hydraulic and pneumatic devices, and electronics using both transistors and vacuum tubes (feedback controllers can also be constructed in this manner). References [1] Kuphaldt, Tony R.. "Chapter 6 LADDER LOGIC" (http:/ / openbookproject. net/ / electricCircuits/ Digital/ DIGI_6. html). Lessons In Electric Circuits -- Volume IV. . Retrieved 22 September 2010. [2] Brady, Ian. "Programmable logic controllers - benefits and applications" (http:/ / www. optimacs. com/ reports/ PLC Report. pdf). PLCs. . Retrieved 5 December 2011. External links • Semiautonomous Flight Direction - Reference unmannedaircraft.org (http://www.unmannedaircraft.org/f/ pat6460810.pdf) Controller (control theory) In control theory, a controller is a device which monitors and affects the operational conditions of a given dynamical system. The operational conditions are typically referred to as output variables of the system which can be affected by adjusting certain input variables. For example, the heating system of a house can be equipped with a thermostat (controller) for sensing air temperature (output variable) which can turn on or off a furnace or heater when the air temperature becomes too low or too high. In this example, the thermostat is the controller and directs the activities of the heater. The heater is the processor that warms the air inside the house to the desired temperature (setpoint). The air temperature reading inside the house is the feedback. And finally, the house is the environment in which the heating system operates. The notion of controllers can be extended to more complex systems. In the natural world, individual organisms also appear to be equipped with controllers that assure the homeostasis necessary for survival of each individual. Both human-made and natural systems exhibit collective behaviors amongst individuals in which the controllers seek some form of equilibrium. Types of Controlling System In control theory there are two basic types of control. These are feedback and feed-forward. The input to a feedback controller is the same as what it is trying to control - the controlled variable is "fed back" into the controller. The thermostat of a house is an example of a feedback controller. This controller relies on measuring the controlled variable, in this case the temperature of the house, and then adjusting the output, whether or not the heater is on. However, feedback control usually results in intermediate periods where the controlled variable is not at the desired 146 Controller (control theory) 147 set-point. With the thermostat example, if the door of the house were opened on a cold day, the house would cool down. After it fell below the desired temperature (set-point), the heater would kick on, but there would be a period when the house was colder than desired. Feed-forward control can avoid the slowness of feedback control. With feed-forward control, the disturbances are measured and accounted for before they have time to affect the system. In the house example, a feed-forward system may measure the fact that the door is opened and automatically turn on the heater before the house can get too cold. The difficulty with feed-forward control is that the effect of the disturbances on the system must be accurately predicted, and there must not be any unmeasured disturbances. For instance, if a window were opened that was not being measured, the feed-forward-controlled thermostat might still let the house cool down. To achieve the benefits of feedback control (controlling unknown disturbances and not having to know exactly how a system will respond to disturbances) and the benefits of feed-forward control (responding to disturbances before they can affect the system), there are combination's of feedback and feed-forward that can be used. Some examples of where feedback and feed-forward control can be used together are dead-time compensation, and inverse response compensation. Dead-time compensation is used to control devices that take a long time to show any change to a change in input, for example, change in composition of flow through a long pipe. A dead-time compensation control uses an element (also called a Smith predictor) to predict how changes made now by the controller will affect the controlled variable in the future. The controlled variable is also measured and used in feedback control. Inverse response compensation involves controlling systems where a change at first affects the measured variable one way but later affects it in the opposite way. An example would be eating candy. At first it will give you lots of energy, but later you will be very tired. As can be imagined, it is difficult to control this system with feedback alone, therefore a predictive feed-forward element is necessary to predict the reverse effect that a change will have in the future. '''' Types of controllers Most control valve systems in the past were implemented using mechanical systems or solid state electronics. Pneumatics were often utilized to transmit information and control using pressure. However, most modern control systems in industrial settings now rely on computers for the controller. Obviously it is much easier to implement complex control algorithms on a computer than using a mechanical system. For feedback controllers there are a few simple types. The most simple is like the thermostat that just turns the heat on if the temperature falls below a certain value and off it exceeds a certain value (on-off control). Another simple type of controller is a proportional controller. With this type of controller, the controller output (control action) is proportional to the error in the measured variable. In feedback control, it is standard to define the error as the difference between the desired value (setpoint) the current value (measured) and . If the error is large, then the control action is large. Mathematically: where represents the control action (controller output), represents the error, represents the controller's gain, and represents the steady state control action (bias) necessary to maintain the variable at the steady state when there is no error. It is important that the control action counteracts the change in the controlled variable There are then two cases depending on the sign of the process gain. (negative feedback). Controller (control theory) 148 In the first case the process gain is positive, so an increase in the controlled variable (measurement) decrease in the control action (reverse-acting control). In this case the controller gain standard definition of the error already contains a negative sign for is positive, because the . In the second case the process gain is negative, so an increase in the controlled variable (measurement) an increase in the control action requires a (direct-acting control). In this case the controller gain A typical example of a reverse-acting system is control of temperature ( requires is negative. ) by use of steam ( ). In this case the process gain is positive, so if the temperature increases, the steam flow must be decreased to maintain the desired temperature. Conversely, a typical example of a direct-acting system is control of temperature using cooling water. In this case the process gain is negative, so if the temperature increases, the cooling water flow must be increased to maintain the desired temperature. Although proportional control is simple to understand, it has drawbacks. The largest problem is that for most systems it will never entirely remove error. This is because when error is 0 the controller only provides the steady state control action so the system will settle back to the original steady state (which is probably not the new set point that we want the system to be at). To get the system to operate near the new steady state, the controller gain, Kc, must be very large so the controller will produce the required output when only a very small error is present. Having large gains can lead to system instability or can require physical impossibilities like infinitely large valves. Alternates to proportional control are proportional-integral (PI) control and proportional-integral-derivative (PID) control. PID control is commonly used to implement closed-loop control. Open-loop control can be used in systems sufficiently well-characterized as to predict what outputs will necessarily achieve the desired states. For example, the rotational velocity of an electric motor may be well enough characterized for the supplied voltage to make feedback unnecessary. The drawback of open-loop control is that it requires perfect knowledge of the system (i.e. one knows exactly what inputs to give in order to get the desired output), and it assumes there are no disturbances to the system. References Process control Process control Process control is a statistics and engineering discipline that deals with architectures, mechanisms and algorithms for maintaining the output of a specific process within a desired range. See also control theory. Background Process control is extensively used in industry and enables mass production of continuous processes such as oil refining, paper manufacturing, chemicals, power plants and many other industries. Process control enables automation, with which a small staff of operating personnel can operate a complex process from a central control room. For example, heating up the temperature in a room is a process that has the specific, desired outcome to reach and maintain a defined temperature (e.g. 20°C), kept constant over time. Here, the temperature is the controlled variable. At the same time, it is the input variable since it is measured by a thermometer and used to decide whether to heat or not to heat. The desired temperature (20°C) is the setpoint. The state of the heater (e.g. the setting of the valve allowing hot water to flow through it) is called the manipulated variable since it is subject to control actions. A commonly used control device called a programmable logic controller, or a PLC, is used to read a set of digital and analog inputs, apply a set of logic statements, and generate a set of analog and digital outputs. Using the example in the previous paragraph, the room temperature would be an input to the PLC. The logical statements would compare the setpoint to the input temperature and determine whether more or less heating was necessary to keep the temperature constant. A PLC output would then either open or close the hot water valve, an incremental amount, depending on whether more or less hot water was needed. Larger more complex systems can be controlled by a Distributed Control System (DCS) or SCADA system. Types of control systems In practice, process control systems can be characterized as one or more of the following forms: • Discrete – Found in many manufacturing, motion and packaging applications. Robotic assembly, such as that found in automotive production, can be characterized as discrete process control. Most discrete manufacturing involves the production of discrete pieces of product, such as metal stamping. • Batch – Some applications require that specific quantities of raw materials be combined in specific ways for particular durations to produce an intermediate or end result. One example is the production of adhesives and glues, which normally require the mixing of raw materials in a heated vessel for a period of time to form a quantity of end product. Other important examples are the production of food, beverages and medicine. Batch processes are generally used to produce a relatively low to intermediate quantity of product per year (a few pounds to millions of pounds). • Continuous – Often, a physical system is represented through variables that are smooth and uninterrupted in time. The control of the water temperature in a heating jacket, for example, is an example of continuous process control. Some important continuous processes are the production of fuels, chemicals and plastics. Continuous processes in manufacturing are used to produce very large quantities of product per year (millions to billions of pounds). Applications having elements of discrete, batch and continuous process control are often called hybrid applications. 149 Process control Statistical Process Control Statistical Process Control (SPC) is an effective method of monitoring a process through the use of control charts. Much of its power lies in the ability to monitor both the current center of a process and the process's variation about that center. By collecting data from samples at various points within the process, variations in the process that may affect the quality of the end product or service can be detected and corrected, thus reducing waste as well as the likelihood that problems will be passed on to the customer. It has an emphasis on early detection and prevention of problems. Multivariable Process Control is a type of Statistical Process Control where a set of variables (manipulated variables and control variables) are identified and the joint variations within this set are captured byng a step test. The Dynamics captured in the model curves are used to control the plant Examples A thermostat is a simple example for a closed control loop: It constantly measures the current temperature and controls the heater's valve setting to increase or decrease the room temperature according to the user-defined setting. A simple method switches the heater either completely on, or completely off, and an overshoot and undershoot of the controlled temperature must be expected. A more expensive method varies the amount of heat provided by the heater depending on the difference between the required temperature (the "setpoint") and the actual temperature. This minimizes over/undershoot. An anti-lock braking system (ABS) is a more complex example, consisting of multiple inputs, conditions and outputs. External links • • • • Jobs in Process Control [1] The Michigan Chemical Engineering Process Dynamics and Controls Open Textbook [2] PID Control Theory and Best Practices [3] Process Control & Instrumentation [4] References [1] [2] [3] [4] http:/ / controljob. com http:/ / controls. engin. umich. edu/ wiki http:/ / www. controlguru. com http:/ / www. flotech. com. sg 150 151 Three-term control Proportionality (mathematics) In mathematics, two variable quantities are proportional if one of them is always the product of the other and a constant quantity, called the coefficient of proportionality or proportionality constant. In other words, x and y are proportional if the ratio is constant. We also say that one of the quantities is proportional to the other. For example, if the speed of an object is constant, it travels a distance that is proportional to the travel time. y is directly proportional to x. If a linear function transforms 0, a and b into 0, c and d, and if the product a b c d is not zero, we say a and b are proportional to c and d. An equality of two ratios such as where no term is zero, is called a proportion. Geometric illustration The two rectangles with stripes are similar, the ratios of their dimensions are horizontally written within the image. The duplication scale of a striped triangle is obliquely written, in a proportion obtained by inverting two terms of another proportion horizontally written. Proportionality (mathematics) 152 When the duplication of a given rectangle preserves its shape, the ratio of the large dimension to the small dimension is a constant number in all the copies, and in the original rectangle. The largest rectangle of the drawing is similar to one or the other rectangle with stripes. From their width to their height, the coefficient is A ratio of their dimensions horizontally written within the image, at the top or the bottom, determines the common shape of the three similar rectangles. The common diagonal of the similar rectangles divides each rectangle into two superposable triangles, with two different kinds of stripes. The four striped triangles and the two striped rectangles have a common vertex: the center of an homothetic transformation with a negative ratio – k or , that transforms one triangle and its stripes into another triangle with the same stripes, enlarged or reduced. The duplication scale of a striped triangle is the proportionality constant between the corresponding sides lengths of the triangles, equal to a positive ratio obliquely written within the image: or In the proportion , the terms a and d are called the extremes, while b and c are the means, because a and d are the extreme terms of the list (a, b, c, d), while b and c are in the middle of the list. From any proportion, we get another proportion by inverting the extremes or the means. And the product of the extremes equals the product of the means. Within the image, a double arrow indicates two inverted terms of the first proportion. Consider dividing the largest rectangle in two triangles, cutting along the diagonal. If we remove two triangles from either half rectangle, we get one of the plain gray rectangles. Above and below this diagonal, the areas of the two biggest triangles of the drawing are equal, because these triangles are superposable. Above and below the subtracted areas are equal for the same reason. Therefore, the two plain gray rectangles have the same area: a d = b c. Symbol The mathematical symbol '∝' is used to indicate that two values are proportional. For example, A ∝ B. In Unicode this is symbol U+221D. Direct proportionality Given two variables x and y, y is (directly) proportional to x (x and y vary directly, or x and y are in direct variation) if there is a non-zero constant k such that The relation is often denoted and the constant ratio is called the proportionality constant or constant of proportionality. Proportionality (mathematics) Examples • If an object travels at a constant speed, then the distance traveled is proportional to the time spent traveling, with the speed being the constant of proportionality. • The circumference of a circle is proportional to its diameter, with the constant of proportionality equal to π. • On a map drawn to scale, the distance between any two points on the map is proportional to the distance between the two locations that the points represent, with the constant of proportionality being the scale of the map. • The force acting on a certain object due to gravity is proportional to the object's mass; the constant of proportionality between the mass and the force is known as gravitational acceleration. Properties Since is equivalent to it follows that if y is proportional to x, with (nonzero) proportionality constant k, then x is also proportional to y with proportionality constant 1/k. If y is proportional to x, then the graph of y as a function of x will be a straight line passing through the origin with the slope of the line equal to the constant of proportionality: it corresponds to linear growth. Inverse proportionality The concept of inverse proportionality can be contrasted against direct proportionality. Consider two variables said to be "inversely proportional" to each other. If all other variables are held constant, the magnitude or absolute value of one inversely proportional variable will decrease if the other variable increases, while their product (the constant of proportionality k) is always the same. Formally, two variables are inversely proportional (or varying inversely, or in inverse variation, or in inverse proportion or in reciprocal proportion) if one of the variables is directly proportional with the multiplicative inverse (reciprocal) of the other, or equivalently if their product is a constant. It follows that the variable y is inversely proportional to the variable x if there exists a non-zero constant k such that The constant can be found by multiplying the original x variable and the original y variable. As an example, the time taken for a journey is inversely proportional to the speed of travel; the time needed to dig a hole is (approximately) inversely proportional to the number of people digging. The graph of two variables varying inversely on the Cartesian coordinate plane is a hyperbola. The product of the X and Y values of each point on the curve will equal the constant of proportionality (k). Since neither x nor y can equal zero (if k is non-zero), the graph will never cross either axis. 153 Proportionality (mathematics) 154 Hyperbolic coordinates The concepts of direct and inverse proportion lead to the location of points in the Cartesian plane by hyperbolic coordinates; the two coordinates correspond to the constant of direct proportionality that locates a point on a ray and the constant of inverse proportionality that locates a point on a hyperbola. Exponential and logarithmic proportionality A variable y is exponentially proportional to a variable x, if y is directly proportional to the exponential function of x, that is if there exist non-zero constants k and a Likewise, a variable y is logarithmically proportional to a variable x, if y is directly proportional to the logarithm of x, that is if there exist non-zero constants k and a Experimental determination To determine experimentally whether two physical quantities are directly proportional, one performs several measurements and plots the resulting data points in a Cartesian coordinate system. If the points lie on or close to a straight line that passes through the origin (0, 0), then the two variables are probably proportional, with the proportionality constant given by the line's slope. Integral Integration is an important concept in mathematics and, together with its inverse, differentiation, is one of the two main operations in calculus. Given a function ƒ of a real variable x and an interval [a, b] of the real line, the definite integral A definite integral of a function can be represented as the signed area of the region bounded by its graph. is defined informally to be the net signed area of the region in the xy-plane bounded by the graph of ƒ, the x-axis, and the vertical lines x = a and x = b. The term integral may also refer to the notion of antiderivative, a function F whose derivative is the given function ƒ. In this case, it is called an indefinite integral and is written: Integral The integrals discussed in this article are termed definite integrals. The principles of integration were formulated independently by Isaac Newton and Gottfried Leibniz in the late 17th century. Through the fundamental theorem of calculus, which they independently developed, integration is connected with differentiation: if ƒ is a continuous real-valued function defined on a closed interval [a, b], then, once an antiderivative F of ƒ is known, the definite integral of ƒ over that interval is given by Integrals and derivatives became the basic tools of calculus, with numerous applications in science and engineering. The founders of the calculus thought of the integral as an infinite sum of rectangles of infinitesimal width. A rigorous mathematical definition of the integral was given by Bernhard Riemann. It is based on a limiting procedure which approximates the area of a curvilinear region by breaking the region into thin vertical slabs. Beginning in the nineteenth century, more sophisticated notions of integrals began to appear, where the type of the function as well as the domain over which the integration is performed has been generalised. A line integral is defined for functions of two or three variables, and the interval of integration [a, b] is replaced by a certain curve connecting two points on the plane or in the space. In a surface integral, the curve is replaced by a piece of a surface in the three-dimensional space. Integrals of differential forms play a fundamental role in modern differential geometry. These generalizations of integrals first arose from the needs of physics, and they play an important role in the formulation of many physical laws, notably those of electrodynamics. There are many modern concepts of integration, among these, the most common is based on the abstract mathematical theory known as Lebesgue integration, developed by Henri Lebesgue. History Pre-calculus integration Integration can be traced as far back as ancient Egypt ca. 1800 BC, with the Moscow Mathematical Papyrus demonstrating knowledge of a formula for the volume of a pyramidal frustum. The first documented systematic technique capable of determining integrals is the method of exhaustion of the ancient Greek astronomer Eudoxus (ca. 370 BC), which sought to find areas and volumes by breaking them up into an infinite number of shapes for which the area or volume was known. This method was further developed and employed by Archimedes in the 3rd century BC and used to calculate areas for parabolas and an approximation to the area of a circle. Similar methods were independently developed in China around the 3rd century AD by Liu Hui, who used it to find the area of the circle. This method was later used in the 5th century by Chinese father-and-son mathematicians Zu Chongzhi and Zu Geng to find the volume of a sphere.[1] The next major step in integral calculus came from the Abbasid Caliphate when the 11th century mathematician Ibn al-Haytham (known as Alhazen in Europe) devised what is now known as "Alhazen's problem", which leads to an equation of the fourth degree, in his Book of Optics. While solving this problem, he applied mathematical induction to find the formula for sums of fourth powers, by a method that can be generalized to sums of arbitrary natural powers; then he used this formula to find the volume of a paraboloid (in modern terminology, he integrated a polynomial of degree 4).[2] Some ideas of integral calculus are also found in the Siddhanta Shiromani, a 12th century astronomy text by Indian mathematician Bhāskara II. The next significant advances in integral calculus did not begin to appear until the 16th century. At this time the work of Cavalieri with his method of indivisibles, and work by Fermat, began to lay the foundations of modern calculus, with Cavalieri computing the integrals of xn up to degree n = 9 in Cavalieri's quadrature formula. Further steps were made in the early 17th century by Barrow and Torricelli, who provided the first hints of a connection between integration and differentiation. Barrow provided the first proof of the fundamental theorem of calculus. 155 Integral Wallis generalized Cavalieri's method, computing integrals of x to a general power, including negative powers and fractional powers. At around the same time, there was also a great deal of work being done by Japanese mathematicians, particularly by Seki Kōwa.[3] He made a number of contributions, namely in methods of determining areas of figures using integrals, extending the method of exhaustion. Newton and Leibniz The major advance in integration came in the 17th century with the independent discovery of the fundamental theorem of calculus by Newton and Leibniz. The theorem demonstrates a connection between integration and differentiation. This connection, combined with the comparative ease of differentiation, can be exploited to calculate integrals. In particular, the fundamental theorem of calculus allows one to solve a much broader class of problems. Equal in importance is the comprehensive mathematical framework that both Newton and Leibniz developed. Given the name infinitesimal calculus, it allowed for precise analysis of functions within continuous domains. This framework eventually became modern calculus, whose notation for integrals is drawn directly from the work of Leibniz. Formalizing integrals While Newton and Leibniz provided a systematic approach to integration, their work lacked a degree of rigour. Bishop Berkeley memorably attacked the vanishing increments used by Newton, calling them "ghosts of departed quantities". Calculus acquired a firmer footing with the development of limits. Integration was first rigorously formalized, using limits, by Riemann. Although all bounded piecewise continuous functions are Riemann integrable on a bounded interval, subsequently more general functions were considered – particularly in the context of Fourier analysis – to which Riemann's definition does not apply, and Lebesgue formulated a different definition of integral, founded in measure theory (a subfield of real analysis). Other definitions of integral, extending Riemann's and Lebesgue's approaches, were proposed. These approaches based on the real number system are the ones most common today, but alternative approaches exist, such as a definition of integral as the standard part of an infinite Riemann sum, based on the hyperreal number system. Notation Isaac Newton used a small vertical bar above a variable to indicate integration, or placed the variable inside a box. The vertical bar was easily confused with or , which Newton used to indicate differentiation, and the box notation was difficult for printers to reproduce, so these notations were not widely adopted. The modern notation for the indefinite integral was introduced by Gottfried Leibniz in 1675 (Burton 1988, p. 359; Leibniz 1899, p. 154). He adapted the integral symbol, ∫, from the letter ſ (long s), standing for summa (written as ſumma; Latin for "sum" or "total"). The modern notation for the definite integral, with limits above and below the integral sign, was first used by Joseph Fourier in Mémoires of the French Academy around 1819–20, reprinted in his book of 1822 (Cajori 1929, pp. 249–250; Fourier 1822, §231). Terminology and notation If a function has an integral, it is said to be integrable. The function for which the integral is calculated is called the integrand. The region over which a function is being integrated is called the domain of integration. Usually this domain will be an interval, in which case it is enough to give the limits of that interval, which are called the limits of integration. If the integral does not have a domain of integration, it is considered indefinite (one with a domain is considered definite). In general, the integrand may be a function of more than one variable, and the domain of integration may be an area, volume, a higher dimensional region, or even an abstract space that does not have a geometric structure in any usual sense (such as a sample space in probability theory). 156 Integral 157 The simplest case, the integral of a real-valued function f of one real variable x on the interval [a, b], is denoted by The ∫ sign represents integration; a and b are the lower limit and upper limit, respectively, of integration, defining the domain of integration; f is the integrand, to be evaluated as x varies over the interval [a,b]; and dx is the variable of integration. In correct mathematical typography, the dx is separated from the integrand by a space (as shown). Some authors use an upright d (that is, dx instead of dx). The variable of integration dx has different interpretations depending on the theory being used. For example, it can be seen as strictly a notation indicating that x is a dummy variable of integration, as a reflection of the weights in the Riemann sum, a measure (in Lebesgue integration and its extensions), an infinitesimal (in non-standard analysis) or as an independent mathematical quantity: a differential form. More complicated cases may vary the notation slightly. In the modern Arabic mathematical notation, which aims at pre-university levels of education in the Arab world and is written from right to left, a reflected integral symbol is used (W3C 2006). Introduction Integrals appear in many practical situations. If a swimming pool is rectangular with a flat bottom, then from its length, width, and depth we can easily determine the volume of water it can contain (to fill it), the area of its surface (to cover it), and the length of its edge (to rope it). But if it is oval with a rounded bottom, all of these quantities call for integrals. Practical approximations may suffice for such trivial examples, but precision engineering (of any discipline) requires exact and rigorous values for these elements. To start off, consider the curve y = f(x) between x = 0 and x = 1, with f(x) = √x. We ask: What is the area under the function f, in the interval from 0 to 1? and call this (yet unknown) area the integral of f. The notation for this integral will be Approximations to integral of √x from 0 to 1, with ■ 5 right samples (above) and ■ 12 left samples (below) As a first approximation, look at the unit square given by the sides x = 0 to x = 1 and y = f(0) = 0 and y = f(1) = 1. Its area is exactly 1. As it is, the true value of the integral must be somewhat less. Decreasing the width of the approximation rectangles shall give a better result; so cross the interval in five steps, using the approximation points 0, 1⁄5, 2⁄5, and so on to 1. Fit a box for each step using the right end height of each curve piece, thus √1⁄5, √2⁄5, and so on to √1 = 1. Summing the areas of these rectangles, we get a better approximation for the sought integral, namely Notice that we are taking a sum of finitely many function values of f, multiplied with the differences of two subsequent approximation points. We can easily see that the approximation is still too large. Using more steps Integral produces a closer approximation, but will never be exact: replacing the 5 subintervals by twelve as depicted, we will get an approximate value for the area of 0.6203, which is too small. The key idea is the transition from adding finitely many differences of approximation points multiplied by their respective function values to using infinitely many fine, or infinitesimal steps. As for the actual calculation of integrals, the fundamental theorem of calculus, due to Newton and Leibniz, is the fundamental link between the operations of differentiating and integrating. Applied to the square root curve, f(x) = x1/2, it says to look at the antiderivative F(x) = 2⁄3x3/2, and simply take F(1) − F(0), where 0 and 1 are the boundaries of the interval [0,1]. So the exact value of the area under the curve is computed formally as (This is a case of a general rule, that for f(x) = xq, with q ≠ −1, the related function, the so-called antiderivative is F(x) = (xq+1)/(q + 1).) The notation conceives the integral as a weighted sum, denoted by the elongated s, of function values, f(x), multiplied by infinitesimal step widths, the so-called differentials, denoted by dx. The multiplication sign is usually omitted. Historically, after the failure of early efforts to rigorously interpret infinitesimals, Riemann formally defined integrals as a limit of weighted sums, so that the dx suggested the limit of a difference (namely, the interval width). Shortcomings of Riemann's dependence on intervals and continuity motivated newer definitions, especially the Lebesgue integral, which is founded on an ability to extend the idea of "measure" in much more flexible ways. Thus the notation refers to a weighted sum in which the function values are partitioned, with μ measuring the weight to be assigned to each value. Here A denotes the region of integration. Differential geometry, with its "calculus on manifolds", gives the familiar notation yet another interpretation. Now f(x) and dx become a differential form, ω = f(x) dx, a new differential operator d, known as the exterior derivative is introduced, and the fundamental theorem becomes the more general Stokes' theorem, from which Green's theorem, the divergence theorem, and the fundamental theorem of calculus follow. More recently, infinitesimals have reappeared with rigor, through modern innovations such as non-standard analysis. Not only do these methods vindicate the intuitions of the pioneers; they also lead to new mathematics. Although there are differences between these conceptions of integral, there is considerable overlap. Thus, the area of the surface of the oval swimming pool can be handled as a geometric ellipse, a sum of infinitesimals, a Riemann integral, a Lebesgue integral, or as a manifold with a differential form. The calculated result will be the same for all. 158 Integral 159 Formal definitions There are many ways of formally defining an integral, not all of which are equivalent. The differences exist mostly to deal with differing special cases which may not be integrable under other definitions, but also occasionally for pedagogical reasons. The most commonly used definitions of integral are Riemann integrals and Lebesgue integrals. Riemann integral The Riemann integral is defined in terms of Riemann sums of functions with respect to tagged partitions of an interval. Let [a,b] be a closed interval of the real line; then a tagged partition of [a,b] is a finite sequence Integral approached as Riemann sum based on tagged partition, with irregular sampling positions and widths (max in red). True value is 3.76; estimate is 3.648. This partitions the interval [a,b] into n sub-intervals [xi−1, xi] indexed by i, each of which is "tagged" with a distinguished point ti ∈ [xi−1, xi]. A Riemann sum of a function f with respect to such a tagged partition is defined as Riemann sums converging as intervals halve, whether sampled at ■ right, ■ minimum, ■ maximum, or ■ left. thus each term of the sum is the area of a rectangle with height equal to the function value at the distinguished point of the given sub-interval, and width the same as the sub-interval width. Let Δi = xi−xi−1 be the width of sub-interval Integral 160 i; then the mesh of such a tagged partition is the width of the largest sub-interval formed by the partition, maxi=1…n Δi. The Riemann integral of a function f over the interval [a,b] is equal to S if: For all ε > 0 there exists δ > 0 such that, for any tagged partition [a,b] with mesh less than δ, we have When the chosen tags give the maximum (respectively, minimum) value of each interval, the Riemann sum becomes an upper (respectively, lower) Darboux sum, suggesting the close connection between the Riemann integral and the Darboux integral. Lebesgue integral The Riemann integral is not defined for a wide range of functions and situations of importance in applications (and of interest in theory). For example, the Riemann integral can easily integrate density to find the mass of a steel beam, but cannot accommodate a steel ball resting on it. This motivates other definitions, under which a broader assortment of functions are integrable (Rudin 1987). The Lebesgue integral, in particular, achieves great flexibility by directing attention to the weights in the weighted sum. Riemann-Darboux's integration (blue) and Lebesgue integration The definition of the Lebesgue integral thus begins (red). with a measure, μ. In the simplest case, the Lebesgue measure μ(A) of an interval A = [a,b] is its width, b − a, so that the Lebesgue integral agrees with the (proper) Riemann integral when both exist. In more complicated cases, the sets being measured can be highly fragmented, with no continuity and no resemblance to intervals. To exploit this flexibility, Lebesgue integrals reverse the approach to the weighted sum. As Folland (1984, p. 56) puts it, "To compute the Riemann integral of f, one partitions the domain [a,b] into subintervals", while in the Lebesgue integral, "one is in effect partitioning the range of f". One common approach first defines the integral of the indicator function of a measurable set A by: . This extends by linearity to a measurable simple function s, which attains only a finite number, n, of distinct non-negative values: (where the image of Ai under the simple function s is the constant value ai). Thus if E is a measurable set one defines Then for any non-negative measurable function f one defines Integral that is, the integral of f is set to be the supremum of all the integrals of simple functions that are less than or equal to f. A general measurable function f is split into its positive and negative values by defining Finally, f is Lebesgue integrable if and then the integral is defined by When the measure space on which the functions are defined is also a locally compact topological space (as is the case with the real numbers R), measures compatible with the topology in a suitable sense (Radon measures, of which the Lebesgue measure is an example) and integral with respect to them can be defined differently, starting from the integrals of continuous functions with compact support. More precisely, the compactly supported functions form a vector space that carries a natural topology, and a (Radon) measure can be defined as any continuous linear functional on this space; the value of a measure at a compactly supported function is then also by definition the integral of the function. One then proceeds to expand the measure (the integral) to more general functions by continuity, and defines the measure of a set as the integral of its indicator function. This is the approach taken by Bourbaki (2004) and a certain number of other authors. For details see Radon measures. Other integrals Although the Riemann and Lebesgue integrals are the most widely used definitions of the integral, a number of others exist, including: • The Riemann–Stieltjes integral, an extension of the Riemann integral. • The Lebesgue-Stieltjes integral, further developed by Johann Radon, which generalizes the Riemann–Stieltjes and Lebesgue integrals. • The Daniell integral, which subsumes the Lebesgue integral and Lebesgue-Stieltjes integral without the dependence on measures. • The Henstock-Kurzweil integral, variously defined by Arnaud Denjoy, Oskar Perron, and (most elegantly, as the gauge integral) Jaroslav Kurzweil, and developed by Ralph Henstock. • The Itō integral and Stratonovich integral, which define integration with respect to semimartingales such as Brownian motion. • The Young integral, which is a kind of Riemann-Stieltjes integral with respect to certain functions of unbounded variation. • The rough path integral defined for functions equipped with some additional "rough path" structure, generalizing stochastic integration against both semimartingales and processes such as the fractional Brownian motion. 161 Integral 162 Properties Linearity • The collection of Riemann integrable functions on a closed interval [a, b] forms a vector space under the operations of pointwise addition and multiplication by a scalar, and the operation of integration is a linear functional on this vector space. Thus, firstly, the collection of integrable functions is closed under taking linear combinations; and, secondly, the integral of a linear combination is the linear combination of the integrals, • Similarly, the set of real-valued Lebesgue integrable functions on a given measure space E with measure μ is closed under taking linear combinations and hence form a vector space, and the Lebesgue integral is a linear functional on this vector space, so that • More generally, consider the vector space of all measurable functions on a measure space (E,μ), taking values in a locally compact complete topological vector space V over a locally compact topological field K, f : E → V. Then one may define an abstract integration map assigning to each function f an element of V or the symbol ∞, that is compatible with linear combinations. In this situation the linearity holds for the subspace of functions whose integral is an element of V (i.e. "finite"). The most important special cases arise when K is R, C, or a finite extension of the field Qp of p-adic numbers, and V is a finite-dimensional vector space over K, and when K=C and V is a complex Hilbert space. Linearity, together with some natural continuity properties and normalisation for a certain class of "simple" functions, may be used to give an alternative definition of the integral. This is the approach of Daniell for the case of real-valued functions on a set X, generalized by Nicolas Bourbaki to functions with values in a locally compact topological vector space. See (Hildebrandt 1953) for an axiomatic characterisation of the integral. Inequalities for integrals A number of general inequalities hold for Riemann-integrable functions defined on a closed and bounded interval [a, b] and can be generalized to other notions of integral (Lebesgue and Daniell). • Upper and lower bounds. An integrable function f on [a, b], is necessarily bounded on that interval. Thus there are real numbers m and M so that m ≤ f (x) ≤ M for all x in [a, b]. Since the lower and upper sums of f over [a, b] are therefore bounded by, respectively, m(b − a) and M(b − a), it follows that • Inequalities between functions. If f(x) ≤ g(x) for each x in [a, b] then each of the upper and lower sums of f is bounded above by the upper and lower sums, respectively, of g. Thus Integral 163 This is a generalization of the above inequalities, as M(b − a) is the integral of the constant function with value M over [a, b]. In addition, if the inequality between functions is strict, then the inequality between integrals is also strict. That is, if f(x) < g(x) for each x in [a, b], then • Subintervals. If [c, d] is a subinterval of [a, b] and f(x) is non-negative for all x, then • Products and absolute values of functions. If f and g are two functions then we may consider their pointwise products and powers, and absolute values: If f is Riemann-integrable on [a, b] then the same is true for |f|, and Moreover, if f and g are both Riemann-integrable then f 2, g 2, and fg are also Riemann-integrable, and This inequality, known as the Cauchy–Schwarz inequality, plays a prominent role in Hilbert space theory, where the left hand side is interpreted as the inner product of two square-integrable functions f and g on the interval [a, b]. • Hölder's inequality. Suppose that p and q are two real numbers, 1 ≤ p, q ≤ ∞ with 1/p + 1/q = 1, and f and g are two Riemann-integrable functions. Then the functions |f|p and |g|q are also integrable and the following Hölder's inequality holds: For p = q = 2, Hölder's inequality becomes the Cauchy–Schwarz inequality. • Minkowski inequality. Suppose that p ≥ 1 is a real number and f and g are Riemann-integrable functions. Then |f|p, |g|p and |f + g|p are also Riemann integrable and the following Minkowski inequality holds: An analogue of this inequality for Lebesgue integral is used in construction of Lp spaces. Conventions In this section f is a real-valued Riemann-integrable function. The integral over an interval [a, b] is defined if a < b. This means that the upper and lower sums of the function f are evaluated on a partition a = x0 ≤ x1 ≤ . . . ≤ xn = b whose values xi are increasing. Geometrically, this signifies that integration takes place "left to right", evaluating f within intervals [x i , x i +1] where an interval with a higher index lies to the right of one with a lower index. The values a and b, the end-points of the interval, are called the limits of integration of f. Integrals can also be defined if a > b: • Reversing limits of integration. If a > b then define Integral 164 This, with a = b, implies: • Integrals over intervals of length zero. If a is a real number then The first convention is necessary in consideration of taking integrals over subintervals of [a, b]; the second says that an integral taken over a degenerate interval, or a point, should be zero. One reason for the first convention is that the integrability of f on an interval [a, b] implies that f is integrable on any subinterval [c, d], but in particular integrals have the property that: • Additivity of integration on intervals. If c is any element of [a, b], then With the first convention the resulting relation is then well-defined for any cyclic permutation of a, b, and c. Instead of viewing the above as conventions, one can also adopt the point of view that integration is performed of differential forms on oriented manifolds only. If M is such an oriented m-dimensional manifold, and M is the same manifold with opposed orientation and ω is an m-form, then one has: These conventions correspond to interpreting the integrand as a differential form, integrated over a chain. In measure theory, by contrast, one interprets the integrand as a function f with respect to a measure and integrates over a subset A, without any notion of orientation; one writes to indicate integration over a subset A. This is a minor distinction in one dimension, but becomes subtler on higher dimensional manifolds; see Differential form: Relation with measures for details. Fundamental theorem of calculus The fundamental theorem of calculus is the statement that differentiation and integration are inverse operations: if a continuous function is first integrated and then differentiated, the original function is retrieved. An important consequence, sometimes called the second fundamental theorem of calculus, allows one to compute integrals by using an antiderivative of the function to be integrated. Statements of theorems • Fundamental theorem of calculus. Let ƒ be a continuous real-valued function defined on a closed interval [a, b]. Let F be the function defined, for all x in [a, b], by Then, F is continuous on [a, b], differentiable on the open interval (a, b), and for all x in (a, b). Integral 165 • Second fundamental theorem of calculus. Let ƒ be a real-valued function defined on a closed interval [a, b] that admits an antiderivative g on [a, b]. That is, ƒ and g are functions such that for all x in [a, b], If ƒ is integrable on [a, b] then Extensions Improper integrals A "proper" Riemann integral assumes the integrand is defined and finite on a closed and bounded interval, bracketed by the limits of integration. An improper integral occurs when one or more of these conditions is not satisfied. In some cases such integrals may be defined by considering the limit of a sequence of proper Riemann integrals on progressively larger intervals. If the interval is unbounded, for instance at its upper end, then the improper integral is the limit as that endpoint goes to infinity. The improper integral has unbounded intervals for both domain and range. If the integrand is only defined or finite on a half-open interval, for instance (a,b], then again a limit may provide a finite result. That is, the improper integral is the limit of proper integrals as one endpoint of the interval of integration approaches either a specified real number, or ∞, or −∞. In more complicated cases, limits are required at both endpoints, or at interior points. Consider, for example, the function integrated from 0 to ∞ (shown right). At the lower bound, as x goes to 0 the function goes to ∞, and the upper bound is itself ∞, though the function goes to 0. Thus this is a doubly improper integral. Integrated, say, from 1 to 3, an ordinary Riemann sum suffices to produce a result of . To integrate from 1 to ∞, a Riemann sum is not possible. However, any finite upper bound, say t (with t > 1), gives a well-defined result, . This has a finite limit as t goes to infinity, namely . Similarly, the integral from 1⁄3 to 1 allows a Riemann sum as well, coincidentally again producing . Replacing 1⁄3 by an arbitrary positive value s (with s < 1) is equally safe, giving . This, too, has a finite limit as s goes to zero, namely . Combining the limits of the two fragments, the result of this improper integral is Integral 166 This process does not guarantee success; a limit may fail to exist, or may be unbounded. For example, over the bounded interval 0 to 1 the integral of does not converge; and over the unbounded interval 1 to ∞ the integral of does not converge. It may also happen that an integrand is unbounded at an interior point, in which case the integral must be split at that point, and the limit integrals on both sides must exist and must be bounded. Thus The improper integral is unbounded internally, but both left and right limits exist. But the similar integral cannot be assigned a value in this way, as the integrals above and below zero do not independently converge. (However, see Cauchy principal value.) Integral 167 Multiple integration Integrals can be taken over regions other than intervals. In general, an integral over a set E of a function f is written: Double integral as volume under a surface. Here x need not be a real number, but can be another suitable quantity, for instance, a vector in R3. Fubini's theorem shows that such integrals can be rewritten as an iterated integral. In other words, the integral can be calculated by integrating one coordinate at a time. Just as the definite integral of a positive function of one variable represents the area of the region between the graph of the function and the x-axis, the double integral of a positive function of two variables represents the volume of the region between the surface defined by the function and the plane which contains its domain. (The same volume can be obtained via the triple integral — the integral of a function in three variables — of the constant function f(x, y, z) = 1 over the above mentioned region between the surface and the plane.) If the number of variables is higher, then the integral represents a hypervolume, a volume of a solid of more than three dimensions that cannot be graphed. For example, the volume of the cuboid of sides 4 × 6 × 5 may be obtained in two ways: • By the double integral of the function f(x, y) = 5 calculated in the region D in the xy-plane which is the base of the cuboid. For example, if a rectangular base of such a cuboid is given via the xy inequalities 3 ≤ x ≤ 7, 4 ≤ y ≤ 10, our above double integral now reads From here, integration is conducted with respect to either x or y first; in this example, integration is first done with respect to x as the interval corresponding to x is the inner integral. Once the first integration is completed via the method or otherwise, the result is again integrated with respect to the other variable. The result will equate to the volume under the surface. • By the triple integral of the constant function 1 calculated on the cuboid itself. Integral 168 Line integrals The concept of an integral can be extended to more general domains of integration, such as curved lines and surfaces. Such integrals are known as line integrals and surface integrals respectively. These have important applications in physics, as when dealing with vector fields. A line integral (sometimes called a path integral) is an integral where the function to be integrated is evaluated along a curve. Various different line integrals are in use. In the case of a closed curve it is also called a contour integral. The function to be integrated may be a scalar field or a vector field. The value of the line integral is the sum of values of the field at all A line integral sums together elements along a points on the curve, weighted by some scalar function on the curve curve. (commonly arc length or, for a vector field, the scalar product of the vector field with a differential vector in the curve). This weighting distinguishes the line integral from simpler integrals defined on intervals. Many simple formulas in physics have natural continuous analogs in terms of line integrals; for example, the fact that work is equal to force, F, multiplied by displacement, s, may be expressed (in terms of vector quantities) as: For an object moving along a path in a vector field such as an electric field or gravitational field, the total work done by the field on the object is obtained by summing up the differential work done in moving from to . This gives the line integral Surface integrals A surface integral is a definite integral taken over a surface (which may be a curved set in space); it can be thought of as the double integral analog of the line integral. The function to be integrated may be a scalar field or a vector field. The value of the surface integral is the sum of the field at all points on the surface. This can be achieved by splitting the surface into surface elements, which provide the partitioning for Riemann sums. For an example of applications of surface integrals, consider a vector The definition of surface integral relies on field v on a surface S; that is, for each point x in S, v(x) is a vector. splitting the surface into small surface elements. Imagine that we have a fluid flowing through S, such that v(x) determines the velocity of the fluid at x. The flux is defined as the quantity of fluid flowing through S in unit amount of time. To find the flux, we need to take the dot product of v with the unit surface normal to S at each point, which will give us a scalar field, which we integrate over the surface: The fluid flux in this example may be from a physical fluid such as water or air, or from electrical or magnetic flux. Thus surface integrals have applications in physics, particularly with the classical theory of electromagnetism. Integral Integrals of differential forms A differential form is a mathematical concept in the fields of multivariable calculus, differential topology and tensors. The modern notation for the differential form, as well as the idea of the differential forms as being the wedge products of exterior derivatives forming an exterior algebra, was introduced by Élie Cartan. We initially work in an open set in Rn. A 0-form is defined to be a smooth function f. When we integrate a function f over an m-dimensional subspace S of Rn, we write it as (The superscripts are indices, not exponents.) We can consider dx1 through dxn to be formal objects themselves, rather than tags appended to make integrals look like Riemann sums. Alternatively, we can view them as covectors, and thus a measure of "density" (hence integrable in a general sense). We call the dx1, …,dxn basic 1-forms. We define the wedge product, "∧", a bilinear "multiplication" operator on these elements, with the alternating property that for all indices a. Note that alternation along with linearity and associativity implies dxb∧dxa = −dxa∧dxb. This also ensures that the result of the wedge product has an orientation. We define the set of all these products to be basic 2-forms, and similarly we define the set of products of the form dxa∧dxb∧dxc to be basic 3-forms. A general k-form is then a weighted sum of basic k-forms, where the weights are the smooth functions f. Together these form a vector space with basic k-forms as the basis vectors, and 0-forms (smooth functions) as the field of scalars. The wedge product then extends to k-forms in the natural way. Over Rn at most n covectors can be linearly independent, thus a k-form with k > n will always be zero, by the alternating property. In addition to the wedge product, there is also the exterior derivative operator d. This operator maps k-forms to (k+1)-forms. For a k-form ω = f dxa over Rn, we define the action of d by: with extension to general k-forms occurring linearly. This more general approach allows for a more natural coordinate-free approach to integration on manifolds. It also allows for a natural generalisation of the fundamental theorem of calculus, called Stokes' theorem, which we may state as where ω is a general k-form, and ∂Ω denotes the boundary of the region Ω. Thus, in the case that ω is a 0-form and Ω is a closed interval of the real line, this reduces to the fundamental theorem of calculus. In the case that ω is a 1-form and Ω is a two-dimensional region in the plane, the theorem reduces to Green's theorem. Similarly, using 2-forms, and 3-forms and Hodge duality, we can arrive at Stokes' theorem and the divergence theorem. In this way we can see that differential forms provide a powerful unifying view of integration. 169 Integral 170 Summations The discrete equivalent of integration is summation. Summations and integrals can be put on the same foundations using the theory of Lebesgue integrals or time scale calculus. Methods Computing integrals The most basic technique for computing definite integrals of one real variable is based on the fundamental theorem of calculus. Let f(x) be the function of x to be integrated over a given interval [a, b]. Then, find an antiderivative of f; that is, a function F such that F' = f on the interval. Provided the integrand and integral have no singularities on the path of integration, by the fundamental theorem of calculus, The integral is not actually the antiderivative, but the fundamental theorem provides a way to use antiderivatives to evaluate definite integrals. The most difficult step is usually to find the antiderivative of f. It is rarely possible to glance at a function and write down its antiderivative. More often, it is necessary to use one of the many techniques that have been developed to evaluate integrals. Most of these techniques rewrite one integral as a different one which is hopefully more tractable. Techniques include: • • • • • • • • • • Integration by substitution Integration by parts Changing the order of integration Integration by trigonometric substitution Integration by partial fractions Integration by reduction formulae Integration using parametric derivatives Integration using Euler's formula Differentiation under the integral sign Contour integration Alternate methods exist to compute more complex integrals. Many nonelementary integrals can be expanded in a Taylor series and integrated term by term. Occasionally, the resulting infinite series can be summed analytically. The method of convolution using Meijer G-functions can also be used, assuming that the integrand can be written as a product of Meijer G-functions. There are also many less common ways of calculating definite integrals; for instance, Parseval's identity can be used to transform an integral over a rectangular region into an infinite sum. Occasionally, an integral can be evaluated by a trick; for an example of this, see Gaussian integral. Computations of volumes of solids of revolution can usually be done with disk integration or shell integration. Specific results which have been worked out by various techniques are collected in the list of integrals. Symbolic algorithms Many problems in mathematics, physics, and engineering involve integration where an explicit formula for the integral is desired. Extensive tables of integrals have been compiled and published over the years for this purpose. With the spread of computers, many professionals, educators, and students have turned to computer algebra systems that are specifically designed to perform difficult or tedious tasks, including integration. Symbolic integration has been one of the motivations for the development of the first such systems, like Macsyma. A major mathematical difficulty in symbolic integration is that in many cases, a closed formula for the antiderivative of a rather simple-looking function does not exist. For instance, it is known that the antiderivatives of the functions exp ( x2), xx and sin x /x cannot be expressed in the closed form involving only rational and exponential functions, Integral logarithm, trigonometric and inverse trigonometric functions, and the operations of multiplication and composition; in other words, none of the three given functions is integrable in elementary functions, which are the functions which may be built from rational functions, roots of a polynomial, logarithm, and exponential functions. Risch algorithm provides a general criterion to determine whether the antiderivative of an elementary function is elementary, and, if it is, to compute it. Unfortunately, it turns out that functions with closed expressions of antiderivatives are the exception rather than the rule. Consequently, computerized algebra systems have no hope of being able to find an antiderivative for a randomly constructed elementary function. On the positive side, if the 'building blocks' for antiderivatives are fixed in advance, it may be still be possible to decide whether the antiderivative of a given function can be expressed using these blocks and operations of multiplication and composition, and to find the symbolic answer whenever it exists. The Risch algorithm, implemented in Mathematica and other computer algebra systems, does just that for functions and antiderivatives built from rational functions, radicals, logarithm, and exponential functions. Some special integrands occur often enough to warrant special study. In particular, it may be useful to have, in the set of antiderivatives, the special functions of physics (like the Legendre functions, the hypergeometric function, the Gamma function, the Incomplete Gamma function and so on - see Symbolic integration for more details). Extending the Risch's algorithm to include such functions is possible but challenging and has been an active research subject. More recently a new approach has emerged, using D-finite function, which are the solutions of linear differential equations with polynomial coefficients. Most of the elementary and special functions are D-finite and the integral of a D-finite function is also a D-finite function. This provide an algorithm to express the antiderivative of a D-finite function as the solution of a differential equation. This theory allows also to compute a definite integrals of a D-function as the sum of a series given by the first coefficients and an algorithm to compute any coefficient.[4] Numerical quadrature The integrals encountered in a basic calculus course are deliberately chosen for simplicity; those found in real applications are not always so accommodating. Some integrals cannot be found exactly, some require special functions which themselves are a challenge to compute, and others are so complex that finding the exact answer is too slow. This motivates the study and application of numerical methods for approximating integrals, which today use floating-point arithmetic on digital electronic computers. Many of the ideas arose much earlier, for hand calculations; but the speed of general-purpose computers like the ENIAC created a need for improvements. The goals of numerical integration are accuracy, reliability, efficiency, and generality. Sophisticated methods can vastly outperform a naive method by all four measures (Dahlquist & Björck 2008; Kahaner, Moler & Nash 1989; Stoer & Bulirsch 2002). Consider, for example, the integral which has the exact answer 94⁄25 = 3.76. (In ordinary practice the answer is not known in advance, so an important task — not explored here — is to decide when an approximation is good enough.) A “calculus book” approach divides the integration range into, say, 16 equal pieces, and computes function values. 171 Integral 172 Spaced function values x −2.00 f(x) 2.22800 x f(x) −1.50 −1.00 −0.50 0.00 0.50 1.00 1.50 2.00 2.45663 2.67200 2.32475 0.64400 −0.92575 −0.94000 −0.16963 0.83600 −1.75 −1.25 −0.75 −0.25 0.25 0.75 1.25 1.75 2.33041 2.58562 2.62934 1.64019 −0.32444 −1.09159 −0.60387 0.31734 Using the left end of each piece, the rectangle method sums 16 function values and multiplies by the step width, h, here 0.25, to get an approximate value of 3.94325 for the integral. The accuracy is not impressive, but calculus formally uses pieces of infinitesimal width, so initially this may seem little cause for concern. Indeed, repeatedly doubling the number of steps eventually produces an approximation of 3.76001. However, 218 pieces are required, a great computational expense for such little accuracy; and a reach for greater accuracy can force steps so small that arithmetic precision becomes an obstacle. A better approach replaces the horizontal tops of the rectangles with slanted tops touching the function at the ends of each piece. This Numerical quadrature methods: ■ Rectangle, trapezium rule is almost as easy to calculate; it sums all 17 function ■ Trapezoid, ■ Romberg, ■ Gauss values, but weights the first and last by one half, and again multiplies by the step width. This immediately improves the approximation to 3.76925, which is noticeably more accurate. Furthermore, only 210 pieces are needed to achieve 3.76000, substantially less computation than the rectangle method for comparable accuracy. Romberg's method builds on the trapezoid method to great effect. First, the step lengths are halved incrementally, giving trapezoid approximations denoted by T(h0), T(h1), and so on, where hk+1 is half of hk. For each new step size, only half the new function values need to be computed; the others carry over from the previous size (as shown in the table above). But the really powerful idea is to interpolate a polynomial through the approximations, and extrapolate to T(0). With this method a numerically exact answer here requires only four pieces (five function values)! The Lagrange polynomial interpolating {hk,T(hk)}k=0…2 = {(4.00,6.128), (2.00,4.352), (1.00,3.908)} is 3.76+0.148h2, producing the extrapolated value 3.76 at h = 0. Gaussian quadrature often requires noticeably less work for superior accuracy. In this example, it can compute the function values at just two x positions, ±2⁄√3, then double each value and sum to get the numerically exact answer. The explanation for this dramatic success lies in error analysis, and a little luck. An n-point Gaussian method is exact for polynomials of degree up to 2n−1. The function in this example is a degree 3 polynomial, plus a term that cancels because the chosen endpoints are symmetric around zero. (Cancellation also benefits the Romberg method.) Shifting the range left a little, so the integral is from −2.25 to 1.75, removes the symmetry. Nevertheless, the trapezoid method is rather slow, the polynomial interpolation method of Romberg is acceptable, and the Gaussian method requires the least work — if the number of points is known in advance. As well, rational interpolation can use the same trapezoid evaluations as the Romberg method to greater effect. Integral 173 Quadrature method cost comparison Method Points Rel. Err. Trapezoid Romberg Rational Gauss 1048577 257 129 36 −5.3×10−13 −6.3×10−15 8.8×10−15 3.1×10−15 Value In practice, each method must use extra evaluations to ensure an error bound on an unknown function; this tends to offset some of the advantage of the pure Gaussian method, and motivates the popular Gauss–Kronrod quadrature formulae. Symmetry can still be exploited by splitting this integral into two ranges, from −2.25 to −1.75 (no symmetry), and from −1.75 to 1.75 (symmetry). More broadly, adaptive quadrature partitions a range into pieces based on function properties, so that data points are concentrated where they are needed most. Simpson's rule, named for Thomas Simpson (1710–1761), uses a parabolic curve to approximate integrals. In many cases, it is more accurate than the trapezoidal rule and others. The rule states that with an error of The computation of higher-dimensional integrals (for example, volume calculations) makes important use of such alternatives as Monte Carlo integration. A calculus text is no substitute for numerical analysis, but the reverse is also true. Even the best adaptive numerical code sometimes requires a user to help with the more demanding integrals. For example, improper integrals may require a change of variable or methods that can avoid infinite function values, and known properties like symmetry and periodicity may provide critical leverage. Practical applications Area under the curve (abbreviated AUC) is frequently used in pharmacokinetics for functions where the x-axis represents time and the y-axis represents drug concentration. For such functions, the area under the curve usually correlates fairly well with the total effect on the body that the drug will have. In standard use, AUC is defined as either: • AUC∞, the integral after a single dose with a hypothetical infinite x-axis • AUCτ, the integral in the time interval between doses given regularly, and after having reached steady state. The time course of drug plasma concentrations over 96 hours following oral administrations every 24 hours. Note that the AUC in steady state equals AUC∞ after the first dose. Integral Notes [1] Shea, Marilyn (May 2007), Biography of Zu Chongzhi (http:/ / hua. umf. maine. edu/ China/ astronomy/ tianpage/ 0014ZuChongzhi9296bw. html), University of Maine, , retrieved 9 January 2009 Katz, Victor J. (2004), A History of Mathematics, Brief Version, Addison-Wesley, pp. 125–126, ISBN 978-0-321-16193-2 [2] Victor J. Katz (1995), "Ideas of Calculus in Islam and India", Mathematics Magazine 68 (3): 163–174 [165–9 & 173–4] [3] http:/ / www2. gol. com/ users/ coynerhm/ 0598rothman. html [4] http:/ / algo. inria. fr/ chyzak/ mgfun. html References • Apostol, Tom M. (1967), Calculus, Vol. 1: One-Variable Calculus with an Introduction to Linear Algebra (2nd ed.), Wiley, ISBN 978-0-471-00005-1 • Bourbaki, Nicolas (2004), Integration I, Springer Verlag, ISBN 3-540-41129-1. In particular chapters III and IV. • Burton, David M. (2005), The History of Mathematics: An Introduction (6th ed.), McGraw-Hill, p. 359, ISBN 978-0-07-305189-5 • Cajori, Florian (1929), A History Of Mathematical Notations Volume II (http://www.archive.org/details/ historyofmathema027671mbp), Open Court Publishing, pp. 247–252, ISBN 978-0-486-67766-8 • Dahlquist, Germund; Björck, Åke (2008), "Chapter 5: Numerical Integration" (http://www.mai.liu.se/~akbjo/ NMbook.html), Numerical Methods in Scientific Computing, Volume I, Philadelphia: SIAM • Folland, Gerald B. (1984), Real Analysis: Modern Techniques and Their Applications (1st ed.), John Wiley & Sons, ISBN 978-0-471-80958-6 • Fourier, Jean Baptiste Joseph (1822), Théorie analytique de la chaleur (http://books.google.com/ books?id=TDQJAAAAIAAJ), Chez Firmin Didot, père et fils, p. §231 Available in translation as Fourier, Joseph (1878), The analytical theory of heat (http://www.archive.org/ details/analyticaltheory00fourrich), Freeman, Alexander (trans.), Cambridge University Press, pp. 200–201 • Heath, T. L., ed. (2002), The Works of Archimedes (http://www.archive.org/details/ worksofarchimede029517mbp), Dover, ISBN 978-0-486-42084-4 (Originally published by Cambridge University Press, 1897, based on J. L. Heiberg's Greek version.) • Hildebrandt, T. H. (1953), "Integration in abstract spaces" (http://projecteuclid.org/euclid.bams/1183517761), Bulletin of the American Mathematical Society 59 (2): 111–139, ISSN 0273-0979 • Kahaner, David; Moler, Cleve; Nash, Stephen (1989), "Chapter 5: Numerical Quadrature", Numerical Methods and Software, Prentice Hall, ISBN 978-0-13-627258-8 • Leibniz, Gottfried Wilhelm (1899), Gerhardt, Karl Immanuel, ed., Der Briefwechsel von Gottfried Wilhelm Leibniz mit Mathematikern. Erster Band (http://name.umdl.umich.edu/AAX2762.0001.001), Berlin: Mayer & Müller • Miller, Jeff, Earliest Uses of Symbols of Calculus (http://jeff560.tripod.com/calculus.html), retrieved 2009-11-22 • O’Connor, J. J.; Robertson, E. F. (1996), A history of the calculus (http://www-history.mcs.st-andrews.ac.uk/ HistTopics/The_rise_of_calculus.html), retrieved 2007-07-09 • Rudin, Walter (1987), "Chapter 1: Abstract Integration", Real and Complex Analysis (International ed.), McGraw-Hill, ISBN 978-0-07-100276-9 • Saks, Stanisław (1964), Theory of the integral (http://matwbn.icm.edu.pl/kstresc.php?tom=7&wyd=10& jez=) (English translation by L. C. Young. With two additional notes by Stefan Banach. Second revised ed.), New York: Dover • Stoer, Josef; Bulirsch, Roland (2002), "Chapter 3: Topics in Integration", Introduction to Numerical Analysis (3rd ed.), Springer, ISBN 978-0-387-95452-3. • W3C (2006), Arabic mathematical notation (http://www.w3.org/TR/arabic-math/) 174 Integral External links • Riemann Sum (http://mathworld.wolfram.com/RiemannSum.html) by Wolfram Research • Introduction to definite integrals (http://www.khanacademy.org/video/ introduction-to-definite-integrals?playlist=Calculus) by Khan Academy Online books • Keisler, H. Jerome, Elementary Calculus: An Approach Using Infinitesimals (http://www.math.wisc.edu/ ~keisler/calc.html), University of Wisconsin • Stroyan, K.D., A Brief Introduction to Infinitesimal Calculus (http://www.math.uiowa.edu/~stroyan/ InfsmlCalculus/InfsmlCalc.htm), University of Iowa • Mauch, Sean, Sean's Applied Math Book (http://www.its.caltech.edu/~sean/book/unabridged.html), CIT, an online textbook that includes a complete introduction to calculus • Crowell, Benjamin, Calculus (http://www.lightandmatter.com/calc/), Fullerton College, an online textbook • Garrett, Paul, Notes on First-Year Calculus (http://www.math.umn.edu/~garrett/calculus/) • Hussain, Faraz, Understanding Calculus (http://www.understandingcalculus.com), an online textbook • Kowalk, W.P., Integration Theory (http://einstein.informatik.uni-oldenburg.de/20910.html), University of Oldenburg. A new concept to an old problem. Online textbook • Sloughter, Dan, Difference Equations to Differential Equations (http://math.furman.edu/~dcs/book), an introduction to calculus • Numerical Methods of Integration (http://numericalmethods.eng.usf.edu/topics/integration.html) at Holistic Numerical Methods Institute • P.S. Wang, Evaluation of Definite Integrals by Symbolic Manipulation (http://www.lcs.mit.edu/publications/ specpub.php?id=660) (1972) - a cookbook of definite integral techniques 175 Derivative 176 Derivative In calculus, a branch of mathematics, the derivative is a measure of how a function changes as its input changes. Loosely speaking, a derivative can be thought of as how much one quantity is changing in response to changes in some other quantity; for example, the derivative of the position of a moving object with respect to time is the object's instantaneous velocity. The derivative of a function at a chosen input value describes the best linear approximation of the function near that input value. For a real-valued function of a single real variable, the derivative at a point equals the slope of the tangent line to the graph of the function at that point. In higher dimensions, the derivative of a function at a point is a linear transformation called the linearization.[1] A closely related notion is the differential of a function. The graph of a function, drawn in black, and a tangent line to that function, drawn in red. The slope of the tangent line is equal to the derivative of the function at the marked point. The process of finding a derivative is called differentiation. The reverse process is called antidifferentiation. The fundamental theorem of calculus states that antidifferentiation is the same as integration. Differentiation and integration constitute the two fundamental operations in single-variable calculus. Differentiation and the derivative Differentiation is a method to compute the rate at which a dependent output y changes with respect to the change in the independent input x. This rate of change is called the derivative of y with respect to x. In more precise language, the dependence of y upon x means that y is a function of x. This functional relationship is often denoted y = ƒ(x), where ƒ denotes the function. If x and y are real numbers, and if the graph of y is plotted against x, the derivative measures the slope of this graph at each point. (Click for larger image) At each point, the derivative of is the slope of a line that is tangent to the curve. The line is always tangent to the blue curve; its slope is the derivative. Note derivative is positive where green, negative where red, and zero where black. The simplest case is when y is a linear function of x, meaning that the graph of y against x is a straight line. In this case, y = ƒ(x) = m x + b, for real numbers m and b, and the slope m is given by Derivative 177 where the symbol Δ (the uppercase form of the Greek letter Delta) is an abbreviation for "change in." This formula is true because y + Δy = ƒ(x+ Δx) = m (x + Δx) + b = m x + b + m Δx = y + mΔx. It follows that Δy = m Δx. This gives an exact value for the slope of a straight line. If the function ƒ is not linear (i.e. its graph is not a straight line), however, then the change in y divided by the change in x varies: differentiation is a method to find an exact value for this rate of change at any given value of x. Figure 1. The tangent line at (x, ƒ(x)) Figure 2. The secant to curve y= ƒ(x) determined by points (x, ƒ(x)) and (x+h, ƒ(x+h)) Figure 3. The tangent line as limit of secants The idea, illustrated by Figures 1-3, is to compute the rate of change as the limiting value of the ratio of the differences Δy / Δx as Δx becomes infinitely small. In Leibniz's notation, such an infinitesimal change in x is denoted by dx, and the derivative of y with respect to x is written suggesting the ratio of two infinitesimal quantities. (The above expression is read as "the derivative of y with respect to x", "d y by d x", or "d y over d x". The oral form "d y d x" is often used conversationally, although it may lead to confusion.) Derivative The most common approach[2] to turn this intuitive idea into a precise definition uses limits, but there are other methods, such as non-standard analysis.[3] Definition via difference quotients Let ƒ be a real valued function. In classical geometry, the tangent line to the graph of the function ƒ at a real number a was the unique line through the point (a, ƒ(a)) that did not meet the graph of ƒ transversally, meaning that the line did not pass straight through the graph. The derivative of y with respect to x at a is, geometrically, the slope of the tangent line to the graph of ƒ at a. The slope of the tangent line is very close to the slope of the line through (a, ƒ(a)) and a nearby point on the graph, for example (a + h, ƒ(a + h)). These lines are called secant lines. A value of h close to zero gives a good approximation to the slope of the tangent line, and smaller values (in absolute value) of h will, in general, give better approximations. The slope m of the secant line is the difference between the y values of these points divided by the difference between the x values, that is, This expression is Newton's difference quotient. The derivative is the value of the difference quotient as the secant lines approach the tangent line. Formally, the derivative of the function ƒ at a is the limit of the difference quotient as h approaches zero, if this limit exists. If the limit exists, then ƒ is differentiable at a. Here ƒ′ (a) is one of several common notations for the derivative (see below). Equivalently, the derivative satisfies the property that which has the intuitive interpretation (see Figure 1) that the tangent line to ƒ at a gives the best linear approximation to ƒ near a (i.e., for small h). This interpretation is the easiest to generalize to other settings (see below). Substituting 0 for h in the difference quotient causes division by zero, so the slope of the tangent line cannot be found directly using this method. Instead, define Q(h) to be the difference quotient as a function of h: Q(h) is the slope of the secant line between (a, ƒ(a)) and (a + h, ƒ(a + h)). If ƒ is a continuous function, meaning that its graph is an unbroken curve with no gaps, then Q is a continuous function away from h = 0. If the limit exists, meaning that there is a way of choosing a value for Q(0) that makes the graph of Q a continuous function, then the function ƒ is differentiable at a, and its derivative at a equals Q(0). In practice, the existence of a continuous extension of the difference quotient Q(h) to h = 0 is shown by modifying the numerator to cancel h in the denominator. This process can be long and tedious for complicated functions, and many shortcuts are commonly used to simplify the process. 178 Derivative 179 Example The squaring function ƒ(x) = x² is differentiable at x = 3, and its derivative there is 6. This result is established by calculating the limit as h approaches zero of the difference quotient of ƒ(3): The last expression shows that the difference quotient equals 6 + h when h ≠ 0 and is undefined when h = 0, because of the definition of the difference quotient. However, the definition of the limit says the difference quotient does not need to be defined when h = 0. The limit is the result of letting h go to zero, meaning it is the value that 6 + h tends to as h becomes very small: Hence the slope of the graph of the squaring function at the point (3, 9) is 6, and so its derivative at x = 3 is ƒ '(3) = 6. More generally, a similar computation shows that the derivative of the squaring function at x = a is ƒ '(a) = 2a. Continuity and differentiability If y = ƒ(x) is differentiable at a, then ƒ must also be continuous at a. As an example, choose a point a and let ƒ be the step function that returns a value, say 1, for all x less than a, and returns a different value, say 10, for all x greater than or equal to a. ƒ cannot have a derivative at a. If h is negative, then a + h is on the low part of the step, so the secant line from a to a + h is very steep, and as h tends to zero the slope tends to infinity. If h is positive, then a + h is on the high part of the step, so the secant line from a to a + h has slope zero. Consequently the secant lines do not approach any single slope, so the limit of the difference quotient does not exist.[4] However, even if a function is continuous at a point, it may not be differentiable there. For example, the absolute value function y = |x| is continuous at x = 0, but it is not differentiable there. If h is positive, then the slope of the secant line from 0 to h is one, whereas if h is negative, then the slope of the secant line from 0 to h is negative one. This can be seen graphically as a "kink" or a "cusp" in the graph at x = 0. Even a function with a smooth graph is not differentiable at a point where its tangent is vertical: For instance, the function y = 3√x is not differentiable at x = 0. In summary: for a function ƒ to have a derivative it is necessary for the function ƒ to be continuous, but continuity alone is not sufficient. This function does not have a derivative at the marked point, as the function is not continuous there. The absolute value function is continuous, but fails to be differentiable at x = 0 since the tangent slopes do not approach the same value from the left as they do from the right. Most functions that occur in practice have derivatives at all points or at almost every point. Early in the history of calculus, many mathematicians assumed that a continuous function was differentiable at most points. Under mild conditions, for example if the function is a monotone function or a Lipschitz function, this is true. However, in 1872 Weierstrass found the first example of a function that is continuous everywhere but differentiable nowhere. This example is now known as the Weierstrass function. In 1931, Stefan Banach proved that the set of functions that have a derivative at some point is a meager set in the space of all Derivative continuous functions.[5] Informally, this means that hardly any continuous functions have a derivative at even one point. The derivative as a function Let ƒ be a function that has a derivative at every point a in the domain of ƒ. Because every point a has a derivative, there is a function that sends the point a to the derivative of ƒ at a. This function is written f′(x) and is called the derivative function or the derivative of ƒ. The derivative of ƒ collects all the derivatives of ƒ at all the points in the domain of ƒ. Sometimes ƒ has a derivative at most, but not all, points of its domain. The function whose value at a equals f′(a) whenever f′(a) is defined and elsewhere is undefined is also called the derivative of ƒ. It is still a function, but its domain is strictly smaller than the domain of ƒ. Using this idea, differentiation becomes a function of functions: The derivative is an operator whose domain is the set of all functions that have derivatives at every point of their domain and whose range is a set of functions. If we denote this operator by D, then D(ƒ) is the function f′(x). Since D(ƒ) is a function, it can be evaluated at a point a. By the definition of the derivative function, D(ƒ)(a) = f′(a). For comparison, consider the doubling function ƒ(x) =2x; ƒ is a real-valued function of a real number, meaning that it takes numbers as inputs and has numbers as outputs: The operator D, however, is not defined on individual numbers. It is only defined on functions: Because the output of D is a function, the output of D can be evaluated at a point. For instance, when D is applied to the squaring function, D outputs the doubling function, which we named ƒ(x). This output function can then be evaluated to get ƒ(1) = 2, ƒ(2) = 4, and so on. Higher derivatives Let ƒ be a differentiable function, and let f′(x) be its derivative. The derivative of f′(x) (if it has one) is written f′′(x) and is called the second derivative of ƒ. Similarly, the derivative of a second derivative, if it exists, is written f′′′(x) and is called the third derivative of ƒ. These repeated derivatives are called higher-order derivatives. If x(t) represents the position of an object at time t, then the higher-order derivatives of x have physical interpretations. The second derivative of x is the derivative of x′(t), the velocity, and by definition this is the object's acceleration. The third derivative of x is defined to be the jerk, and the fourth derivative is defined to be the jounce. A function ƒ need not have a derivative, for example, if it is not continuous. Similarly, even if ƒ does have a derivative, it may not have a second derivative. For example, let Calculation shows that ƒ is a differentiable function whose derivative is 180 Derivative f′(x) is twice the absolute value function, and it does not have a derivative at zero. Similar examples show that a function can have k derivatives for any non-negative integer k but no (k + 1)-order derivative. A function that has k successive derivatives is called k times differentiable. If in addition the kth derivative is continuous, then the function is said to be of differentiability class Ck. (This is a stronger condition than having k derivatives. For an example, see differentiability class.) A function that has infinitely many derivatives is called infinitely differentiable or smooth. On the real line, every polynomial function is infinitely differentiable. By standard differentiation rules, if a polynomial of degree n is differentiated n times, then it becomes a constant function. All of its subsequent derivatives are identically zero. In particular, they exist, so polynomials are smooth functions. The derivatives of a function ƒ at a point x provide polynomial approximations to that function near x. For example, if ƒ is twice differentiable, then in the sense that If ƒ is infinitely differentiable, then this is the beginning of the Taylor series for ƒ. Inflection point A point where the second derivative of a function changes sign is called an inflection point.[6] At an inflection point, the second derivative may be zero, as in the case of the inflection point x=0 of the function y=x3, or it may fail to exist, as in the case of the inflection point x=0 of the function y=x1/3. At an inflection point, a function switches from being a convex function to being a concave function or vice versa. Notations for differentiation Leibniz's notation The notation for derivatives introduced by Gottfried Leibniz is one of the earliest. It is still commonly used when the equation y = ƒ(x) is viewed as a functional relationship between dependent and independent variables. Then the first derivative is denoted by and was once thought of as an infinitesimal quotient. Higher derivatives are expressed using the notation for the nth derivative of y = ƒ(x) (with respect to x). These are abbreviations for multiple applications of the derivative operator. For example, With Leibniz's notation, we can write the derivative of y at the point x = a in two different ways: 181 Derivative 182 Leibniz's notation allows one to specify the variable for differentiation (in the denominator). This is especially relevant for partial differentiation. It also makes the chain rule easy to remember:[7] Lagrange's notation Sometimes referred to as prime notation,[8] one of the most common modern notations for differentiation is due to Joseph-Louis Lagrange and uses the prime mark, so that the derivative of a function ƒ(x) is denoted ƒ′(x) or simply ƒ′. Similarly, the second and third derivatives are denoted and To denote the number of derivatives beyond this point, some authors use Roman numerals in superscript, whereas others place the number in parentheses: or The latter notation generalizes to yield the notation ƒ (n) for the nth derivative of ƒ — this notation is most useful when we wish to talk about the derivative as being a function itself, as in this case the Leibniz notation can become cumbersome. Newton's notation Newton's notation for differentiation, also called the dot notation, places a dot over the function name to represent a time derivative. If y = ƒ(t), then and denote, respectively, the first and second derivatives of y with respect to t. This notation is used exclusively for time derivatives, meaning that the independent variable of the function represents time. It is very common in physics and in mathematical disciplines connected with physics such as differential equations. While the notation becomes unmanageable for high-order derivatives, in practice only very few derivatives are needed. Euler's notation Euler's notation uses a differential operator D, which is applied to a function ƒ to give the first derivative Df. The second derivative is denoted D2ƒ, and the nth derivative is denoted Dnƒ. If y = ƒ(x) is a dependent variable, then often the subscript x is attached to the D to clarify the independent variable x. Euler's notation is then written or , although this subscript is often omitted when the variable x is understood, for instance when this is the only variable present in the expression. Euler's notation is useful for stating and solving linear differential equations. Derivative 183 Computing the derivative The derivative of a function can, in principle, be computed from the definition by considering the difference quotient, and computing its limit. In practice, once the derivatives of a few simple functions are known, the derivatives of other functions are more easily computed using rules for obtaining derivatives of more complicated functions from simpler ones. Derivatives of elementary functions Most derivative computations eventually require taking the derivative of some common functions. The following incomplete list gives some of the most frequently used functions of a single real variable and their derivatives. • Derivatives of powers: if where r is any real number, then wherever this function is defined. For example, if , then and the derivative function is defined only for positive x, not for x = 0. When r = 0, this rule implies that f′(x) is zero for x ≠ 0, which is almost the constant rule (stated below). • Exponential and logarithmic functions: • Trigonometric functions: • Inverse trigonometric functions: Derivative 184 Rules for finding the derivative In many cases, complicated limit calculations by direct application of Newton's difference quotient can be avoided using differentiation rules. Some of the most basic rules are the following. • Constant rule: if ƒ(x) is constant, then • Sum rule: for all functions ƒ and g and all real numbers a and b. • Product rule: for all functions ƒ and g. • Quotient rule: for all functions ƒ and g where g ≠ 0. • Chain rule: If , then Example computation The derivative of is Here the second term was computed using the chain rule and third using the product rule. The known derivatives of the elementary functions x2, x4, sin(x), ln(x) and exp(x) = ex, as well as the constant 7, were also used. Derivatives in higher dimensions Derivatives of vector valued functions A vector-valued function y(t) of a real variable sends real numbers to vectors in some vector space Rn. A vector-valued function can be split up into its coordinate functions y1(t), y2(t), …, yn(t), meaning that y(t) = (y1(t), ..., yn(t)). This includes, for example, parametric curves in R2 or R3. The coordinate functions are real valued functions, so the above definition of derivative applies to them. The derivative of y(t) is defined to be the vector, called the tangent vector, whose coordinates are the derivatives of the coordinate functions. That is, Equivalently, if the limit exists. The subtraction in the numerator is subtraction of vectors, not scalars. If the derivative of y exists for every value of t, then y′ is another vector valued function. If e1, …, en is the standard basis for Rn, then y(t) can also be written as y1(t)e1 + … + yn(t)en. If we assume that the derivative of a vector-valued function retains the linearity property, then the derivative of y(t) must be Derivative because each of the basis vectors is a constant. This generalization is useful, for example, if y(t) is the position vector of a particle at time t; then the derivative y′(t) is the velocity vector of the particle at time t. Partial derivatives Suppose that ƒ is a function that depends on more than one variable. For instance, ƒ can be reinterpreted as a family of functions of one variable indexed by the other variables: In other words, every value of x chooses a function, denoted fx, which is a function of one real number.[9] That is, Once a value of x is chosen, say a, then f(x,y) determines a function fa that sends y to a² + ay + y²: In this expression, a is a constant, not a variable, so fa is a function of only one real variable. Consequently the definition of the derivative for a function of one variable applies: The above procedure can be performed for any choice of a. Assembling the derivatives together into a function gives a function that describes the variation of ƒ in the y direction: This is the partial derivative of ƒ with respect to y. Here ∂ is a rounded d called the partial derivative symbol. To distinguish it from the letter d, ∂ is sometimes pronounced "der", "del", or "partial" instead of "dee". In general, the partial derivative of a function ƒ(x1, …, xn) in the direction xi at the point (a1 …, an) is defined to be: In the above difference quotient, all the variables except xi are held fixed. That choice of fixed values determines a function of one variable and, by definition, In other words, the different choices of a index a family of one-variable functions just as in the example above. This expression also shows that the computation of partial derivatives reduces to the computation of one-variable derivatives. An important example of a function of several variables is the case of a scalar-valued function ƒ(x1,...xn) on a domain in Euclidean space Rn (e.g., on R² or R³). In this case ƒ has a partial derivative ∂ƒ/∂xj with respect to each variable xj. At the point a, these partial derivatives define the vector This vector is called the gradient of ƒ at a. If ƒ is differentiable at every point in some domain, then the gradient is a vector-valued function ∇ƒ that takes the point a to the vector ∇f(a). Consequently the gradient determines a vector 185 Derivative field. Directional derivatives If ƒ is a real-valued function on Rn, then the partial derivatives of ƒ measure its variation in the direction of the coordinate axes. For example, if ƒ is a function of x and y, then its partial derivatives measure the variation in ƒ in the x direction and the y direction. They do not, however, directly measure the variation of ƒ in any other direction, such as along the diagonal line y = x. These are measured using directional derivatives. Choose a vector The directional derivative of ƒ in the direction of v at the point x is the limit In some cases it may be easier to compute or estimate the directional derivative after changing the length of the vector. Often this is done to turn the problem into the computation of a directional derivative in the direction of a unit vector. To see how this works, suppose that v = λu. Substitute h = k/λ into the difference quotient. The difference quotient becomes: This is λ times the difference quotient for the directional derivative of f with respect to u. Furthermore, taking the limit as h tends to zero is the same as taking the limit as k tends to zero because h and k are multiples of each other. Therefore Dv(ƒ) = λDu(ƒ). Because of this rescaling property, directional derivatives are frequently considered only for unit vectors. If all the partial derivatives of ƒ exist and are continuous at x, then they determine the directional derivative of ƒ in the direction v by the formula: This is a consequence of the definition of the total derivative. It follows that the directional derivative is linear in v, meaning that Dv + w(ƒ) = Dv(ƒ) + Dw(ƒ). The same definition also works when ƒ is a function with values in Rm. The above definition is applied to each component of the vectors. In this case, the directional derivative is a vector in Rm. Total derivative, total differential and Jacobian matrix When ƒ is a function from an open subset of Rn to Rm, then the directional derivative of ƒ in a chosen direction is the best linear approximation to ƒ at that point and in that direction. But when n > 1, no single directional derivative can give a complete picture of the behavior of ƒ. The total derivative, also called the (total) differential, gives a complete picture by considering all directions at once. That is, for any vector v starting at a, the linear approximation formula holds: Just like the single-variable derivative, ƒ ′(a) is chosen so that the error in this approximation is as small as possible. If n and m are both one, then the derivative ƒ ′(a) is a number and the expression ƒ ′(a)v is the product of two numbers. But in higher dimensions, it is impossible for ƒ ′(a) to be a number. If it were a number, then ƒ ′(a)v would be a vector in Rn while the other terms would be vectors in Rm, and therefore the formula would not make sense. For the linear approximation formula to make sense, ƒ ′(a) must be a function that sends vectors in Rn to vectors in Rm, and ƒ ′(a)v must denote this function evaluated at v. To determine what kind of function it is, notice that the linear approximation formula can be rewritten as 186 Derivative Notice that if we choose another vector w, then this approximate equation determines another approximate equation by substituting w for v. It determines a third approximate equation by substituting both w for v and a + v for a. By subtracting these two new equations, we get If we assume that v is small and that the derivative varies continuously in a, then ƒ ′(a + v) is approximately equal to ƒ ′(a), and therefore the right-hand side is approximately zero. The left-hand side can be rewritten in a different way using the linear approximation formula with v + w substituted for v. The linear approximation formula implies: This suggests that ƒ ′(a) is a linear transformation from the vector space Rn to the vector space Rm. In fact, it is possible to make this a precise derivation by measuring the error in the approximations. Assume that the error in these linear approximation formula is bounded by a constant times ||v||, where the constant is independent of v but depends continuously on a. Then, after adding an appropriate error term, all of the above approximate equalities can be rephrased as inequalities. In particular, ƒ ′(a) is a linear transformation up to a small error term. In the limit as v and w tend to zero, it must therefore be a linear transformation. Since we define the total derivative by taking a limit as v goes to zero, ƒ ′(a) must be a linear transformation. In one variable, the fact that the derivative is the best linear approximation is expressed by the fact that it is the limit of difference quotients. However, the usual difference quotient does not make sense in higher dimensions because it is not usually possible to divide vectors. In particular, the numerator and denominator of the difference quotient are not even in the same vector space: The numerator lies in the codomain Rm while the denominator lies in the domain Rn. Furthermore, the derivative is a linear transformation, a different type of object from both the numerator and denominator. To make precise the idea that ƒ ′ (a) is the best linear approximation, it is necessary to adapt a different formula for the one-variable derivative in which these problems disappear. If ƒ : R → R, then the usual definition of the derivative may be manipulated to show that the derivative of ƒ at a is the unique number ƒ ′(a) such that This is equivalent to because the limit of a function tends to zero if and only if the limit of the absolute value of the function tends to zero. This last formula can be adapted to the many-variable situation by replacing the absolute values with norms. The definition of the total derivative of ƒ at a, therefore, is that it is the unique linear transformation ƒ ′(a) : Rn → Rm such that Here h is a vector in Rn, so the norm in the denominator is the standard length on Rn. However, ƒ′(a)h is a vector in Rm, and the norm in the numerator is the standard length on Rm. If v is a vector starting at a, then ƒ ′(a)v is called the pushforward of v by ƒ and is sometimes written ƒ*v. If the total derivative exists at a, then all the partial derivatives and directional derivatives of ƒ exist at a, and for all v, ƒ ′(a)v is the directional derivative of ƒ in the direction v. If we write ƒ using coordinate functions, so that ƒ = (ƒ1, ƒ2, ..., ƒm), then the total derivative can be expressed using the partial derivatives as a matrix. This matrix is called the Jacobian matrix of ƒ at a: 187 Derivative 188 The existence of the total derivative ƒ′(a) is strictly stronger than the existence of all the partial derivatives, but if the partial derivatives exist and are continuous, then the total derivative exists, is given by the Jacobian, and depends continuously on a. The definition of the total derivative subsumes the definition of the derivative in one variable. That is, if ƒ is a real-valued function of a real variable, then the total derivative exists if and only if the usual derivative exists. The Jacobian matrix reduces to a 1×1 matrix whose only entry is the derivative ƒ′(x). This 1×1 matrix satisfies the property that ƒ(a + h) − ƒ(a) − ƒ ′(a)h is approximately zero, in other words that Up to changing variables, this is the statement that the function is the best linear approximation to ƒ at a. The total derivative of a function does not give another function in the same way as the one-variable case. This is because the total derivative of a multivariable function has to record much more information than the derivative of a single-variable function. Instead, the total derivative gives a function from the tangent bundle of the source to the tangent bundle of the target. The natural analog of second, third, and higher-order total derivatives is not a linear transformation, is not a function on the tangent bundle, and is not built by repeatedly taking the total derivative. The analog of a higher-order derivative, called a jet, cannot be a linear transformation because higher-order derivatives reflect subtle geometric information, such as concavity, which cannot be described in terms of linear data such as vectors. It cannot be a function on the tangent bundle because the tangent bundle only has room for the base space and the directional derivatives. Because jets capture higher-order information, they take as arguments additional coordinates representing higher-order changes in direction. The space determined by these additional coordinates is called the jet bundle. The relation between the total derivative and the partial derivatives of a function is paralleled in the relation between the kth order jet of a function and its partial derivatives of order less than or equal to k. Generalizations The concept of a derivative can be extended to many other settings. The common thread is that the derivative of a function at a point serves as a linear approximation of the function at that point. • An important generalization of the derivative concerns complex functions of complex variables, such as functions from (a domain in) the complex numbers C to C. The notion of the derivative of such a function is obtained by replacing real variables with complex variables in the definition. If C is identified with R² by writing a complex number z as x + i y, then a differentiable function from C to C is certainly differentiable as a function from R² to R² (in the sense that its partial derivatives all exist), but the converse is not true in general: the complex derivative only exists if the real derivative is complex linear and this imposes relations between the partial derivatives called the Cauchy Riemann equations — see holomorphic functions. • Another generalization concerns functions between differentiable or smooth manifolds. Intuitively speaking such a manifold M is a space that can be approximated near each point x by a vector space called its tangent space: the prototypical example is a smooth surface in R³. The derivative (or differential) of a (differentiable) map ƒ: M → N between manifolds, at a point x in M, is then a linear map from the tangent space of M at x to the tangent space of N at ƒ(x). The derivative function becomes a map between the tangent bundles of M and N. This definition is fundamental in differential geometry and has many uses — see pushforward (differential) and pullback (differential geometry). • Differentiation can also be defined for maps between infinite dimensional vector spaces such as Banach spaces and Fréchet spaces. There is a generalization both of the directional derivative, called the Gâteaux derivative, and Derivative • • • • of the differential, called the Fréchet derivative. One deficiency of the classical derivative is that not very many functions are differentiable. Nevertheless, there is a way of extending the notion of the derivative so that all continuous functions and many other functions can be differentiated using a concept known as the weak derivative. The idea is to embed the continuous functions in a larger space called the space of distributions and only require that a function is differentiable "on average". The properties of the derivative have inspired the introduction and study of many similar objects in algebra and topology — see, for example, differential algebra. The discrete equivalent of differentiation is finite differences. The study of differential calculus is unified with the calculus of finite differences in time scale calculus. Also see arithmetic derivative. Notes [1] Differential calculus, as discussed in this article, is a very well-established mathematical discipline for which there are many sources. Almost all of the material in this article can be found in Apostol 1967, Apostol 1969, and Spivak 1994. [2] Spivak 1994, chapter 10. [3] See Differential (infinitesimal) for an overview. Further approaches include the Radon–Nikodym theorem, and the universal derivation (see Kähler differential). [4] Despite this, it is still possible to take the derivative in the sense of distributions. The result is nine times the Dirac measure centered at a. [5] Banach, S. (1931), "Uber die Baire'sche Kategorie gewisser Funktionenmengen", Studia. Math. (3): 174–179.. Cited by Hewitt, E and Stromberg, K (1963), Real and abstract analysis, Springer-Verlag, Theorem 17.8 [6] Apostol 1967, §4.18 [7] In the formulation of calculus in terms of limits, the du symbol has been assigned various meanings by various authors. Some authors do not assign a meaning to du by itself, but only as part of the symbol du/dx. Others define "dx" as an independent variable, and define du by du = dx•ƒ′ (x). In non-standard analysis du is defined as an infinitesimal. It is also interpreted as the exterior derivative du of a function u. See differential (infinitesimal) for further information. [8] (http:/ / web. mit. edu/ wwmath/ calculus/ differentiation/ notation. html) [9] This can also be expressed as the adjointness between the product space and function space constructions. References Print • Anton, Howard; Bivens, Irl; Davis, Stephen (February 2, 2005), Calculus: Early Transcendentals Single and Multivariable (8th ed.), New York: Wiley, ISBN 978-0-471-47244-5 • Apostol, Tom M. (June 1967), Calculus, Vol. 1: One-Variable Calculus with an Introduction to Linear Algebra, 1 (2nd ed.), Wiley, ISBN 978-0-471-00005-1 • Apostol, Tom M. (June 1969), Calculus, Vol. 2: Multi-Variable Calculus and Linear Algebra with Applications, 1 (2nd ed.), Wiley, ISBN 978-0-471-00007-5 • Courant, Richard; John, Fritz (December 22, 1998), Introduction to Calculus and Analysis, Vol. 1, Springer-Verlag, ISBN 978-3-540-65058-4 • Eves, Howard (January 2, 1990), An Introduction to the History of Mathematics (6th ed.), Brooks Cole, ISBN 978-0-03-029558-4 • Larson, Ron; Hostetler, Robert P.; Edwards, Bruce H. (February 28, 2006), Calculus: Early Transcendental Functions (4th ed.), Houghton Mifflin Company, ISBN 978-0-618-60624-5 • Spivak, Michael (September 1994), Calculus (3rd ed.), Publish or Perish, ISBN 978-0-914098-89-8 • Stewart, James (December 24, 2002), Calculus (5th ed.), Brooks Cole, ISBN 978-0-534-39339-7 • Thompson, Silvanus P. (September 8, 1998), Calculus Made Easy (Revised, Updated, Expanded ed.), New York: St. Martin's Press, ISBN 978-0-312-18548-0 189 Derivative Online books • Crowell, Benjamin (2003), Calculus (http://www.lightandmatter.com/calc/) • Garrett, Paul (2004), Notes on First-Year Calculus (http://www.math.umn.edu/~garrett/calculus/), University of Minnesota • Hussain, Faraz (2006), Understanding Calculus (http://www.understandingcalculus.com/) • Keisler, H. Jerome (2000), Elementary Calculus: An Approach Using Infinitesimals (http://www.math.wisc. edu/~keisler/calc.html) • Mauch, Sean (2004), Unabridged Version of Sean's Applied Math Book (http://www.its.caltech.edu/~sean/ book/unabridged.html) • Sloughter, Dan (2000), Difference Equations to Differential Equations (http://synechism.org/drupal/de2de/) • Strang, Gilbert (1991), Calculus (http://ocw.mit.edu/ans7870/resources/Strang/strangtext.htm) • Stroyan, Keith D. (1997), A Brief Introduction to Infinitesimal Calculus (http://www.math.uiowa.edu/ ~stroyan/InfsmlCalculus/InfsmlCalc.htm) • Wikibooks, Calculus (http://en.wikibooks.org/wiki/Calculus) Web pages • Khan Academy: Derivative lesson 1 (http://www.khanacademy.org/video/ calculus--derivatives-1--new-hd-version?playlist=Calculus) • Weisstein, Eric W. " Derivative. (http://mathworld.wolfram.com/Derivative.html)" From MathWorld • Derivatives of Trigonometric functions (http://www.ugrad.math.ubc.ca/coursedoc/math100/notes/ derivative/trig2.html), UBC • Solved Problems in Derivatives (http://calculus.solved-problems.com/category/derivative/) 190 191 Control methods OLE for process control OLE for Process Control (OPC), which stands for Object Linking and Embedding (OLE) for Process Control, is the original name for a standards specification developed in 1996 by an industrial automation industry task force. The standard specifies the communication of real-time plant data between control devices from different manufacturers. After the initial release, the OPC Foundation was created to maintain the standard. Since then, standards have been added and names have been changed. As of June, 2006, "OPC is a series of standards specifications". (Seven current standards and two emerging standards.) "The first standard (originally called simply the OPC Specification"), is "now called the Data Access Specification", or (later on the same page) "OPC Data Access", or OPC Data Access Specification. [1] While OPC originally stood for "OLE for Process Control", the official stance of the OPC Foundation is that OPC is no longer an acronym and the technology is simply known as "OPC". One of the reasons behind this is while OPC is heavily used within the process industries, it can be, and is, widely used in discrete manufacturing as well. Hence, OPC is known for more than just its applications within process control. Origin and uses The OPC Specification was based on the OLE, COM, and DCOM technologies developed by Microsoft for the Microsoft Windows operating system family. The specification defined a standard set of objects, interfaces and methods for use in process control and manufacturing automation applications to facilitate interoperability. The most common OPC specification is OPC Data Access, which is used to read and write real-time data. When vendors refer to OPC generically, they typically mean OPC Data Access. OPC DA itself has gone through 3 major revisions since its inception. Versions are backwards compatible, in that a version 3 OPC Server can still be accessed by a version 1 OPC Client, since the specifications add functionality but still require the older version to be implemented as well. However, a Client could be written that does not support the older functions since everything can be done using the newer ones, so a DA 3 compatible Client will not necessarily work with a DA 1.0 Server. OPC was designed to provide a common bridge for Windows based software applications and process control hardware. Standards define consistent methods of accessing field data from plant floor devices. This method remains the same regardless of the type and source of data. An OPC Server for one hardware device provides the same methods for an OPC Client to access its data as any and every other OPC Server for that same and any other hardware device. The aim was to reduce the amount of duplicated effort required from hardware manufacturers and their software partners, and from the SCADA and other HMI producers in order to interface the two. Once a hardware manufacturer had developed their OPC Server for the new hardware device their work was done to allow any 'top end' to access their device, and once the SCADA producer had developed their OPC Client their work was done to allow access to any hardware, existing or yet to be created, with an OPC compliant server. OPC servers provide a method for many different software packages (so long as it is an OPC Client) to access data from a process control device, such as a PLC or DCS. Traditionally, any time a package needed access to data from a device, a custom interface, or driver, had to be written. The purpose of OPC is to define a common interface that is written once and then reused by any business, SCADA, HMI, or custom software packages. OLE for process control There is nothing in the OPC Specs. to restrict the Server to providing access to a process control device. OPC Servers can be written for anything from getting the internal temperature of a microprocessor to the current temperature in Monument Valley. Once an OPC server is written for a particular device, it can be reused by any application that is able to act as an OPC client. OPC servers use Microsoft’s OLE technology (also known as the Component Object Model, or COM) to communicate with clients. COM technology permits a standard for real-time information exchange between software applications and process hardware to be defined. It is important to note that some OPC specifications are published, others are available only to member of the OPC Foundation. So whilst no company "owns" OPC and anyone can develop an OPC server, whether or not they are a member of the OPC Foundation, non-members will not necessarily be using the latest specifications. Anyone can integrate OPC products, and there is no pre-requisite for the system integrator to belong to any organization. It is therefore up to each company that requires OPC products to ensure that their products are certified and that their system integrators have the necessary training. Future The OPC Unified Architecture (UA) has been specified and is being tested and implemented through its Early Adopters program. It can be implemented with Java, Microsoft .NET, or C, eliminating the need to use a Microsoft Windows based platform of earlier OPC versions. UA combines the functionality of the existing OPC interfaces with new technologies such as XML and Web Services to deliver higher level MES and ERP support. On September 16, 2010, The OPC Foundation and the MTConnect Institute announced a cooperation to ensure interoperability and consistency between the two standards.[2] References [1] What is OPC? (http:/ / www. opcfoundation. org/ Default. aspx/ 01_about/ 01_whatis. asp?MID=AboutOPC) opcfoundation.org (http:/ / www. opcfoundation. org/ ) [2] "OPC Foundation and MTConnect Institute Announce a Memorandum of Understanding" (http:/ / www. opcfoundation. org/ Default. aspx/ 02_news/ 02_news_display. asp?id=845& MID=News). OPC Foundation. 2010-10-21. . Retrieved 2010-10-26. External links • • • • • • OPC Foundation (http://www.opcfoundation.org/) OPC Tutorials (http://matrikonopc.ca/resources/opc-tutorials.aspx) OPC Programmers' Connection (http://www.opcconnect.com) FatRat Library - free OPC server developers toolkit (http://code.google.com/p/frl/) OpenOPC - Open Source OPC client development in Python (http://openopc.sourceforge.net) OPC Unified Architecture Address Space e-book (http://www.commsvr.com/UAModelDesigner/Index.aspx) 192 Nonlinear control Nonlinear control Nonlinear control is the area of control engineering specifically involved with systems that are nonlinear, time-variant, or both. Many well-established analysis and design techniques exist for LTI systems (e.g., root-locus, Bode plot, Nyquist criterion, state-feedback, pole placement); however, one or both of the controller and the system under control in a general control system may not be an LTI system, and so these methods cannot necessarily be applied directly. Nonlinear control theory studies how to apply existing linear methods to these more general control systems. Additionally, it provides novel control methods that cannot be analyzed using LTI system theory. Even when LTI system theory can be used for the analysis and design of a controller, a nonlinear controller can have attractive characteristics (e.g., simpler implementation, increased speed, or decreased control energy); however, nonlinear control theory usually requires more rigorous mathematical analysis to justify its conclusions. Properties of nonlinear systems Some properties of nonlinear dynamic systems are • They do not follow the principle of superposition (linearity and homogeneity). • They may have multiple isolated equilibrium points. • They may exhibit properties such as limit-cycle, bifurcation, chaos. • Finite escape time: Solutions of nonlinear systems may not exist for all times. Analysis and control of nonlinear systems There are several well-developed techniques for analyzing nonlinear feedback systems: • • • • • • • • Describing function method Phase plane method Lyapunov stability analysis Singular perturbation method Popov criterion (described in The Lur'e Problem below) Center manifold theorem Small-gain theorem Passivity analysis Control design techniques for nonlinear systems also exist. These can be subdivided into techniques which attempt to treat the system as a linear system in a limited range of operation and use (well-known) linear design techniques for each region: • Gain scheduling Those that attempt to introduce auxiliary nonlinear feedback in such a way that the system can be treated as linear for purposes of control design: • Feedback linearization And Lyapunov based methods: • • • • Lyapunov redesign Nonlinear damping Backstepping Sliding mode control 193 Nonlinear control 194 Nonlinear feedback analysis – The Lur'e problem An early nonlinear feedback system analysis problem was formulated by A. I. Lur'e. Control systems described by the Lur'e problem have a forward path that is linear and time-invariant, and a feedback path that contains a memory-less, possibly time-varying, static nonlinearity. Lur'e problem block diagram The linear part can be characterized by four matrices (A,B,C,D), while the nonlinear part is Φ(y) with (a sector nonlinearity). Absolute stability problem Consider: 1. (A,B) is controllable and (C,A) is observable 2. two real numbers a, b with a<b, defining a sector for function Φ The problem is to derive conditions involving only the transfer matrix H(s) and {a,b} such that x=0 is a globally uniformly asymptotically stable equilibrium of the system. This is known as the Lur'e problem. There are two main theorems concerning the problem: • The Circle criterion • The Popov criterion. Popov criterion The sub-class of Lur'e systems studied by Popov is described by: where x ∈ Rn, ξ,u,y are scalars and A,b,c,d have commensurate dimensions. The nonlinear element Φ: R → R is a time-invariant nonlinearity belonging to open sector (0, ∞). This means that Φ(0) = 0, y Φ(y) > 0, ∀ y ≠ 0; The transfer function from u to y is given by Theorem: Consider the system (1)-(2) and suppose 1. A is Hurwitz 2. (A,b) is controllable 3. (A,c) is observable 4. d>0 and 5. Φ ∈ (0,∞) Nonlinear control 195 then the system is globally asymptotically stable if there exists a number r>0 such that infω ∈ R Re[(1+jωr)h(jω)] > 0 . Things to be noted: • The Popov criterion is applicable only to autonomous systems • The system studied by Popov has a pole at the origin and there is no direct pass-through from input to output • The nonlinearity Φ must satisfy an open sector condition Theoretical results in nonlinear control Frobenius Theorem The Frobenius theorem is a deep result in Differential Geometry. When applied to Nonlinear Control, it says the following: Given a system of the form where , integral curves of are vector fields belonging to a distribution are restricted to a manifold of dimension and if span( are control functions, the and is an involutive distribution. Further reading • A. I. Lur'e and V. N. Postnikov, "On the theory of stability of control systems," Applied mathematics and mechanics, 8(3), 1944, (in Russian). • M. Vidyasagar, Nonlinear Systems Analysis, 2nd edition, Prentice Hall, Englewood Cliffs, New Jersey 07632. • A. Isidori, Nonlinear Control Systems, 3rd edition, Springer Verlag, London, 1995. • H. K. Khalil, Nonlinear Systems, 3rd edition, Prentice Hall, Upper Saddle River, New Jersey, 2002. ISBN 0130673897 • B. Brogliato, R. Lozano, B. Maschke, O. Egeland, "Dissipative Systems Analysis and Control", Springer Verlag, London, 2nd edition, 2007. Proportional control 196 Proportional control A proportional control system is a type of linear feedback control system. Two classic mechanical examples are the toilet bowl float proportioning valve and the fly-ball governor. The proportional control system is more complex than an on-off control system like a bi-metallic domestic thermostat, but simpler than a proportional-integral-derivative (PID) control system used in something like an automobile cruise control. On-off control will work where the overall system has a relatively long response time, but will result in instability if the system being controlled has a rapid response time. Proportional control overcomes this by modulating the output to the controlling device, such as a continuously variable valve. The fly-ball governor is a classic example of proportional control. An analogy to on-off control is driving a car by applying either full power or no power and varying the duty cycle, to control speed. The power would be on until the target speed is reached, and then the power would be removed, so the car reduces speed. When the speed falls below the target, with a certain hysteresis, full power would again be applied. It can be seen that this looks like pulse-width modulation, but would obviously result in poor control and large variations in speed. The more powerful the engine; the greater the instability, the heavier the car; the greater the stability. Stability may be expressed as correlating to the power-to-weight ratio of the vehicle. Proportional control is how most drivers control the speed of a car. If the car is at target speed and the speed increases slightly, the power is reduced slightly, or in proportion to the error (the actual versus target speed), so that the car reduces speed gradually and reaches the target point with very little, if any, "overshoot", so the result is much smoother control than on-off control. Further refinements like PID control would help compensate for additional variables like hills, where the amount of power needed for a given speed change would vary, which would be accounted for by the integral function of the PID control. Proportional Control Theory In the proportional control algorithm, the controller output is proportional to the error signal, which is the difference between the set point and the process variable. In other words, the output of a proportional controller is the multiplication product of the error signal and the proportional gain. This can be mathematically expressed as where • : Output of the proportional controller • : Proportional gain • : Instantaneous process error at time 't'. • SP: Set point • PV: Process variable Proportional control 197 External links • Proportional control compared to ON-OFF or bang-bang control [1] References [1] http:/ / www. fourmilab. ch/ hackdiet/ www/ subsection1_2_3_0_5. html Optimal control Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. The method is largely due to the work of Lev Pontryagin and his collaborators in the Soviet Union[1] and Richard Bellman in the United States. General method Optimal control deals with the problem of finding a control law for a given system such that a certain optimality criterion is achieved. A control problem includes a cost functional that is a function of state and control variables. An optimal control is a set of differential equations describing the paths of the control variables that minimize the cost functional. The optimal control can be derived using Pontryagin's maximum principle (a necessary condition also known as Pontryagin's minimum principle or simply Pontryagin's Principle[2] ), or by solving the Hamilton-Jacobi-Bellman equation (a sufficient condition). We begin with a simple example. Consider a car traveling on a straight line through a hilly road. The question is, how should the driver press the accelerator pedal in order to minimize the total traveling time? Clearly in this example, the term control law refers specifically to the way in which the driver presses the accelerator and shifts the gears. The "system" consists of both the car and the road, and the optimality criterion is the minimization of the total traveling time. Control problems usually include ancillary constraints. For example the amount of available fuel might be limited, the accelerator pedal cannot be pushed through the floor of the car, speed limits, etc. A proper cost functional is a mathematical expression giving the traveling time as a function of the speed, geometrical considerations, and initial conditions of the system. It is often the case that the constraints are interchangeable with the cost functional. Another optimal control problem is to find the way to drive the car so as to minimize its fuel consumption, given that it must complete a given course in a time not exceeding some amount. Yet another control problem is to minimize the total monetary cost of completing the trip, given assumed monetary prices for time and fuel. A more abstract framework goes as follows. Minimize the continuous-time cost functional subject to the first-order dynamic constraints the algebraic path constraints and the boundary conditions where is the state, initial time, and is the control, is the independent variable (generally speaking, time), is the terminal time. The terms and is the are called the endpoint cost and Lagrangian, Optimal control 198 respectively. Furthermore, it is noted that the path constraints are in general inequality constraints and thus may not be active (i.e., equal to zero) at the optimal solution. It is also noted that the optimal control problem as stated above may have multiple solutions (i.e., the solution may not be unique). Thus, it is most often the case that any solution the optimal control problem is locally minimizing. Linear quadratic control A special case of the general nonlinear optimal control problem given in the previous section is the linear quadratic (LQ) optimal control problem. The LQ problem is stated as follows. Minimize the quadratic continuous-time cost functional Subject to the linear first-order dynamic constraints and the initial condition A particular form of the LQ problem that arises in many control system problems is that of the linear quadratic regulator (LQR) where all of the matrices (i.e., , , , and ) are constant, the initial time is arbitrarily set to zero, and the terminal time is taken in the limit (this last assumption is what is known as infinite horizon). The LQR problem is stated as follows. Minimize the infinite horizon quadratic continuous-time cost functional Subject to the linear time-invariant first-order dynamic constraints and the initial condition In the finite-horizon case the matrices are restricted in that and are positive semi-definite and positive definite, respectively. In the infinite-horizon case, however, the matrices and are not only positive-semidefinite and positive-definite, respectively, but are also constant. These additional restrictions on and in the infinite-horizon case are enforced to ensure that the cost functional remains positive. Furthermore, in order to ensure that the cost function is bounded, the additional restriction is imposed that the pair is controllable. Note that the LQ or LQR cost functional can be thought of physically as attempting to minimize the control energy (measured as a quadratic form). The infinite horizon problem (i.e., LQR) may seem overly restrictive and essentially useless because it assumes that the operator is driving the system to zero-state and hence driving the output of the system to zero. This is indeed correct. However the problem of driving the output to a desired nonzero level can be solved after the zero output one is. In fact, it can be proved that this secondary LQR problem can be solved in a very straightforward manner. It has been shown in classical optimal control theory that the LQ (or LQR) optimal control has the feedback form where and is a properly dimensioned matrix, given as is the solution of the differential Riccati equation. The differential Riccati equation is given as Optimal control 199 For the finite horizon LQ problem, the Riccati equation is integrated backward in time using the terminal boundary condition For the infinite horizon LQR problem, the differential Riccati equation is replaced with the algebraic Riccati equation (ARE) given as Understanding that the ARE arises from infinite horizon problem, the matrices , , , and are all constant. It is noted that there are in general multiple solutions to the algebraic Riccati equation and the positive definite (or positive semi-definite) solution is the one that is used to compute the feedback gain. The LQ (LQR) problem was elegantly solved by Rudolf Kalman.[3] Numerical methods for optimal control Optimal control problems are generally nonlinear and therefore, generally do not have analytic solutions (e.g., like the linear-quadratic optimal control problem). As a result, it is necessary to employ numerical methods to solve optimal control problems. In the early years of optimal control (circa 1950s to 1980s) the favored approach for solving optimal control problems was that of indirect methods. In an indirect method, the calculus of variations is employed to obtain the first-order optimality conditions. These conditions result in a two-point (or, in the case of a complex problem, a multi-point) boundary-value problem. This boundary-value problem actually has a special structure because it arises from taking the derivative of a Hamiltonian. Thus, the resulting dynamical system is a Hamiltonian system of the form where is the augmented Hamiltonian and in an indirect method, the boundary-value problem is solved (using the appropriate boundary or transversality conditions). The beauty of using an indirect method is that the state and adjoint (i.e., ) are solved for and the resulting solution is readily verified to be an extremal trajectory. The disadvantage of indirect methods is that the boundary-value problem is often extremely difficult to solve (particularly for problems that span large time intervals or problems with interior point constraints). A well-known software program that implements indirect methods is BNDSCO.[4] The approach that has risen to prominence in numerical optimal control over the past two decades (i.e., from the 1980s to the present) is that of so called direct methods. In a direct method, the state and/or control are approximated using an appropriate function approximation (e.g., polynomial approximation or piecewise constant parameterization). Simultaneously, the cost functional is approximated as a cost function. Then, the coefficients of the function approximations are treated as optimization variables and the problem is "transcribed" to a nonlinear optimization problem of the form: Minimize subject to the algebraic constraints Depending upon the type of direct method employed, the size of the nonlinear optimization problem can be quite small (e.g., as in a direct shooting or quasilinearization method) or may be quite large (e.g., a direct collocation method[5] ). In the latter case (i.e., a collocation method), the nonlinear optimization problem may be literally Optimal control 200 thousands to tens of thousands of variables and constraints. Given the size of many NLPs arising from a direct method, it may appear somewhat counter-intuitive that solving the nonlinear optimization problem is easier than solving the boundary-value problem. It is, however, the fact that the NLP is easier to solve than the boundary-value problem. The reason for the relative ease of computation, particularly of a direct collocation method, is that the NLP is sparse and many well-known software programs exist (e.g., SNOPT[6] ) to solve large sparse NLPs. As a result, the range of problems that can be solved via direct methods (particularly direct collocation methods which are very popular these days) is significantly larger than the range of problems that can be solved via indirect methods. In fact, direct methods have become so popular these days that many people have written elaborate software programs that employ these methods. In particular, many such programs written in FORTRAN include DIRCOL,[7] SOCS,[8] OTIS,[9] GESOP/ASTOS[10] and DITAN.[11] In recent years, due to the advent of the MATLAB programming language, optimal control software in MATLAB has become more common. Examples of academically developed MATLAB software tools implementing direct methods include RIOTS [12],[13] DIDO,[14] DIRECT,[15] and GPOPS [16] [17] , while an example of an industry developed MATLAB tool is PROPT.[18] These software tools have increased significantly the opportunity for people to explore complex optimal control problems both for academic research and industrial-strength problems. Finally, it is noted that general-purpose MATLAB optimization environments such as TOMLAB have made coding complex optimal control problems significantly easier than was previously possible in languages such as C and FORTRAN. Discrete-time optimal control The examples thus far have shown continuous time systems and control solutions. In fact, as optimal control solutions are now often implemented digitally, contemporary control theory is now primarily concerned with discrete time systems and solutions. The Theory of Consistent Approximations[19] provides conditions under which solutions to a series of increasingly accurate discretized optimal control problem converge to the solution of the original, continuous-time problem. Not all discretization methods have this property, even seemingly obvious ones. For instance, using a variable step-size routine to integrate the problem's dynamic equations may generate a gradient which does not converge to zero (or point in the right direction) as the solution is approached. The direct method RIOTS [20] is based on the Theory of Consistent Approximation. Examples A common solution strategy in many optimal control problems is to solve for the costate (sometimes called the shadow price) . The costate summarizes in one number the marginal value of expanding or contracting the state variable next turn. The marginal value is not only the gains accruing to it next turn but associated with the duration of the program. It is nice when can be solved analytically, but usually the most one can do is describe it sufficiently well that the intuition can grasp the character of the solution and an equation solver can solve numerically for the values. Having obtained , the turn-t optimal value for the control can usually be solved as a differential equation conditional on knowledge of . Again it is infrequent, especially in continuous-time problems, that one obtains the value of the control or the state explicitly. Usually the strategy is to solve for thresholds and regions that characterize the optimal control and use a numerical solver to isolate the actual choice values in time. Finite time Consider the problem of a mine owner who must decide at what rate to extract ore from his mine. He owns rights to the ore from date to date . At date there is ore in the ground, and the instantaneous stock of ore declines at the rate the mine owner extracts it u(t). The mine owner extracts ore at cost constant price . He does not value the ore remaining in the ground at time and sells ore at a (there is no "scrap value"). He chooses the rate of extraction in time u(t) to maximize profits over the period of ownership with no time discounting. Optimal control 201 1. Discrete-time version The manager maximizes profit 2. Continuous-time version : The manager maximizes profit : subject to the law of evolution for the state variable subject to the law of evolution for the state variable Form the Hamiltonian and differentiate: Form the Hamiltonian and differentiate: As the mine owner does not value the ore remaining at time , Using the above equations, it is easy to solve for the series and using the initial and turn-T conditions, the explicitly, giving . and As the mine owner does not value the ore remaining at time , Using the above equations, it is easy to solve for the differential equations governing and series can be solved and using the initial and turn-T conditions, the functions can be solved numerically. References [1] L. S. Pontryagin, 1962. The Mathematical Theory of Optimal Processes. [2] I. M. Ross, 2009. A Primer on Pontryagin's Principle in Optimal Control, Collegiate Publishers. ISBN 978-0-9843571-0-9. [3] Kalman, Rudolf. A new approach to linear filtering and prediction problems. Transactions of the ASME, Journal of Basic Engineering, 82:34–45, 1960 [4] Oberle, H. J. and Grimm, W., "BNDSCO-A Program for the Numerical Solution of Optimal Control Problems," Institute for Flight Systems Dynamics, DLR, Oberpfaffenhofen, 1989 [5] Betts, J. T., Practical Methods for Optimal Control Using Nonlinear Programming, SIAM Press, Philadelphia, Pennsylvania, 2001 [6] Gill, P. E., Murray, W. M., and Saunders, M. A., User's Manual for SNOPT Version 7: Software for Large-Scale Nonlinear Programming, University of California, San Diego Report, 24 April 2007 [7] von Stryk, O., User's Guide for DIRCOL (version 2.1): A Direct Collocation Method for the Numerical Solution of Optimal Control Problems, Fachgebiet Simulation und Systemoptimierung (SIM), Technische Universität Darmstadt (2000, Version of November 1999). [8] Betts, J.T. and Huffman, W. P., Sparse Optimal Control Software, SOCS, Boeing Information and Support Services, Seattle, Washington, July 1997 [9] Hargraves, C. R. and Paris, S. W., "Direct Trajectory Optimization Using Nonlinear Programming and Collocation", Journal of Guidance, Control, and Dynamics, Vol. 10, No. 4., 1987, pp. 338-342 [10] Gath, P.F., Well, K.H., "Trajectory Optimization Using a Combination of Direct Multiple Shooting and Collocation", AIAA 2001-4047, AIAA Guidance, Navigation, and Control Conference, Montréal, Québec, Canada, 6–9 August 2001 [11] Vasile M., Bernelli-Zazzera F., Fornasari N., Masarati P., "Design of Interplanetary and Lunar Missions Combining Low-Thrust and Gravity Assists", Final Report of the ESA/ESOC Study Contract No. 14126/00/D/CS, September 2002 [12] http:/ / www. schwartz-home. com/ RIOTS/ [13] Schwartz, Adam, Theory and Implementation of Methods based on Runge-Kutta Integration for Solving Optimal Control Problems, University of California at Berkeley, PhD Dissertation, 1996. [14] Ross, I. M. and Fahroo, F., User's Manual for DIDO: A MATLAB Package for Dynamic Optimization, Dept. of Aeronautics and Astronautics, Naval Postgraduate School Technical Report, 2002 [15] Williams, P., User's Guide to DIRECT, Version 2.00, Melbourne, Australia, 2008 [16] http:/ / gpops. sourceforge. net [17] Rao, A. V., Benson, D. A., Huntington, G. T., Francolin, C., Darby, C. L., and Patterson, M. A., User's Manual for GPOPS: A MATLAB Package for Dynamic Optimization Using the Gauss Pseudospectral Method, University of Florida Report, August 2008. Optimal control [18] Rutquist, P. and Edvall, M. M, PROPT - MATLAB Optimal Control Software," 1260 S.E. Bishop Blvd Ste E, Pullman, WA 99163, USA: Tomlab Optimization, Inc. [19] E. Polak, On the use of consistent approximations in the solution of semi-infinite optimization and optimal control problems Math. Prog. 62 pp. 385-415 (1993). [20] http:/ / www. schwartz-home. com/ RIOTS Further reading Books • Athans, M. A. and Falb, P. L., Optimal Control, McGraw-Hill, New York, 1966. • Becerra, V.M., 2008, Optimal control (http://www.scholarpedia.org/article/Optimal_control). Scholarpedia (http://www.scholarpedia.org), 3(1):5354 • Bryson, A. E., 1969. Applied Optimal Control: Optimization, Estimation, & Control. • Evans, L.C., An Introduction to Optimal Control Theory ( available free online (http://math.berkeley.edu/ ~evans/control.course.pdf)) • Ross, I. M. A Primer on Pontryagin's Principle in Optimal Control, Collegiate Publishers, 2009. ISBN 978-0-9843571-0-9. (http://www.ElissarGlobal.com free chapter available online) • H. O. Fattorini and S. S. Sritharan, "Existence of Optimal Controls for Viscous Flow Problems," Proceedings of the Royal Society of London Series A, Vol. 439, 1992, pp. 81–102. • H. O. Fattorini and S. S. Sritharan, "Necessary and Sufficient Conditions for Optimal Controls in Viscous Flow," Proceedings of the Royal Society of Edinburgh, Series A, Vol. 124A, 1994, pp. 211–251. • H. O. Fattorini and S. S. Sritharan, "Optimal chattering controls for viscous flow," Nonlinear analysis, Theory, Methods and Applications, Vol. 25, No. 8, pp. 763–797, 1995. • H. O. Fattorini and S. S. Sritharan,"Optimal control problems with state constraints in fluid mechanics and combustion," Applied Mathematics and Optimization, Vol. 38(2), 1998, pp. 159–192. • Kirk, D. E., 2004. Optimal Control Theory: An Introduction. • Lebedev, L. P., and Cloud, M. J., 2003. The Calculus of Variations and Functional Analysis with Optimal Control and Applications in Mechanics. World Scientific. Especially chpt. 2. • Lewis, F. L., and Syrmos, V. L., 19nn. Optimal Control, 2nd ed. John Wiley & Sons. • S. S. Sritharan, "Optimal Control of Viscous Flow", SIAM, 1998. (http://www.ec-securehost.com/SIAM/ot59. html) • S. S. Sritharan, "An Optimal Control Problem in Exterior Hydrodynamics," Proceedings of the Royal Society of Edinburgh, Series 121A, 1992, pp. 5–32. • S. S. Sritharan,"Deterministic and stochastic control of viscous flow with linear, monotone and hyper viscosities," Applied Mathematics and Optimization, Vol. 41(2), pp. 255–308, 2000. • Stengel, R. F., 1994. Optimal Control and Estimation. (http://www.princeton.edu/~stengel/OptConEst.html) Dover. • Sethi, S. P., and Thompson, G. L., 2000. Optimal Control Theory: Applications to Management Science and Economics, 2nd edition, Springer (ISBN 0387280928 and ISBN 0792386086). Slides are available at http:// www.utdallas.edu/~sethi/OPRE7320presentation.html • Sontag, Eduardo D. Mathematical Control Theory: Deterministic Finite Dimensional Systems. Second Edition. Springer. (ISBN 0-387-984895) ( available free online (http://www.math.rutgers.edu/~sontag/FTP_DIR/ sontag_mathematical_control_theory_springer98.pdf)) • Brogan, William L. 1990. Modern Control Theory. ISBN 0135897637 • Bryson, A.E.; Ho, Y.C. (1975). Applied optimal control. Washington, DC: Hemisphere. Journals • Optimal Control Applications and Methods. (http://www3.interscience.wiley.com/cgi-bin/jhome/ 2133?CRETRY=1&SRETRY=0) John Wiley & Sons, Inc. 202 Optimal control • SIAM Journal of Control and Optimization. External links • ACADO Toolkit - Open Source Toolkit for Automatic Control and Dynamic Optimization (C++, MATLAB interface available) (http://www.acadotoolkit.org) • Dr. Benoît CHACHUAT: Automatic Control Laboratory (http://lawww.epfl.ch/page4234.html) - Nonlinear Programming, Calculus of Variations and Optimal Control. • Flood: An open source neural networks C++ library (http://www.cimne.com/flood) • GPOPS - Open Source Pseudospectral Optimal Control Solver in MATLAB (http://gpops.sourceforge.net) • Elmer G. Wiens: Optimal Control (http://www.egwald.ca/optimalcontrol/index.php) - Applications of Optimal Control Theory Using the Pontryagin Maximum Principle with interactive models. • PPOPT - MATLAB Optimal Control Software (http://tomdyn.com/) • PSOPT - Open Source Pseudospectral Optimal Control Solver in C++ (https://sites.google.com/a/psopt.org/ psopt/) • GESOP - Graphical Environment for Simulation and OPtimization (http://www.astos.de/products/gesop) Robust control Robust control is a branch of control theory that explicitly deals with uncertainty in its approach to controller design. Robust control methods are designed to function properly so long as uncertain parameters or disturbances are within some (typically compact) set. Robust methods aim to achieve robust performance and/or stability in the presence of bounded modelling errors. The early methods of Bode and others were fairly robust; the state-space methods invented in the 1960s and 1970s were sometimes found to lack robustness,[1] prompting research to improve them. This was the start of the theory of Robust Control, which took shape in the 1980s and 1990s and is still active today. In contrast with an adaptive control policy, a robust control policy is static; rather than adapting to measurements of variations, the controller is designed to work assuming that certain variables will be unknown but, for example, bounded.[2] [3] When is a control method said to be robust? Informally, a controller designed for a particular set of parameters is said to be robust if it would also work well under a different set of assumptions. High-gain feedback is a simple example of a robust control method; with sufficiently high gain, the effect of any parameter variations will be negligible. High-gain feedback is the principle that allows simplified models of operational amplifiers and emitter-degenerated bipolar transistors to be used in a variety of different settings. This idea was already well understood by Bode and Black in 1927. The modern theory of robust control The theory of robust control began in the late 1970s and early 1980s and soon developed a number of techniques for dealing with bounded system uncertainty.[4] [5] Probably the most important example of a robust control technique is H-infinity loop-shaping, which was developed by Duncan McFarlane and Keith Glover of Cambridge University; this method minimizes the sensitivity of a system over its frequency spectrum, and this guarantees that the system will not greatly deviate from expected trajectories when disturbances enter the system. Another example is LQG/LTR, which was developed to overcome the robustness problems of LQG control. 203 Robust control References [1] [2] [3] [4] [5] M. Athans, Editorial on the LQG problem, IEEE Trans. Autom. Control 16 (1971), no. 6, 528. J. Ackermann: Robuste Regelung. Springer-Verlag, 1993. (Section 1.5) In German; an english version is also available Manfred Morari : Homepage (http:/ / control. ee. ethz. ch/ ~morari/ ) Safonov: editorial (http:/ / routh. usc. edu/ pub/ safonov/ safo97d. pdf) Kemin Zhou: Essentials of Robust Control (http:/ / www. ee. lsu. edu/ kemin/ essentials. htm) Further reading • Ray, L.R.; Stengel, R.F. (1991). "Stochastic Robustness of Linear-Time-Invariant Control Systems" (http:// www.princeton.edu/~stengel/RayTAC1991.pdf). IEEE Trans. Automatic Control 36 (1): 82–87. doi:10.1109/9.62270. • V. Barbu and S. S. Sritharan (1998). "H-infinity Control Theory of Fluid Dynamics" (http://www.nps.edu/ Academics/Schools/GSEAS/SRI/R19.pdf). Proceedings of the Royal Royal Society of London, Ser. A. 545: 3009–3033. • Dullerud, G.E.; Paganini, F. (2000). A Course in Robust Control Theory: A Convex Approach. Springer Verlag New York. ISBN 0-387-98945-5. • Bhattacharya, Apellat,Keel (2000). Robust Control-The Parametric Approach. Prentice Hall PTR. ISBN 0-13-781576-X. • Zhou, Kemin; Doyle C., John (1999). Essentials of Robust Control. Prentice Hall. ISBN 0-13-525833-2. • Morari, Manfred; Zafiriou, Evanghelos (1989). Robust Process Control Theory. Prentice Hall. ISBN 0-13-782153-0. • Mahmoud S., Magdi; Munro, Neil (1989). Robust Control and Filtering for Time-Delay Systems. Marcel Dekker Inc.. ISBN 0-8247-0327-8. • Calafiore, G.; Dabbene, F. (ed.) (2006). Probabilistic and Randomized Methods for Design under Uncertainty. Springer Verlag London Ltd.. ISBN 978-1-84628-094-8. 204 Adaptive control Adaptive control Adaptive control is the control method used by a controller which must adapt to a controlled system with parameters which vary, or are initially uncertain. For example, as an aircraft flies, its mass will slowly decrease as a result of fuel consumption; a control law is needed that adapts itself to such changing conditions. Adaptive control is different from robust control in that it does not need a priori information about the bounds on these uncertain or time-varying parameters; robust control guarantees that if the changes are within given bounds the control law need not be changed, while adaptive control is concerned with control law changes themselves. Parameter estimation The foundation of adaptive control is parameter estimation. Common methods of estimation include recursive least squares and gradient descent. Both of these methods provide update laws which are used to modify estimates in real time (i.e., as the system operates). Lyapunov stability is used to derive these update laws and show convergence criterion (typically persistent excitation). Projection (mathematics) and normalization are commonly used to improve the robustness of estimation algorithms. Classification of adaptive control techniques In general one should distinguish between: 1. Feedforward Adaptive Control 2. Feedback Adaptive Control as well as between 1. Direct Methods and 2. Indirect Methods Direct methods are ones wherein the estimated parameters are those directly used in the adaptive controller. In contrast, indirect methods are those in which the estimated parameters are used to calculate required controller parameters[1] There are several broad categories of feedback adaptive control (classification can vary): • Dual Adaptive Controllers [based on Dual control theory] • Optimal Dual Controllers [difficult to design] • Suboptimal Dual Controllers • Nondual Adaptive Controllers • • • • • Adaptive Pole Placement Extremum Seeking Controllers Iterative learning control Gain scheduling Model Reference Adaptive Controllers (MRACs) [incorporate a reference model defining desired closed loop performance] 205 Adaptive control 206 • Gradient Optimization MRACs [use local rule for adjusting params when performance differs from reference. Ex.: "MIT rule".] • Stability Optimized MRACs • Model Identification Adaptive Controllers (MIACs) [perform System identification while the system is running] MRAC • Cautious Adaptive Controllers [use current SI to modify control law, allowing for SI uncertainty] • Certainty Equivalent Adaptive Controllers [take current SI to be the true system, assume no uncertainty] • Nonparametric Adaptive Controllers • Parametric Adaptive Controllers MIAC • Explicit Parameter Adaptive Controllers • Implicit Parameter Adaptive Controllers Some special topics in adaptive control can be introduced as well: 1. 2. 3. 4. 5. Adaptive Control Based on Discrete-Time Process Identification Adaptive Control Based on the Model Reference Technique Adaptive Control based on Continuous-Time Process Models Adaptive Control of Multivariable Processes Adaptive Control of Nonlinear Processes Applications When designing adaptive control systems, special consideration is necessary of convergence and robustness issues. Lyapunov stability is typically used to derive control adaptation laws and show convergence. Typical applications of adaptive control are (in general): • Self-tuning of subsequently fixed linear controllers during the implementation phase for one operating point; • Self-tuning of subsequently fixed robust controllers during the implementation phase for whole range of operating points; • Self-tuning of fixed controllers on request if the process behaviour changes due to ageing, drift, wear etc.; • Adaptive control of linear controllers for nonlinear or time-varying processes; • Adaptive control or self-tuning control of nonlinear controllers for nonlinear processes; • Adaptive control or self-tuning control of multivariable controllers for multivariable processes (MIMO systems); Usually these methods adapt the controllers to both the process statics and dynamics. In special cases the adaptation can be limited to the static behavior alone, leading to adaptive control based on characteristic curves for the steady-states or to extremum value control, optimizing the steady state. Hence, there are several ways to apply adaptive control algorithms. Adaptive control References [1] Astrom, Karl (2008). Adaptive Control. Dover. pp. 25–26. Further reading • K. J. Astrom and B. Wittenmark, Adaptive Control, Addison-Wesley, 1989, 2d ed. 1994. External links • Shankar Sastry and Marc Bodson, Adaptive Control: Stability, Convergence, and Robustness, Prentice-Hall, 1989-1994 (book) (http://www.ece.utah.edu/~bodson/acscr/) • K. Sevcik: Tutorial on Model Reference Adaptive Control (Drexel University) (http://www.pages.drexel.edu/ ~kws23/tutorials/MRAC/MRAC.html) Stochastic control Stochastic control is a subfield of control theory which deals with the existence of uncertainty in the data. The designer assumes, in a Bayesian probability-driven fashion, that a random noise with known probability distribution affects the state evolution and the observation of the controllers. Stochastic control aims to design the optimal controller that performs the desired control task with minimum average cost despite the presence of these noises.[1] An extremely well studied formulation in stochastic control is that of linear-quadratic-Gaussian problem. Here the model is linear, and the objective function is the expected value of a quadratic form, and the additive disturbances are distributed in a Gaussian manner. A basic result for discrete time centralized systems is the certainty equivalence property:[2] that the optimal control solution in this case is the same as would be obtained in the absence of the additive disturbances. This property is applicable to all systems that are merely linear and quadratic (LQ), and the Gaussian assumption allows for the optimal control laws, that are based on the certainty-equivalence property, to be linear functions of the observations of the controllers. This property fails to hold for decentralized control, as was demonstrated by Witsenhausen in the celebrated Witsenhausen's counterexample. Any deviation from the above assumptions—a nonlinear state equation, a non-quadratic objective function, or noise in the multiplicative parameters of the model—would cause the certainty equivalence property not to hold. In the discrete-time case with uncertainty about the parameter values in the transition matrix and/or the control response matrix of the state equation, but still with a linear state equation and quadratic objective function, a matrix Riccati equation can still be obtained for iterating to each period's solution.[3] [2] ch.13 The discrete-time case of a non-quadratic loss function but only additive disturbances can also be handled, albeit with more complications.[4] References [1] Definition from Answers.com (http:/ / www. answers. com/ topic/ stochastic-control-theory?cat=technology) [2] Chow, Gregory P., Analysis and Control of Dynamic Economic Systems, Wiley, 1976. [3] Turnovsky, Stephen, "Optimal stabilization policies for stochastic linear systems: The case of correlated multiplicative and additive disturbances," Review of Economic Studies 43(1), 1976, 191-94. [4] Mitchell, Douglas W., "Tractable risk sensitive control bassed on approximate expected utility," Economic Modelling, April 1990, 161-164. 207 Neural network Neural network The term neural network was traditionally used to refer to a network or circuit of biological neurons.[1] The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes. Thus the term has two distinct usages: 1. Biological neural networks are made up of real biological neurons that are connected or functionally related in a nervous system. In the field of neuroscience, they are often identified as groups of neurons that perform a specific physiological function in laboratory analysis. 2. Artificial neural networks are composed of interconnecting artificial neurons (programming constructs that mimic the properties of biological neurons). Artificial neural networks may either be used to gain an understanding of biological neural networks, or for solving artificial intelligence problems without necessarily creating a model of a real biological system. The real, biological nervous system is highly complex: artificial neural network algorithms attempt to Simplified view of a feedforward artificial neural abstract this complexity and focus on what may hypothetically network matter most from an information processing point of view. Good performance (e.g. as measured by good predictive ability, low generalization error), or performance mimicking animal or human error patterns, can then be used as one source of evidence towards supporting the hypothesis that the abstraction really captured something important from the point of view of information processing in the brain. Another incentive for these abstractions is to reduce the amount of computation required to simulate artificial neural networks, so as to allow one to experiment with larger networks and train them on larger data sets. This article focuses on the relationship between the two concepts; for detailed coverage of the two different concepts refer to the separate articles: biological neural network and artificial neural network. Overview A biological neural network is composed of a group or groups of chemically connected or functionally associated neurons. A single neuron may be connected to many other neurons and the total number of neurons and connections in a network may be extensive. Connections, called synapses, are usually formed from axons to dendrites, though dendrodendritic microcircuits[2] and other connections are possible. Apart from the electrical signaling, there are other forms of signaling that arise from neurotransmitter diffusion, which have an effect on electrical signaling. As such, neural networks are extremely complex. Artificial intelligence and cognitive modeling try to simulate some properties of biological neural networks. While similar in their techniques, the former has the aim of solving particular tasks, while the latter aims to build mathematical models of biological neural systems. In the artificial intelligence field, artificial neural networks have been applied successfully to speech recognition, image analysis and adaptive control, in order to construct software agents (in computer and video games) or autonomous robots. Most of the currently employed artificial neural networks for artificial intelligence are based on statistical estimations, classification optimization and control theory. The cognitive modelling field involves the physical or mathematical modeling of the behavior of neural systems; ranging from the individual neural level (e.g. modeling the spike response curves of neurons to a stimulus), through the neural cluster level (e.g. modelling the release and effects of dopamine in the basal ganglia) to the complete 208 Neural network 209 organism (e.g. behavioral modelling of the organism's response to stimuli). Artificial intelligence, cognitive modelling, and neural networks are information processing paradigms inspired by the way biological neural systems process data. History of the neural network analogy In the brain, spontaneous order appears to arise out of decentralized networks of simple units (neurons). Neural network theory has served both to better identify how the neurons in the brain function and to provide the basis for efforts to create artificial intelligence. The preliminary theoretical base for contemporary neural networks was independently proposed by Alexander Bain[3] (1873) and William James[4] (1890). In their work, both thoughts and body activity resulted from interactions among neurons within the brain. For Bain,[3] every activity led to the firing of a certain set of neurons. When activities were repeated, the connections between those neurons strengthened. According to his theory, this repetition was what led to the formation of memory. The general scientific community at the time was skeptical of Bain’s[3] theory because it required what appeared to be an inordinate number of neural connections within the brain. It is now apparent that the brain is exceedingly complex and that the same brain “wiring” can handle multiple problems and inputs. James’s[4] theory was similar to Bain’s,[3] however, he suggested that memories and actions resulted from electrical currents flowing among the neurons in the brain. His model, by focusing on the flow of electrical currents, did not require individual neural connections for each memory or action. C. S. Sherrington[5] (1898) conducted experiments to test James’s theory. He ran electrical currents down the spinal cords of rats. However, instead of the demonstrating an increase in electrical current as projected by James, Sherrington found that the electrical current strength decreased as the testing continued over time. Importantly, this work led to the discovery of the concept of habituation. McCullouch and Pitts[6] (1943) created a computational model for neural networks based on mathematics and algorithms. They called this model threshold logic. The model paved the way for neural network research to split into two distinct approaches. One approach focused on biological processes in the brain and the other focused on the application of neural networks to artificial intelligence. In the late 1940s psychologist Donald Hebb[7] created a hypothesis of learning based on the mechanism of neural plasticity that is now known as Hebbian learning. Hebbian learning is considered to be a 'typical' unsupervised learning rule and its later variants were early models for long term potentiation. These ideas started being applied to computational models in 1948 with Turing's B-type machines. Farley and Clark[8] (1954) first used computational machines, then called calculators, to simulate a Hebbian network at MIT. Other neural network computational machines were created by Rochester, Holland, Habit, and Duda[9] (1956). Rosenblatt[10] (1958) created the perceptron, an algorithm for pattern recognition based on a two-layer learning computer network using simple addition and subtraction. With mathematical notation, Rosenblatt also described circuitry not in the basic perceptron, such as the exclusive-or circuit, a circuit whose mathematical computation could not be processed until after the backpropogation algorithm was created by Werbos[11] (1975). The perceptron is essentially a linear classifier for classifying data and an output function specified by parameters . Its parameters are adapted with an ad-hoc rule similar to stochastic steepest gradient descent. Because the inner product is a linear operator in the input space, the perceptron can only perfectly classify a set of data for which different classes are linearly separable in the input space, while it often fails completely for non-separable data. While the development of the algorithm initially generated some enthusiasm, partly because of its apparent relation to biological mechanisms, the later discovery of this inadequacy caused such models to be abandoned until the introduction of non-linear models into the field. Neural network Neural network research stagnated after the publication of research of machine learning research by Minsky and Papert[12] (1969). They discovered two key issues with the computational machines that processed neural networks. The first issue was that single-layer neural networks were incapable of processing the exclusive-or circuit. The second significant issue was that computers were not sophisticated enough to effectively handle the long run time required by large neural networks. Neural network research slowed until computers achieved greater processing power. Also key in later advances was the backpropogation algorithm which effectively solved the exclusive-or problem (Werbos 1975).[11] The cognitron (1975) designed by Kunihiko Fukushima[13] was an early multilayered neural network with a training algorithm. The actual structure of the network and the methods used to set the interconnection weights change from one neural strategy to another, each with its advantages and disadvantages. Networks can propagate information in one direction only, or they can bounce back and forth until self-activation at a node occurs and the network settles on a final state. The ability for bi-directional flow of inputs between neurons/nodes was produced with the Hopfield's network (1982), and specialization of these node layers for specific purposes was introduced through the first hybrid network. The parallel distributed processing of the mid-1980s became popular under the name connectionism. The text by Rummelhart and McClelland[14] (1986) provided a full exposition of the use connectionism in computers to simulate neural processes. The rediscovery of the backpropagation algorithm was probably the main reason behind the repopularisation of neural networks after the publication of "Learning Internal Representations by Error Propagation" in 1986 (Though backpropagation itself dates from 1969). The original network utilized multiple layers of weight-sum units of the type , where was a sigmoid function or logistic function such as used in logistic regression. Training was done by a form of stochastic gradient descent. The employment of the chain rule of differentiation in deriving the appropriate parameter updates results in an algorithm that seems to 'backpropagate errors', hence the nomenclature. However it is essentially a form of gradient descent. Determining the optimal parameters in a model of this type is not trivial, and local numerical optimization methods such as gradient descent can be sensitive to initialization because of the presence of local minima of the training criterion. In recent times, networks with the same architecture as the backpropagation network are referred to as multilayer perceptrons. This name does not impose any limitations on the type of algorithm used for learning. The backpropagation network generated much enthusiasm at the time and there was much controversy about whether such learning could be implemented in the brain or not, partly because a mechanism for reverse signaling was not obvious at the time, but most importantly because there was no plausible source for the 'teaching' or 'target' signal. However, since 2006, several unsupervised learning procedures have been proposed for neural networks with one or more layers, using so-called deep learning algorithms. These algorithms can be used to learn intermediate representations, with or without a target signal, that capture the salient features of the distribution of sensory signals arriving at each layer of the neural network. 210 Neural network The brain, neural networks and computers Neural networks, as used in artificial intelligence, have traditionally been viewed as simplified models of neural processing in the brain, even though the relation between this model and brain biological architecture is debated, as little is known about how the brain actually works. A subject of current research in theoretical neuroscience is the question surrounding the degree of complexity and the properties that individual neural elements should have to reproduce something resembling animal intelligence. Historically, computers evolved from the von Neumann architecture, which is based on sequential processing and execution of explicit Computer simulation of the branching instructions. On the other hand, the origins of neural networks are architecture of the dendrites of pyramidal based on efforts to model information processing in biological systems, [15] neurons. which may rely largely on parallel processing as well as implicit instructions based on recognition of patterns of 'sensory' input from external sources. In other words, at its very heart a neural network is a complex statistical processor (as opposed to being tasked to sequentially process and execute). Neural coding is concerned with how sensory and other information is represented in the brain by neurons. The main goal of studying neural coding is to characterize the relationship between the stimulus and the individual or ensemble neuronal responses and the relationship among electrical activity of the neurons in the ensemble.[16] It is thought that neurons can encode both digital and analog information.[17] Neural networks and artificial intelligence A neural network (NN), in the case of artificial neurons called artificial neural network (ANN) or simulated neural network (SNN), is an interconnected group of natural or artificial neurons that uses a mathematical or computational model for information processing based on a connectionistic approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network. In more practical terms neural networks are non-linear statistical data modeling or decision making tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data. However, the paradigm of neural networks - i.e., implicit, not explicit , learning is stressed - seems more to correspond to some kind of natural intelligence than to the traditional symbol-based Artificial Intelligence, which would stress, instead, rule-based learning. Background An artificial neural network involves a network of simple processing elements (artificial neurons) which can exhibit complex global behavior, determined by the connections between the processing elements and element parameters. Artificial neurons were first proposed in 1943 by Warren McCulloch, a neurophysiologist, and Walter Pitts, a logician, who first collaborated at the University of Chicago.[18] One classical type of artificial neural network is the recurrent Hopfield net. In a neural network model simple nodes (which can be called by a number of names, including "neurons", "neurodes", "Processing Elements" (PE) and "units"), are connected together to form a network of nodes — hence the term "neural network". While a neural network does not have to be adaptive per se, its practical use comes with algorithms designed to alter the strength (weights) of the connections in the network to produce a desired signal flow. 211 Neural network In modern software implementations of artificial neural networks the approach inspired by biology has more or less been abandoned for a more practical approach based on statistics and signal processing. In some of these systems, neural networks, or parts of neural networks (such as artificial neurons), are used as components in larger systems that combine both adaptive and non-adaptive elements. The concept of a neural network appears to have first been proposed by Alan Turing in his 1948 paper "Intelligent Machinery". Applications of natural and of artificial neural networks The utility of artificial neural network models lies in the fact that they can be used to infer a function from observations and also to use it. Unsupervised neural networks can also be used to learn representations of the input that capture the salient characteristics of the input distribution, e.g., see the Boltzmann machine (1983), and more recently, deep learning algorithms, which can implicitly learn the distribution function of the observed data. Learning in neural networks is particularly useful in applications where the complexity of the data or task makes the design of such functions by hand impractical. The tasks to which artificial neural networks are applied tend to fall within the following broad categories: • Function approximation, or regression analysis, including time series prediction and modeling. • Classification, including pattern and sequence recognition, novelty detection and sequential decision making. • Data processing, including filtering, clustering, blind signal separation and compression. Application areas of ANNs include system identification and control (vehicle control, process control), game-playing and decision making (backgammon, chess, racing), pattern recognition (radar systems, face identification, object recognition), sequence recognition (gesture, speech, handwritten text recognition), medical diagnosis, financial applications, data mining (or knowledge discovery in databases, "KDD"), visualization and e-mail spam filtering. Neural networks and neuroscience Theoretical and computational neuroscience is the field concerned with the theoretical analysis and computational modeling of biological neural systems. Since neural systems are intimately related to cognitive processes and behaviour, the field is closely related to cognitive and behavioural modeling. The aim of the field is to create models of biological neural systems in order to understand how biological systems work. To gain this understanding, neuroscientists strive to make a link between observed biological processes (data), biologically plausible mechanisms for neural processing and learning (biological neural network models) and theory (statistical learning theory and information theory). Types of models Many models are used in the field, each defined at a different level of abstraction and trying to model different aspects of neural systems. They range from models of the short-term behaviour of individual neurons, through models of how the dynamics of neural circuitry arise from interactions between individual neurons, to models of how behaviour can arise from abstract neural modules that represent complete subsystems. These include models of the long-term and short-term plasticity of neural systems and its relation to learning and memory, from the individual neuron to the system level. 212 Neural network Current research While initially research had been concerned mostly with the electrical characteristics of neurons, a particularly important part of the investigation in recent years has been the exploration of the role of neuromodulators such as dopamine, acetylcholine, and serotonin on behaviour and learning. Biophysical models, such as BCM theory, have been important in understanding mechanisms for synaptic plasticity, and have had applications in both computer science and neuroscience. Research is ongoing in understanding the computational algorithms used in the brain, with some recent biological evidence for radial basis networks and neural backpropagation as mechanisms for processing data. Computational devices have been created in CMOS for both biophysical simulation and neuromorphic computing. More recent efforts show promise for creating nanodevices[19] for very large scale principal components analyses and convolution. If successful, these efforts could usher in a new era of neural computing[20] that is a step beyond digital computing, because it depends on learning rather than programming and because it is fundamentally analog rather than digital even though the first instantiations may in fact be with CMOS digital devices. Architecture The basic architecture consists of three types of neuron layers: input, hidden, and output. In feed-forward networks, the signal flow is from input to output units, strictly in a feed-forward direction. The data processing can extend over multiple layers of units, but no feedback connections are present. Recurrent networks contain feedback connections. Contrary to feed-forward networks, the dynamical properties of the network are important. In some cases, the activation values of the units undergo a relaxation process such that the network will evolve to a stable state in which these activations do not change anymore. In other applications, the changes of the activation values of the output neurons are significant, such that the dynamical behavior constitutes the output of the network. Other neural network architectures include adaptive resonance theory maps and competitive networks. Criticism A common criticism of neural networks, particularly in robotics, is that they require a large diversity of training for real-world operation. This is not surprising, since any learning machine needs sufficient representative examples in order to capture the underlying structure that allows it to generalize to new cases. Dean Pomerleau, in his research presented in the paper "Knowledge-based Training of Artificial Neural Networks for Autonomous Robot Driving," uses a neural network to train a robotic vehicle to drive on multiple types of roads (single lane, multi-lane, dirt, etc.). A large amount of his research is devoted to (1) extrapolating multiple training scenarios from a single training experience, and (2) preserving past training diversity so that the system does not become overtrained (if, for example, it is presented with a series of right turns – it should not learn to always turn right). These issues are common in neural networks that must decide from amongst a wide variety of responses, but can be dealt with in several ways, for example by randomly shuffling the training examples, by using a numerical optimization algorithm that does not take too large steps when changing the network connections following an example, or by grouping examples in so-called mini-batches. A. K. Dewdney, a former Scientific American columnist, wrote in 1997, "Although neural nets do solve a few toy problems, their powers of computation are so limited that I am surprised anyone takes them seriously as a general problem-solving tool." (Dewdney, p. 82) Arguments for Dewdney's position are that to implement large and effective software neural networks, much processing and storage resources need to be committed. While the brain has hardware tailored to the task of processing signals through a graph of neurons, simulating even a most simplified form on Von Neumann technology may compel a NN designer to fill many millions of database rows for its connections - which can consume vast 213 Neural network amounts of computer memory and hard disk space. Furthermore, the designer of NN systems will often need to simulate the transmission of signals through many of these connections and their associated neurons - which must often be matched with incredible amounts of CPU processing power and time. While neural networks often yield effective programs, they too often do so at the cost of efficiency (they tend to consume considerable amounts of time and money). Arguments against Dewdney's position are that neural nets have been successfully used to solve many complex and diverse tasks, ranging from autonomously flying aircraft [21] to detecting credit card fraud [22]. Technology writer Roger Bridgman commented on Dewdney's statements about neural nets: Neural networks, for instance, are in the dock not only because they have been hyped to high heaven, (what hasn't?) but also because you could create a successful net without understanding how it worked: the bunch of numbers that captures its behaviour would in all probability be "an opaque, unreadable table...valueless as a scientific resource". In spite of his emphatic declaration that science is not technology, Dewdney seems here to pillory neural nets as bad science when most of those devising them are just trying to be good engineers. An unreadable table that a useful machine could read would still be well worth having.[23] In response to this kind of criticism, one should note that although it is true that analyzing what has been learned by an artificial neural network is difficult, it is much easier to do so than to analyze what has been learned by a biological neural network. Furthermore, researchers involved in exploring learning algorithms for neural networks are gradually uncovering generic principles which allow a learning machine to be successful. For example, Bengio and LeCun (2007) wrote an article regarding local vs non-local learning, as well as shallow vs deep architecture [24]. Some other criticisms came from believers of hybrid models (combining neural networks and symbolic approaches). They advocate the intermix of these two approaches and believe that hybrid models can better capture the mechanisms of the human mind (Sun and Bookman 1990). References [1] J. J. HOPFIELD Neural networks and physical systems with emergent collective computational abilities. Proc. NatL Acad. Sci. USA Vol. 79, pp. 2554-2558, April 1982 Biophysics (http:/ / www. pnas. org/ content/ 79/ 8/ 2554. full. pdf) [2] Arbib, p.666 [3] Bain (1873). Mind and Body: The Theories of Their Relation. New York: D. Appleton and Company. [4] James (1890). The Principles of Psychology. New York: H. Holt and Company. [5] Sherrington, C.S.. "Experiments in Examination of the Peripheral Distribution of the Fibers of the Posterior Roots of Some Spinal Nerves". Proceedings of the Royal Society of London 190: 45–186. [6] McCullock, Warren; Walter Pitts (1943). "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics 5 (4): 115–133. doi:10.1007/BF02478259. [7] Hebb, Donald (1949). The Organization of Behavior. New York: Wiley. [8] Farley, B; W.A. Clark (1954). "Simulation of Self-Organizing Systems by Digital Computer". IRE Transactions on Information Theory 4 (4): 76–84. doi:10.1109/TIT.1954.1057468. [9] Rochester, N.; J.H. Holland, L.H. Habit, and W.L. Duda (1956). "Tests on a cell assembly theory of the action of the brain, using a large digital computer". IRE Transactions on Information Theory 2 (3): 80–93. doi:10.1109/TIT.1956.1056810. [10] Rosenblatt, F. (1958). "The Perceptron: A Probalistic Model For Information Storage And Organization In The Brain". Psychological Review 65 (6): 386–408. doi:10.1037/h0042519. PMID 13602029. [11] Werbos, P.J. (1975). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. [12] Minsky, M.; S. Papert (1969). An Introduction to Computational Geometry. MIT Press. ISBN 0262630222. [13] Fukushima, Kunihiko (1975). "Cognitron: A self-organizing multilayered neural network". Biological Cybernetics 20 (3–4): 121–136. doi:10.1007/BF00342633. PMID 1203338. [14] Rummelhart, D.E; James McClelland (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge: MIT Press. [15] "PLoS Computational Biology Issue Image". PLoS Computational Biology 6 (8): ev06.ei08. 2010. doi:10.1371/image.pcbi.v06.i08. [16] Brown EN, Kass RE, Mitra PP. (2004). "Multiple neural spike train data analysis: state-of-the-art and future challenges". Nature Neuroscience 7 (5): 456–61. doi:10.1038/nn1228. PMID 15114358. 214 Neural network [17] Spike arrival times: A highly efficient coding scheme for neural networks (http:/ / pop. cerco. ups-tlse. fr/ fr_vers/ documents/ thorpe_sj_90_91. pdf), SJ Thorpe - Parallel processing in neural systems, 1990 [18] McCulloch, Warren; Pitts, Walter, "A Logical Calculus of Ideas Immanent in Nervous Activity", 1943, Bulletin of Mathematical Biophysics 5:115-133. [19] Yang, J. J.; Pickett, M. D.; Li, X. M.; Ohlberg, D. A. A.; Stewart, D. R.; Williams, R. S. Nat. Nanotechnol. 2008, 3, 429–433. [20] Strukov, D. B.; Snider, G. S.; Stewart, D. R.; Williams, R. S. Nature 2008, 453, 80–83. [21] http:/ / www. nasa. gov/ centers/ dryden/ news/ NewsReleases/ 2003/ 03-49. html [22] http:/ / www. visa. ca/ en/ about/ visabenefits/ innovation. cfm [23] Roger Bridgman's defence of neural networks (http:/ / members. fortunecity. com/ templarseries/ popper. html) [24] http:/ / www. iro. umontreal. ca/ ~lisa/ publications2/ index. php/ publications/ show/ 4 Further reading • Arbib, Michael A. (Ed.) (1995). The Handbook of Brain Theory and Neural Networks. • Alspector, U.S. Patent 4874963 (http://www.google.com/patents?vid=4874963) "Neuromorphic learning networks". October 17, 1989. • Agre, Philip E. (1997). Computation and Human Experience. Cambridge University Press. ISBN 0-521-38603-9., p. 80 • Yaneer Bar-Yam (2003). Dynamics of Complex Systems, Chapter 2 (http://necsi.edu/publications/dcs/ Bar-YamChap2.pdf). • Yaneer Bar-Yam (2003). Dynamics of Complex Systems, Chapter 3 (http://necsi.edu/publications/dcs/ Bar-YamChap3.pdf). • Yaneer Bar-Yam (2005). Making Things Work (http://necsi.edu/publications/mtw/). See chapter 3. • Bertsekas, Dimitri P. (1999). Nonlinear Programming. ISBN 1886529000. • Bertsekas, Dimitri P. & Tsitsiklis, John N. (1996). Neuro-dynamic Programming. ISBN 1886529108. • Bhadeshia H. K. D. H. (1992). "Neural Networks in Materials Science" (http://www.msm.cam.ac.uk/ phase-trans/abstracts/neural.review.pdf). ISIJ International 39 (10): 966–979. doi:10.2355/isijinternational.39.966. • Boyd, Stephen & Vandenberghe, Lieven (2004). Convex Optimization (http://www.stanford.edu/~boyd/ cvxbook/). • Dewdney, A. K. (1997). Yes, We Have No Neutrons: An Eye-Opening Tour through the Twists and Turns of Bad Science. Wiley, 192 pp. ISBN 0471108065. See chapter 5. • Egmont-Petersen, M., de Ridder, D., Handels, H. (2002). "Image processing with neural networks - a review". Pattern Recognition 35 (10): 2279–2301. doi:10.1016/S0031-3203(01)00178-9. • Fukushima, K. (1975). "Cognitron: A Self-Organizing Multilayered Neural Network". Biological Cybernetics 20 (3–4): 121–136. doi:10.1007/BF00342633. PMID 1203338. • Frank, Michael J. (2005). "Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Non-medicated Parkinsonism". Journal of Cognitive Neuroscience 17 (1): 51–72. doi:10.1162/0898929052880093. PMID 15701239. • Gardner, E.J., & Derrida, B. (1988). "Optimal storage properties of neural network models". Journal of Physics a 21: 271–284. doi:10.1088/0305-4470/21/1/031. • Hadzibeganovic, Tarik & Cannas, Sergio A. (2009). "A Tsallis' statistics based neural network model for novel word learning". Physica A: Statistical Mechanics and its Applications 388 (5): 732–746. doi:10.1016/j.physa.2008.10.042. • Krauth, W., & Mezard, M. (1989). "Storage capacity of memory with binary couplings". Journal de Physique 50 (20): 3057–3066. doi:10.1051/jphys:0198900500200305700. • Maass, W., & Markram, H. (2002). "On the computational power of recurrent circuits of spiking neurons" (http:// www.igi.tugraz.at/maass/publications.html). Journal of Computer and System Sciences 69 (4): 593–616. doi:10.1016/j.jcss.2004.04.001. 215 Neural network • MacKay, David (2003). Information Theory, Inference, and Learning Algorithms (http://www.inference.phy. cam.ac.uk/mackay/itprnn/book.html). • Mandic, D. & Chambers, J. (2001). Recurrent Neural Networks for Prediction: Architectures, Learning algorithms and Stability. Wiley. ISBN 0471495174. • Minsky, M. & Papert, S. (1969). An Introduction to Computational Geometry. MIT Press. ISBN 0262630222. • Muller, P. & Insua, D.R. (1995). "Issues in Bayesian Analysis of Neural Network Models". Neural Computation 10 (3): 571–592. PMID 9527841. • Reilly, D.L., Cooper, L.N. & Elbaum, C. (1982). "A Neural Model for Category Learning". Biological Cybernetics 45: 35–41. doi:10.1007/BF00387211. • Rosenblatt, F. (1962). Principles of Neurodynamics. Spartan Books. • Sun, R. & Bookman,L. (eds.) (1994.). Computational Architectures Integrating Neural and Symbolic Processes.. Kluwer Academic Publishers, Needham, MA.. • Sutton, Richard S. & Barto, Andrew G. (1998). Reinforcement Learning : An introduction (http://www.cs. ualberta.ca/~sutton/book/the-book.html). • Van den Bergh, F. Engelbrecht, AP. Cooperative Learning in Neural Networks using Particle Swarm Optimizers. CIRG 2000. • Wilkes, A.L. & Wade, N.J. (1997). "Bain on Neural Networks". Brain and Cognition 33 (3): 295–305. doi:10.1006/brcg.1997.0869. PMID 9126397. • Wasserman, P.D. (1989). Neural computing theory and practice. Van Nostrand Reinhold. ISBN 0442207433. • Jeffrey T. Spooner, Manfredi Maggiore, Raul Ord onez, and Kevin M. Passino, Stable Adaptive Control and Estimation for Nonlinear Systems: Neural and Fuzzy Approximator Techniques, John Wiley and Sons, NY, 2002. • Peter Dayan, L.F. Abbott. Theoretical Neuroscience. MIT Press. ISBN 0262041995. • Wulfram Gerstner, Werner Kistler. Spiking Neuron Models:Single Neurons, Populations, Plasticity. Cambridge University Press. ISBN 0521890799. • Steeb, W-H (2008). The Nonlinear Workbook: Chaos, Fractals, Neural Networks, Genetic Algorithms, Gene Expression Programming, Support Vector Machine, Wavelets, Hidden Markov Models, Fuzzy Logic with C++, Java and SymbolicC++ Programs: 4th edition. World Scientific Publishing. ISBN 981-281-852-9. External links • A Brief Introduction to Neural Networks (D. Kriesel) (http://www.dkriesel.com/en/science/neural_networks) - Illustrated, bilingual manuscript about artificial neural networks; Topics so far: Perceptrons, Backpropagation, Radial Basis Functions, Recurrent Neural Networks, Self Organizing Maps, Hopfield Networks. • LearnArtificialNeuralNetworks (http://www.learnartificialneuralnetworks.com/robotcontrol.html) - Robot control and neural networks • Review of Neural Networks in Materials Science (http://www.msm.cam.ac.uk/phase-trans/abstracts/neural. review.html) • Artificial Neural Networks Tutorial in three languages (Univ. Politécnica de Madrid) (http://www.gc.ssr.upm. es/inves/neural/ann1/anntutorial.html) • Introduction to Neural Networks and Knowledge Modeling (http://www.makhfi.com/tutorial/introduction. htm) • Another introduction to ANN (http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html) • Next Generation of Neural Networks (http://youtube.com/watch?v=AyzOUbkUf3M) - Google Tech Talks • Performance of Neural Networks (http://www.msm.cam.ac.uk/phase-trans/2009/performance.html) • Neural Networks and Information (http://www.msm.cam.ac.uk/phase-trans/2009/ review_Bhadeshia_SADM.pdf) • PMML Representation (http://www.dmg.org/v4-0/NeuralNetwork.html) - Standard way to represent neural networks 216 Fuzzy logic 217 Fuzzy logic Fuzzy logic is a form of many-valued logic; it deals with reasoning that is approximate rather than fixed and exact. In contrast with traditional logic theory, where binary sets have two-valued logic: true or false, fuzzy logic variables may have a truth value that ranges in degree between 0 and 1. Fuzzy logic has been extended to handle the concept of partial truth, where the truth value may range between completely true and completely false.[1] Furthermore, when linguistic variables are used, these degrees may be managed by specific functions. Fuzzy logic began with the 1965 proposal of fuzzy set theory by Lotfi Zadeh.[2] many fields, from control theory to artificial intelligence. [3] Fuzzy logic has been applied to Overview The reasoning in fuzzy logic is similar to human reasoning. It allows for approximate values and inferences as well as incomplete or ambiguous data (fuzzy data) as opposed to only relying on crisp data (binary yes/no choices). Fuzzy logic is able to process incomplete data and provide approximate solutions to problems other methods find difficult to solve. Terminology used in fuzzy logic not used in other methods are: very high, increasing, somewhat decreased, reasonable and very low. [4] Degrees of truth Fuzzy logic and probabilistic logic are mathematically similar – both have truth values ranging between 0 and 1 – but conceptually distinct, due to different interpretations—see interpretations of probability theory. Fuzzy logic corresponds to "degrees of truth", while probabilistic logic corresponds to "probability, likelihood"; as these differ, fuzzy logic and probabilistic logic yield different models of the same real-world situations. Both degrees of truth and probabilities range between 0 and 1 and hence may seem similar at first. For example, let a 100 ml glass contain 30 ml of water. Then we may consider two concepts: Empty and Full. The meaning of each of them can be represented by a certain fuzzy set. Then one might define the glass as being 0.7 empty and 0.3 full. Note that the concept of emptiness would be subjective and thus would depend on the observer or designer. Another designer might equally well design a set membership function where the glass would be considered full for all values down to 50 ml. It is essential to realize that fuzzy logic uses truth degrees as a mathematical model of the vagueness phenomenon while probability is a mathematical model of ignorance. Applying truth values A basic application might characterize subranges of a continuous variable. For instance, a temperature measurement for anti-lock brakes might have several separate membership functions defining particular temperature ranges needed to control the brakes properly. Each function maps the same temperature value to a truth value in the 0 to 1 range. These truth values can then be used to determine how the brakes should be controlled. Fuzzy logic temperature Fuzzy logic In this image, the meaning of the expressions cold, warm, and hot is represented by functions mapping a temperature scale. A point on that scale has three "truth values"—one for each of the three functions. The vertical line in the image represents a particular temperature that the three arrows (truth values) gauge. Since the red arrow points to zero, this temperature may be interpreted as "not hot". The orange arrow (pointing at 0.2) may describe it as "slightly warm" and the blue arrow (pointing at 0.8) "fairly cold". Linguistic variables While variables in mathematics usually take numerical values, in fuzzy logic applications, the non-numeric linguistic variables are often used to facilitate the expression of rules and facts.[5] A linguistic variable such as age may have a value such as young or its antonym old. However, the great utility of linguistic variables is that they can be modified via linguistic hedges applied to primary terms. The linguistic hedges can be associated with certain functions. Example Fuzzy set theory defines fuzzy operators on fuzzy sets. The problem in applying this is that the appropriate fuzzy operator may not be known. For this reason, fuzzy logic usually uses IF-THEN rules, or constructs that are equivalent, such as fuzzy associative matrices. Rules are usually expressed in the form: IF variable IS property THEN action For example, a simple temperature regulator that uses a fan might look like this: IF temperature IS very cold THEN stop fan IF temperature IS cold THEN turn down fan IF temperature IS normal THEN maintain level IF temperature IS hot THEN speed up fan There is no "ELSE" – all of the rules are evaluated, because the temperature might be "cold" and "normal" at the same time to different degrees. The AND, OR, and NOT operators of boolean logic exist in fuzzy logic, usually defined as the minimum, maximum, and complement; when they are defined this way, they are called the Zadeh operators. So for the fuzzy variables x and y: NOT x = (1 - truth(x)) x AND y = minimum(truth(x), truth(y)) x OR y = maximum(truth(x), truth(y)) There are also other operators, more linguistic in nature, called hedges that can be applied. These are generally adverbs such as "very", or "somewhat", which modify the meaning of a set using a mathematical formula. 218 Fuzzy logic Logical analysis In mathematical logic, there are several formal systems of "fuzzy logic"; most of them belong among so-called t-norm fuzzy logics. Propositional fuzzy logics The most important propositional fuzzy logics are: • Monoidal t-norm-based propositional fuzzy logic MTL is an axiomatization of logic where conjunction is defined by a left continuous t-norm, and implication is defined as the residuum of the t-norm. Its models correspond to MTL-algebras that are prelinear commutative bounded integral residuated lattices. • Basic propositional fuzzy logic BL is an extension of MTL logic where conjunction is defined by a continuous t-norm, and implication is also defined as the residuum of the t-norm. Its models correspond to BL-algebras. • Łukasiewicz fuzzy logic is the extension of basic fuzzy logic BL where standard conjunction is the Łukasiewicz t-norm. It has the axioms of basic fuzzy logic plus an axiom of double negation, and its models correspond to MV-algebras. • Gödel fuzzy logic is the extension of basic fuzzy logic BL where conjunction is Gödel t-norm. It has the axioms of BL plus an axiom of idempotence of conjunction, and its models are called G-algebras. • Product fuzzy logic is the extension of basic fuzzy logic BL where conjunction is product t-norm. It has the axioms of BL plus another axiom for cancellativity of conjunction, and its models are called product algebras. • Fuzzy logic with evaluated syntax (sometimes also called Pavelka's logic), denoted by EVŁ, is a further generalization of mathematical fuzzy logic. While the above kinds of fuzzy logic have traditional syntax and many-valued semantics, in EVŁ is evaluated also syntax. This means that each formula has an evaluation. Axiomatization of EVŁ stems from Łukasziewicz fuzzy logic. A generalization of classical Gödel completeness theorem is provable in EVŁ. Predicate fuzzy logics These extend the above-mentioned fuzzy logics by adding universal and existential quantifiers in a manner similar to the way that predicate logic is created from propositional logic. The semantics of the universal (resp. existential) quantifier in t-norm fuzzy logics is the infimum (resp. supremum) of the truth degrees of the instances of the quantified subformula. Decidability issues for fuzzy logic The notions of a "decidable subset" and "recursively enumerable subset" are basic ones for classical mathematics and classical logic. Then, the question of a suitable extension of such concepts to fuzzy set theory arises. A first proposal in such a direction was made by E.S. Santos by the notions of fuzzy Turing machine, Markov normal fuzzy algorithm and fuzzy program (see Santos 1970). Successively, L. Biacino and G. Gerla showed that such a definition is not adequate and therefore proposed the following one. Ü denotes the set of rational numbers in [0,1]. A fuzzy subset s : S [0,1] of a set S is recursively enumerable if a recursive map h : S×N Ü exists such that, for every x in S, the function h(x,n) is increasing with respect to n and s(x) = lim h(x,n). We say that s is decidable if both s and its complement –s are recursively enumerable. An extension of such a theory to the general case of the L-subsets is proposed in Gerla 2006. The proposed definitions are well related with fuzzy logic. Indeed, the following theorem holds true (provided that the deduction apparatus of the fuzzy logic satisfies some obvious effectiveness property). Theorem. Any axiomatizable fuzzy theory is recursively enumerable. In particular, the fuzzy set of logically true formulas is recursively enumerable in spite of the fact that the crisp set of valid formulas is not recursively enumerable, in general. Moreover, any axiomatizable and complete theory is decidable. 219 Fuzzy logic It is an open question to give supports for a Church thesis for fuzzy logic claiming that the proposed notion of recursive enumerability for fuzzy subsets is the adequate one. To this aim, further investigations on the notions of fuzzy grammar and fuzzy Turing machine should be necessary (see for example Wiedermann's paper). Another open question is to start from this notion to find an extension of Gödel’s theorems to fuzzy logic. Fuzzy databases Once fuzzy relations are defined, it is possible to develop fuzzy relational databases. The first fuzzy relational database, FRDB, appeared in Maria Zemankova's dissertation. Later, some other models arose like the Buckles-Petry model, the Prade-Testemale Model, the Umano-Fukami model or the GEFRED model by J.M. Medina, M.A. Vila et al. In the context of fuzzy databases, some fuzzy querying languages have been defined, highlighting the SQLf by P. Bosc et al. and the FSQL by J. Galindo et al. These languages define some structures in order to include fuzzy aspects in the SQL statements, like fuzzy conditions, fuzzy comparators, fuzzy constants, fuzzy constraints, fuzzy thresholds, linguistic labels and so on. Comparison to probability Fuzzy logic and probability are different ways of expressing uncertainty. While both fuzzy logic and probability theory can be used to represent subjective belief, fuzzy set theory uses the concept of fuzzy set membership (i.e., how much a variable is in a set), and probability theory uses the concept of subjective probability (i.e., how probable do I think that a variable is in a set). While this distinction is mostly philosophical, the fuzzy-logic-derived possibility measure is inherently different from the probability measure, hence they are not directly equivalent. However, many statisticians are persuaded by the work of Bruno de Finetti that only one kind of mathematical uncertainty is needed and thus fuzzy logic is unnecessary. On the other hand, Bart Kosko argues that probability is a subtheory of fuzzy logic, as probability only handles one kind of uncertainty. He also claims to have proven a derivation of Bayes' theorem from the concept of fuzzy subsethood. Lotfi Zadeh argues that fuzzy logic is different in character from probability, and is not a replacement for it. He fuzzified probability to fuzzy probability and also generalized it to what is called possibility theory. (cf.[6] ) Notes [1] Novák, V., Perfilieva, I. and Močkoř, J. (1999) Mathematical principles of fuzzy logic Dodrecht: Kluwer Academic. ISBN 0-7923-8595-0 [2] "Fuzzy Logic" (http:/ / plato. stanford. edu/ entries/ logic-fuzzy/ ). Stanford Encyclopedia of Philosophy. Stanford University. 2006-07-23. . Retrieved 2008-09-29. [3] Zadeh, L.A. (1965). "Fuzzy sets", Information and Control 8 (3): 338–353. [4] James A. O'Brien; George M. Marakas (2011). Management Information Systesm (10th ed.). New York: McGraw Hill. pp. 431. [5] Zadeh, L. A. et al. 1996 Fuzzy Sets, Fuzzy Logic, Fuzzy Systems, World Scientific Press, ISBN 9810224214 [6] Novák, V. Are fuzzy sets a reasonable tool for modeling vague phenomena?, Fuzzy Sets and Systems 156 (2005) 341—348. Bibliography • Von Altrock, Constantin (1995). Fuzzy logic and NeuroFuzzy applications explained. Upper Saddle River, NJ: Prentice Hall PTR. ISBN 0-13-368465-2. • Arabacioglu, B. C. (2010). "Using fuzzy inference system for architectural space analysis" (http://www. sciencedirect.com/science/article/pii/S1568494609002014). Applied Soft Computing 10 (3): 926–937. • Biacino, L.; Gerla, G. (2002). "Fuzzy logic, continuity and effectiveness". Archive for Mathematical Logic 41 (7): 643–667. doi:10.1007/s001530100128. ISSN 0933-5846. • Cox, Earl (1994). The fuzzy systems handbook: a practitioner's guide to building, using, maintaining fuzzy systems. Boston: AP Professional. ISBN 0-12-194270-8. • Gerla, Giangiacomo (2006). "Effectiveness and Multivalued Logics". Journal of Symbolic Logic 71 (1): 137–162. doi:10.2178/jsl/1140641166. ISSN 0022-4812. 220 Fuzzy logic • Hájek, Petr (1998). Metamathematics of fuzzy logic. Dordrecht: Kluwer. ISBN 0792352386. • Hájek, Petr (1995). "Fuzzy logic and arithmetical hierarchy". Fuzzy Sets and Systems 3 (8): 359–363. doi:10.1016/0165-0114(94)00299-M. ISSN 0165-0114. • Halpern, Joseph Y. (2003). Reasoning about uncertainty. Cambridge, Mass: MIT Press. ISBN 0-262-08320-5. • Höppner, Frank; Klawonn, F.; Kruse, R.; Runkler, T. (1999). Fuzzy cluster analysis: methods for classification, data analysis and image recognition. New York: John Wiley. ISBN 0-471-98864-2. • Ibrahim, Ahmad M. (1997). Introduction to Applied Fuzzy Electronics. Englewood Cliffs, N.J: Prentice Hall. ISBN 0-13-206400-6. • Klir, George J.; Folger, Tina A. (1988). Fuzzy sets, uncertainty, and information. Englewood Cliffs, N.J: Prentice Hall. ISBN 0-13-345984-5. • Klir, George J.; St Clair, Ute H.; Yuan, Bo (1997). Fuzzy set theory: foundations and applications. Englewood Cliffs, NJ: Prentice Hall. ISBN 0133410587. • Klir, George J.; Yuan, Bo (1995). Fuzzy sets and fuzzy logic: theory and applications. Upper Saddle River, NJ: Prentice Hall PTR. ISBN 0-13-101171-5. • Kosko, Bart (1993). Fuzzy thinking: the new science of fuzzy logic. New York: Hyperion. ISBN 0-7868-8021-X. • Kosko, Bart; Isaka, Satoru (July 1993). "Fuzzy Logic". Scientific American 269 (1): 76–81. doi:10.1038/scientificamerican0793-76. • Montagna, F. (2001). "Three complexity problems in quantified fuzzy logic". Studia Logica 68 (1): 143–152. doi:10.1023/A:1011958407631. ISSN 0039-3215. • Mundici, Daniele; Cignoli, Roberto; D'Ottaviano, Itala M. L. (1999). Algebraic foundations of many-valued reasoning. Dodrecht: Kluwer Academic. ISBN 0-7923-6009-5. • Novák, Vilém (1989). Fuzzy Sets and Their Applications. Bristol: Adam Hilger. ISBN 0-85274-583-4. • Novák, Vilém (2005). "On fuzzy type theory". Fuzzy Sets and Systems 149 (2): 235–273. doi:10.1016/j.fss.2004.03.027. • Novák, Vilém; Perfilieva, Irina; Močkoř, Jiří (1999). Mathematical principles of fuzzy logic. Dordrecht: Kluwer Academic. ISBN 0-7923-8595-0. • Onses, Richard (1996). Second Order Experton: A new Tool for Changing Paradigms in Country Risk Calculation. ISBN 8477195587. • Onses, Richard (1994). Détermination de l´incertitude inhérente aux investissements en Amérique Latine sur la base de la théorie des sous ensembles flous. Barcelona. ISBN 8447508811. • Passino, Kevin M.; Yurkovich, Stephen (1998). Fuzzy control. Boston: Addison-Wesley. ISBN 020118074X. • Pedrycz, Witold; Gomide, Fernando (2007). Fuzzy systems engineering: Toward Human-Centerd Computing. Hoboken: Wiley-Interscience. ISBN 978047178857-7. • Pu, Pao Ming; Liu, Ying Ming (1980). "Fuzzy topology. I. Neighborhood structure of a fuzzy point and Moore-Smith convergence". Journal of Mathematical Analysis and Applications 76 (2): 571–599. doi:10.1016/0022-247X(80)90048-7. ISSN 0022-247X • Santos, Eugene S. (1970). "Fuzzy Algorithms". Information and Control 17 (4): 326–339. doi:10.1016/S0019-9958(70)80032-8. • Scarpellini, Bruno (1962). "Die Nichaxiomatisierbarkeit des unendlichwertigen Prädikatenkalküls von Łukasiewicz". Journal of Symbolic Logic (Association for Symbolic Logic) 27 (2): 159–170. doi:10.2307/2964111. ISSN 0022-4812. JSTOR 2964111. • Steeb, Willi-Hans (2008). The Nonlinear Workbook: Chaos, Fractals, Cellular Automata, Neural Networks, Genetic Algorithms, Gene Expression Programming, Support Vector Machine, Wavelets, Hidden Markov Models, Fuzzy Logic with C++, Java and SymbolicC++ Programs: 4edition. World Scientific. ISBN 981-281-852-9. • Wiedermann, J. (2004). "Characterizing the super-Turing computing power and efficiency of classical fuzzy Turing machines". Theor. Comput. Sci. 317 (1-3): 61–69. doi:10.1016/j.tcs.2003.12.004. 221 Fuzzy logic • Yager, Ronald R.; Filev, Dimitar P. (1994). Essentials of fuzzy modeling and control. New York: Wiley. ISBN 0-471-01761-2. • Van Pelt, Miles (2008). Fuzzy Logic Applied to Daily Life. Seattle, WA: No No No No Press. ISBN 0-252-16341-9. • Wilkinson, R.H. (1963). "A method of generating functions of several variables using analog diode logic". IEEE Transactions on Electronic Computers 12 (2): 112–129. doi:10.1109/PGEC.1963.263419. • Zadeh, L.A. (1968). "Fuzzy algorithms". Information and Control 12 (2): 94–102. doi:10.1016/S0019-9958(68)90211-8. ISSN 0019-9958. • Zadeh, L.A. (1965). "Fuzzy sets". Information and Control 8 (3): 338–353. doi:10.1016/S0019-9958(65)90241-X. ISSN 0019-9958. • Zemankova-Leech, M. (1983). Fuzzy Relational Data Bases. Ph. D. Dissertation. Florida State University. • Zimmermann, H. (2001). Fuzzy set theory and its applications. Boston: Kluwer Academic Publishers. ISBN 0-7923-7435-5. External links Additional articles • • • • • Formal fuzzy logic (http://en.citizendium.org/wiki/Formal_fuzzy_logic) - article at Citizendium Fuzzy Logic (http://www.scholarpedia.org/article/Fuzzy_Logic) - article at Scholarpedia Modeling With Words (http://www.scholarpedia.org/article/Modeling_with_words) - article at Scholarpedia Fuzzy logic (http://plato.stanford.edu/entries/logic-fuzzy/) - article at Stanford Encyclopedia of Philosophy Fuzzy Math (http://blog.peltarion.com/2006/10/25/fuzzy-math-part-1-the-theory) - Beginner level introduction to Fuzzy Logic. • Fuzzy Logic and the Internet of Things: I-o-T (http://www.i-o-t.org/post/WEB_3) Links pages • Web page about FSQL (http://www.lcc.uma.es/~ppgg/FSQL/): References and links about FSQL Applications • Research conference paper providing a way to establish a fuzzy workload plan while dealing with project planning problem under uncertainty (http://www.enim.fr/mosim2010/articles/279.pdf) • Research article that describes how industrial foresight could be integrated into capital budgeting with intelligent agents and Fuzzy Logic (http://econpapers.repec.org/paper/amrwpaper/398.htm) • A doctoral dissertation describing how Fuzzy Logic can be applied in profitability analysis of very large industrial investments (http://econpapers.repec.org/paper/pramprapa/4328.htm) • A method for asset valuation that uses fuzzy logic and fuzzy numbers for real option valuation (http://users.abo. fi/mcollan/fuzzypayoff.html) 222 Expert system Expert system In artificial intelligence, an expert system is a computer system that emulates the decision-making ability of a human expert.[1] Expert systems are designed to solve complex problems by reasoning about knowledge, like an expert, and not by following the procedure of a developer as is the case in conventional programming.[2] [3] [4] The first expert systems were created in the 1970s and then proliferated in the 1980s.[5] Expert systems were among the first truly successful forms of AI software.[6] [7] [8] [9] [10] [11] An expert system has a unique structure, different from traditional programs. It is divided into two parts, one fixed, independent of the expert system: the inference engine, and one variable: the knowledge base. To run an expert system, the engine reasons about the knowledge base like a human.[12] In the 80's a third part appeared: a dialog interface to communicate with users.[13] This ability to conduct a conversation with users was later called "conversational".[14] [15] History Expert systems were introduced by researchers in the Stanford Heuristic Programming Project, including the "father of expert systems" Edward Feigenbaum, with the Dendral and Mycin systems. Principal contributors to the technology were Bruce Buchanan, Edward Shortliffe, Randall Davis, William vanMelle, Carli Scott and others at Stanford. Expert systems were among the first truly successful forms of AI software.[6] [7] [8] [9] [10] [11] Research is also very active in France, where researchers focus on the automation of reasoning and logic engines. The French Prolog computer language, designed in 1972, marks a real advance over expert systems like Dendral or Mycin: it is a shell,[16] that's to say a software structure ready to receive any expert system and to run it. It integrates an engine using First-Order logic, with rules and facts. It's a tool for mass production of expert systems and was the first operational declarative language,[17] later becoming the best selling IA language in the world.[18] However Prolog is not particularly user friendly and is an order of logic away from human logic.[19] [20] [21] In the 1980s, expert systems proliferated as they were recognized as a practical tool for solving real-world problems. Universities offered expert system courses and two thirds of the Fortune 1000 companies applied the technology in daily business activities.[5] [22] Interest was international with the Fifth Generation Computer Systems project in Japan and increased research funding in Europe. Growth in the field continued into the 1990s. The development of expert systems was aided by the development of the symbolic processing languages Lisp and Prolog. To avoid re-inventing the wheel, expert system shells were created that had more specialized features for building large expert systems.[23] In 1981 the first IBM PC was introduced, with MS-DOS operating system. Its low price started to multiply users and opened a new market for computing and expert systems. In the 80's the image of IA was very good and people believed it would succeed within a short time[15]. Many companies began to market expert systems shells from universities, renamed "generators" because they added to the shell a tool for writing rules in plain language and thus, theoretically, allowed to write expert systems without a programming language nor any other software[16]. The best known: Guru (USA) inspired by Mycin[17][18], Personal Consultant Plus (USA)[19][20], Nexpert Object (developed by Neuron Data, company founded in California by three French)[21][22], Genesia (developed by French public company Electricité de France and marketed by Steria)[23], VP Expert (USA)[24]. But eventually the tools were only used in research projects. They did not penetrate the business market, showing that AI technology was not mature. In 1986, a new expert system generator for PCs appeared on the market, derived from the French academic research: Intelligence Service,[24] [25] sold by GSI-TECSI software company. This software showed a radical innovation: it used propositional logic ("Zeroth order logic") to execute expert systems, reasoning on a knowledge base written with everyday language rules, producing explanations and detecting logic contradictions between the facts. It was the 223 Expert system first tool showing the AI defined by Edward Feigenbaum in his book about the Japanese Fifth Generation, Artificial Intelligence and Japan's Computer Challenge to the World [26] (1983): "The machines will have reasoning power: they will automatically engineer vast amounts of knowledge to serve whatever purpose humans propose, from medical diagnosis to product design, from management decisions to education", "The reasoning animal has, perhaps inevitably, fashioned the reasoning machine", "the reasoning power of these machines matches or exceeds the reasoning power of the humans who instructed them and, in some cases, the reasoning power of any human performing such tasks". Intelligence Service was in fact "Pandora" (1985),[27] a software developed for their thesis by two academic students of Jean-Louis Laurière,[28] one of the most famous and prolific French AI researcher.[29] Unfortunately, as this software was not developed by his own IT developers, GSI-TECSI was unable to make it evolve. Sales became scarce and marketing stopped after a few years. Software architecture The rule base or knowledge base In expert system technology, the knowledge base is expressed with natural language rules IF ... THEN ... For examples : • "IF it is living THEN it is mortal" • "IF his age = known THEN his year of birth = date of today - his age in years" • "IF the identity of the germ is not known with certainty AND the germ is gram-positive AND the morphology of the organism is "rod" AND the germ is aerobic THEN there is a strong probability (0.8) that the germ is of type enterobacteriacae"[30] This formulation has the advantage of speaking in everyday language which is very rare in computer science (a classic program is coded). Rules express the knowledge to be exploited by the expert system. There exists other formulations of rules, which are not in everyday language, understandable only to computer scientists. Each rule style is adapted to an engine style. The whole problem of expert systems is to collect this knowledge, usually unconscious, from the experts. There are methods but almost all are usable only by computer scientists. The inference engine The inference engine is a computer program designed to produce a reasoning on rules. In order to produce a reasoning, it is based on logic. There are several kinds of logic: propositional logic, predicates of order 1 or more, epistemic logic, modal logic, temporal logic, fuzzy logic, etc. Except propositional logic, all are complex and can only be understood by mathematicians, logicians or computer scientists. Propositional logic is the basic human logic, that expressed in the syllogism. The expert system that uses that logic are also called zeroth-order expert system. With logic, the engine is able to generate new information from the knowledge contained in the rule base and data to be processed. The engine has two ways to run: batch or conversational. In batch, expert system has all the necessary data to process from the beginning. For the user, the program works as a classical program: he provides data and receives results immediately. Reasoning is invisible. The conversational becomes necessary when the developer knows he can't ask the user all the necessary data at the start, the problem being too complex. The software must "invent" the way to solve the problem, request the user missing data, gradually, approaching the goal as quickly as possible. The result gives the impression of a dialogue led by an expert. To guide a dialogue, the engine may have several levels of sophistication : "forward chaining", "backward chaining" and "mixed chaining". Forward chaining is the questioning of an expert who has no idea of the solution and investigates progressively (e.g. fault diagnosis). In backward chaining, the engine has an idea of the target (e.g. is it okay or not? Or: there is danger but what is the level?). It starts from the goal in hopes of finding the solution as soon as possible. In mixed chaining the engine has an idea of the goal but it is not enough: it deduces in forward chaining from previous user responses all that is possible before 224 Expert system asking the next question. So, quite often, he deduces the answer to the next question before asking it. A strong interest in using logic is that this kind of software is able to give to user clear explanation of what it is doing (the "Why?") and what it has deduced (the "How?" ). Better yet, thanks to logic the most sophisticated expert system are able to detect contradictions[31] into user information or in the knowledge and can explain them clearly, revealing at the same time the expert knowledge and his way of thinking. Advantages Expert systems offer many advantages for users when compared to traditional programs because they operate like a human brain. Quick availability and opportunity to program itself As the rule base is in everyday language (the engine is untouchable), expert system can be written much faster than a conventional program, by users or experts, bypassing professional developers and avoiding the need to explain the subject. Ability to exploit a considerable amount of knowledge The expert system uses a rule base, unlike conventional programs, which means that the volume of knowledge to program is not a major concern. Whether the rule base has 10 rules or 10 000, the engine operation is the same. Reliability The reliability of an expert system is the same as the reliability of a database, i.e. good, higher than that of a classical program. Scalability Evolving an expert system is to add, modify or delete rules. Since the rules are written in plain language, it is easy to identify those to be removed or modified. Pedagogy The engines that are run by a true logic are able to explain to the user in plain language why they ask a question and how they arrived at each deduction. In doing so, they show knowledge of the expert contained in the expert system. So, user can learn this knowledge in its context. Moreover, they can communicate their deductions step by step. So, the user has information about their problem even before the final answer of the expert system. Preservation and improvement of knowledge Valuable knowledge can disappear with the death, resignation or retirement of an expert. Recorded in an expert system, it becomes eternal. To develop an expert system is to interview an expert and make the system aware of their knowledge. In doing so, it reflects and enhances it. New areas neglected by conventional computing Automating a vast knowledge, the developer may meet a classic problem: "combinatorial explosion" commonly known as "information overload" that greatly complicates his work and results in a complex and time consuming program. The reasoning expert system does not encounter that problem since the engine automatically loads combinatorics between rules. This ability can address areas where combinatorics are enormous: highly interactive or conversational applications, fault diagnosis, decision support in complex systems, educational software, logic simulation of machines or systems, constantly changing software. 225 Expert system Disadvantages The expert system has a major flaw which explains its low success although the principle has existed for 70 years: knowledge collection and interpretation into rules, the knowledge engineering. Most developers have no method to perform this task. They work "manually" what opens to many possibilities for errors. Expert knowledge is not well understood, there is a lack of rules, rules are contradictory, some are poorly written and unusable. Worse, they most often use an engine unable to reasoning. Result: the expert system works badly and the project is abandoned.[32] Correct development methodology can mitigate these problems. There exists software to interview the expert step by step which automatically write the rules and simultaneously run the expert system before his eyes, performing a consistency of the rules control.[33] [34] [35] So expert and users can check the quality of the software before it is finished. Many expert systems are also penalized by the logic used. Most of the logics operate on "variables" facts, i.e. whose value changes several times during one reasoning, considered as a property belonging to a more powerful logic. Exactly like in classical computing, a way of programming where developers are in fact comfortable. This is the case of Mycin, Dendral, fuzzy logic, predicate logic (Prolog), symbolic logic, mathematical logic, etc.. Propositional logic uses only not variable facts.[36] It turns out that in human mind, the facts used must remain invariable as long as the brain reasons on them. This makes possible detection of contradictions and production of explanations, two ways of controlling consistency of the knowledge.[37] [38] That is why expert systems using variable facts, more understandable for IT developers so the most numerous, are less easy to develop, less clear to users, less reliable and why they don't produce explanation or contradiction detection. Application field Expert systems address areas where combinatorics is enormous: • • • • • • • highly interactive or conversational applications, IVR, voice server, chatterbot fault diagnosis, medical diagnosis decision support in complex systems, process control, interactive user guide educational and tutorial software logic simulation of machines or systems knowledge management constantly changing software. They can also be used in software engineering for rapid prototyping applications (RAD). Indeed, the expert system quickly developed in front of the expert shows him if the future application should be programmed. Indeed, any program contains expert knowledge and classic programming always begins with an expert interview. A program written in the form of expert system receives all the specific benefits of expert system, among others things it can be developed by anyone without computer training and without programming languages. But this solution has a defect: expert system runs slower than a traditional program because he consistently "thinks" when in fact a classic software just follows paths traced by the programmer. Examples of applications Expert systems are designed to facilitate tasks in the fields of accounting, medicine, process control, financial service, production, human resources, among others. Typically, the problem area is complex enough that a more simple traditional algorithm cannot provide a proper solution. The foundation of a successful expert system depends on a series of technical procedures and development that may be designed by technicians and related experts. As such, expert systems do not typically provide a definitive answer, but provide probabilistic recommendations. An example of the application of expert systems in the financial field is expert systems for mortgages. Loan departments are interested in expert systems for mortgages because of the growing cost of labour, which makes the 226 Expert system handling and acceptance of relatively small loans less profitable. They also see a possibility for standardized, efficient handling of mortgage loan by applying expert systems, appreciating that for the acceptance of mortgages there are hard and fast rules which do not always exist with other types of loans. Another common application in the financial area for expert systems are in trading recommendations in various marketplaces. These markets involve numerous variables and human emotions which may be impossible to deterministically characterize, thus expert systems based on the rules of thumb from experts and simulation data are used. Expert system of this type can range from ones providing regional retail recommendations, like Wishabi, to ones used to assist monetary decisions by financial institutions and governments. Another 1970s and 1980s application of expert systems, which we today would simply call AI, was in computer games. For example, the computer baseball games Earl Weaver Baseball and Tony La Russa Baseball each had highly detailed simulations of the game strategies of those two baseball managers. When a human played the game against the computer, the computer queried the Earl Weaver or Tony La Russa Expert System for a decision on what strategy to follow. Even those choices where some randomness was part of the natural system (such as when to throw a surprise pitch-out to try to trick a runner trying to steal a base) were decided based on probabilities supplied by Weaver or La Russa. Today we would simply say that "the game's AI provided the opposing manager's strategy". Knowledge engineering The building, maintaining and development of expert systems is known as knowledge engineering.[39] Knowledge engineering is a "discipline that involves integrating knowledge into computer systems in order to solve complex problems normally requiring a high level of human expertise".[40] There are generally three individuals having an interaction in an expert system. Primary among these is the end-user, the individual who uses the system for its problem solving assistance. In the construction and maintenance of the system there are two other roles: the problem domain expert who builds the system and supplies the knowledge base, and a knowledge engineer who assists the experts in determining the representation of their knowledge, enters this knowledge into an explanation module and who defines the inference technique required to solve the problem. Usually the knowledge engineer will represent the problem solving activity in the form of rules. When these rules are created from domain expertise, the knowledge base stores the rules of the expert system. References [1] Jackson, Peter (1998), Introduction To Expert Systems (3 ed.), Addison Wesley, p. 2, ISBN 978-0-201-87686-4 [2] Nwigbo Stella and Agbo Okechuku Chuks (http:/ / www. hrmars. com/ admin/ pics/ 261. pdf), School of Science Education, Expert system: a catalyst in educational development in Nigeria: "The ability of this system to explain the reasoning process through back-traces (...) provides an additional feature that conventional programming does not handle" [3] Regina Barzilay, Daryl McCullough, Owen Rambow, Jonathan DeCristofaro, Tanya Korelsky, Benoit Lavoie (http:/ / www. cogentex. com/ papers/ explanation-iwnlg98. pdf): "A new approach to expert system explanations" [4] Conventional programming (http:/ / www. pcmag. com/ encyclopedia_term/ 0,2542,t=conventional+ programming& i=40325,00. asp) [5] Cornelius T. Leondes (2002). Expert systems: the technology of knowledge management and decision making for the 21st century. pp. 1–22. ISBN 978-0-12-443880-4. [6] ACM 1998, I.2.1 [7] Russell & Norvig 2003, pp. 22−24 [8] Luger & Stubblefield 2004, pp. 227–331 [9] Nilsson 1998, chpt. 17.4 [10] McCorduck 2004, pp. 327–335, 434–435 [11] Crevier 1993, pp. 145–62, 197−203 [12] Nwigbo Stella and Agbo Okechuku Chuks (http:/ / www. hrmars. com/ admin/ pics/ 261. pdf), School of Science Education, Expert system: a catalyst in educational development in Nigeria: "Knowledge-based systems collect the small fragments of human know-how into a knowledge-base which is used to reason through a problem, using the knowledge that is appropriated" [13] Koch, C. G.; Isle, B. A.; Butler, A. W. (1988). "Intelligent user interface for expert systems applied to power plant maintenance and troubleshooting". IEEE Transactions on Energy Conversion 3: 71. doi:10.1109/60.4202. 227 Expert system [14] McTear, M. F. (2002). "Spoken dialogue technology: Enabling the conversational user interface". ACM Computing Surveys 34: 90. doi:10.1145/505282.505285. [15] Lowgren, J. (1992). "The Ignatius environment: Supporting the design and development of expert-system user interfaces". IEEE Expert 7 (4): 49–57. doi:10.1109/64.153464. [16] George F. Luger and William A. Stubblefield, Benjamin/Cummings Publishers, Rule Based Expert System Shell: example of code using the Prolog rule based expert system shell [17] A. MICHIELS (http:/ / promethee. philo. ulg. ac. be/ engdep1/ download/ prolog/ htm_docs/ prolog. htm), Université de Liège, Belgique: "PROLOG, the first declarative language [18] Carnegie Mellon University's AI Web Site (http:/ / www. prenhall. com/ divisions/ bp/ app/ turban/ dss/ html/ chap16. html): "Prolog was the most popular AI language in Japan and probably in Europe" [19] Ivana Berković, Biljana Radulović and Petar Hotomski (http:/ / www. proceedings2007. imcsit. org/ pliks/ 33. pdf), University of Novi Sad, 2007, Extensions of Deductive Concept in Logic Programing and Some Applications: "the defects of PROLOG-system: the expansion concerning Horn clauses, escaping negation treatment as definite failure" [20] "Software developed in Prolog has been criticized for having a high performance penalty compared to conventional programming languages" [21] Dr. Nikolai Bezroukov (http:/ / www. softpanorama. org/ Lang/ prolog. shtml), Softpanorama: "I think that most people exposed to Prolog remember strongly the initial disappointment. Language was/is so hyped but all you can see initially are pretty trivial examples that are solved by complex, obscure notation that lacks real expressive power: some of simple examples can be expressed no less concisely is many other languages" [22] Durkin, J. Expert Systems: Catalog of Applications. Intelligent Computer Systems, Inc., Akron, OH, 1993. [23] page 21. Giarratano & Riley, 3rd ed. [24] Flamant B. and Girard G (http:/ / cat. inist. fr/ ?aModele=afficheN& cpsidt=7001328)., GSI-TECSI, Intelligence Service: build your own expert system : "Intelligence Service is a development environment for expert systems that requires no experience of classic programming that offers to everyone the opportunity to develop its own expert system" [25] Bertrand Savatier (http:/ / www. tree-logic. com/ Articles/ 01, seul IS fait de l'IA, par un universitaire. jpg), Le Monde Informatique, November 23, 1987: "Expert systems accessible to all" [26] http:/ / www. atarimagazines. com/ creative/ v10n8/ 103_The_fifth_generation_Jap. php [27] Jean-Philippe de Lespinay (http:/ / www. tree-logic. com/ scienceetvie. htm), Science et Vie, "From total zero to Zero Plus [logic]", May 1991 [28] Death of Jean-Louis Laurière (http:/ / www. lip6. fr/ actualite/ information-fiche. php?RECORD_KEY(informations)=id& id(informations)=18) [29] Journey " In honor of Jean-Louis Laurière (http:/ / www. lip6. fr/ Laboratoire/ 2006-03-22/ 2006-03-22-Affiche. pdf)", Université Pierre et Marie Curie in Paris (March 22, 2006) [30] Mycin rule [31] Nabil Arman (http:/ / www. ccis2k. org/ iajit/ PDF/ vol. 4,no. 1/ 9-Nabil. pdf), Polytechnic University of Palestine, January 2007, Fault Detection in Dynamic Rule Bases Using Spanning Trees and Disjoin Sets: "" [32] Kenneth Laudon, Jane Laudon, Eric Fimbel, "Management Information Systems: Managing the Digital Firm", Business & Economics, 2010 edition, chapter 11-3.5: The implementation of a large number of expert systems requires the deployment of considerable development efforts, lengthy and expensive. Hiring and training a larger number of experts may be less expensive than building an expert system .(...) Some expert systems, particularly the largest, are so complex that over years, the costs of curative and adaptive maintenance become as high as the cost of development. [33] Systèmes Experts, April 15, 1990, Miao, authentic expert system generator of fault diagnosis: "MIAO can explain, again in [plain] language, all of his logical approach: why he is asking such a question and how it came to such a conclusion. And that because he is constantly reasoning and not because an IT developer programmed in advance all the possible explanations." [34] Olivier Rafal (http:/ / www. tree-logic. com/ Articles/ LMI 2001 cf Maieutique. jpg), Le Monde Informatique, Programming for all (T.Rex generator): "This software allows to develop a conversational application (...) leading to a self-learning" (i.e. thanks to the automatic explanations) [35] French Technology Survey (http:/ / www. tree-logic. com/ Articles/ Maïeutica par FTS (91). jpg), MAIEUTICA, An Expert System Generator which writes its own rules, July 1991: "checking the coherence of the knowledge", "it can detect contradictions", "it react appropriately to changes of minds" [36] RGU: School of Computing (http:/ / www. comp. rgu. ac. uk/ docs/ info/ index. php), More Complex Inference: "propositional logic, where variables are not allowed". [37] Ong K. and Lee R.M, Texas University-Austin, A logic model for maintaining consistency of bureaucratic policies, 1993: "Inconsistencies can be detected if any of the integrity constraints is proven false, and an explanation can be provided based on the proof tree. A more general inference mechanism is presented based on the theory of abduction for checking potential inconsistency of policies" [38] Carl G. Hempel and Paul Oppenheim (http:/ / people. cohums. ohio-state. edu/ tennant9/ hempel_oppenheim_PS1948. pdf), Philosophy of Science, Studies in the Logic of Explanation, 1948: "The sentences constituting the explanans must be true" [39] Kendal, S.L.; Creen, M. (2007), An introduction to knowledge engineering, London: Springer, ISBN 978-1-84628-475-5, OCLC 70987401 228 Expert system [40] Feigenbaum, Edward A.; McCorduck, Pamela (1983), The fifth generation (1st ed.), Reading, MA: Addison-Wesley, ISBN 978-0-201-11519-2, OCLC 9324691 Bibliography Textbooks • Darlington, Keith (2000). The Essence of Expert Systems. Pearson Education. ISBN 978-0-13-022774-4. • Ignizio, James (1991). Introduction to Expert Systems. McGraw-Hill Companies. ISBN 978-0-07-909785-9. • Giarratano, Joseph C. and Riley, Gary; Gary Riley (2005). Expert Systems, Principles and Programming. Course Technology Ptr. ISBN 978-0-534-38447-0. • Jackson, Peter (1998). Introduction to Expert Systems. Addison Wesley. ISBN 978-0-201-87686-4. • Walker, Adrian et al. (1990). Knowledge Systems and Prolog. Addison-Wesley. ISBN 978-0-201-52424-6. • Naylor, Chris. (1983). Build your own Expert System. Sigma Technical Press. ISBN 978-0-905-10441-6. History of AI • Crevier, Daniel (1993), AI: The Tumultuous Search for Artificial Intelligence, New York, NY: BasicBooks, ISBN 0-465-02997-3 • McCorduck, Pamela (2004), Machines Who Think (http://www.pamelamc.com/html/machines_who_think. html) (2nd ed.), Natick, MA: A. K. Peters, Ltd., ISBN 1-56881-205-1 • Luger, George; William A. Stubblefield (2004). Artificial Intelligence: Structures and Strategies for Complex Problem Solving (http://www.cs.unm.edu/~luger/ai-final/tocfull.html) (5th ed.). The Benjamin/Cummings Publishing Company, Inc.. ISBN 978-0-8053-4780-7. • Nilsson, Nils (1998-04-01). Artificial Intelligence: A New Synthesis. Morgan Kaufmann Publishers. ISBN 978-1-55860-467-4. • Russell, Stuart J.; Norvig, Peter (2003), Artificial Intelligence: A Modern Approach (http://aima.cs.berkeley. edu/) (2nd ed.), Upper Saddle River, New Jersey: Prentice Hall, ISBN 0-13-790395-2 • Winston, Patrick Henry (1984-04). Artificial Intelligence. Addison-Wesley. ISBN 978-0-201-08259-3. Other • "ACM Computing Classification System: Artificial intelligence" (http://www.acm.org/class/1998/I.2.html). ACM. 1998. Retrieved 2007-08-30. • Jean-Philippe de Lespinay, Admiroutes, December 2008: Reasoning AI (http://www.admiroutes.asso.fr/ larevue/2008/93/lespinay.htm) • Automates Intelligents, 2009: Conversational and Call centers (http://www.automatesintelligents.com/ interviews/2009/lespinay.html) • US patent 4763277 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US4763277), Ashford, Thomas J. et al., "Method for obtaining information in an expert system", published 1988-08-09, issued 1988-08-09 229 Expert system External links • Artificial Intelligence (http://www.dmoz.org/Computers/Artificial_Intelligence//) at the Open Directory Project • Expert System tutorial on Code Project (http://www.codeproject.com/KB/recipes/ArtificialAdvice-1.aspx) 230 231 The Root-Locus Design Method Root locus Root locus analysis is a graphical method for examining how the roots of a system change with variation of a certain system parameter, commonly the gain of a feedback system. This is a technique used in the field of control systems developed by Walter R. Evans. Uses In addition to determining the stability of the system, the root locus can be used to design for the damping ratio and natural frequency of a feedback system. Lines of constant damping ratio can be drawn radially from the origin and lines of constant natural frequency can be drawn as arcs whose center points coincide with the origin. By selecting a point along the root locus that coincides with a desired damping ratio and natural frequency a gain, K, can be calculated and implemented in the controller. More elaborate techniques of controller design using the root locus are available in most control textbooks: for instance, lag, lead, PI, PD and PID controllers can be designed approximately with this technique. The definition of the damping ratio and natural frequency presumes that the overall feedback system is well approximated by a second order system, that is, the system has a dominant pair of poles. This often doesn't happen and so it's good practice to simulate the final design to check if the project goals are satisfied. Example Suppose there is a plant (process) with a transfer function expression P(s), and a forward controller with both an adjustable gain K and output expression C(s) as shown in the block diagram below. RL = root locus Root locus 232 A unity feedback loop is constructed to complete this feedback system. For this system, the overall transfer function is given by Thus the closed-loop poles (roots of the characteristic equation) of the transfer function are the solutions to the equation 1+ KC(s)P(s) = 0. The principal feature of this equation is that roots may be found wherever KCP = -1. The variability of K, the gain for the controller, removes amplitude from the equation, meaning the complex valued evaluation of the polynomial in s C(s)P(s) needs to have net phase of 180 deg, wherever there is a closed loop pole. The geometrical construction adds angle contributions from the vectors extending from each of the poles of KC to a prospective closed loop root (pole) and subtracts the angle contributions from similar vectors extending from the zeros, requiring the sum be 180. The vector formulation arises from the fact that each polynomial term in the factored CP,(s-a) for example, represents the vector from a which is one of the roots, to s which is the prospective closed loop pole we are seeking. Thus the entire polynomial is the product of these terms, and according to vector mathematics the angles add (or subtract, for terms in the denominator) and lengths multiply (or divide). So to test a point for inclusion on the root locus, all you do is add the angles to all the open loop poles and zeros. Indeed a form of protractor, the "spirule" was once used to draw exact root loci. From the function T(s), we can also see that the zeros of the open loop system (CP) are also the zeros of the closed loop system. It is important to note that the root locus only gives the location of closed loop poles as the gain K is varied, given the open loop transfer function. The zeros of a system can not be moved. Using a few basic rules, the root locus method can plot the overall shape of the path (locus) traversed by the roots as the value of K varies. The plot of the root locus then gives an idea of the stability and dynamics of this feedback system for different values of k.[1] [2] Sketching root locus • Mark open-loop poles and zeros • Mark real axis portion to the left of an odd number of poles and zeros • Find asymptotes Let P be the number of poles and Z be the number of zeros: number of asymptotes The asymptotes intersect the real axis at where and depart at angle is the sum of all the locations of the poles, and given by: is the sum of all the locations of the explicit zeros. Root locus • Phase condition on test point to find angle of deparature • Compute breakaway/break-in points The breakaway points are located at the roots of the following equation: Once you solve for z, the real roots give you the breakaway/reentry points. Complex roots correspond to a lack of breakaway/reentry. The break-away (break-in) points are obtained by solving a polynomial equation z-plane versus s-plane The root locus can also be computed in the z-plane, the discrete counterpart of the s-plane. An equation (z = esT) maps continuous s-plane poles (not zeros) into the z-domain, where T is the sampling period. The stable, left half s-plane maps into the interior of the unit circle of the z-plane, with the s-plane origin equating to |z| = 1 (because e0 = 1). A diagonal line of constant damping in the s-plane maps around a spiral from (1,0) in the z plane as it curves in toward the origin. Note also that the Nyquist aliasing criteria is expressed graphically in the z-plane by the x-axis, where (wnT = π). The line of constant damping just described spirals in indefinitely but in sampled data systems, frequency content is aliased down to lower frequencies by integral multiples of the Nyquist frequency. That is, the sampled response appears as a lower frequency and better damped as well since the root in the z-plane maps equally well to the first loop of a different, better damped spiral curve of constant damping. Many other interesting and relevant mapping properties can be described, not least that z-plane controllers, having the property that they may be directly implemented from the z-plane transfer function (zero/pole ratio of polynomialls), can be imagined graphically on a z-plane plot of the open loop transfer function, and immediately analyzed utilizing root locus. Since root locus is a graphical angle technique, root locus rules work the same in the z and s planes. The idea of a root locus can be applied to many systems where a single parameter K is varied. For example, it is useful to sweep any system parameter for which the exact value is uncertain, in order to determine its behavior. References [1] Evans, W. R. (January 1948), "Graphical Analysis of Control Systems", Trans. AIEE 67 (1): 547–551, doi:10.1109/T-AIEE.1948.5059708, ISSN 0096-3860 [2] Evans, W. R. (January 1950), "Control Systems Synthesis by Root Locus Method", Trans. AIEE 69 (1): 66–69, doi:10.1109/T-AIEE.1950.5060121, ISSN 0096-3860 • Ash, R. H.; Ash, G. H. (October 1968), "Numerical Computation of Root Loci Using the Newton-Raphson Technique", IEEE Trans. Automatic Control 13 (5), doi:10.1109/TAC.1968.1098980 • Williamson, S. E. (May 1968), "Design Data to assist the Plotting of Root Loci (Part I)", Control Magazine 12 (119): 404–407 • Williamson, S. E. (June 1968), "Design Data to assist the Plotting of Root Loci (Part II)", Control Magazine 12 (120): 556–559 • Williamson, S. E. (July 1968), "Design Data to assist the Plotting of Root Loci (Part III)", Control Magazine 12 (121): 645–647 • Williamson, S. E. (May 15, 1969), "Computer Program to Obtain the Time Response of Sampled Data Systems", IEE Electronics Letters 5 (10): 209–210, doi:10.1049/el:19690159 • Williamson, S. E. (July 1969), "Accurate Root Locus Plotting Including the Effects of Pure Time Delay" (http:// ieeexplore.ieee.org/Xplore/login.jsp?url=http://ieeexplore.ieee.org/iel5/5247218/5248781/05249527. pdf?tp=&arnumber=5249527&punumber=5247218&authDecision=-203), Proc. IEE 116 (7): 1269–1271, doi:10.1049/piee.1969.0235 233 Root locus External links • Wikibooks: Control Systems/Root Locus (http://en.wikibooks.org/wiki/Control_Systems/Root_Locus) • Carnegie Mellon / University of Michigan Tutorial (http://www.engin.umich.edu/group/ctm/rlocus/rlocus. html) • Excellent examples. Start with example 5 and proceed backwards through 4 to 1. Also visit the main page (http:// www.swarthmore.edu/NatSci/echeeve1/Ref/LPSA/Root_Locus/RLocusExamples.html#ex5) • The root-locus method: Drawing by hand techniques (http://www.atp.ruhr-uni-bochum.de/rt1/syscontrol/ node46.html) • "RootLocs": A free multi-featured root-locus plotter for Mac and Windows platforms (http://www.coppice. myzen.co.uk) • "Root Locus": A free root-locus plotter/analyzer for Windows (http://web.archive.org/web/20091027092528/ http://geocities.com/aseldawy/root_locus.html) • Root Locus at ControlTheoryPro.com (http://wikis.controltheorypro.com/index.php?title=Root_Locus) • Root Locus Analysis of Control Systems (http://www.roymech.co.uk/Related/Control/root_locus.html) • Wechsler, E. R. (January-March 1983), Root Locus Algorithms for Programmable Pocket Calculators (http:// ipnpr.jpl.nasa.gov/progress_report/42-73/73F.PDF), NASA, pp. 60–64, TDA Progress Report 42-73 Compensation (engineering) In engineering, compensation is planning for side effects or other unintended issues in a design. The design of an invention can itself also be to compensate for some other existing issue or exception. One example is in a voltage-controlled crystal oscillator (VCXO), which is normally affected not only by voltage, but to a lesser extent by temperature. A temperature-compensated version (a TCVCXO) is designed so that heat buildup within the enclosure of a transmitter or other such device will not alter the piezoelectric effect, thereby causing frequency drift. Another example is motion compensation on digital cameras and video cameras, which keep a picture steady and not blurry. Other examples in electrical engineering include: • A constant voltage device compensates for low or high voltage in an electrical circuit, keeping its output the same within a given range of input. • Error correction compensates for data corruption. • Gray coding compensates for errors on rotary encoders and linear encoders. • Debouncing compensates for jitter in an electrical switch (see Contact Bounce section in Switch article). • A resistor or inductor compensates for negative resistance in gas-discharge lighting. • Frequency compensation is used in feedback control systems to avert oscillations. There are also examples in civil engineering: • Expansion joints in sidewalks, buildings, and bridges compensate for expansion and contraction. • Various devices between a structure and its Foundation compensate for earthquake movements, either actively or passively. 234 Compensation (engineering) Railways • On railways, steep gradients on sharp curves have an equivalent gradient slightly steeper than the stated value. This is because of extra friction of the wheel on the rails, and because wagons stretch out on the chord rather than on the arc, and may thus cause trains to stall. To compensate, the gradient of ruling grades is slightly reduced from say 1 in 50 to 1 in 53. Frequency compensation In electrical engineering, frequency compensation is a technique used in amplifiers, and especially in amplifiers employing negative feedback. It usually has two primary goals: To avoid the unintentional creation of positive feedback, which will cause the amplifier to oscillate, and to control overshoot and ringing in the amplifier's step response. Explanation Most amplifiers use negative feedback to trade gain for other desirable properties, such as decreased distortion or improved noise reduction. Ideally, the phase characteristic of an amplifier's frequency response would be linear; however, device limitations make this goal physically unattainable. More particularly, capacitances within the amplifier's gain stages cause the Figure 1: Step response of a two-pole amplifier for various degrees of compensation. output signal to lag behind the input Parameter ζ is set by a compensation capacitor: smaller ζ results in faster response, but more ringing and overshoot. signal by 90° for each pole they [1] create. If the sum of these phase lags reaches 360°, the output signal will be in phase with the input signal. Feeding back any portion of this output signal to the input when the gain of the amplifier is sufficient will cause the amplifier to oscillate. This is because the feedback signal will reinforce the input signal. That is, the feedback is then positive rather than negative. Frequency compensation is implemented to avoid this result. Another goal of frequency compensation is to control the step response of an amplifier circuit as shown in Figure 1. For example, if a step in voltage is input to a voltage amplifier, ideally a step in output voltage would occur. However, the output is not ideal because of the frequency response of the amplifier, and ringing occurs. Several figures of merit to describe the adequacy of step response are in common use. One is the rise time of the output, which ideally would be short. A second is the time for the output to lock into its final value, which again should be short. The success in reaching this lock-in at final value is described by overshoot (how far the response exceeds final value) and settling time (how long the output swings back and forth about its final value). These various measures of the step response usually conflict with one another, requiring optimization methods. Frequency compensation is implemented to optimize step response, one method being pole splitting. 235 Frequency compensation Use in operational amplifiers Because operational amplifiers are so ubiquitous and are designed to be used with feedback, the following discussion will be limited to frequency compensation of these devices. It should be expected that the outputs of even the simplest operational amplifiers will have at least two poles. An unfortunate consequence of this is that at some critical frequency, the phase of the amplifier's output = -180° compared to the phase of its input signal. The amplifier will oscillate if it has a gain of one or more at this critical frequency. This is because (a) the feedback is implemented through the use of an inverting input that adds an additional -180° to the output phase making the total phase shift -360° and (b) the gain is sufficient to induce oscillation. A more precise statement of this is the following: An operational amplifier will oscillate at the frequency at which its open loop gain equals its closed loop gain if, at that frequency, 1. The open loop gain of the amplifier is ≥ 1 and 2. The difference between the phase of the open loop signal and phase response of the network creating the closed loop output = -180°. Mathematically, ΦOL – ΦCLnet = -180° Practice Frequency compensation is implemented by modifying the gain and phase characteristics of the amplifier's open loop output or of its feedback network, or both, in such a way as to avoid the conditions leading to oscillation. This is usually done by the internal or external use of resistance-capacitance networks. Dominant-pole compensation The method most commonly used is called dominant-pole compensation, which is a form of lag compensation. A pole placed at an appropriate low frequency in the open-loop response reduces the gain of the amplifier to one (0 dB) for a frequency at or just below the location of the next highest frequency pole. The lowest frequency pole is called the dominant pole because it dominates the effect of all of the higher frequency poles. The result is that the difference between the open loop output phase and the phase response of a feedback network having no reactive elements never falls below −180° while the amplifier has a gain of one or more, ensuring stability. Dominant-pole compensation can be implemented for general purpose operational amplifiers by adding an integrating capacitance to the stage that provides the bulk of the amplifier's gain. This capacitor creates a pole that is set at a frequency low enough to reduce the gain to one (0 dB) at or just below the frequency where the pole next highest in frequency is located. The result is a phase margin of ≈ 45°, depending on the proximity of still higher poles.[2] This margin is sufficient to prevent oscillation in the most commonly used feedback configurations. In addition, dominant-pole compensation allows control of overshoot and ringing in the amplifier step response, which can be a more demanding requirement than the simple need for stability. Though simple and effective, this kind of conservative dominant pole compensation has two drawbacks: 1. It reduces the bandwidth of the amplifier, thereby reducing available open loop gain at higher frequencies. This, in turn, reduces the amount of feedback available for distortion correction, etc. at higher frequencies. 2. It reduces the amplifier's slew rate. This reduction results from the time it takes the finite current driving the compensated stage to charge the compensating capacitor. The result is the inability of the amplifier to reproduce high amplitude, rapidly changing signals accurately. A strong example of this can be found at http://www.ece.utah.edu/~harrison/ece3110/Compensation.pdf Often, the implementation of dominant-pole compensation results in the phenomenon of Pole splitting. This results in the lowest frequency pole of the uncompensated amplifier "moving" to an even lower frequency to become the 236 Frequency compensation dominant pole, and the higher-frequency pole of the uncompensated amplifier "moving" to a higher frequency. Other methods Some other compensation methods are: lead compensation, lead–lag compensation and feed-forward compensation. Lead compensation. Whereas dominant pole compensation places or moves poles in the open loop response, lead compensation places a zero[3] in the open loop response to cancel one of the existing poles. Lead–lag compensation places both a zero and a pole in the open loop response, with the pole usually being at an open loop gain of less than one. Feed-forward compensation uses a capacitor to bypass a stage in the amplifier at high frequencies, thereby eliminating the pole that stage creates. The purpose of these three methods is to allow greater open loop bandwidth while still maintaining amplifier closed loop stability. They are often used to compensate high gain, wide bandwidth amplifiers. Footnotes [1] In this context, a pole is the point in a frequency response curve where the amplitude decreases by 3db due to an integrating resistance and capacitive reactance. Ultimately, each pole will result in a phase lag of 90°, i.e., the output signal will lag behind the input signal by 90° at this point. For the mathematical concept of a pole, see, Pole (complex analysis). [2] The dominant pole produces a phase shift approximating -90° from approx. 10 times the pole frequency to a frequency a factor of ten below the next higher pole position. The next higher pole, in turn, adds another -45° for a frequency at its location for a total of -135° (neglecting the still higher poles). [3] In this context, a zero is the point in a frequency response curve where the amplitude increases by 3db due to a differentiating resistance and capacitive reactance. Ultimately, each zero will result in a phase lead of 90°, i.e., the phase of the output signal will be 90° ahead of the phase of the input signal at this point. For the mathematical concept of a zero, see, Zero (complex analysis). Error detection and correction In information theory and coding theory with applications in computer science and telecommunication, error detection and correction or error control are techniques that enable reliable delivery of digital data over unreliable communication channels. Many communication channels are subject to channel noise, and thus errors may be introduced during transmission from the source to a receiver. Error detection techniques allow detecting such errors, while error correction enables reconstruction of the original data. Definitions The general definitions of the terms are as follows: • Error detection is the detection of errors caused by noise or other impairments during transmission from the transmitter to the receiver.[1] • Error correction is the detection of errors and reconstruction of the original, error-free data. Introduction The general idea for achieving error detection and correction is to add some redundancy (i.e., some extra data) to a message, which receivers can use to check consistency of the delivered message, and to recover data determined to be erroneous. Error-detection and correction schemes can be either systematic or non-systematic: In a systematic scheme, the transmitter sends the original data, and attaches a fixed number of check bits (or parity data), which are derived from the data bits by some deterministic algorithm. If only error detection is required, a receiver can simply apply the same algorithm to the received data bits and compare its output with the received check bits; if the values 237 Error detection and correction do not match, an error has occurred at some point during the transmission. In a system that uses a non-systematic code, the original message is transformed into an encoded message that has at least as many bits as the original message. Good error control performance requires the scheme to be selected based on the characteristics of the communication channel. Common channel models include memory-less models where errors occur randomly and with a certain probability, and dynamic models where errors occur primarily in bursts. Consequently, error-detecting and correcting codes can be generally distinguished between random-error-detecting/correcting and burst-error-detecting/correcting. Some codes can also be suitable for a mixture of random errors and burst errors. If the channel capacity cannot be determined, or is highly varying, an error-detection scheme may be combined with a system for retransmissions of erroneous data. This is known as automatic repeat request (ARQ), and is most notably used in the Internet. An alternate approach for error control is hybrid automatic repeat request (HARQ), which is a combination of ARQ and error-correction coding. Implementation Error correction may generally be realized in two different ways: • Automatic repeat request (ARQ) (sometimes also referred to as backward error correction): This is an error control technique whereby an error detection scheme is combined with requests for retransmission of erroneous data. Every block of data received is checked using the error detection code used, and if the check fails, retransmission of the data is requested – this may be done repeatedly, until the data can be verified. • Forward error correction (FEC): The sender encodes the data using an error-correcting code (ECC) prior to transmission. The additional information (redundancy) added by the code is used by the receiver to recover the original data. In general, the reconstructed data is what is deemed the "most likely" original data. ARQ and FEC may be combined, such that minor errors are corrected without retransmission, and major errors are corrected via a request for retransmission: this is called hybrid automatic repeat-request (HARQ). Error detection schemes Error detection is most commonly realized using a suitable hash function (or checksum algorithm). A hash function adds a fixed-length tag to a message, which enables receivers to verify the delivered message by recomputing the tag and comparing it with the one provided. There exists a vast variety of different hash function designs. However, some are of particularly widespread use because of either their simplicity or their suitability for detecting certain kinds of errors (e.g., the cyclic redundancy check's performance in detecting burst errors). Random-error-correcting codes based on minimum distance coding can provide a suitable alternative to hash functions when a strict guarantee on the minimum number of errors to be detected is desired. Repetition codes, described below, are special cases of error-correcting codes: although rather inefficient, they find applications for both error correction and detection due to their simplicity. Repetition codes A repetition code is a coding scheme that repeats the bits across a channel to achieve error-free communication. Given a stream of data to be transmitted, the data is divided into blocks of bits. Each block is transmitted some predetermined number of times. For example, to send the bit pattern "1011", the four-bit block can be repeated three times, thus producing "1011 1011 1011". However, if this twelve-bit pattern was received as "1010 1011 1011" – where the first block is unlike the other two – it can be determined that an error has occurred. Repetition codes are very inefficient, and can be susceptible to problems if the error occurs in exactly the same place for each group (e.g., "1010 1010 1010" in the previous example would be detected as correct). The advantage of 238 Error detection and correction repetition codes is that they are extremely simple, and are in fact used in some transmissions of numbers stations. Parity bits A parity bit is a bit that is added to a group of source bits to ensure that the number of set bits (i.e., bits with value 1) in the outcome is even or odd. It is a very simple scheme that can be used to detect single or any other odd number (i.e., three, five, etc.) of errors in the output. An even number of flipped bits will make the parity bit appear correct even though the data is erroneous. Extensions and variations on the parity bit mechanism are horizontal redundancy checks, vertical redundancy checks, and "double," "dual," or "diagonal" parity (used in RAID-DP). Checksums A checksum of a message is a modular arithmetic sum of message code words of a fixed word length (e.g., byte values). The sum may be negated by means of a ones'-complement operation prior to transmission to detect errors resulting in all-zero messages. Checksum schemes include parity bits, check digits, and longitudinal redundancy checks. Some checksum schemes, such as the Luhn algorithm and the Verhoeff algorithm, are specifically designed to detect errors commonly introduced by humans in writing down or remembering identification numbers. Cyclic redundancy checks (CRCs) A cyclic redundancy check (CRC) is a single-burst-error-detecting cyclic code and non-secure hash function designed to detect accidental changes to digital data in computer networks. It is characterized by specification of a so-called generator polynomial, which is used as the divisor in a polynomial long division over a finite field, taking the input data as the dividend, and where the remainder becomes the result. Cyclic codes have favorable properties in that they are well suited for detecting burst errors. CRCs are particularly easy to implement in hardware, and are therefore commonly used in digital networks and storage devices such as hard disk drives. Even parity is a special case of a cyclic redundancy check, where the single-bit CRC is generated by the divisor x + 1. Cryptographic hash functions A cryptographic hash function can provide strong assurances about data integrity, provided that changes of the data are only accidental (i.e., due to transmission errors). Any modification to the data will likely be detected through a mismatching hash value. Furthermore, given some hash value, it is infeasible to find some input data (other than the one given) that will yield the same hash value. Message authentication codes, also called keyed cryptographic hash functions, provide additional protection against intentional modification by an attacker. Error-correcting codes Any error-correcting code can be used for error detection. A code with minimum Hamming distance, d, can detect up to d − 1 errors in a code word. Using minimum-distance-based error-correcting codes for error detection can be suitable if a strict limit on the minimum number of errors to be detected is desired. Codes with minimum Hamming distance d = 2 are degenerate cases of error-correcting codes, and can be used to detect single errors. The parity bit is an example of a single-error-detecting code. The Berger code is an early example of a unidirectional error(-correcting) code that can detect any number of errors on an asymmetric channel, provided that only transitions of cleared bits to set bits or set bits to cleared bits can occur. 239 Error detection and correction Error correction Automatic repeat request Automatic Repeat reQuest (ARQ) is an error control method for data transmission that makes use of error-detection codes, acknowledgment and/or negative acknowledgment messages, and timeouts to achieve reliable data transmission. An acknowledgment is a message sent by the receiver to indicate that it has correctly received a data frame. Usually, when the transmitter does not receive the acknowledgment before the timeout occurs (i.e., within a reasonable amount of time after sending the data frame), it retransmits the frame until it is either correctly received or the error persists beyond a predetermined number of retransmissions. Three types of ARQ protocols are Stop-and-wait ARQ, Go-Back-N ARQ, and Selective Repeat ARQ. ARQ is appropriate if the communication channel has varying or unknown capacity, such as is the case on the Internet. However, ARQ requires the availability of a back channel, results in possibly increased latency due to retransmissions, and requires the maintenance of buffers and timers for retransmissions, which in the case of network congestion can put a strain on the server and overall network capacity.[2] Error-correcting code An error-correcting code (ECC) or forward error correction (FEC) code is a system of adding redundant data, or parity data, to a message, such that it can be recovered by a receiver even when a number of errors (up to the capability of the code being used) were introduced, either during the process of transmission, or on storage. Since the receiver does not have to ask the sender for retransmission of the data, a back-channel is not required in forward error correction, and it is therefore suitable for simplex communication such as broadcasting. Error-correcting codes are frequently used in lower-layer communication, as well as for reliable storage in media such as CDs, DVDs, hard disks, and RAM. Error-correcting codes are usually distinguished between convolutional codes and block codes: • Convolutional codes are processed on a bit-by-bit basis. They are particularly suitable for implementation in hardware, and the Viterbi decoder allows optimal decoding. • Block codes are processed on a block-by-block basis. Early examples of block codes are repetition codes, Hamming codes and multidimensional parity-check codes. They were followed by a number of efficient codes, Reed-Solomon codes being the most notable due to their current widespread use. Turbo codes and low-density parity-check codes (LDPC) are relatively new constructions that can provide almost optimal efficiency. Shannon's theorem is an important theorem in forward error correction, and describes the maximum information rate at which reliable communication is possible over a channel that has a certain error probability or signal-to-noise ratio (SNR). This strict upper limit is expressed in terms of the channel capacity. More specifically, the theorem says that there exist codes such that with increasing encoding length the probability of error on a discrete memoryless channel can be made arbitrarily small, provided that the code rate is smaller than the channel capacity. The code rate is defined as the fraction k/n of k source symbols and n encoded symbols. The actual maximum code rate allowed depends on the error-correcting code used, and may be lower. This is because Shannon's proof was only of existential nature, and did not show how to construct codes which are both optimal and have efficient encoding and decoding algorithms. 240 Error detection and correction Hybrid schemes Hybrid ARQ is a combination of ARQ and forward error correction. There are two basic approaches:[2] • Messages are always transmitted with FEC parity data (and error-detection redundancy). A receiver decodes a message using the parity information, and requests retransmission using ARQ only if the parity data was not sufficient for successful decoding (identified through a failed integrity check). • Messages are transmitted without parity data (only with error-detection information). If a receiver detects an error, it requests FEC information from the transmitter using ARQ, and uses it to reconstruct the original message. The latter approach is particularly attractive on an erasure channel when using a rateless erasure code. Applications Applications that require low latency (such as telephone conversations) cannot use Automatic Repeat reQuest (ARQ); they must use Forward Error Correction (FEC). By the time an ARQ system discovers an error and re-transmits it, the re-sent data will arrive too late to be any good. Applications where the transmitter immediately forgets the information as soon as it is sent (such as most television cameras) cannot use ARQ; they must use FEC because when an error occurs, the original data is no longer available. (This is also why FEC is used in data storage systems such as RAID and distributed data store). Applications that use ARQ must have a return channel. Applications that have no return channel cannot use ARQ. Applications that require extremely low error rates (such as digital money transfers) must use ARQ. Internet In a typical TCP/IP stack, error control is performed at multiple levels: • Each Ethernet frame carries a CRC-32 checksum. Frames received with incorrect checksums are discarded by the receiver hardware. • The IPv4 header contains a checksum protecting the contents of the header. Packets with mismatching checksums are dropped within the network or at the receiver. • The checksum was omitted from the IPv6 header in order to minimize processing costs in network routing and because current link layer technology is assumed to provide sufficient error detection (see also RFC 3819). • UDP has an optional checksum covering the payload and addressing information from the UDP and IP headers. Packets with incorrect checksums are discarded by the operating system network stack. The checksum is optional under IPv4, only, because the Data-Link layer checksum may already provide the desired level of error protection. • TCP provides a checksum for protecting the payload and addressing information from the TCP and IP headers. Packets with incorrect checksums are discarded within the network stack, and eventually get retransmitted using ARQ, either explicitly (such as through triple-ack) or implicitly due to a timeout. Deep-space telecommunications Development of error-correction codes was tightly coupled with the history of deep-space missions due to the extreme dilution of signal power over interplanetary distances, and the limited power availability aboard space probes. Whereas early missions sent their data uncoded, starting from 1968 digital error correction was implemented in the form of (sub-optimally decoded) convolutional codes and Reed-Muller codes.[3] The Reed-Muller code was well suited to the noise the spacecraft was subject to (approximately matching a bell curve), and was implemented at the Mariner spacecraft for missions between 1969 and 1977. The Voyager 1 and Voyager 2 missions, which started in 1977, were designed to deliver color imaging amongst scientific information of Jupiter and Saturn.[4] This resulted in increased coding requirements, and thus the spacecraft 241 Error detection and correction were supported by (optimally Viterbi-decoded) convolutional codes that could be concatenated with an outer Golay (24,12,8) code. The Voyager 2 probe additionally supported an implementation of a Reed-Solomon code: the concatenated Reed-Solomon-Viterbi (RSV) code allowed for very powerful error correction, and enabled the spacecraft's extended journey to Uranus and Neptune. The CCSDS currently recommends usage of error correction codes with performance similar to the Voyager 2 RSV code as a minimum. Concatenated codes are increasingly falling out of favor with space missions, and are replaced by more powerful codes such as Turbo codes or LDPC codes. The different kinds of deep space and orbital missions that are conducted suggest that trying to find a "one size fits all" error correction system will be an ongoing problem for some time to come. For missions close to earth the nature of the channel noise is different from that of a spacecraft on an interplanetary mission experiences. Additionally, as a spacecraft increases its distance from earth, the problem of correcting for noise gets larger. Satellite broadcasting (DVB) The demand for satellite transponder bandwidth continues to grow, fueled by the desire to deliver television (including new channels and High Definition TV) and IP data. Transponder availability and bandwidth constraints have limited this growth, because transponder capacity is determined by the selected modulation scheme and Forward error correction (FEC) rate. Overview • QPSK coupled with traditional Reed Solomon and Viterbi codes have been used for nearly 20 years for the delivery of digital satellite TV. • Higher order modulation schemes such as 8PSK, 16QAM and 32QAM have enabled the satellite industry to increase transponder efficiency by several orders of magnitude. • This increase in the information rate in a transponder comes at the expense of an increase in the carrier power to meet the threshold requirement for existing antennas. • Tests conducted using the latest chipsets demonstrate that the performance achieved by using Turbo Codes may be even lower than the 0.8 dB figure assumed in early designs. Data storage Error detection and correction codes are often used to improve the reliability of data storage media. A "parity track" was present on the first magnetic tape data storage in 1951. The "Optimal Rectangular Code" used in group code recording tapes not only detects but also corrects single-bit errors. Some file formats, particularly archive formats, include a checksum (most often CRC32) to detect corruption and truncation and can employ redundancy and/or parity files to recover portions of corrupted data. Reed Solomon codes are used in compact discs to correct errors caused by scratches. Modern hard drives use CRC codes to detect and Reed-Solomon codes to correct minor errors in sector reads, and to recover data from sectors that have "gone bad" and store that data in the spare sectors.[5] RAID systems use a variety of error correction techniques, to correct errors when a hard drive completely fails. 242 Error detection and correction Error-correcting memory DRAM memory may provide increased protection against soft errors by relying on error correcting codes. Such error-correcting memory, known as ECC or EDAC-protected memory, is particularly desirable for high fault-tolerant applications, such as servers, as well as deep-space applications due to increased radiation. Error-correcting memory controllers traditionally use Hamming codes, although some use triple modular redundancy. Interleaving allows distributing the effect of a single cosmic ray potentially upsetting multiple physically neighboring bits across multiple words by associating neighboring bits to different words. As long as a single event upset (SEU) does not exceed the error threshold (e.g., a single error) in any particular word between accesses, it can be corrected (e.g., by a single-bit error system may be maintained.[6] Notes [1] See also Encyclopedia of Microcomputers: Volume 6 (http:/ / books. google. com. br/ books?id=ajpoqYqFrkQC& lpg=PA343& dq=Error detection and correction is& as_brr=3& client=firefox-a& pg=PA343#v=snippet& q="error detection is"& f=false), p. 343. [2] A. J. McAuley, Reliable Broadband Communication Using a Burst Erasure Correcting Code, ACM SIGCOMM, 1990. [3] K. Andrews et al., The Development of Turbo and LDPC Codes for Deep-Space Applications, Proceedings of the IEEE, Vol. 95, No. 11, Nov. 2007. [4] W. Huffman, V. Pless, Fundamentals of error-correcting codes, Cambridge University Press, ISBN 9780521782807, 2003. [5] My Hard Drive Died | Scott A. Moulton (http:/ / www. myharddrivedied. com/ presentations_whitepaper. html) [6] "Using StrongArm SA-1110 in the On-Board Computer of Nanosatellite" (http:/ / www. apmcsta. org/ File/ doc/ Conferences/ 6th meeting/ Chen Zhenyu. doc). Tsinghua Space Center, Tsinghua University, Beijing. . Retrieved 2009-02-16. References • Shu Lin, Daniel J. Costello, Jr. (1983). Error Control Coding: Fundamentals and Applications. Prentice Hall. ISBN 0-13-283796-X. External links • The on-line textbook: Information Theory, Inference, and Learning Algorithms (http://www.inference.phy. cam.ac.uk/mackay/itila/), by David MacKay, contains chapters on elementary error-correcting codes; on the theoretical limits of error-correction; and on the latest state-of-the-art error-correcting codes, including low-density parity-check codes, turbo codes, and fountain codes. • Compute parameters of linear codes (http://ipsit.bu.edu/comp.html) - an on-line interface for generating and computing parameters (e.g. minimum distance, covering radius) of linear error-correcting codes. • ECC Page (http://www.eccpage.com/) 243 Linear encoder 244 Linear encoder A linear encoder is a sensor, transducer or readhead paired with a scale that encodes position. The sensor reads the scale in order to convert the encoded position into an analog or digital signal, which can then be decoded into position by a digital readout (DRO) or motion controller. The encoder can be either incremental or absolute. Motion can be determined by change in position over time. Linear encoder technologies include optical, magnetic, inductive, capacitive and eddy current. Optical technologies include shadow, self imaging and interferometric. Linear encoders are used in metrology instruments, motion systems and high precision machining tools ranging from digital calipers and coordinate measuring machines to stages, CNC Mills, manufacturing gantry tables and semiconductor steppers. Typical linear optical encoders. Physical principle Linear encoders are transducers that exploit many different physical properties in order to encode position: Visualization of magnetic structures of a linear encoder (Recorded with MagView). Scale / reference based Optical Optical linear encoders [1], [2] dominate the high resolution market and may employ shuttering / Moiré, diffraction or holographic principles. Typical incremental scale periods vary from hundreds down to sub-micrometre and following interpolation can provide resolutions as fine as a nanometre. Light sources used include infrared LEDs, visible LEDs, miniature light-bulbs and laser diodes. Magnetic Magnetic linear encoders [3] employ either active (magnetized) or passive (variable reluctance) scales and position may be sensed using sense-coils, Hall Effect or magnetoresistive readheads. With coarser scale periods than optical encoders (typically a few hundred micrometers to several millimeters) resolutions in the order of a micrometer are the norm. Linear encoder Capacitive Capacitive linear encoders work by sensing the capacitance between a reader and scale. Typical applications are digital calipers. One of the disadvantage is the sensitivity to uneven dirt, which can locally change the relative permittivity. Inductive Inductive technology is robust to contaminants, allowing calipers and other measurement tools that are coolant-proof [4]. A popular application of the inductive measuring principle is the Inductosyn [5]. In effect it is a resolver unwound into a linear system. The Spherosyn encoder [6] is based on the principle of electromagnetic induction and uses coils to sense nickel-chrome ball-bearings mounted within a tube. Eddy current US Patent 3820110, "Eddy current type digital encoder and position reference", gives an example of this type of encoder, which uses a scale coded with high and low permeability, non-magnetic materials, which is detected and decoded by monitoring changes in inductance of an AC circuit that includes an inductive coil sensor. Maxon [7] makes an example (rotary encoder) product (the MILE encoder). Without scales Optical image sensor The sensors are based on an image correlation method. The Sensor takes subsequent pictures from the surface being measured and compares the images for displacement [8]. Resolutions down to 1 nm are possible. [9] Applications There are two main areas of application for linear encoders: Measurement Measurement application include coordinate-measuring machines (CMM), laser scanners, calipers, gear measurement [10], tension testers [11] and Digital read outs (DROs). Motion systems Servo controlled motion systems employ linear encoder so as to provide accurate, high-speed movement. Typical applications include robotics, machine tools, pick-and-place PCB assembly equipment; semiconductors handling and test equipment, wire bonders, printers and digital presses.[12] 245 Linear encoder Output Signal Formats Incremental signals Linear encoders can have analog or digital outputs. Analog The industry standard, analog output for linear encoders is sine and cosine quadrature signals. These are usually transmitted differentially so as to improve noise immunity. An early industry standard was 12 µA peak-peak current signals but more recently this has been replaced with 1V peak to peak voltage signals. Compared to digital transmission, the analog signals' lower bandwidth helps to minimise emc emissions. Quadrature sine/cosine signals can be monitored easily by using an oscilloscope in XY mode to display a circular Lissajous Figure. Highest accuracy signals are obtained if the Lissajous Figure is circular (no gain or phase error) and perfectly centred. Modern encoder systems employ circuitry to trim these error mechanisms automatically. The overall accuracy of the linear encoder is a combination of the scale accuracy and errors introduced by the readhead. Scale contributions to the error budget include linearity and slope (scaling factor error). Readhead error mechanisms are usually described as cyclic error or sub-divisional error (SDE) as they repeat every scale period. The largest contributor to readhead inaccuracy is signal offset, followed by signal imbalance (ellipticity) and phase error (the quadrature The sine and cosine outputs. signals not being exactly 90° apart). Overall signal size does not affect encoder accuracy, however, signal-to-noise and jitter performance may degrade with smaller signals. Automatic signal compensation mechanisms can include automatic offset compensation (AOC), automatic balance compensation (ABC) and automatic gain control (AGC). Phase is more difficult to compensate dynamically and is usually applied as one time compensation during installation or calibration. Other forms of inaccuracy include signal distortion (frequently harmonic distortion of the sine/cosine signals). 246 Linear encoder Digital Many linear encoders interpolate the analogue sine/cosine signals in order to sub-divide the scale period, providing a higher measurement resolution. The output of the interpolation process is quadrature squarewaves – the distance between edges of the two channels being the resolution of the encoder. The A and B quadrature channels The reference mark or index pulse will also be processed digitally and will be a pulse, usually one to four units-of-resolution wide. The major advantage of encoders with built-in interpolation and digital signal transmission is improved noise immunity. However, the high frequency, fast edge speed signals may produce more emc emissions. Incremental encoders with built-in digital processing make it possible to transmit position to any subsequent electronics such as a position counter. Absolute reference signals As well as analog or digital incremental output signals, linear encoders can provide absolute reference or positioning signals. Reference mark Most incremental, linear encoders can produce an index or reference mark pulse providing a datum position along the scale for use at power-up or following a loss of power. This index signal must be able to identify position within one, unique period of the scale. The reference mark may comprise a single feature on the scale, an autocorrelator pattern (typically a Barker code) or a chirp pattern. Distance coded reference marks (DCRM) are placed onto the scale in a unique pattern allowing a minimal movement (typically moving past two reference marks) to define the readhead's position. Multiple, equally spaced reference marks may also be placed onto the scale such that following installation, the desired marker can be selected - usually via a magnet or optically. Absolute code With suitably encoded scales (multitrack, vernier, digital code, or pseudo-random code) an encoder can determine its position without movement or needing to find a reference position. Such absolute encoders also communicate using serial communication protocols. Many of these protocols are proprietary – Fanuc, Mitsubishi, EnDat, DriveCliq, Panasonic, Yaskawa – but open standards such as BiSS [13] are now appearing, which avoid tying users to a particular supplier. 247 Linear encoder Limit switches Many linear encoders include built-in limit switches - either optical or magnetic. Two limit switches are frequently included such that on power-up the controller can determine if the encoder is at an end-of-travel and in which direction to drive the axis. Physical arrangement / protection Linear encoders may be either enclosed or open. Enclosed linear encoders are employed in dirty, hostile environments such as machine-tools. They typically comprise an aluminium extrusion enclosing a glass or metal scale. Flexible lip seals allow an internal, guided readhead to read the scale. Accuracy is limited due to the friction and hysteresis imposed by this mechanical arrangement. For the highest accuracy, lowest measurement hysteresis and lowest friction applications, open linear encoders are used. Linear encoders may use transmissive (glass) or reflective scales, employing Ronchi or phase gratings. Scale materials include chrome on glass, metal (stainless steel, gold plated steel, Invar), ceramics (Zerodur) and plastics. The scale may be self supporting, be thermally mastered to the substrate (via adhesive or adhesive tape) or track mounted. Track mounting may allow the scale to maintain its own coefficient of thermal expansion and allows large equipment to be broken down for shipment. Encoder terms Resolution Repeatability Hysteresis Signal-to-noise ratio / Noise / Jitter Lissajous Figure Quadrature Index / Reference Mark / Datum / Fiducial Distance Coded Reference Marks (DCRM) Book • David S. Nyce: Linear Position Sensors: Theory and Application, New Jersey, John Wiley & Sons Inc. (2003) • Walcher Hans: Position Sensing: Angle and Distance Measurement for Engineers, Butterworth Heinemann (1994) References [1] [2] [3] [4] [5] [6] [7] [8] http:/ / www. microesys. com/ m2/ index. html http:/ / www. renishaw. com/ en/ 6433. aspx http:/ / www. rls. si/ default. asp?prod=LMencoders http:/ / www. mitutoyo. com/ pdf/ ABS1813-293. pdf http:/ / www. ruhle. com/ bar_scale. htm http:/ / www. newall. com/ LEDs/ operation. htm http:/ / www. maxonmotor. com/ downloads/ Flyer_EC6_MILE_e_03. 09. pdf http:/ / www. intacton. com/ us/ products/ INTACTON/ OpticalMotionSensors_Context/ LengthSpeedSensors_Optical_Context_Technology_PopupBase. html [9] http:/ / www. mitutoyo. com/ pdf/ 1976_MICSYS. pdf [10] http:/ / www. wenzel-cmm. co. uk/ Industries. asp?SE=9 [11] http:/ / www. instron. co. uk/ wa/ product/ Tension-Testers. aspx [12] http:/ / global. oce. com/ products/ productionprinting/ digitalpresses/ color/ default. aspx [13] http:/ / www. biss-interface. com/ 248 Gray code 249 Gray code Gray code by bit width 2-bit 4-bit 00 01 11 10 3-bit 000 001 011 010 110 111 101 100 0000 0001 0011 0010 0110 0111 0101 0100 1100 1101 1111 1110 1010 1011 1001 1000 The reflected binary code, also known as Gray code after Frank Gray, is a binary numeral system where two successive values differ in only one bit. It is a non-weighted code. The reflected binary code was originally designed to prevent spurious output from electromechanical switches. Today, Gray codes are widely used to facilitate error correction in digital communications such as digital terrestrial television and some cable TV systems. Name Bell Labs researcher Frank Gray introduced the term reflected binary code in his 1947 patent application, remarking that the code had "as yet no recognized name".[1] He derived the name from the fact that it "may be built up from the conventional binary code by a sort of reflection process". The code was later named after Gray by others who used it. Two different 1953 patent applications give "Gray code" as an alternative name for the "reflected binary code";[2] [3] one of those also lists "minimum error code" and "cyclic permutation code" among the names.[3] A 1954 patent application refers to "the Bell Telephone Gray code".[4] Gray's patent introduces the term "reflected binary code" Gray code 250 Motivation Many devices indicate position by closing and opening switches. If that device uses natural binary codes, these two positions would be right next to each other: ... 011 100 ... The problem with natural binary codes is that, with real (mechanical) switches, it is very unlikely that switches will change states exactly in synchrony. In the transition between the two states shown above, all three switches change state. In the brief period while all are changing, the switches will read some spurious position. Even without keybounce, the transition might look like 011 — 001 — 101 — 100. When the switches appear to be in position 001, the observer cannot tell if that is the "real" position 001, or a transitional state between two other positions. If the output feeds into a sequential system (possibly via combinational logic) then the sequential system may store a false value. The reflected binary code solves this problem by changing only one switch at a time, so there is never any ambiguity of position, Dec 0 1 2 3 4 5 6 7 Gray 000 001 011 010 110 111 101 100 Binary 000 001 010 011 100 101 110 111 Notice that state 7 can roll over to state 0 with only one switch change. This is called the "cyclic" property of a Gray code. A good way to remember Gray coding is by being aware that the least significant bit follows a repetitive pattern of 2. That is 11, 00, 11 etc. and the second digit follows a pattern of fours. More formally, a Gray code is a code assigning to each of a contiguous set of integers, or to each member of a circular list, a word of symbols such that each two adjacent code words differ by one symbol. These codes are also known as single-distance codes, reflecting the Hamming distance of 1 between adjacent codes. There can be more than one Gray code for a given word length, but the term was first applied to a particular binary code for the non-negative integers, the binary-reflected Gray code, or BRGC, the three-bit version of which is shown above. History and practical application Reflected binary codes were applied to mathematical puzzles before they became known to engineers. The French engineer Émile Baudot used Gray codes in telegraphy in 1878. He received the French Legion of Honor medal for his work. The Gray code is sometimes attributed, incorrectly,[5] to Elisha Gray (in Principles of Pulse Code Modulation, K. W. Cattermole,[6] for example). Frank Gray, who became famous for inventing the signaling method that came to be used for compatible color television, invented a method to convert analog signals to reflected binary code groups using vacuum tube-based apparatus. The method and apparatus were patented in 1953 and the name of Gray stuck to the codes. The "PCM tube" apparatus that Gray patented was made by Raymond W. Sears of Bell Labs, working with Gray and William M. Goodall, who credited Gray for the idea of the reflected binary code.[7] Gray code 251 Part of front page of Gray's patent, showing PCM tube (10) with reflected binary code in plate (15) The use of his eponymous codes that Gray was most interested in was to minimize the effect of error in the conversion of analog signals to digital; his codes are still used today for this purpose, and others. Position encoders Gray codes are used in position encoders (linear encoders and rotary encoders), in preference to straightforward binary encoding. This avoids the possibility that, when several bits change in the binary representation of an angle, a misread will result from some of the bits changing before others. Originally, the code pattern was electrically conductive, supported (in a rotary encoder) by an insulating disk. Each track had its own stationary metal spring contact; one more contact made the connection to the pattern. That common contact was connected by the pattern to whichever of the track contacts were resting on the conductive pattern. However, sliding contacts wear out and need maintenance, which favors optical encoders. Rotary encoder for angle-measuring devices Regardless of the care in aligning the contacts, and accuracy of the marked in 3-bit binary-reflected Gray code pattern, a natural-binary code would have errors at specific disk (BRGC) positions, because it is impossible to make all bits change at exactly the same time as the disk rotates. The same is true of an optical encoder; transitions between opaque and transparent cannot be made to happen simultaneously for certain exact positions. Rotary encoders benefit from the cyclic nature of Gray codes, because consecutive positions of the sequence differ by only one bit. Gray code Towers of Hanoi The binary-reflected Gray code can also be used to serve as a solution guide for the Towers of Hanoi problem, as well as the classical Chinese rings puzzle, a sequential mechanical puzzle mechanism.[5] It also forms a Hamiltonian cycle on a hypercube, where each bit is seen as one dimension. Genetic algorithms Due to the Hamming distance properties of Gray codes, they are sometimes used in genetic algorithms. They are very useful in this field, since mutations in the code allow for mostly incremental changes, but occasionally a single bit-change can cause a big leap and lead to new properties. Karnaugh maps Gray codes are also used in labelling the axes of Karnaugh maps.[8] Error correction In modern digital communications, Gray codes play an important role A Gray code absolute rotary encoder with 13 in error correction. For example, in a digital modulation scheme such tracks. At the top can be seen the housing, as QAM where data is typically transmitted in symbols of 4 bits or interrupter disk, and light source; at the bottom more, the signal's constellation diagram is arranged so that the bit can be seen the sensing element and support patterns conveyed by adjacent constellation points differ by only one components. bit. By combining this with forward error correction capable of correcting single-bit errors, it is possible for a receiver to correct any transmission errors that cause a constellation point to deviate into the area of an adjacent point. This makes the transmission system less susceptible to noise. Communication between clock domains Digital logic designers use Gray codes extensively for passing multi-bit count information between synchronous logic that operates at different clock frequencies. The logic is considered operating in different "clock domains". It is fundamental to the design of large chips that operate with many different clocking frequencies. Gray code counters and arithmetic A typical use of Gray code counters is building a FIFO (first-in, first-out) data buffer that has read and write ports that exist in different clock domains. The input and output counters inside such a dual-port FIFO are often stored using Gray code to prevent invalid transient states from being captured when the count crosses clock domains.[9] The updated read and write pointers need to be passed between clock domains when they change, to be able to track FIFO empty and full status in each domain. Each bit of the pointers is sampled non-deterministically for this clock domain transfer. So for each bit, either the old value or the new value is propagated. Therefore, if more than one bit in the multi-bit pointer is changing at the sampling point, a "wrong" binary value (neither new nor old) can be propagated. By guaranteeing only one bit can be changing, Gray codes guarantee that the only possible sampled values are the new or old multi-bit value. Typically Gray codes of power-of-two length are used. 252 Gray code 253 Sometimes digital buses in electronic systems are used to convey quantities that can only increase or decrease by one at a time, for example the output of an event counter which is being passed between clock domains or to a digital-to-analog converter. The advantage of Gray codes in these applications is that differences in the propagation delays of the many wires that represent the bits of the code cannot cause the received value to go through states that are out of the Gray code sequence. This is similar to the advantage of Gray codes in the construction of mechanical encoders, however the source of the Gray code is an electronic counter in this case. The counter itself must count in Gray code, or if the counter runs in binary then the output value from the counter must be reclocked after it has been converted to Gray code, because when a value is converted from binary to Gray code, it is possible that differences in the arrival times of the binary data bits into the binary-to-Gray conversion circuit will mean that the code could go briefly through states that are wildly out of sequence. Adding a clocked register after the circuit that converts the count value to Gray code may introduce a clock cycle of latency, so counting directly in Gray code may be advantageous. A Gray code counter was patented in 1962 US3020481 [10], and there have been many others since. In recent times a Gray code counter can be implemented as a state machine in Verilog. In order to produce the next count value, it is necessary to have some combinational logic that will increment the current count value that is stored in Gray code. Probably the most obvious way to increment a Gray code number is to convert it into ordinary binary code, add one to it with a standard binary adder, and then convert the result back to Gray code. This approach was discussed in a paper in 1996 [11] and then subsequently patented by someone else in 1998 US5754614 [12]. Other methods of counting in Gray code are discussed in a report by R. W. Doran, including taking the output from the first latches of the master-slave flip flops in a binary ripple counter.[13] Perhaps the most common electronic counter with the "only one bit changes at a time" property is the Johnson counter. Constructing an n-bit Gray code The binary-reflected Gray code list for n bits can be generated recursively from the list for n−1 bits by reflecting the list (i.e. listing the entries in reverse order), concatenating the original list with the reversed list, prefixing the entries in the original list with a binary 0, and then prefixing the entries in the reflected list with a binary 1. For example, generating the n = 3 list from the n = 2 list: The first few steps of the reflect-and-prefix method. Gray code 254 4-bit Gray code permutation 2-bit list: 00, 01, 11, 10 Reflected: 10, 11, 01, 00 Prefix old entries with 0: 000, 001, 011, 010, Prefix new entries with 1: Concatenated: 110, 111, 101, 100 000, 001, 011, 010, 110, 111, 101, 100 The one-bit Gray code is G1 = (0, 1). This can be thought of as built recursively as above from a zero-bit Gray code G0 = { Λ } consisting of a single entry of zero length. This iterative process of generating Gn+1 from Gn makes the following properties of the standard reflecting code clear: • Gn is a permutation of the numbers 0, ... , 2n−1. (Each number appears exactly once in the list.) • Gn is embedded as the first half of Gn+1. • Therefore the coding is stable, in the sense that once a binary number appears in Gn it appears in the same position in all longer lists; so it makes sense to talk about the reflective Gray code value of a number: G(m) = the m-th reflecting Gray code, counting from 0. • Each entry in Gn differs by only one bit from the previous entry. (The Hamming distance is 1.) • The last entry in Gn differs by only one bit from the first entry. (The code is cyclic.) These characteristics suggest a simple and fast method of translating a binary value into the corresponding Gray code. Each bit is inverted if the next higher bit of the input value is set to one. This can be performed in parallel by a bit-shift and exclusive-or operation if they are available: the nth Gray code is obtained by computing A similar method can be used to perform the reverse translation, but the computation of each bit depends on the computed value of the next higher bit so it cannot be performed in parallel. Assuming is the th gray-coded bit ( being the most significant bit), and is the reverse translation can be given recursively: th binary-coded bit ( , and being the most-significant bit), the . Alternatively, decoding a Gray code into a binary number can be described as a prefix sum of the bits in the Gray code, where each individual summation operation in the prefix sum is performed modulo two. To construct the binary-reflected Gray code iteratively, start with the code 0, and at step i find the bit position of the least significant '1' in the binary representation of i - flip the bit at that position in the previous code to get the next code. The bit positions start 0, 1, 0, 2, 0, 1, 0, 3, ... (sequence A007814 in OEIS). See find first set for efficient Gray code 255 algorithms to compute these values. Converting to and from Gray code /* The purpose of this function is to convert an unsigned binary number to reflected binary Gray code. */ unsigned short binaryToGray(unsigned short num) { return (num>>1) ^ num; } /* The purpose of this function is to convert a reflected binary Gray code number to a binary number. */ unsigned short grayToBinary(unsigned short num) { unsigned short temp = num ^ (num>>8); temp ^= (temp>>4); temp ^= (temp>>2); temp ^= (temp>>1); return temp; } Special types of Gray codes In practice, a "Gray code" almost always refers to a binary-reflected Gray code (BRGC). However, mathematicians have discovered other kinds of Gray codes. Like BRGCs, each consists of a lists of words, where each word differs from the next in only one digit (each word has a Hamming distance of 1 from the next word). n-ary Gray code There are many specialized types of Gray codes other than the binary-reflected Gray code. One such type of Gray code is the n-ary Gray code, also known as a non-Boolean Gray code. As the name implies, this type of Gray code uses non-Boolean values in its encodings. For example, a 3-ary (ternary) Gray code would use the values {0, 1, 2}. The (n, k)-Gray code is the n-ary Gray code with k digits.[14] The sequence of elements in the (3, 2)-Gray code is: {00, 01, 02, 12, 10, 11, 21, 22, 20}. The (n, k)-Gray code may be constructed recursively, as the BRGC, or may be constructed iteratively. An algorithm to iteratively generate the (N, k)-Gray code is presented (in C): // parameters: value, base, digits // Convert a value to a graycode with the given base and digits. Iterating // through a sequence of values would result in a sequence of graycodes in // which only one digit changes at a time. Gray code int baseN[digits]; per entry int gray[digits]; int i; 256 // Stores the ordinary base-N number, one digit // Stores the base-N graycode number // The loop variable // Put the normal baseN number into the baseN array. For base 10, 109 // would be stored as [9,0,1] int tempvalue = value; for(i = 0; i < digits; i++) { baseN[i] = tempvalue % base; tempvalue /= base; } // Convert the normal baseN number into the graycode equivalent. Note that // the loop starts at the most significant digit and goes down. int shift = 0; for(i = digits - 1; i >= 0; i--) { // The gray digit gets shifted down equal to the sum of the higher // digits. gray[i] = (baseN[i] - shift) % base; shift += gray[i] - base; // - base to prevent negative in mod operation } // EXAMPLES // input: value = 1899, base = 10, digits = 4 // output: baseN[] = [9,9,8,1], gray[] = [0,1,7,1] // input: value = 1900, base = 10, digits = 4 // output: baseN[] = [0,0,9,1], gray[] = [0,1,8,1] Gray code 257 Ternary number -> ternary Gray code 0 1 2 10 11 12 20 21 22 100 101 102 110 111 112 120 121 122 200 201 202 210 211 212 220 221 222 -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> -> 000 001 002 012 010 011 021 022 020 120 121 122 102 100 101 111 112 110 210 211 212 222 220 221 201 202 200 There are other graycode algorithms for (n,k)-Gray codes. It is important to note that the (n,k)-Gray codes produced by the above algorithm is always cyclical; some algorithms, such as that by Guan,[14] lack this property when k is odd. On the other hand, while only one digit at a time changes with this method, it can change by wrapping (looping from n-1 to 0). In Guan's algorithm, the count alternately rises and falls, so that the numeric difference between two graycode digits is always one. Gray codes are not uniquely defined, because a permutation of the columns of such a code is a Gray code too. The above procedure produces a code in which the lower the significance of a digit, the more often it changes, making it similar to normal counting methods. Balanced Gray code Although the binary reflected Gray code is useful in many scenarios, it is not optimal in certain cases because of a lack of "uniformity".[15] In balanced Gray codes, we seek to make the number of changes in each coordinate position as close as possible. To make this more precise, let G be a R-ary complete Gray cycle having transition sequence ; the transition counts (spectrum) of are the collection of integers defined by We say that a Gray code is uniform or uniformly balanced if its transition counts are all equal, in which case we have for all k. Clearly, when , such codes exist only if n is a power of 2. Otherwise, if n does not divide evenly, the best we can do is to construct well-balanced codes where every transition count is either or . Gray codes can also be exponentially balanced if all of their transition counts are adjacent powers of two, and such codes exist for every power of two.[16] We will now show a construction for well-balanced binary Gray codes which allows us to generate a n-digit balanced Gray code for every n.[17] The main principle is to inductively construct a -digit Gray code Gray code 258 given an n-digit Gray code G in such a way that the balanced property is preserved. To do this, we consider partitions of into an even number L of non-empty blocks of the form where , and (mod ). This partition induces a -digit Gray code given by If we define the transition multiplicities to be the number of times the digit in position i changes between consecutive blocks in a partition, then for the this partition the transition spectrum -digit Gray code induced by is The delicate part of this construction is to find an adequate partitioning of a balanced n-digit Gray code such that the code induced by it remains balanced. Uniform codes can be found when and , and this construction can be extended to the R-ary case as well.[17] Monotonic Gray codes Monotonic codes are useful in the theory of interconnection networks, especially for minimizing dilation for linear arrays of processors.[18] If we define the weight of a binary string to be the number of 1's in the string, then although we clearly cannot have a Gray code with strictly increasing weight, we may want to approximate this by having the code run through two adjacent weights before reaching the next one. We can formalize the concept of monotone Gray codes as follows: consider the partition of the hypercube into levels of vertices that have equal weight, i.e. for . These levels satisfy , and let be the edges in . Let be the subgraph of induced by . A monotonic Gray code is then a Hamiltonian path in such that whenever comes before in the path, then . An elegant construction of monotonic n-digit Gray codes for any n is based on the idea of recursively building subpaths of length or otherwise. Here, having edges in .[18] We define first few values of whenever , and is a suitably defined permutation and . These paths give rise to two monotonic n-digit Gray codes The choice of , refers to the path P with its coordinates permuted by and given by which ensures that these codes are indeed Gray codes turns out to be are shown in the table below. . The Gray code 259 Subpaths in the Savage-Winkler algorithm j=0 j=1 n=1 0, 1 n=2 00, 01 10, 11 n=3 000, 001 100, 110, 010, 011 j=2 j=3 101, 111 n = 4 0000, 0001 1000, 1100, 0100, 0110, 0010, 0011 1010, 1011, 1001, 1101, 0101, 0111 1110, 1111 These monotonic Gray codes can be efficiently implemented in such a way that each subsequent element can be generated in O(n) time. The algorithm is most easily described using coroutines. Monotonic codes have an interesting connection to the Lovász conjecture, which states that every connected vertex-transitive graph contains a Hamiltonian path. The "middle-level" subgraph is vertex-transitive (that is, its automorphism group is transitive, so that each vertex has the same "local environment"" and cannot be differentiated from the others, since we can relabel the coordinates as well as the binary digits to obtain an automorphism) and the problem of finding a Hamiltonian path in this subgraph is called the "middle-levels problem", which can provide insights into the more general conjecture. The question has been answered affirmatively for , and the preceding construction for monotonic codes ensures a Hamiltonian path of length at least 0.839N where N is the number of vertices in the middle-level subgraph.[19] Beckett–Gray code Another type of Gray code, the Beckett–Gray code, is named for Irish playwright Samuel Beckett, who was interested in symmetry. His play "Quad" features four actors and is divided into sixteen time periods. Each period ends with one of the four actors entering or leaving the stage. The play begins with an empty stage, and Beckett wanted each subset of actors to appear on stage exactly once.[20] Clearly the set of actors currently on stage can be represented by a 4-bit binary Gray code. Beckett, however, placed an additional restriction on the script: he wished the actors to enter and exit so that the actor who had been on stage the longest would always be the one to exit. The actors could then be represented by a first in, first out queue, so that (of the actors onstage) the actor being dequeued is always the one who was enqueued first.[20] Beckett was unable to find a Beckett–Gray code for his play, and indeed, an exhaustive listing of all possible sequences reveals that no such code exists for n = 4. It is known today that such codes do exist for n = 2, 5, 6, 7, and 8, and do not exist for n = 3 or 4. An example of an 8-bit Beckett–Gray code can be found in Knuth's Art of Computer Programming.[5] According to Sawada and Wong, the search space for n = 6 can be explored in 15 hours, and more than 9,500 solutions for the case n = 7 have been found.[21] Snake-in-the-box codes Snake-in-the-box codes, or snakes, are the sequences of nodes of induced paths in an n-dimensional hypercube graph, and coil-in-the-box codes, or coils, are the sequences of nodes of induced cycles in a hypercube. Viewed as Gray codes, these sequences have the property of being able to detect any single-bit coding error. Codes of this type were first described by W. H. Kautz in the late 1950s;[22] since then, there has been much research on finding the code with the largest possible number of codewords for a given hypercube dimension. Gray code Single-track Gray code Yet another kind of Gray code is the single-track Gray code (STGC) developed by N. B. Spedding (NZ Patent 264738 - October 28, 1994)[23] and refined by Hiltgen, Paterson and Brandestini in "Single-track Gray codes" (1996).[24] The STGC is a cyclical list of P unique binary encodings of length n such that two consecutive words differ in exactly one position, and when the list is examined as a P x n matrix, each column is a cyclic shift of the first column.[25] The name comes from their use with rotary encoders, where a number of tracks are being sensed by contacts, resulting for each in an output of 0 or 1. To reduce noise due to different contacts not switching at exactly the same moment in time, one preferably sets up the tracks so that the data output by the contacts are in Gray code. To get high angular accuracy, one needs lots of contacts; in order to achieve at least 1 degree accuracy, one needs at least 360 distinct positions per revolution, which requires a minimum of 9 bits of data, and thus the same number of contacts. If all contacts are placed at the same angular position, then 9 tracks are needed to get a standard BRGC with at least 1 degree accuracy. However, if the manufacturer moves a contact to a different angular position (but at the same distance from the center shaft), then the corresponding "ring pattern" needs to be rotated the same angle to give the same output. If the most significant bit (the inner ring in Figure 1) is rotated enough, it exactly matches the next ring out. Since both rings are then identical, the inner ring can be cut out, and the sensor for that ring moved to the remaining, identical ring (but offset at that angle from the other sensor on that ring). Those 2 sensors on a single ring make a quadrature encoder. That reduces the number of tracks for a "1 degree resolution" angular encoder to 8 tracks. Reducing the number of tracks still further can't be done with BRGC. For many years, Torsten Sillke and other mathematicians believed that it was impossible to encode position on a single track such that consecutive positions differed at only a single sensor, except for the 2-sensor, 1-track quadrature encoder. So for applications where 8 tracks were too bulky, people used single-track incremental encoders (quadrature encoders) or 2-track "quadrature encoder + reference notch" encoders. N. B. Spedding, however, registered a patent in 1994 with several examples showing that it was possible.[23] Hiltgen and Paterson published a paper in 2001 exhibiting a single-track gray code with exactly 360 angular positions, constructed using 9 sensors,[26] the same number as a BRGC with the same resolution. (It would be impossible to discriminate that many positions with any fewer sensors.) An STGC for P = 30 and n = 5 is reproduced here: 10000 10100 11100 11110 260 Gray code 11010 11000 01000 01010 01110 01111 01101 01100 00100 00101 00111 10111 10110 00110 00010 10010 10011 11011 01011 00011 00001 01001 11001 11101 10101 10001 Note that each column is a cyclic shift of the first column, and from any row to the next row only one bit changes.[27] The single-track nature (like a code chain) is useful in the fabrication of these wheels (compared to BRGC), as only one track is needed, thus reducing their cost and size. The Gray code nature is useful (compared to chain codes, also called De Bruijn sequences), as only one sensor will change at any one time, so the uncertainty during a transition between two discrete states will only be plus or minus one unit of angular measurement the device is capable of resolving.[28] Notes [1] F. Gray. Pulse code communication, March 17, 1953 (filed Nov. 1947). U.S. Patent 2632058 (http:/ / www. google. com/ patents?vid=2632058) [2] J. Breckman. Encoding Circuit, Jan 31, 1956 (filed Dec. 1953). U.S. Patent 2733432 (http:/ / www. google. com/ patents?vid=2733432) [3] E. A. Ragland et al. Direction-Sensitive Binary Code Position Control System, Feb. 11, 1958 (filed Oct. 1953). U.S. Patent 2823345 (http:/ / www. google. com/ patents?vid=2823345) [4] S. Reiner et al. Automatic Rectification System, Jun 24, 1958 (filed Jan. 1954). U.S. Patent 2839974 (http:/ / www. google. com/ patents?vid=2839974) [5] Knuth, Donald E. "Generating all n-tuples." The Art of Computer Programming, Volume 4A: Enumeration and Backtracking, pre-fascicle 2a, October 15, 2004. (http:/ / www-cs-faculty. stanford. edu/ ~knuth/ fasc2a. ps. gz) [6] Cattermole, K. W. (1969). Principles of Pulse Code Modulation. New York: American Elsevier. ISBN 0-444-19747-8. [7] Goodall, W. M. (1951). "Television by Pulse Code Modulation". Bell Sys. Tech. J. 30: 33–49. [8] Wakerly, John F (1994). Digital Design: Principles & Practices. New Jersey: Prentice Hall. pp. 222, 48–49. ISBN 0132114593. Note that the two page sections taken together say that K-maps are labeled with Gray code. The first section says that they are labeled with a code that changes only one bit between entries and the second section says that such a code is called Gray code. 261 Gray code [9] "Synchronization in Digital Logic Circuits (http:/ / www. stanford. edu/ class/ ee183/ handouts_spr2003/ synchronization_pres. pdf) by Ryan Donohue [10] http:/ / www. google. com/ patents?vid=3020481 [11] Mehta, H.; Owens, R.M. & Irwin, M.J. (1996), Some issues in gray code addressing (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?tp=& arnumber=497616& isnumber=10625), in the Proceedings of the 6th Great Lakes Symposium on VLSI (GLSVLSI 96), IEEE Computer Society,pp. 178 [12] http:/ / www. google. com/ patents?vid=5754614 [13] The Gray Code by R. W. Doran (http:/ / www. cs. auckland. ac. nz/ CDMTCS/ / researchreports/ 304bob. pdf) [14] Guan, Dah-Jyh (1998). "Generalized Gray Codes with Applications" (http:/ / nr. stpi. org. tw/ ejournal/ ProceedingA/ v22n6/ 841-848. pdf) (PDF). Proc. Natl. Sci. Counc. Repub. Of China (A) 22: 841–848. . [15] Girish S. Bhat and Carla D. Savage (1996). "Balanced Gray codes" (http:/ / www. combinatorics. org/ Volume_3/ Abstracts/ v3i1r25. html). The Electronic Journal of Combinatorics 3 (1): R25. . [16] I. N Suparta (2005). "A simple proof for the existence of exponentially balanced Gray codes". Electron. J. Combin 12. [17] M. Flahive and B. Bose (2007). "Balancing cyclic R-ary Gray codes". the electronic journal of combinatorics 14. [18] C. D Savage and P. Winkler (1995). "Monotone Gray codes and the middle levels problem". Journal of Combinatorial Theory, Series A 70 (2): 230–248. doi:10.1016/0097-3165(95)90091-8. ISSN 0097-3165. [19] C. D Savage (1997). Long cycles in the middle two levels of the Boolean lattice. [20] Goddyn, Luis (1999). "MATH 343 Applied Discrete Math Supplementary Materials" (http:/ / www. math. sfu. ca/ ~goddyn/ Courses/ 343/ supMaterials. pdf) (PDF). Dept. of Math, Simon Fraser U. . [21] Wong, J. (2007). "A Fast Algorithm to generate Beckett-Gray codes". Electronic Notes in Discrete Mathematics 29: 571–577. doi:10.1016/j.endm.2007.07.091. [22] Kautz, W. H. (1958). "Unit-distance error-checking codes". IRE Trans. Elect. Comput. 7: 177–180. [23] A position encoder (http:/ / www. winzurf. co. nz/ Single_Track_Grey_Code_Patent/ Single_track_Grey_code_encoder_patent. pdf). 1994. . [24] Hiltgen, Alain P.; Kenneth G. Paterson, Marco Brandestini (1996). "Single-Track Gray Codes" (http:/ / ieeexplore. ieee. org/ iel1/ 18/ 11236/ 00532900. pdf) (PDF). IEEE Transactions on Information Theory 42 (5): 1555–1561. doi:10.1109/18.532900. . [25] Etzion, Tuvi; Moshe Schwartz (1999). "The Structure of Single-Track Gray Codes" (http:/ / www. cs. technion. ac. il/ ~etzion/ PUB/ Gray2. pdf) (PDF). IEEE Transactions on Information Theory 45 (7): 2383–2396. doi:10.1109/18.796379. . [26] Hiltgen, Alain P.; Kenneth G. Paterson (2001). "Single-Track Circuit Codes" (http:/ / www. hpl. hp. com/ techreports/ 2000/ HPL-2000-81. pdf) (PDF). IEEE Transactions on Information Theory 47 (6): 2587–2595. doi:10.1109/18.945274. . [27] "Venn Diagram Survey — Symmetric Diagrams" (http:/ / www. combinatorics. org/ Surveys/ ds5/ VennSymmEJC. html). The Electronic Journal of Combinatorics. 2001. . [28] Alciatore, David G.; Michael B. Histand (1999). McGraw-Hill Education - Europe. ISBN 9780071314442. http:/ / mechatronics. colostate. edu/ . References • Black, Paul E. Gray code (http://www.nist.gov/dads/HTML/graycode.html). 25 February 2004. NIST. • Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007). "Section 22.3. Gray Codes" (http://apps. nrbook.com/empanel/index.html#pg=1166). Numerical Recipes: The Art of Scientific Computing (3rd ed.). New York: Cambridge University Press. ISBN 978-0-521-88068-8. • Savage, Carla (1997). "A Survey of Combinatorial Gray Codes" (http://www.csc.ncsu.edu/faculty/savage/ AVAILABLE_FOR_MAILING/survey.ps). SIAM Rev. 39 (4): 605–629. doi:10.1137/S0036144595295272. JSTOR 2132693. • Wilf, Herbert S. (1989). "Chapters 1-3". Combinatorial algorithms: an update. SIAM. ISBN 0-89871-231-9. 262 Gray code External links • "Gray Code" demonstration (http://demonstrations.wolfram.com/BinaryGrayCode/) by Michael Schreiber, Wolfram Demonstrations Project (with Mathematica implementation). 2007. • NIST Dictionary of Algorithms and Data Structures: Gray code (http://www.nist.gov/dads/HTML/graycode. html) • Hitch Hiker's Guide to Evolutionary Computation, Q21: What are Gray codes, and why are they used? (http:// www.aip.de/~ast/EvolCompFAQ/Q21.htm), including C code to convert between binary and BRGC • Subsets or Combinations (http://www.theory.cs.uvic.ca/~cos/gen/comb.html) Can generate BRGC strings • "The Structure of Single-Track Gray Codes" (http://www.cs.technion.ac.il/users/wwwb/cgi-bin/tr-info. cgi?1998/CS/CS0937) by Moshe Schwartz, Tuvi Etzion • Single-Track Circuit Codes (http://www.hpl.hp.com/techreports/2000/HPL-2000-81.html) by Hiltgen, Alain P.; Paterson, Kenneth G. • Dragos A. Harabor uses Gray codes in a 3D digitizer (http://www.ugcs.caltech.edu/~dragos/3DP/coord. html). • single-track gray codes, binary chain codes ( Lancaster 1994 (http://tinaja.com/text/chain01.html)), and linear feedback shift registers are all useful in finding one's absolute position on a single-track rotary encoder (or other position sensor). • Computing Binary Combinatorial Gray Codes Via Exhaustive Search With SAT Solvers (http://ieeexplore.ieee. org/xpls/abs_all.jsp?isnumber=4475352&arnumber=4475394&count=44&index=39) by Zinovik, I.; Kroening, D.; Chebiryak, Y. • AMS Column: Gray codes (http://www.ams.org/featurecolumn/archive/gray.html) • Optical Encoder Wheel Generator (http://www.bushytails.net/~randyg/encoder/encoderwheel.html) • ProtoTalk.net - Understanding Quadrature Encoding (http://prototalk.net/forums/showthread.php?t=78) Covers quadrature encoding in more detail with a focus on robotic applications 263 Rotary encoder 264 Rotary encoder A rotary encoder, also called a shaft encoder, is an electro-mechanical device that converts the angular position or motion of a shaft or axle to an analog or digital code. The output of incremental encoders provides information about the motion of the shaft which is typically further processed elsewhere into information such as speed, distance, RPM and position. The output of absolute encoders indicates the current position of the shaft, making them angle transducers. Rotary encoders are used in many applications that require precise shaft unlimited rotation—including industrial controls, robotics, special purpose photographic lenses,[1] computer input devices (such as optomechanical mice and trackballs), controlled stress rheometers, and rotating radar platforms. There are two main types: absolute and incremental (relative). A Gray code absolute rotary encoder with 13 tracks. At the top can be seen the housing, interrupter disk, and light source; at the bottom can be seen the sensing element and support components. Absolute rotary encoder Construction Digital absolute encoders produce a unique digital code for each distinct angle of the shaft. They come in two basic types: optical and mechanical. Mechanical absolute encoders A metal disc containing a set of concentric rings of openings is fixed to an insulating disc, which is rigidly fixed to the shaft. A row of sliding contacts is fixed to a stationary object so that each contact wipes against the metal disc at a different distance from the shaft. As the disc rotates with the shaft, some of the contacts touch metal, while others Absolute rotary encoder ROD 425 Rotary encoder 265 fall in the gaps where the metal has been cut out. The metal sheet is connected to a source of electric current, and each contact is connected to a separate electrical sensor. The metal pattern is designed so that each possible position of the axle creates a unique binary code in which some of the contacts are connected to the current source (i.e. switched on) and others are not (i.e. switched off). Optical absolute encoders The optical encoder's disc is made of glass or plastic with transparent and opaque areas. A light source and photo detector array reads the optical pattern that results from the disc's position at any one time. This code can be read by a controlling device, such as a microprocessor or microcontroller to determine the angle of the shaft. The absolute analog type produces a unique dual analog code that can be translated into an absolute angle of the shaft (by using a special algorithm). Standard binary encoding An example of a binary code, in an extremely simplified encoder with only three contacts, is shown below. Rotary encoder for angle-measuring devices marked in 3-bit binary. The inner ring corresponds to Contact 1 in the table. Black sectors are "on". Zero degrees is on the right-hand side, with angle increasing counterclockwise. Standard Binary Encoding Sector Contact 1 Contact 2 Contact 3 Angle 1 off off off 0° to 45° 2 off off ON 45° to 90° 3 off ON off 90° to 135° 4 off ON ON 135° to 180° 5 ON off off 180° to 225° 6 ON off ON 225° to 270° 7 ON ON off 270° to 315° 8 ON ON ON 315° to 360° Rotary encoder 266 In general, where there are n contacts, the number of distinct positions of the shaft is 2n. In this example, n is 3, so there are 2³ or 8 positions. In the above example, the contacts produce a standard binary count as the disc rotates. However, this has the drawback that if the disc stops between two adjacent sectors, or the contacts are not perfectly aligned, it can be impossible to determine the angle of the shaft. To illustrate this problem, consider what happens when the shaft angle changes from 179.9° to 180.1° (from sector 4 to sector 5). At some instant, according to the above table, the contact pattern changes from off-on-on to on-off-off. However, this is not what happens in reality. In a practical device, the contacts are never perfectly aligned, so each switches at a different moment. If contact 1 switches first, followed by contact 3 and then contact 2, for example, the actual sequence of codes is: off-on-on (starting position) on-on-on (first, contact 1 switches on) on-on-off (next, contact 3 switches off) on-off-off (finally, contact 2 switches off) Now look at the sectors corresponding to these codes in the table. In order, they are 4, 8, 7 and then 5. So, from the sequence of codes produced, the shaft appears to have jumped from sector 4 to sector 8, then gone backwards to sector 7, then backwards again to sector 5, which is where we expected to find it. In many situations, this behaviour is undesirable and could cause the system to fail. For example, if the encoder were used in a robot arm, the controller would think that the arm was in the wrong position, and try to correct the error by turning it through 180°, perhaps causing damage to the arm. Gray encoding To avoid the above problem, Gray encoding is used. This is a system of binary counting in which adjacent codes differ in only one position. For the three-contact example given above, the Gray-coded version would be as follows. Rotary encoder for angle-measuring devices marked in 3-bit binary-reflected Gray code (BRGC). The inner ring corresponds to Contact 1 in the table. Black sectors are "on". Zero degrees is on the right-hand side, with angle increasing anticlockwise. Rotary encoder 267 Gray Coding Sector Contact 1 Contact 2 Contact 3 Angle 1 off off off 0° to 45° 2 off off ON 45° to 90° 3 off ON ON 90° to 135° 4 off ON off 135° to 180° 5 ON ON off 180° to 225° 6 ON ON ON 225° to 270° 7 ON off ON 270° to 315° 8 ON off off 315° to 360° In this example, the transition from sector 4 to sector 5, like all other transitions, involves only one of the contacts changing its state from on to off or vice versa. This means that the sequence of incorrect codes shown in the previous illustration cannot happen. Single-track Gray encoding If the designer moves a contact to a different angular position (but at the same distance from the center shaft), then the corresponding "ring pattern" needs to be rotated the same angle to give the same output. If the most significant bit (the inner ring in Figure 1) is rotated enough, it exactly matches the next ring out. Since both rings are then identical, the inner ring can be omitted, and the sensor for that ring moved to the remaining, identical ring (but offset at that angle from the other sensor on that ring). Those two sensors on a single ring make a quadrature encoder. For many years, Torsten Sillke [2] and other mathematicians believed that it was impossible to encode position on a single track so that consecutive positions differed at only a single sensor, except for the two-sensor, one-track quadrature encoder. However, in 1994 N. B. Spedding registered a patent (NZ Patent 264738) showing it was possible with several examples. See Single-track Gray code for details. Absolute encoder output formats In commercial absolute encoders there are several formats for transmission of absolute encoder data, including parallel binary, SSI, "BiSS" [3], ISI, Profibus, DeviceNet, CANopen, Endat and Hiperface, depending on the manufacturer of the device. Incremental rotary encoder Encoder ROD 420 An incremental rotary encoder provides cyclical outputs (only) when the encoder is rotated. They can be either mechanical or optical. The mechanical type requires debouncing and is typically used as digital potentiometers on equipment including consumer devices. Most modern home and car stereos use mechanical rotary encoders for volume control. Due to the fact the mechanical switches require debouncing, the mechanical type are limited in the rotational speeds they can handle. The incremental rotary encoder is the most widely used of all rotary encoders due to its low cost and ability to provide signals that can be easily interpreted to provide motion related information such as velocity and RPM. Rotary encoder 268 The fact that incremental encoders use only two sensors does not compromise their accuracy. One can find in the market incremental encoders with up to 10,000 counts per revolution, or more. There can be an optional third output: reference, which happens once every turn. This is used when there is the need of an absolute reference, such as positioning systems. The optical type is used when higher RPMs are encountered or a higher degree of precision is required. Incremental encoders are used to track motion and can be used to determine position and velocity. This can be either linear or rotary motion. Because the direction can be determined, very accurate measurements can be made. They employ two outputs called A & B, which are called quadrature outputs, as they are 90 degrees out of phase. The state diagram: Phase A B Phase A B 1 0 0 1 1 0 2 0 1 2 1 1 3 1 1 3 0 1 4 1 0 4 0 0 The two output wave forms are 90 degrees out of phase, which is all that the quadrature term means. These signals are decoded to produce a count up pulse or a count down pulse. For decoding in software, the A & B outputs are read by software, either via an interrupt on any edge or polling, and the Two square waves in quadrature (clockwise rotation). above table is used to decode the direction. For example, if the last value was 00 and the current value is 01, the device has moved one half step in the clockwise direction. The mechanical types would be debounced first by requiring that the same (valid) value be read a certain number of times before recognizing a state change. On encoders with detents there are different ways to switch states. In some, both A and B are always open circuit at the detents, and an entire 00 → 00 switching cycle occurs while transitioning from one detent to the next. Others have detents of alternating 00 and 11 value, with staggered switching times during the transition between detents. If the encoder is turning too fast, an invalid transition may occur, such as 00 → 11. There is no way to know which way the encoder turned; if it was 00 → 01 → 11, or 00 → 10 → 11. If the encoder is turning even faster, a backward count may occur. Example: consider the 00 → 01 → 11 → 10 transition (3 steps forward). If the encoder is turning too fast, the system might read only the 00 and then the 10, which yields a 00 → 10 transition (1 step backward). This same principle is used in ball mice to track whether the mouse is moving to the right/left or forward/backward. Rotary encoders with a single output cannot be used to sense direction of motion. They are well-suited for systems that measure rate-of-movement variables such as velocity and RPM. In certain applications they may be used to measure distance of motion (e.g. feet of movement). Rotary encoder Incremental versus absolute encoder terminology There seem to be some grey areas as to what constitutes an incremental encoder as opposed to an absolute encoder. Traditional absolute encoders Traditional absolute encoders have multiple code rings with various binary weightings which provide a data word representing the absolute position of the encoder within one revolution. This type of encoder is often referred to as a parallel absolute encoder. The distinguishing feature of the absolute encoder is that it reports the absolute position of the encoder to the electronics immediately upon power-up with no need for indexing.[4] Traditional incremental encoders A traditional incremental encoder works differently by providing an A and a B pulse output that provide no usable count information in their own right. Rather, the counting is done in the external electronics. The point where the counting begins depends on the counter in the external electronics and not on the position of the encoder. To provide useful position information, the encoder position must be referenced to the device to which it is attached, generally using an index pulse. The distinguishing feature of the incremental encoder is that it reports an incremental change in position of the encoder to the counting electronics.[4] Battery backed incremental encoders Some encoder manufacturers, such as Fanuc, have taken a different approach to this terminology. These manufacturers use absolute as their terminology for incremental encoders with a battery backed up memory to store count information and provide an absolute count immediately upon power up.[4] Sine wave encoder A variation on the Incremental encoder is the Sinewave Encoder. Instead of producing two quadrature square waves, the outputs are quadrature sine waves (a Sine and a Cosine). By performing the arctangent function, arbitrary levels of resolution can be achieved. Use in industry Encoders used on servomotors Rotary encoders are often used to track the position of the motor shaft on permanent magnet brushless motors, which are commonly used on CNC machines, robots, and other industrial equipment. Incremental (Quadrature) encoders are used on Induction Motor type servomotors, but absolute encoders are used in Permanent Magnet Brushless Motors, where applicable. In these applications, the feedback device (encoder) plays a vital role in ensuring that the equipment operates properly. The encoder synchronizes the relative rotor magnet and stator winding positions to the current provided by the drive. Maximum torque results if the current is applied to the windings when the rotor magnets are in a particular position range relative to the stator windings. The motor will perform poorly or not at all if this timing is not adjusted correctly. Improper encoder alignment on the motor can actually cause it to run backwards sometimes resulting in a hazardous run away condition. Correct alignment is essential to proper operation of these motors.[5] 269 Rotary encoder 270 Encoder technologies Encoders may be implemented using a variety of technologies: • Conductive tracks. A series of copper pads etched onto a PCB is used to encode the information. Contact brushes sense the conductive areas. This form of encoder is now rarely seen. • Optical. This uses a light shining onto a photodiode through slits in a metal or glass disc. Reflective versions also exist. This is one of the most common technologies. • Magnetic. Strips of magnetised material are placed on the rotating disc and are sensed by a Hall-effect sensor or magnetoresistive sensor. Hall effect sensors are also used to sense gear teeth directly, without the need for a separate encoder disc. References [1] (http:/ / www. canon. com/ bctv/ faq/ rotary. html) Canon video camera lens, used for zoom and aperture control [2] http:/ / www. mathematik. uni-bielefeld. de/ ~sillke/ PROBLEMS/ gray [3] http:/ / www. biss-interface. com Hall-effect quadrature encoder, sensing gear teeth on the driveshaft of a robot vehicle. [4] TI-5000EX Serial/Incremental Encoder Test System User Manual (http:/ / www. mitchell-electronics. com/ downloads/ Catalog_PriceList/ TI5000EXManual. pdf), Mitchell Electronics, Inc. [5] PM Brushless Servo Motor Feedback Commutation Series – Part 1 (http:/ / www. mitchell-electronics. com/ downloads/ AN5000-PD01. pdf), Mitchell Electronics, Inc. External links • "Choosing a code wheel: A detailed look at how encoders work" (http://www.smallformfactors.com/articles/ id/?3039) article by Steve Trahey 2008-03-25 describes "rotary encoders". • "Encoders provide a sense of place" (http://embedded.com/showArticle.jhtml?articleID=166400808) article by Jack Ganssle 2005-07-19 describes "nonlinear encoders". • "Robot Encoders" (http://www.societyofrobots.com/sensors_encoder.shtml). • Introductory Tutorial (http://web.archive.org/web/20110717090211/http://www.embedded.com.au/pages/ Motor_Interface.html) on PWM and Quadrature Encoding • ProtoTalk.net - Understanding Quadrature Encoding (http://prototalk.net/forums/showthread.php?t=78) Covers details of rotary and quadrature encoding with a focus on robotic applications 271 The Frequency-Response Design Method Bode plot A Bode plot is a graph of the transfer function of a linear, time-invariant system versus frequency, plotted with a log-frequency axis, to show the system's frequency response. It is usually a combination of a Bode magnitude plot, expressing the magnitude of the frequency response gain, and a Bode phase plot, expressing the frequency response phase shift. Overview Among his several important contributions to circuit theory and control theory, engineer Hendrik Wade Bode (1905–1982), while working at Figure 1(a): The Bode plot for a first-order (one-pole) highpass filter; the straight-line Bell Labs in the United States in the approximations are labeled "Bode pole"; phase varies from 90° at low frequencies (due to 1930s, devised a simple but accurate the contribution of the numerator, which is 90° at all frequencies) to 0° at high frequencies (where the phase contribution of the denominator is −90° and cancels the method for graphing gain and contribution of the numerator). phase-shift plots. These bear his name, Bode gain plot and Bode phase plot (pronounced Boh-dee in English, Bow-duh in Dutch).[1] The magnitude axis of the Bode plot is usually expressed as decibels of power, that is by the 20 log rule: 20 times the common (base 10) logarithm of the amplitude gain. With the magnitude gain being logarithmic, Bode plots make multiplication of magnitudes a simple matter of adding distances on the graph (in decibels), since Bode plot 272 Figure 1(b): The Bode plot for a first-order (one-pole) lowpass filter; the straight-line approximations are labeled "Bode pole"; phase is 90° lower than for Figure 1(a) because the phase contribution of the numerator is 0° at all frequencies. A Bode phase plot is a graph of phase versus frequency, also plotted on a log-frequency axis, usually used in conjunction with the magnitude plot, to evaluate how much a signal will be phase-shifted. For example a signal described by: Asin(ωt) may be attenuated but also phase-shifted. If the system attenuates it by a factor x and phase shifts it by −Φ the signal out of the system will be (A/x) sin(ωt − Φ). The phase shift Φ is generally a function of frequency. Phase can also be added directly from the graphical values, a fact that is mathematically clear when phase is seen as the imaginary part of the complex logarithm of a complex gain. In Figure 1(a), the Bode plots are shown for the one-pole highpass filter function: where f is the frequency in Hz, and f1 is the pole position in Hz, f1 = 100 Hz in the figure. Using the rules for complex numbers, the magnitude of this function is while the phase is: Care must be taken that the inverse tangent is set up to return degrees, not radians. On the Bode magnitude plot, decibels are used, and the plotted magnitude is: In Figure 1(b), the Bode plots are shown for the one-pole lowpass filter function: Bode plot 273 Also shown in Figure 1(a) and 1(b) are the straight-line approximations to the Bode plots that are used in hand analysis, and described later. The magnitude and phase Bode plots can seldom be changed independently of each other — changing the amplitude response of the system will most likely change the phase characteristics and vice versa. For minimum-phase systems the phase and amplitude characteristics can be obtained from each other with the use of the Hilbert transform. If the transfer function is a rational function with real poles and zeros, then the Bode plot can be approximated with straight lines. These asymptotic approximations are called straight line Bode plots or uncorrected Bode plots and are useful because they can be drawn by hand following a few simple rules. Simple plots can even be predicted without drawing them. The approximation can be taken further by correcting the value at each cutoff frequency. The plot is then called a corrected Bode plot. Rules for hand-made Bode plot The premise of a Bode plot is that one can consider the log of a function in the form: as a sum of the logs of its poles and zeros: This idea is used explicitly in the method for drawing phase diagrams. The method for drawing amplitude plots implicitly uses this idea, but since the log of the amplitude of each pole or zero always starts at zero and only has one asymptote change (the straight lines), the method can be simplified. Straight-line amplitude plot Amplitude decibels is usually done using the where and are constants, version. Given a transfer function in the form , , and H is the transfer function: • at every value of s where (a zero), increase the slope of the line by per decade. • at every value of s where (a pole), decrease the slope of the line by per decade. • The initial value of the graph depends on the boundaries. The initial point is found by putting the initial angular frequency ω into the function and finding |H(jω)|. • The initial slope of the function at the initial value depends on the number and order of zeros and poles that are at values below the initial value, and are found using the first two rules. To handle irreducible 2nd order polynomials, can, in many cases, be approximated as . Note that zeros and poles happen when ω is equal to a certain or . This is because the function in question is the magnitude of H(jω), and since it is a complex function, is a zero or pole involving the . term . Thus at any place where there , the magnitude of that term is Bode plot 274 Corrected amplitude plot To correct a straight-line amplitude plot: • at every zero, put a point above the line, • at every pole, put a point below the line, • draw a smooth curve through those points using the straight lines as asymptotes (lines which the curve approaches). Note that this correction method does not incorporate how to handle complex values of or . In the case of an irreducible polynomial, the best way to correct the plot is to actually calculate the magnitude of the transfer function at the pole or zero corresponding to the irreducible polynomial, and put that dot over or under the line at that pole or zero. Straight-line phase plot Given a transfer function in the same form as above: the idea is to draw separate plots for each pole and zero, then add them up. The actual phase curve is given by . To draw the phase plot, for each pole and zero: • if A is positive, start line (with zero slope) at 0 degrees • if A is negative, start line (with zero slope) at 180 degrees • at every (for stable zeros – beginning one decade before • at every (E.g.: (E.g.: degrees per decade, ), decrease the slope by degrees per decade, ) (for stable poles – beginning one decade before ), increase the slope by ) • "unstable" (right half plane) poles and zeros ( • flatten the slope again when the phase has changed by ) have opposite behavior degrees (for a zero) or degrees (for a pole), • After plotting one line for each pole or zero, add the lines together to obtain the final phase plot; that is, the final phase plot is the superposition of each earlier phase plot. Example A passive (unity pass band gain) lowpass RC filter, for instance has the following transfer function expressed in the frequency domain: From the transfer function it can be determined that the cutoff frequency point fc (in hertz) is at the frequency or (equivalently) at where is the angular cutoff frequency in radians per second. The transfer function in terms of the angular frequencies becomes: Bode plot 275 The above equation is the normalized form of the transfer function. The Bode plot is shown in Figure 1(b) above, and construction of the straight-line approximation is discussed next. Magnitude plot The magnitude (in decibels) of the transfer function above, (normalized and converted to angular frequency form), given by the decibel gain expression : when plotted versus input frequency on a logarithmic scale, can be approximated by two lines and it forms the asymptotic (approximate) magnitude Bode plot of the transfer function: • for angular frequencies below it is a horizontal line at 0 dB since at low frequencies the term is small and can be neglected, making the decibel gain equation above equal to zero, • for angular frequencies above it is a line with a slope of −20 dB per decade since at high frequencies the term dominates and the decibel gain expression above simplifies to which is a straight line with a slope of −20 dB per decade. These two lines meet at the corner frequency. From the plot, it can be seen that for frequencies well below the corner frequency, the circuit has an attenuation of 0 dB, corresponding to a unity pass band gain, i.e. the amplitude of the filter output equals the amplitude of the input. Frequencies above the corner frequency are attenuated – the higher the frequency, the higher the attenuation. Phase plot The phase Bode plot is obtained by plotting the phase angle of the transfer function given by versus , where and lower than corner, the ratio are the input and cutoff angular frequencies respectively. For input frequencies much is small and therefore the phase angle is close to zero. As the ratio increases the absolute value of the phase increases and becomes –45 degrees when . As the ratio increases for input frequencies much greater than the corner frequency, the phase angle asymptotically approaches −90 degrees. The frequency scale for the phase plot is logarithmic. Bode plot 276 Normalized plot The horizontal frequency axis, in both the magnitude and phase plots, can be replaced by the normalized (nondimensional) frequency ratio . In such a case the plot is said to be normalized and units of the frequencies are no longer used since all input frequencies are now expressed as multiples of the cutoff frequency . An example with pole and zero Figures 2-5 further illustrate construction of Bode plots. This example with both a pole and a zero shows how to use superposition. To begin, the components are presented separately. Figure 2 shows the Bode magnitude plot for a zero and a low-pass pole, and compares the two with the Bode straight line plots. The straight-line plots are horizontal up to the pole (zero) location and then drop (rise) at 20 dB/decade. The second Figure 3 does the same for the phase. The phase plots are horizontal up to a frequency factor of ten below the pole (zero) location and then drop (rise) at 45°/decade until the frequency is ten times higher than the pole (zero) location. The plots then are again horizontal at higher frequencies at a final, total phase change of 90°. Figure 4 and Figure 5 show how superposition (simple addition) of a pole and zero plot is done. The Bode straight line plots again are compared with the exact plots. The zero has been moved to higher frequency than the pole to make a more interesting example. Notice in Figure 4 that the 20 dB/decade drop of the pole is arrested by the 20 dB/decade rise of the zero resulting in a horizontal magnitude plot for frequencies above the zero location. Notice in Figure 5 in the phase plot that the straight-line approximation is pretty approximate in the region where both pole and zero affect the phase. Notice also in Figure 5 that the range of frequencies where the phase changes in the straight line plot is limited to frequencies a factor of ten above and below the pole (zero) location. Where the phase of the pole and the zero both are present, the straight-line phase plot is horizontal because the 45°/decade drop of the pole is arrested by the overlapping 45°/decade rise of the zero in the limited range of frequencies where both are active contributors to the phase. Example with pole and zero Figure 2: Bode magnitude plot for zero and low-pass pole; curves labeled "Bode" are the straight-line Bode plots Figure 3: Bode phase plot for zero and low-pass pole; curves labeled "Bode" are the straight-line Bode plots Bode plot Figure 4: Bode magnitude plot for pole-zero combination; the location of the zero is ten times higher than in Figures 2&3; curves labeled "Bode" are the straight-line Bode plots 277 Figure 5: Bode phase plot for pole-zero combination; the location of the zero is ten times higher than in Figures 2&3; curves labeled "Bode" are the straight-line Bode plots Gain margin and phase margin Bode plots are used to assess the stability of negative feedback amplifiers by finding the gain and phase margins of an amplifier. The notion of gain and phase margin is based upon the gain expression for a negative feedback amplifier given by where AFB is the gain of the amplifier with feedback (the closed-loop gain), β is the feedback factor and AOL is the gain without feedback (the open-loop gain). The gain AOL is a complex function of frequency, with both magnitude and phase.[2] Examination of this relation shows the possibility of infinite gain (interpreted as instability) if the product βAOL = −1. (That is, the magnitude of βAOL is unity and its phase is −180°, the so-called Barkhausen stability criterion). Bode plots are used to determine just how close an amplifier comes to satisfying this condition. Key to this determination are two frequencies. The first, labeled here as f180, is the frequency where the open-loop gain flips sign. The second, labeled here f0dB, is the frequency where the magnitude of the product | β AOL | = 1 (in dB, magnitude 1 is 0 dB). That is, frequency f180 is determined by the condition: where vertical bars denote the magnitude of a complex number (for example, | a + j b | = [ a2 + b2]1/2 ), and frequency f0dB is determined by the condition: One measure of proximity to instability is the gain margin. The Bode phase plot locates the frequency where the phase of βAOL reaches −180°, denoted here as frequency f180. Using this frequency, the Bode magnitude plot finds the magnitude of βAOL. If |βAOL|180 = 1, the amplifier is unstable, as mentioned. If |βAOL|180 < 1, instability does not occur, and the separation in dB of the magnitude of |βAOL|180 from |βAOL| = 1 is called the gain margin. Because a magnitude of one is 0 dB, the gain margin is simply one of the equivalent forms: 20 log10( |βAOL|180) = 20 log10( |AOL|180) − 20 log10( 1 / β ). Another equivalent measure of proximity to instability is the phase margin. The Bode magnitude plot locates the frequency where the magnitude of |βAOL| reaches unity, denoted here as frequency f0dB. Using this frequency, the Bode phase plot finds the phase of βAOL. If the phase of βAOL( f0dB) > −180°, the instability condition cannot be met at any frequency (because its magnitude is going to be < 1 when f = f180), and the distance of the phase at f0dB in degrees above −180° is called the phase margin. If a simple yes or no on the stability issue is all that is needed, the amplifier is stable if f0dB < f180. This criterion is sufficient to predict stability only for amplifiers satisfying some restrictions on their pole and zero positions (minimum phase systems). Although these restrictions usually are met, if they are not another method must be used, such as the Nyquist plot.[3] [4] Bode plot Examples using Bode plots Figures 6 and 7 illustrate the gain behavior and terminology. For a three-pole amplifier, Figure 6 compares the Bode plot for the gain without feedback (the open-loop gain) AOL with the gain with feedback AFB (the closed-loop gain). See negative feedback amplifier for more detail. In this example, AOL = 100 dB at low frequencies, and 1 / β = 58 dB. At low frequencies, AFB ≈ 58 dB as well. Because the open-loop gain AOL is plotted and not the product β AOL, the condition AOL = 1 / β decides f0dB. The feedback gain at low frequencies and for large AOL is AFB ≈ 1 / β (look at the formula for the feedback gain at the beginning of this section for the case of large gain AOL), so an equivalent way to find f0dB is to look where the feedback gain intersects the open-loop gain. (Frequency f0dB is needed later to find the phase margin.) Near this crossover of the two gains at f0dB, the Barkhausen criteria are almost satisfied in this example, and the feedback amplifier exhibits a massive peak in gain (it would be infinity if β AOL = −1). Beyond the unity gain frequency f0dB, the open-loop gain is sufficiently small that AFB ≈ AOL (examine the formula at the beginning of this section for the case of small AOL). Figure 7 shows the corresponding phase comparison: the phase of the feedback amplifier is nearly zero out to the frequency f180 where the open-loop gain has a phase of −180°. In this vicinity, the phase of the feedback amplifier plunges abruptly downward to become almost the same as the phase of the open-loop amplifier. (Recall, AFB ≈ AOL for small AOL.) Comparing the labeled points in Figure 6 and Figure 7, it is seen that the unity gain frequency f0dB and the phase-flip frequency f180 are very nearly equal in this amplifier, f180 ≈ f0dB ≈ 3.332 kHz, which means the gain margin and phase margin are nearly zero. The amplifier is borderline stable. Figures 8 and 9 illustrate the gain margin and phase margin for a different amount of feedback β. The feedback factor is chosen smaller than in Figure 6 or 7, moving the condition | β AOL | = 1 to lower frequency. In this example, 1 / β = 77 dB, and at low frequencies AFB ≈ 77 dB as well. Figure 8 shows the gain plot. From Figure 8, the intersection of 1 / β and AOL occurs at f0dB = 1 kHz. Notice that the peak in the gain AFB near f0dB is almost gone.[5] [6] Figure 9 is the phase plot. Using the value of f0dB = 1 kHz found above from the magnitude plot of Figure 8, the open-loop phase at f0dB is −135°, which is a phase margin of 45° above −180°. Using Figure 9, for a phase of −180° the value of f180 = 3.332 kHz (the same result as found earlier, of course[7] ). The open-loop gain from Figure 8 at f180 is 58 dB, and 1 / β = 77 dB, so the gain margin is 19 dB. Stability is not the sole criterion for amplifier response, and in many applications a more stringent demand than stability is good step response. As a rule of thumb, good step response requires a phase margin of at least 45°, and often a margin of over 70° is advocated, particularly where component variation due to manufacturing tolerances is an issue.[8] See also the discussion of phase margin in the step response article. 278 Bode plot 279 Examples Figure 6: Gain of feedback amplifier AFB in dB and corresponding open-loop amplifier AOL. Parameter 1/β = 58 dB, and at low frequencies AFB ≈ 58 dB as well. The gain margin in this amplifier is nearly zero because | βAOL| = 1 occurs at almost f = f180°. Figure 7: Phase of feedback amplifier °AFB in degrees and corresponding open-loop amplifier °AOL. The phase margin in this amplifier is nearly zero because the phase-flip occurs at almost the unity gain frequency f = f0dB where | βAOL| = 1. Figure 8: Gain of feedback amplifier AFB in dB and corresponding open-loop amplifier AOL. In this example, 1 / β = 77 dB. The gain margin in this amplifier is 19 dB. Figure 9: Phase of feedback amplifier AFB in degrees and corresponding open-loop amplifier AOL. The phase margin in this amplifier is 45°. Bode plotter The Bode plotter is an electronic instrument resembling an oscilloscope, which produces a Bode diagram, or a graph, of a circuit's voltage gain or phase shift plotted against frequency in a feedback control system or a filter. An example of this is shown in Figure 10. It is extremely useful for analyzing and testing filters and the stability of feedback control systems, through the measurement of corner (cutoff) frequencies and gain and phase margins. Figure 10: Amplitude diagram of a 10th order Chebyshev filter plotted using a Bode Plotter application. The chebyshev transfer function is defined by poles and zeros which are added by clicking on a graphical complex diagram. This is identical to the function performed by a vector network analyzer, but the network analyzer is typically used at much higher frequencies. For education/research purposes, plotting Bode diagrams for given transfer functions facilitates better understanding and getting faster results (see external links). Bode plot 280 Related plots Two related plots that display the same data in different coordinate systems are the Nyquist plot and the Nichols plot. These are parametric plots, with frequency as the input and magnitude and phase of the frequency response as the output. The Nyquist plot displays these in polar coordinates, with magnitude mapping to radius and phase to argument (angle). The Nichols plot displays these in rectangular coordinates, on the log scale. Related Plots A Nyquist plot. A Nichols plot of the same response. Notes [1] Van Valkenburg, M. E. University of Illinois at Urbana-Champaign, "In memoriam: Hendrik W. Bode (1905-1982)", IEEE Transactions on Automatic Control, Vol. AC-29, No 3., March 1984, pp. 193-194. Quote: "Something should be said about his name. To his colleagues at Bell Laboratories and the generations of engineers that have followed, the pronunciation is boh-dee. The Bode family preferred that the original Dutch be used as boh-dah." [2] Ordinarily, as frequency increases the magnitude of the gain drops and the phase becomes more negative, although these are only trends and may be reversed in particular frequency ranges. Unusual gain behavior can render the concepts of gain and phase margin inapplicable. Then other methods such as the Nyquist plot have to be used to assess stability. [3] Thomas H. Lee (2004). The design of CMOS radio-frequency integrated circuits (http:/ / worldcat. org/ isbn/ 0-521-83539-9) (Second Edition ed.). Cambridge UK: Cambridge University Press. p. §14.6 pp. 451–453. ISBN 0-521-83539-9. . [4] William S Levine (1996). The control handbook: the electrical engineering handbook series (http:/ / books. google. com/ books?id=2WQP5JGaJOgC& pg=RA1-PA163& lpg=RA1-PA163& dq=stability+ "minimum+ phase"& source=web& ots=P3fFTcyfzM& sig=ad5DJ7EvVm6In_zhI0MlF_6vHDA) (Second Edition ed.). Boca Raton FL: CRC Press/IEEE Press. p. §10.1 p. 163. ISBN 0849385709. . [5] The critical amount of feedback where the peak in the gain just disappears altogether is the maximally flat or Butterworth design. [6] Willy M C Sansen (2006). Analog design essentials (http:/ / worldcat. org/ isbn/ 0-387-25746-2). Dordrecht, The Netherlands: Springer. p. §0517-§0527 pp. 157–163. ISBN 0-387-25746-2. . [7] The frequency where the open-loop gain flips sign f180 does not change with a change in feedback factor; it is a property of the open-loop gain. The value of the gain at f180 also does not change with a change in β. Therefore, we could use the previous values from Figures 6 and 7. However, for clarity the procedure is described using only Figures 8 and 9. [8] Willy M C Sansen. §0526 p. 162 (http:/ / worldcat. org/ isbn/ 0-387-25746-2). ISBN 0-387-25746-2. . Bode plot References External links • Explanation of Bode plots with movies and examples (http://www.facstaff.bucknell.edu/mastascu/ eControlHTML/Freq/Freq5.html) • How to draw piecewise asymptotic Bode plots (http://lpsa.swarthmore.edu/Bode/BodeHow.html) • Summarized drawing rules (http://lims.mech.northwestern.edu/~lynch/courses/ME391/2003/bodesketching. pdf) (PDF) • Bode plot applet (http://www.uwm.edu/People/msw/BodePlot/) - Accepts transfer function coefficients as input, and calculates magnitude and phase response • Circuit analysis in electrochemistry (http://www.abc.chemistry.bsu.by/vi/fit.htm) • Tim Green: Operational amplifier stability (http://www.en-genius.net/includes/files/acqt_013105.pdf) Includes some Bode plot introduction • Gnuplot code for generating Bode plot: DIN-A4 printing template (pdf) Nyquist stability criterion When designing a feedback control system, it is generally necessary to determine whether the closed-loop system will be stable. An example of a destabilizing feedback control system would be a car steering system that overcompensates -- if the car drifts in one direction, the control system overcorrects in the opposite direction, and even further back in the first, until the car goes off the road. In contrast, for a stable system the vehicle would continue to track the control input. The Nyquist stability criterion, named after Harry Nyquist, is a graphical technique for determining the stability of a system. Because it only looks at the Nyquist plot of the open loop systems, it can be applied without explicitly computing the poles and zeros of either the closed-loop or open-loop The Nyquist plot for . system (although the number of each type of right-half-plane singularities must be known). As a result, it can be applied to systems defined by non-rational functions, such as systems with delays. In contrast to Bode plots, it can handle transfer functions with right half-plane singularities. In addition, there is a natural generalization to more complex systems with multiple inputs and multiple outputs, such as control systems for airplanes. While Nyquist is one of the most general stability tests, it is still restricted to linear, time-invariant systems. Non-linear systems must use more complex stability criteria, such as Lyapunov. While Nyquist is a graphical technique, it only provides a limited amount of intuition for why a system is stable or unstable, or how to modify an 281 Nyquist stability criterion 282 unstable system to be stable. Techniques like Bode plots, while less general, are sometimes a more useful design tool. Background We consider a system whose open loop transfer function (OLTF) is feedback ; when placed in a closed loop with , the closed loop transfer function (CLTF) then becomes . The case where H=1 is usually taken, when investigating stability, and then the characteristic equation, used to predict stability, becomes . Stability can be determined by examining the roots of this equation e.g. using the Routh array, but this method is somewhat tedious. Conclusions can also be reached by examining the OLTF, using its Bode plots or, as here, polar plot of the OLTF using the Nyquist criterion, as follows. Any Laplace domain transfer function can be expressed as the ratio of two polynomials We define: • Zero: the zeros of are the roots of • Pole: the poles of are the roots of Stability of , and . is determined by its poles or simply the roots of the characteristic equation: stability, the real part of every pole must be negative. If around the open-loop transfer function zeros of . For is formed by closing a negative unity feedback loop , then the roots of the characteristic equation are also the , or simply the roots of . Cauchy's argument principle From complex analysis, specifically the argument principle, we know that a contour drawn in the complex plane, encompassing but not passing through any number of zeros and poles of a function another plane (the plane plane) by the function times, where inside the contour . . The resulting contour and , can be mapped to will encircle the origin of the are respectively the number of zeros and poles of . Note that we count encirclements in the plane in the same sense as the contour and that encirclements in the opposite direction are negative encirclements. Instead of Cauchy's argument principle, the original paper by Harry Nyquist in 1932 uses a less elegant approach. The approach explained here is similar to the approach used by Leroy MacColl (Fundamental theory of servomechanisms 1945) or by Hendrik Bode (Network analysis and feedback amplifier design 1945), both of whom also worked for Bell Laboratories. This approach appears in most modern textbooks on control theory. The Nyquist criterion We first construct The Nyquist Contour, a contour that encompasses the right-half of the complex plane: • a path traveling up the axis, from • a semicircular arc, with radius to , that starts at The Nyquist Contour mapped through the function . and travels clock-wise to yields a plot of . in the complex plane. By the Argument Principle, the number of clock-wise encirclements of the origin must be the number of zeros of in the right-half complex plane minus the poles of the contour is mapped through the open-loop transfer function in the right-half complex plane. If instead, , the result is the Nyquist Plot of . By counting the resulting contour's encirclements of -1, we find the difference between the number of poles and zeros in Nyquist stability criterion 283 the right-half complex plane of the poles of are same as the poles of Given a Nyquist contour zeros of . Recalling that the zeros of , let encircled by are the poles of the closed-loop system, and noti , we now state The Nyquist Criterion: be the number of poles of encircled by . Alternatively, and more importantly, loop system in the right half plane. The resultant contour in the , and be the number of is the number of poles of the closed -plane, shall encircle (clock-wise) the point times such that . If the system is originally open-loop unstable, feedback is necessary to stabilize the system. Right-half-plane (RHP) poles represent that instability. For closed-loop stability of a system, the number of closed-loop roots in the right half of the s-plane must be zero. Hence, the number of counter-clockwise encirclements about must be equal to the number of open-loop poles in the RHP. Any clockwise encirclements of the critical point by the open-loop frequency response (when judged from low frequency to high frequency) would indicate that the feedback control system would be destabilizing if the loop were closed. (Using RHP zeros to "cancel out" RHP poles does not remove the instability, but rather ensures that the system will remain unstable even in the presence of feedback, since the closed-loop roots travel between open-loop poles and zeros in the presence of feedback. In fact, the RHP zero can make the unstable pole unobservable and therefor not stabilizable through feedback.) The Nyquist criterion for systems with poles on the imaginary axis The above consideration was conducted with an assumption that the open-loop transfer function have any pole on the imaginary axis (i.e. poles of the form does not ). This results from the requirement of the argument principle that the contour cannot pass through any pole of the mapping function. The most common case are systems with integrators (poles at zero). To be able to analyze systems with poles on the imaginary axis, the Nyquist Contour can be modified to avoid passing through the point . One way to do it is to construct a semicircular arc with radius around , that starts at that the phasor and travels anticlockwise to travels along an arc of infinite radius by . Such a modification implies , where is the multiplicity of the pole on the imaginary axis. Summary • If the open-loop transfer function discontinuity at has a zero pole of multiplicity , then the Nyquist plot has a . During further analysis it should be assumed that the phasor travels times clock-wise along a semicircle of infinite radius. After applying this rule, the zero poles should be neglected, i.e. if there are no other unstable poles, then the open-loop transfer function should be considered stable. • If the open-loop transfer function is stable, then the closed-loop system is unstable for any encirclement of the point -1. • If the open-loop transfer function is unstable, then there must be one counter clock-wise encirclement of -1 for each pole of in the right-half of the complex plane. • The number of surplus encirclements (greater than N+P) is exactly the number of unstable poles of the closed-loop system • However, if the graph happens to pass through the point , then deciding upon even the marginal stability of the system becomes difficult and the only conclusion that can be drawn from the graph is that there exist zeros on the axis. Nyquist stability criterion 284 References • • • • Faulkner, E.A. (1969): Introduction to the Theory of Linear Systems; Chapman & Hall; ISBN 0-412-09400-2 Pippard, A.B. (1985): Response & Stability; Cambridge University Press; ISBN 0-521-31994-3 Gessing, R. (2004): Control fundamentals; Silesian University of Technology; ISBN 83-7335-176-0 Franklin, G. (2002): Feedback Control of Dynamic Systems; Prentice Hall, ISBN 0-130-32393-4 Instability In numerous fields of study, the component of instability within a system is generally characterized by some of the outputs or internal states growing without bounds. Not all systems that are not stable are unstable; systems can also be marginally stable or exhibit limit cycle behavior. In control theory, a system is unstable if any of the roots of its A ball on the top of a hill is an unstable situation. characteristic equation has real part greater than zero (or if zero is a repeated root). This is equivalent to any of the eigenvalues of the state matrix having either real part greater than zero, or, for the eigenvalues on the imaginary axis, the algebraic multiplicity being larger than the geometric multiplicity. In structural engineering, a structure can become unstable when excessive load is applied. Beyond a certain threshold, structural deflections magnify stresses, which in turn increases deflections. This can take the form of buckling or crippling. The general field of study is called structural stability. Atmospheric instability is a major component of all weather systems on Earth. Fluid instabilities Fluid instabilities occur in liquids, gases and plasmas, and are often characterized by the shape that form; they are studied in fluid dynamics and magnetohydrodynamics. Fluid instabilities include: • Ballooning mode instability (some analogy to the Rayleigh–Taylor instability); found in the magnetosphere • Atmospheric instability • Hydrodynamic instability or dynamic instability (atmospheric dynamics) • Inertial instability; baroclinic instability; symmetric instability, conditional symmetric or convective symmetric instability; barotropic instability; Helmholtz or shearing instability; rotational instability Hydrodynamics simulation of the [1] Rayleigh–Taylor instability • Hydrostatic instability or static instability/vertical instability (parcel instability), thermodynamic instability (atmospheric thermodynamics) • Conditional or static instability, buoyant instability, latent instability, nonlocal static instability, conditional-symmetric Instability 285 instability; convective, potential, or thermal instability, convective instability of the first and second kind; absolute or mechanical instability • Bénard instability • Drift mirror instability • Kelvin–Helmholtz instability (similar, but different from the diocotron instability in plasmas) • Rayleigh–Taylor instability • Plateau-Rayleigh instability (similar to the Rayleigh–Taylor instability) • Richtmyer-Meshkov instability (similar to the Rayleigh–Taylor instability) Plasma instabilities Plasma instabilities can be divided into two general groups (1) hydrodynamic instabilities (2) kinetic instabilities. Plasma instabilities are also categorised into different modes - see this paragraph in plasma stability. Unstable flow structure generated from the collision of two impinging jets. Instabilities of stellar systems Galaxies and star clusters can be unstable, if small perturbations in the gravitational potential cause changes in the density that reinforce the original perturbation. Such instabilities usually require that the motions of stars be highly correlated, so that the perturbation is not "smeared out" by random motions. After the instability has run its course, the system is typically "hotter" (the motions are more random) or rounder than before. Instabilities in stellar systems include: • Bar instability of rapidly-rotating disks • Jeans instability • • • • Firehose instability[2] Gravothermal instability Radial-orbit instability Various instabilities in cold rotating disks Notes [1] Shengtai Li, Hui Li "Parallel AMR Code for Compressible MHD or HD Equations" (Los Alamos National Laboratory) (http:/ / math. lanl. gov/ Research/ Highlights/ amrmhd. shtml) [2] Merritt, D.; Sellwood, J. (1994), "Bending Instabilities of Stellar Systems", The Astrophysical Journal 425: 551–567, Bibcode 1994ApJ...425..551M, doi:10.1086/174005 External links • eFluids Fluid Flow Image Gallery (http://www.efluids.com/efluids/pages/gallery.htm) Frequency response 286 Frequency response Frequency response is the quantitative measure of the output spectrum of a system or device in response to a stimulus, and is used to characterize the dynamics of the system. It is a measure of magnitude and phase of the output as a function of frequency, in comparison to the input. In simplest terms, if a sine wave is injected into a system at a given frequency, a linear system will respond at that same frequency with a certain magnitude and a certain phase angle relative to the input. Also for a linear system, doubling the amplitude of the input will double the amplitude of the output. In addition, if the system is time-invariant, then the frequency response also will not vary with time. Two applications of frequency response analysis are related but have different objectives. For an audio system, the objective may be to reproduce the input signal with no distortion. That would require a uniform (flat) magnitude of response up to the bandwidth limitation of the system, with the signal delayed by precisely the same amount of time at all frequencies. That amount of time could be seconds, or weeks or months in the case of recorded media. In contrast, for a feedback apparatus used to control a dynamical system, the objective is to give the closed-loop system improved response as compared to the uncompensated system. The feedback generally needs to respond to system dynamics within a very small number of cycles of oscillation (usually less than one full cycle), and with a definite phase angle relative to the commanded control input. For feedback of sufficient amplification, getting the phase angle wrong can lead to instability for an open-loop stable system, or failure to stabilize a system that is open-loop unstable. Digital filters may be used for both audio systems and feedback control systems, but since the objectives are different, generally the phase characteristics of the filters will be significantly different for the two applications. Estimation and plotting Estimating the frequency response for a physical system generally involves exciting the system with an input signal, measuring both input and output time histories, and comparing the two through a process such as the Fast Fourier Transform (FFT). One thing to keep in mind for the analysis is that the frequency content of the input signal must cover the frequency range of interest or the results will not be valid for the portion of the frequency range not covered. The frequency response of a system can be measured by applying a test signal, for example: Frequency response of a low pass filter with 6 dB per octave or 20 dB per decade • applying an impulse to the system and measuring its response (see impulse response) • sweeping a constant-amplitude pure tone through the bandwidth of interest and measuring the output level and phase shift relative to the input • applying a signal with a wide frequency spectrum (for example digitally-generated maximum length sequence noise, or analog filtered white noise equivalent, like pink noise), and calculating the impulse response by deconvolution of this input signal and the output signal of the system. The frequency response is characterized by the magnitude of the system's response, typically measured in decibels (dB) or as a decimal, and the phase, measured in radians or degrees, versus frequency in radians/sec or Hertz (Hz). Frequency response These response measurements can be plotted in three ways: by plotting the magnitude and phase measurements on two rectangular plots as functions of frequency to obtain a Bode plot; by plotting the magnitude and phase angle on a single polar plot with frequency as a parameter to obtain a Nyquist plot; or by plotting magnitude and phase on a single rectangular plot with frequency as a parameter to obtain a Nichols plot. For audio systems with nearly uniform time delay at all frequencies, the magnitude versus frequency portion of the Bode plot may be all that is of interest. For design of control systems, any of the three types of plots [Bode, Nyquist, Nichols] can be used to infer closed-loop stability and stability margins (gain and phase margins) from the open-loop frequency response, provided that for the Bode analysis the phase-versus-frequency plot is included. Applications In electronics this stimulus would be an input signal.[1] In the audible range it is usually referred to in connection with electronic amplifiers, microphones and loudspeakers. Radio spectrum frequency response can refer to measurements of coaxial cable, twisted-pair cable, video switching equipment, wireless communications devices, and antenna systems. Infrasonic frequency response measurements include earthquakes and electroencephalography (brain waves). Frequency response requirements differ depending on the application.[2] In high fidelity audio, an amplifier requires a frequency response of at least 20–20,000 Hz, with a tolerance as tight as ±0.1 dB in the mid-range frequencies around 1000 Hz, however, in telephony, a frequency response of 400–4,000 Hz, with a tolerance of ±1 dB is sufficient for intelligibility of speech.[2] Frequency response curves are often used to indicate the accuracy of electronic components or systems.[1] When a system or component reproduces all desired input signals with no emphasis or attenuation of a particular frequency band, the system or component is said to be "flat", or to have a flat frequency response curve.[1] Once a frequency response has been measured (e.g., as an impulse response), providing the system is linear and time-invariant, its characteristic can be approximated with arbitrary accuracy by a digital filter. Similarly, if a system is demonstrated to have a poor frequency response, a digital or analog filter can be applied to the signals prior to their reproduction to compensate for these deficiencies. References Notes [1] Stark, 2002, p. 51. [2] Luther, 1999, p. 141. Bibliography • Luther, Arch C.; Inglis, Andrew F. Video engineering (http://books.google.com/books?id=VRailj6TKqUC), McGraw-Hill, 1999. ISBN 0-07-135017-9 • Stark, Scott Hunter. Live Sound Reinforcement (http://books.google.com/books?id=7QOcDeGFx4UC), Vallejo, California, Artistpro.com, 1996–2002. ISBN 0-918371-07-4 287 Frequency response 288 External links • University of Michigan: Frequency Response Analysis and Design Tutorial (http://www.engin.umich.edu/ group/ctm/freq/freq.html) • Smith, Julius O. III: Introduction to Digital Filters with Audio Applications (http://ccrma.stanford.edu/~jos/ filters/) has a nice chapter on Frequency Response (http://ccrma.stanford.edu/~jos/filters/ Frequency_Response_I.html) Nyquist plot A Nyquist plot is a parametric plot of a transfer function used in automatic control and signal processing. The most common use of Nyquist plots is for assessing the stability of a system with feedback. In Cartesian coordinates, the real part of the transfer function is plotted on the X axis. The imaginary part is plotted on the Y axis. The frequency is swept as a parameter. Alternatively, in polar coordinates, the gain of the transfer function is plotted as the radial coordinate, while the phase of the transfer function is plotted as the angular coordinate. The Nyquist plot is named after Harry Nyquist, a former engineer at Bell Laboratories. A Nyquist plot. Uses Assessment of the stability of a closed-loop negative feedback system is done by applying the Nyquist stability criterion to the Nyquist plot of the open-loop system (i.e. the same system without its feedback loop). This method is easily applicable even for systems with delays and other non-rational transfer functions, which may appear difficult to analyze by means of other methods. Stability is determined by looking at the number of encirclements of the point at (-1,0). Range of gains over which the system will be stable can be determined by looking at crossing of the real axis. The Nyquist plot can provide some information about the shape of the transfer function. For instance, the plot provides information on the difference between the number of poles and zeros of the transfer function[1] by the angle at which the curve approaches the origin. When drawn by hand, a cartoon version of the Nyquist plot is sometimes used, which shows the shape of the curve, but where coordinates are distorted to show more detail in regions of interest. When plotted computationally, one needs to be careful to cover all frequencies of interest. This typically means that the parameter is swept logarithmically, in order to cover a wide range of values. Nyquist plot 289 References [1] Nyquist Plots (http:/ / www. facstaff. bucknell. edu/ mastascu/ econtrolhtml/ Freq/ Freq6. html) External links • Applets with modifiable parameters (http://controlcan.homestead.com/files/idxpages.htm) • EIS Spectrum Analyser - a freeware program for analysis and simulation of impedance spectra (http://www.abc. chemistry.bsu.by/vi/analyser/) Nichols plot The Nichols plot is a plot used in signal processing and control design [1] [2] [3] . Use in Control Design Given a transfer function, A Nichols plot. with the closed-loop transfer function defined as, the Nichols plots displays versus . Loci of constant and are overlaid to allow the designer to obtain the closed loop transfer function directly from the open loop transfer function. Thus, the frequency is the parameter along the curve. This plot may be compared to the Bode plot in which the two inter-related graphs versus and versus ) - are plotted. Note that the decibel convention shown above, is not unique; another convention sometimes used is . In feedback control design, the plot is useful for assessing the stability and robustness of a linear system. This application of the Nichols plot is central to the Quantitative feedback theory (QFT) of Horowitz and Sidi, which is a well known method for robust control system design. Nichols plot 290 References [1] Isaac M. Howowitz, Synthesis of Feedback Systems, Academic Press, 1963, Lib Congress 63-12033 p. 194-198 [2] Boris J. Lurie and Paul J. Enright, Classical Feedback Control, Marcel Dekker, 2000, ISBN 0-8247-0370-7 p. 10 [3] Allen Stubberud, Ivan Williams, and Joseph DeStefano, Shaums Outline Feedback and Control Systems, McGraw-Hill, 1995, ISBN-10: 0070170525 ch. 17 Bandwidth (signal processing) Bandwidth is the difference between the upper and lower frequencies in a contiguous set of frequencies. It is typically measured in hertz, and may sometimes refer to passband bandwidth, sometimes to baseband bandwidth, depending on context. Passband bandwidth is the difference between the upper and lower cutoff frequencies of, for example, an electronic filter, a communication channel, or a signal spectrum. In case of a low-pass filter or baseband signal, the bandwidth is equal to its upper cutoff frequency. The term baseband bandwidth always refers to the upper cutoff frequency, regardless of whether the filter is bandpass or low-pass. Baseband bandwidth. Here the bandwidth equals the upper frequency. Bandwidth in hertz is a central concept in many fields, including electronics, information theory, radio communications, signal processing, and spectroscopy. A key characteristic of bandwidth is that a band of a given width can carry the same amount of information, regardless of where that band is located in the frequency spectrum (assuming equivalent noise level). For example, a 5 kHz band can carry a telephone conversation whether that band is at baseband (as in your POTS telephone line) or modulated to some higher (passband) frequency. In computer networking and other digital fields, the term bandwidth often refers to a data rate measured in bits per second, for example network throughput, sometimes denoted network bandwidth, data bandwidth or digital bandwidth. The reason is that according to Hartley's law, the digital data rate limit (or channel capacity) of a physical communication link is proportional to its bandwidth in hertz, sometimes denoted radio frequency (RF) bandwidth, signal bandwidth, frequency bandwidth, spectral bandwidth or analog bandwidth. For bandwidth as a computing term, less ambiguous terms are bit rate, throughput, maximum throughput, goodput or channel capacity. Overview Bandwidth is a key concept in many telephony applications. In radio communications, for example, bandwidth is the frequency range occupied by a modulated carrier wave, whereas in optics it is the width of an individual spectral line or the entire spectral range. In many signal processing contexts, bandwidth is a valuable and limited resource. For example, an FM radio receiver's tuner spans a limited range of frequencies. A government agency (such as the Federal Communications Commission in the United States) may apportion the regionally available bandwidth to broadcast license holders so that their signals do not mutually interfere. Each transmitter owns a slice of bandwidth, a valuable (if intangible) commodity. Bandwidth (signal processing) For different applications there are different precise definitions. For example, one definition of bandwidth could be the range of frequencies beyond which the frequency function is zero. This would correspond to the mathematical notion of the support of a function (i.e., the total "length" of values for which the function is nonzero). A less strict and more practically useful definition will refer to the frequencies where the frequency function is small. Small could mean less than 3 dB below (i.e., power output < 1/2 or voltage output < 0.707 of) the maximum value, or more rarely 10 dB below, or it could mean below a certain absolute value. As with any definition of the width of a function, many definitions are suitable for different purposes. Bandwidth typically refers to baseband bandwidth in the context of, for example, sampling theorem and Nyquist sampling rate, while it refers to passband bandwidth in the context of Nyquist symbol rate or Shannon-Hartley channel capacity for communication systems. X-dB bandwidth In some contexts, the signal bandwidth in hertz refers to the frequency range in which the signal's spectral density is nonzero or above a small threshold value. That definition is used in calculations of the lowest sampling rate that will satisfy the sampling theorem. Because this range of non-zero amplitude may be very broad or infinite, this definition is typically relaxed so that the bandwidth is defined as the range of frequencies in which the signal's spectral density is above a certain threshold relative to its maximum. Most commonly, A graph of a bandpass filter's gain magnitude, illustrating the concept of −3 dB bandwidth refers to the 3-dB bandwidth, bandwidth at a gain of 0.707. The frequency axis of this symbolic diagram can be that is, the frequency range within which the linear or logarithmically scaled. spectral density (in W/Hz or V2/Hz) is above half its maximum value (or the spectral amplitude, in V or V/Hz, is more than 70.7% of its maximum); that is, above −3 dB relative to the peak.[1] The word bandwidth applies to signals as described above, but it could also apply to systems, for example filters or communication channels. To say that a system has a certain bandwidth means that the system can process signals of that bandwidth, or that the system reduces the bandwidth of a white noise input to that bandwidth. The 3 dB bandwidth of an electronic filter or communication channel is the part of the system's frequency response that lies within 3 dB of the response at its peak, which in the passband filter case is typically at or near its center frequency, and in the lowpass filter is near 0 hertz. If the maximum gain is 0 dB, the 3 dB gain is the range where the gain is more than -3dB, or the attenuation is less than + 3dB. This is also the range of frequencies where the amplitude gain is above 70.7% of the maximum amplitude gain, and above half the maximum power gain. This same "half power gain" convention is also used in spectral width, and more generally for extent of functions as full width at half maximum (FWHM). In electronic filter design, a filter specification may require that within the filter passband, the gain is nominally 0 dB +/- a small number of dB, for example within the +/- 1 dB interval. In the stopband(s), the required attenuation in dB is above a certain level, for example >100 dB. In a transition band the gain is not specified. In this case, the filter bandwidth corresponds to the passband width, which in this example is the 1dB-bandwidth. If the filter shows amplitude ripple within the passband, the x dB point refers to the point where the gain is x dB below the nominal 291 Bandwidth (signal processing) 292 passband gain rather than x dB below the maximum gain. A commonly used quantity is fractional bandwidth. This is the bandwidth of a device divided by its center frequency. E.g., a passband filter that has a bandwidth of 2 MHz with center frequency 10 MHz will have a fractional bandwidth of 2/10, or 20%. In communication systems, in calculations of the Shannon–Hartley channel capacity, bandwidth refers to the 3dB-bandwidth. In calculations of the maximum symbol rate, the Nyquist sampling rate, and maximum bit rate according to the Hartley formula, the bandwidth refers to the frequency range within which the gain is non-zero, or the gain in dB is below a very large value. The fact that in equivalent baseband models of communication systems, the signal spectrum consists of both negative and positive frequencies, can lead to confusion about bandwidth, since they are sometimes referred to only by the positive half, and one will occasionally see expressions such as , where is the total bandwidth (i.e. the maximum passband bandwidth of the carrier-modulated RF signal and the minimum passband bandwidth of the physical passband channel), and is the positive bandwidth (the baseband bandwidth of the equivalent channel model). For instance, the baseband model of the signal would require a lowpass filter with cutoff frequency of at least to stay intact, and the physical passband channel would require a passband filter of at least to stay intact. In signal processing and control theory the bandwidth is the frequency at which the closed-loop system gain drops 3 dB below peak. In basic electric circuit theory, when studying band-pass and band-reject filters, the bandwidth represents the distance between the two points in the frequency domain where the signal is of the maximum signal amplitude (half power). Antenna systems In the field of antennas, two different methods of expressing relative bandwidth are used for narrowband and wideband antennas.[2] For either, a set of criteria is established to define the extents of the bandwidth, such as input impedance, pattern, or polarization. Percent bandwidth, usually used for narrowband antennas, is used defined as . The theoretical limit to percent bandwidth is 200%, which occurs for . Fractional bandwidth or Ratio bandwidth, usually used for wideband antennas, is defined as typically presented in the form of and is . Fractional bandwidth is used for wideband antennas because of the compression of the percent bandwidth that occurs mathematically with percent bandwidths above 100%, which corresponds to a fractional bandwidth of 3:1. . Bandwidth (signal processing) Photonics In photonics, the term bandwidth occurs in a variety of meanings: • the bandwidth of the output of some light source, e.g., an ASE source or a laser; the bandwidth of ultrashort optical pulses can be particularly large • the width of the frequency range that can be transmitted by some element, e.g. an optical fiber • the gain bandwidth of an optical amplifier • the width of the range of some other phenomenon (e.g., a reflection, the phase matching of a nonlinear process, or some resonance) • the maximum modulation frequency (or range of modulation frequencies) of an optical modulator • the range of frequencies in which some measurement apparatus (e.g., a powermeter) can operate • the data rate (e.g., in Gbit/s) achieved in an optical communication system; see bandwidth (computing). A related concept is the spectral linewidth of the radiation emitted by excited atoms. References [1] Van Valkenburg, M. E.. Network Analysis (http:/ / www. amazon. com/ Network-Analysis-Mac-Van-Valkenburg/ dp/ 0136110959) (3rd edition ed.). pp. 383–384. ISBN 0-13-611095-9. . Retrieved 2008-06-22. [2] Stutzman, Warren L., and Gary A. Theiele. Antenna Theory and Design. 2nd Ed. New York: 1998. ISBN 0-471-02590-9 Lead–lag compensator A lead–lag compensator is a component in a control system that improves an undesirable frequency response in a feedback and control system. It is a fundamental building block in classical control theory. Applications Lead–lag compensators influence disciplines as varied as robotics, satellite control, automobile diagnostics, laser frequency stabilization, and many more. They are an important building block in analog control systems, and can also be used in digital control. Given a control plant, desired specifications can be achieved using compensators. I, D, PI, PD, and PID, are optimizing controllers which are used to improve system parameters (such as reducing steady state error, reducing resonant peak, improving system response by reducing rise time). All these operations can be done by compensators as well. Theory Both lead compensators and lag compensators introduce a pole–zero pair into the open loop transfer function. The transfer function can be written in the Laplace domain as where X is the input to the compensator, Y is the output, s is the complex Laplace transform variable, z is the zero frequency and p is the pole frequency. The pole and zero are both typically negative. In a lead compensator, the pole is left of the zero in the complex plane, , while in a lag compensator . A lead-lag compensator consists of a lead compensator cascaded with a lag compensator. The overall transfer function can be written as 293 Leadlag compensator Typically 294 , where z1 and p1 are the zero and pole of the lead compensator and z2 and p2 are the zero and pole of the lag compensator. The lead compensator provides phase lead at high frequencies. This shifts the poles to the left, which enhances the responsiveness and stability of the system. The lag compensator provides phase lag at low frequencies which reduces the steady state error. The precise locations of the poles and zeros depend on both the desired characteristics of the closed loop response and the characteristics of the system being controlled. However, the pole and zero of the lag compensator should be close together so as not to cause the poles to shift right, which could cause instability or slow convergence. Since their purpose is to affect the low frequency behaviour, they should be near the origin. Implementation Both analog and digital control systems use lead-lag compensators. The technology used for the implementation is different in each case, but the underlying principles are the same. The transfer function is rearranged so that the output is expressed in terms of sums of terms involving the input, and integrals of the input and output. For example, In analog control systems, where integrators are expensive, it is common to group terms together to minimize the number of integrators required: In analog control, the control signal is typically an electrical voltage or current (although other signals such as hydraulic pressure can be used). In this case a lead-lag compensator will consist of a network of operational amplifiers ("op-amps") connected as integrators and weighted adders. In digital control, the operations are performed numerically. The reason for expressing the transfer function as an integral equation is that differentiating signals amplify the noise on the signal, since even very small amplitude noise has a high derivative if its frequency is high, while integrating a signal averages out the noise. This makes implementations in terms of integrators the most numerically stable. Intuitive explanation To begin designing a lead-lag compensator, an engineer must consider whether the system needing correction can be classified as a lead-network, a lag-network, or a combination of the two: a lead-lag network (hence the name "lead-lag compensator"). The electrical response of this network to an input signal is expressed by the network's Laplace-domain transfer function, a complex mathematical function which itself can be expressed as one of two ways: as the current-gain ratio transfer function or as the voltage-gain ratio transfer function. Remember that a complex function can be in general written as , where is the "Real Part" and is the "Imaginary Part" of the single-variable function,, The "phase angle" of the network is the argument of . ; in the left half plane this is . If the phase angle is negative for all signal frequencies in the network then the network is classified as a "lag network". If the phase angle is positive for all signal frequencies in the network then the network is classified as a "lead network". If the total network phase angle has a combination of positive and negative phase as a function of frequency then it is a "lead-lag network". Depending upon the nominal operation design parameters of a system under an active feedback control, a lag or lead network can cause instability and poor speed and response times. Leadlag compensator References 1. Nise, Norman S. (2004); Control Systems Engineering (4 ed.); Wiley & Sons; ISBN 0-471-44577-0 2. Horowitz, P. & Hill, W. (2001); The Art of Electronics (2 ed.); Cambridge University Press; ISBN 0-521-37095-7 3. Cathey, J.J. (1988); Electronic Devices and Circuits (Schaum's Outlines Series); McGraw-Hill ISBN 0-07-010274-0 External links • • • • http://www.library.cmu.edu/ctms/ctms/extras/leadlag.htm lead controller using Matlab [1] Lead-Lag Frequency Response [2] at MathPages Lead-Lag Algorithms [3] at MathPages References [1] http:/ / www. sccs. swarthmore. edu/ users/ 06/ adem/ engin/ e58/ lab6/ index. php [2] http:/ / www. mathpages. com/ home/ kmath249/ kmath249. htm [3] http:/ / www. mathpages. com/ home/ kmath198/ kmath198. htm Static VAR compensator A static var compensator (or SVC) is an electrical device for providing fast-acting reactive power on high-voltage electricity transmission networks.[1] [2] SVCs are part of the Flexible AC transmission system device family, regulating voltage and stabilising the system. The term "static" refers to the fact that the SVC has no moving parts (other than circuit breakers and disconnects, which do not move under normal SVC operation), unlike the Synchronous condenser, which is a rotating electrical machine. Prior to the invention of the SVC, power factor compensation was the preserve of large rotating machines such as synchronous condensers.[3] The SVC is an automated impedance matching device, designed to bring the system closer to unity power factor. SVCs are used in two main situations: • Connected to the power system, to regulate the transmission voltage ("Transmission SVC") • Connected near large industrial loads, to improve power quality ("Industrial SVC") In transmission applications, the SVC is used to regulate the grid voltage. If the power system's reactive load is capacitive (leading), the SVC will use reactors (usually in the form of Thyristor-Controlled Reactors) to consume vars from the system, lowering the system voltage. Under inductive (lagging) conditions, the capacitor banks are automatically switched in, thus providing a higher system voltage. In industrial applications, SVCs are typically placed near high and rapidly varying loads, such as arc furnaces, where they can smooth flicker voltage.[1] [4] 295 Static VAR compensator Description Principle Typically, an SVC comprises one or more banks of fixed or switched shunt capacitors or reactors, of which at least one bank is switched by thyristors. Elements which may be used to make an SVC typically include: • • • • Thyristor controlled reactor (TCR), where the reactor may be air- or iron-cored Thyristor Switched Capacitor (TSC) Harmonic filter(s) Mechanically switched capacitors and/or reactors (switched by a circuit breaker) One-line diagram of a typical SVC configuration; here employing a thyristor-controlled reactor, a thyristor switched capacitor, a harmonic filter, a mechanically switched capacitor and a mechanically switched reactor By means of phase angle modulation switched by the thyristors, the reactor may be variably switched into the circuit and so provide a continuously variable Mvar injection (or absorption) to the electrical network.[2] In this configuration, coarse voltage control is provided by the capacitors; the thyristor-controlled reactor is to provide smooth control. Smoother control and more flexibility can be provided with thyristor-controlled capacitor switching.[5] The thyristors are electronically controlled. Thyristors, like all semiconductors, generate heat and deionized water is commonly used to cool them.[3] Chopping reactive load into the circuit in this manner injects undesirable odd-order harmonics and so banks of high-power filters are usually provided to smooth the waveform. Since the filters themselves are capacitive, they also export Mvars to the power system. More complex arrangements are practical where precise voltage regulation is required. Voltage regulation is provided by means of a closed-loop controller.[5] Remote supervisory control and manual adjustment of the voltage set-point are also common. 296 Static VAR compensator Connection Generally, static var compensation is not done at line voltage; a bank of transformers steps the transmission voltage (for example, 230 kV) down to a much lower level (for example, 9.5 kV).[3] This reduces the size and number of components needed in the SVC, although the conductors must be very large to handle the high currents associated with the lower voltage. In some static var compensators for industrial applications such as electric arc furnaces, where there may be an existing medium-voltage busbar present (for example at 33kV or 34.5kV), the static var compensator may be directly connected in order to save the cost of the transformer. The dynamic nature of the SVC lies in the use of thyristors connected in series and inverse-parallel, forming "thyristor valves"). The disc-shaped semiconductors, usually several inches in diameter, are usually located indoors in a "valve house". Advantages The main advantage of SVCs over simple mechanically-switched compensation schemes is their near-instantaneous response to changes in the system voltage.[5] For this reason they are often operated at close to their zero-point in order to maximise the reactive power correction they can rapidly provide when required. They are, in general, cheaper, higher-capacity, faster and more reliable than dynamic compensation schemes such as synchronous condensers [5] . However, static var compensators are more expensive than mechanically switched capacitors, so many system operators use a combination of the two technologies (sometimes in the same installation), using the static var compensator to provide support for fast changes and the mechanically switched capacitors to provide steady-state Vars. References [1] De Kock, Jan; Strauss, Cobus (2004). Practical Power Distribution for Industry (http:/ / books. google. co. uk/ books?id=N8bJpt1wSd4C& pg=PA74). Elsevier. pp. 74–75. ISBN 9780750663960. . [2] Deb, Anjan K.. Power Line Ampacity System (http:/ / books. google. co. uk/ books?id=ebZHT8gzpksC& pg=PA169). CRC Press. pp. 169–171. ISBN 9780849313066. . [3] Ryan, H.M. (2001). High Voltage Engineering and Testing (http:/ / books. google. co. uk/ books?id=Jg1xA65n56oC& pg=PA160). IEE. pp. 160–161. ISBN 9780852967751. . [4] Arrillaga,, J.; Watson, N. R.. Power System Harmonics (http:/ / books. google. co. uk/ books?id=1h9aqRj4o8EC& pg=PA126). Wiley. pp. 126. ISBN 9780470851296. . [5] Padiyar, K. R. (1998). Analysis of Subsynchronous Resonance in Power Systems (http:/ / books. google. co. uk/ books?id=QMSELoMjsg0C& pg=PA169). Springer. pp. 169–177. ISBN 9780792383192. . 297 PID controller 298 PID controller A proportional–integral–derivative controller (PID controller) is a generic control loop feedback mechanism (controller) widely used in industrial control systems – a PID is the most commonly used feedback controller. A PID controller calculates an "error" value as the difference between a measured process variable and a desired setpoint. The controller attempts to minimize the error by adjusting the process control inputs. A block diagram of a PID controller The PID controller calculation (algorithm) involves three separate constant parameters, and is accordingly sometimes called three-term control: the proportional, the integral and derivative values, denoted P, I, and D. Heuristically, these values can be interpreted in terms of time: P depends on the present error, I on the accumulation of past errors, and D is a prediction of future errors, based on current rate of change.[1] The weighted sum of these three actions is used to adjust the process via a control element such as the position of a control valve, or the power supplied to a heating element. In the absence of knowledge of the underlying process, a PID controller is the best controller.[2] By tuning the three parameters in the PID controller algorithm, the controller can provide control action designed for specific process requirements. The response of the controller can be described in terms of the responsiveness of the controller to an error, the degree to which the controller overshoots the setpoint and the degree of system oscillation. Note that the use of the PID algorithm for control does not guarantee optimal control of the system or system stability. Some applications may require using only one or two actions to provide the appropriate system control. This is achieved by setting the other parameters to zero. A PID controller will be called a PI, PD, P or I controller in the absence of the respective control actions. PI controllers are fairly common, since derivative action is sensitive to measurement noise, whereas the absence of an integral term may prevent the system from reaching its target value due to the control action. Control loop basics Further information: Control system A familiar example of a control loop is the action taken when adjusting hot and cold faucets (valves) to maintain the water at a desired temperature. This typically involves the mixing of two process streams, the hot and cold water. The person touches the water to sense or measure its temperature. Based on this feedback they perform a control action to adjust the hot and cold water valves until the process temperature stabilizes at the desired value. The sensed water temperature is the process variable or process value (PV). The desired temperature is called the setpoint (SP). The input to the process (the water valve position) is called the manipulated variable (MV). The difference between the temperature measurement and the set point is the error (e) and quantifies whether the water is too hot or too cold and by how much. After measuring the temperature (PV), and then calculating the error, the controller decides when to change the tap position (MV) and by how much. When the controller first turns the valve on, it may turn the hot valve only slightly if warm water is desired, or it may open the valve all the way if very hot water is desired. This is an example of a simple proportional control. In the event that hot water does not arrive quickly, the controller may try to speed-up PID controller 299 the process by opening up the hot water valve more-and-more as time goes by. This is an example of an integral control. Making a change that is too large when the error is small is equivalent to a high gain controller and will lead to overshoot. If the controller were to repeatedly make changes that were too large and repeatedly overshoot the target, the output would oscillate around the setpoint in either a constant, growing, or decaying sinusoid. If the oscillations increase with time then the system is unstable, whereas if they decrease the system is stable. If the oscillations remain at a constant magnitude the system is marginally stable. In the interest of achieving a gradual convergence at the desired temperature (SP), the controller may wish to damp the anticipated future oscillations. So in order to compensate for this effect, the controller may elect to temper its adjustments. This can be thought of as a derivative control method. If a controller starts from a stable state at zero error (PV = SP), then further changes by the controller will be in response to changes in other measured or unmeasured inputs to the process that impact on the process, and hence on the PV. Variables that impact on the process other than the MV are known as disturbances. Generally controllers are used to reject disturbances and/or implement setpoint changes. Changes in feedwater temperature constitute a disturbance to the faucet temperature control process. In theory, a controller can be used to control any process which has a measurable output (PV), a known ideal value for that output (SP) and an input to the process (MV) that will affect the relevant PV. Controllers are used in industry to regulate temperature, pressure, flow rate, chemical composition, speed and practically every other variable for which a measurement exists. PID controller theory This section describes the parallel or non-interacting form of the PID controller. For other forms please see the section Alternative nomenclature and PID forms. The PID control scheme is named after its three correcting terms, whose sum constitutes the manipulated variable (MV). The proportional, integral, and derivative terms are summed to calculate the output of the PID controller. Defining as the controller output, the final form of the PID algorithm is: where : Proportional gain, a tuning parameter : Integral gain, a tuning parameter : Derivative gain, a tuning parameter : Error : Time or instantaneous time (the present) PID controller 300 Proportional term The proportional term makes a change to the output that is proportional to the current error value. The proportional response can be adjusted by multiplying the error by a constant Kp, called the proportional gain. The proportional term is given by: Plot of PV vs time, for three values of Kp (Ki and Kd held constant) A high proportional gain results in a large change in the output for a given change in the error. If the proportional gain is too high, the system can become unstable (see the section on loop tuning). In contrast, a small gain results in a small output response to a large input error, and a less responsive or less sensitive controller. If the proportional gain is too low, the control action may be too small when responding to system disturbances. Tuning theory and industrial practice indicate that the proportional term should contribute the bulk of the output change. Droop A pure proportional controller will not always settle at its target value, but may retain a steady-state error. Specifically, drift in the absence of control, such as cooling of a furnace towards room temperature, biases a pure proportional controller. If the drift is downwards, as in cooling, then the bias will be below the set point, hence the term "droop". Droop is proportional to the process gain and inversely proportional to proportional gain. Specifically the steady-state error is given by: Droop is an inherent defect of purely proportional control. Droop may be mitigated by adding a compensating bias term (setting the setpoint above the true desired value), or corrected by adding an integral term. PID controller 301 Integral term The contribution from the integral term is proportional to both the magnitude of the error and the duration of the error. The integral in a PID controller is the sum of the instantaneous error over time and gives the accumulated offset that should have been corrected previously. The accumulated error is then multiplied by the integral gain ( ) and added to the controller output. The integral term is given by: Plot of PV vs time, for three values of Ki (Kp and Kd held constant) The integral term accelerates the movement of the process towards setpoint and eliminates the residual steady-state error that occurs with a pure proportional controller. However, since the integral term responds to accumulated errors from the past, it can cause the present value to overshoot the setpoint value (see the section on loop tuning). Derivative term The derivative of the process error is calculated by determining the slope of the error over time and multiplying this rate of change by the derivative gain . The magnitude of the contribution of the derivative term to the overall control action is termed the derivative gain, . The derivative term is given by: Plot of PV vs time, for three values of Kd (Kp and Ki held constant) PID controller The derivative term slows the rate of change of the controller output. Derivative control is used to reduce the magnitude of the overshoot produced by the integral component and improve the combined controller-process stability. However, the derivative term slows the transient response of the controller. Also, differentiation of a signal amplifies noise and thus this term in the controller is highly sensitive to noise in the error term, and can cause a process to become unstable if the noise and the derivative gain are sufficiently large. Hence an approximation to a differentiator with a limited bandwidth is more commonly used. Such a circuit is known as a phase-lead compensator. Loop tuning Tuning a control loop is the adjustment of its control parameters (gain/proportional band, integral gain/reset, derivative gain/rate) to the optimum values for the desired control response. Stability (bounded oscillation) is a basic requirement, but beyond that, different systems have different behavior, different applications have different requirements, and requirements may conflict with one another. PID tuning is a difficult problem, even though there are only three parameters and in principle is simple to describe, because it must satisfy complex criteria within the limitations of PID control. There are accordingly various methods for loop tuning, and more sophisticated techniques are the subject of patents; this section describes some traditional manual methods for loop tuning. Designing and tuning a PID controller appears to be conceptually intuitive, but can be hard in practice, if multiple (and often conflicting) objectives such as short transient and high stability are to be achieved. Usually, initial designs need to be adjusted repeatedly through computer simulations until the closed-loop system performs or compromises as desired. Some processes have a degree of non-linearity and so parameters that work well at full-load conditions don't work when the process is starting up from no-load; this can be corrected by gain scheduling (using different parameters in different operating regions). PID controllers often provide acceptable control using default tunings, but performance can generally be improved by careful tuning, and performance may be unacceptable with poor tuning. Stability If the PID controller parameters (the gains of the proportional, integral and derivative terms) are chosen incorrectly, the controlled process input can be unstable, i.e., its output diverges, with or without oscillation, and is limited only by saturation or mechanical breakage. Instability is caused by excess gain, particularly in the presence of significant lag. Generally, stability of response is required and the process must not oscillate for any combination of process conditions and setpoints, though sometimes marginal stability (bounded oscillation) is acceptable or desired. Optimum behavior The optimum behavior on a process change or setpoint change varies depending on the application. Two basic requirements are regulation (disturbance rejection – staying at a given setpoint) and command tracking (implementing setpoint changes) – these refer to how well the controlled variable tracks the desired value. Specific criteria for command tracking include rise time and settling time. Some processes must not allow an overshoot of the process variable beyond the setpoint if, for example, this would be unsafe. Other processes must minimize the energy expended in reaching a new setpoint. 302 PID controller 303 Overview of methods There are several methods for tuning a PID loop. The most effective methods generally involve the development of some form of process model, then choosing P, I, and D based on the dynamic model parameters. Manual tuning methods can be relatively inefficient, particularly if the loops have response times on the order of minutes or longer. The choice of method will depend largely on whether or not the loop can be taken "offline" for tuning, and the response time of the system. If the system can be taken offline, the best tuning method often involves subjecting the system to a step change in input, measuring the output as a function of time, and using this response to determine the control parameters. Choosing a Tuning Method Method Advantages Disadvantages Manual Tuning No math required. Online method. Requires experienced personnel. Ziegler–Nichols Proven Method. Online method. Process upset, some trial-and-error, very aggressive tuning. Software Tools Consistent tuning. Online or offline method. May include valve and sensor analysis. Allow simulation before downloading. Can support Non-Steady State (NSS) Tuning. Some cost and training involved. Cohen-Coon Good process models. Some math. Offline method. Only good for first-order processes. Manual tuning If the system must remain online, one tuning method is to first set and values to zero. Increase the until the output of the loop oscillates, then the should be set to approximately half of that value for a "quarter amplitude decay" type response. Then increase until any offset is corrected in sufficient time for the process. However, too much will cause instability. Finally, increase , if required, until the loop is acceptably quick to reach its reference after a load disturbance. However, too much will cause excessive response and overshoot. A fast PID loop tuning usually overshoots slightly to reach the setpoint more quickly; however, some systems cannot accept overshoot, in which case an over-damped closed-loop system is required, which will require a significantly less than half that of the setting causing oscillation. Effects of increasing a parameter independently Parameter [3] Rise time Overshoot Settling time Steady-state error Decrease Increase Small change Decrease Degrade Increase Increase Decrease significantly Degrade Decrease [4] Minor decrease Minor decrease Minor decrease No effect in theory Stability Improve if small setting PID controller 304 Ziegler–Nichols method Another heuristic tuning method is formally known as the Ziegler–Nichols method, introduced by John G. Ziegler and Nathaniel B. Nichols in the 1940s. As in the method above, the and gains are first set to zero. The P gain is increased until it reaches the ultimate gain, the oscillation period , at which the output of the loop starts to oscillate. and are used to set the gains as shown: Ziegler–Nichols method Control Type P PI - - PID These gains apply to the ideal, parallel form of the PID controller. When applied to the standard PID form, the integral and derivative time parameters and are only dependent on the oscillation period . Please see the section "Alternative nomenclature and PID forms". PID tuning software Most modern industrial facilities no longer tune loops using the manual calculation methods shown above. Instead, PID tuning and loop optimization software are used to ensure consistent results. These software packages will gather the data, develop process models, and suggest optimal tuning. Some software packages can even develop tuning by gathering data from reference changes. Mathematical PID loop tuning induces an impulse in the system, and then uses the controlled system's frequency response to design the PID loop values. In loops with response times of several minutes, mathematical loop tuning is recommended, because trial and error can literally take days just to find a stable set of loop values. Optimal values are harder to find. Some digital loop controllers offer a self-tuning feature in which very small setpoint changes are sent to the process, allowing the controller itself to calculate optimal tuning values. Other formulas are available to tune the loop according to different performance criteria. Many patented formulas are now embedded within PID tuning software and hardware modules. Advances in automated PID Loop Tuning software also deliver algorithms for tuning PID Loops in a dynamic or Non-Steady State (NSS) scenario. The software will model the dynamics of a process, through a disturbance, and calculate PID control parameters in response. Modifications to the PID algorithm The basic PID algorithm presents some challenges in control applications that have been addressed by minor modifications to the PID form. Integral windup One common problem resulting from the ideal PID implementations is integral windup, where a large change in setpoint occurs (say a positive change) and the integral term accumulates an error larger than the maximal value for the regulation variable (windup), thus the system overshoots and continues to increase as this accumulated error is unwound. This problem can be addressed by: • Initializing the controller integral to a desired value • Increasing the setpoint in a suitable ramp • Disabling the integral function until the PV has entered the controllable region • Limiting the time period over which the integral error is calculated PID controller • Preventing the integral term from accumulating above or below pre-determined bounds Overshooting from known disturbances For example, a PID loop is used to control the temperature of an electric resistance furnace, the system has stabilized. Now the door is opened and something cold is put into the furnace the temperature drops below the setpoint. The integral function of the controller tends to compensate this error by introducing another error in the positive direction. This overshoot can be avoided by freezing of the integral function after the opening of the door for the time the control loop typically needs to reheat the furnace. Replacing the integral function by a model based part Often the time-response of the system is approximately known. Then it is an advantage to simulate this time-response with a model and to calculate some unknown parameter from the actual response of the system. If for instance the system is an electrical furnace the response of the difference between furnace temperature and ambient temperature to changes of the electrical power will be similar to that of a simple RC low-pass filter multiplied by an unknown proportional coefficient. The actual electrical power supplied to the furnace is delayed by a low-pass filter to simulate the response of the temperature of the furnace and then the actual temperature minus the ambient temperature is divided by this low-pass filtered electrical power. Then, the result is stabilized by another low-pass filter leading to an estimation of the proportional coefficient. With this estimation, it is possible to calculate the required electrical power by dividing the set-point of the temperature minus the ambient temperature by this coefficient. The result can then be used instead of the integral function. This also achieves a control error of zero in the steady-state, but avoids integral windup and can give a significantly improved control action compared to an optimized PID controller. This type of controller does work properly in an open loop situation which causes integral windup with an integral function. This is an advantage if, for example, the heating of a furnace has to be reduced for some time because of the failure of a heating element, or if the controller is used as an advisory system to a human operator who may not switch it to closed-loop operation. It may also be useful if the controller is inside a branch of a complex control system that may be temporarily inactive. Many PID loops control a mechanical device (for example, a valve). Mechanical maintenance can be a major cost and wear leads to control degradation in the form of either stiction or a deadband in the mechanical response to an input signal. The rate of mechanical wear is mainly a function of how often a device is activated to make a change. Where wear is a significant concern, the PID loop may have an output deadband to reduce the frequency of activation of the output (valve). This is accomplished by modifying the controller to hold its output steady if the change would be small (within the defined deadband range). The calculated output must leave the deadband before the actual output will change. The proportional and derivative terms can produce excessive movement in the output when a system is subjected to an instantaneous step increase in the error, such as a large setpoint change. In the case of the derivative term, this is due to taking the derivative of the error, which is very large in the case of an instantaneous step change. As a result, some PID algorithms incorporate the following modifications: Derivative of the Process Variable In this case the PID controller measures the derivative of the measured process variable (PV), rather than the derivative of the error. This quantity is always continuous (i.e., never has a step change as a result of changed setpoint). For this technique to be effective, the derivative of the PV must have the opposite sign of the derivative of the error, in the case of negative feedback control. Setpoint ramping In this modification, the setpoint is gradually moved from its old value to a newly specified value using a linear or first order differential ramp function. This avoids the discontinuity present in a simple step change. Setpoint weighting 305 PID controller Setpoint weighting uses different multipliers for the error depending on which element of the controller it is used in. The error in the integral term must be the true control error to avoid steady-state control errors. This affects the controller's setpoint response. These parameters do not affect the response to load disturbances and measurement noise. History PID controllers date to 1890s governor design.[2] [5] PID controllers were subsequently developed in automatic ship steering. One of the earliest examples of a PID-type controller was developed by Elmer Sperry in 1911,[6] while the first published theoretical analysis of a PID controller was by Russian American engineer Nicolas Minorsky, in (Minorsky 1922). Minorsky was designing automatic steering systems for the US Navy, and based his analysis on observations of a helmsman, observing that the helmsman controlled the ship not only based on the current error, but also on past error and current rate of PID theory developed by observing the action of change;[7] this was then made mathematical by Minorsky. His goal was helmsmen. stability, not general control, which significantly simplified the problem. While proportional control provides stability against small disturbances, it was insufficient for dealing with a steady disturbance, notably a stiff gale (due to droop), which required adding the integral term. Finally, the derivative term was added to improve control. Trials were carried out on the USS New Mexico, with the controller controlling the angular velocity (not angle) of the rudder. PI control yielded sustained yaw (angular error) of ±2°, while adding D yielded yaw of ±1/6°, better than most helmsmen could achieve.[8] The Navy ultimately did not adopt the system, due to resistance by personnel. Similar work was carried out and published by several others in the 1930s. Limitations of PID control While PID controllers are applicable to many control problems, and often perform satisfactorily without any improvements or even tuning, they can perform poorly in some applications, and do not in general provide optimal control. The fundamental difficulty with PID control is that it is a feedback system, with constant parameters, and no direct knowledge of the process, and thus overall performance is reactive and a compromise – while PID control is the best controller with no model of the process,[2] better performance can be obtained by incorporating a model of the process. The most significant improvement is to incorporate feed-forward control with knowledge about the system, and using the PID only to control error. Alternatively, PIDs can be modified in more minor ways, such as by changing the parameters (either gain scheduling in different use cases or adaptively modifying them based on performance), improving measurement (higher sampling rate, precision, and accuracy, and low-pass filtering if necessary), or cascading multiple PID controllers. PID controllers, when used alone, can give poor performance when the PID loop gains must be reduced so that the control system does not overshoot, oscillate or hunt about the control setpoint value. They also have difficulties in the presence of non-linearities, may trade-off regulation versus response time, do not react to changing process behavior (say, the process changes after it has warmed up), and have lag in responding to large disturbances. 306 PID controller Linearity Another problem faced with PID controllers is that they are linear, and in particular symmetric. Thus, performance of PID controllers in non-linear systems (such as HVAC systems) is variable. For example, in temperature control, a common use case is active heating (via a heating element) but passive cooling (heating off, but no cooling), so overshoot can only be corrected slowly – it cannot be forced downward. In this case the PID should be tuned to be overdamped, to prevent or reduce overshoot, though this reduces performance (it increases settling time). Noise in derivative A problem with the derivative term is that small amounts of measurement or process noise can cause large amounts of change in the output. It is often helpful to filter the measurements with a low-pass filter in order to remove higher-frequency noise components. However, low-pass filtering and derivative control can cancel each other out, so reducing noise by instrumentation means is a much better choice. Alternatively, a nonlinear median filter may be used, which improves the filtering efficiency and practical performance.[9] In some case, the differential band can be turned off in many systems with little loss of control. This is equivalent to using the PID controller as a PI controller. Improvements Feed-forward The control system performance can be improved by combining the feedback (or closed-loop) control of a PID controller with feed-forward (or open-loop) control. Knowledge about the system (such as the desired acceleration and inertia) can be fed forward and combined with the PID output to improve the overall system performance. The feed-forward value alone can often provide the major portion of the controller output. The PID controller can be used primarily to respond to whatever difference or error remains between the setpoint (SP) and the actual value of the process variable (PV). Since the feed-forward output is not affected by the process feedback, it can never cause the control system to oscillate, thus improving the system response and stability. For example, in most motion control systems, in order to accelerate a mechanical load under control, more force or torque is required from the prime mover, motor, or actuator. If a velocity loop PID controller is being used to control the speed of the load and command the force or torque being applied by the prime mover, then it is beneficial to take the instantaneous acceleration desired for the load, scale that value appropriately and add it to the output of the PID velocity loop controller. This means that whenever the load is being accelerated or decelerated, a proportional amount of force is commanded from the prime mover regardless of the feedback value. The PID loop in this situation uses the feedback information to change the combined output to reduce the remaining difference between the process setpoint and the feedback value. Working together, the combined open-loop feed-forward controller and closed-loop PID controller can provide a more responsive, stable and reliable control system. Other improvements In addition to feed-forward, PID controllers are often enhanced through methods such as PID gain scheduling (changing parameters in different operating conditions), fuzzy logic or computational verb logic. [10] [11] Further practical application issues can arise from instrumentation connected to the controller. A high enough sampling rate, measurement precision, and measurement accuracy are required to achieve adequate control performance. Another new method for improvement of PID controller is to increase the degree of freedom by using fractional order. The order of the integrator and differentiator add increased flexibility to the controller. 307 PID controller 308 Cascade control One distinctive advantage of PID controllers is that two PID controllers can be used together to yield better dynamic performance. This is called cascaded PID control. In cascade control there are two PIDs arranged with one PID controlling the set point of another. A PID controller acts as outer loop controller, which controls the primary physical parameter, such as fluid level or velocity. The other controller acts as inner loop controller, which reads the output of outer loop controller as set point, usually controlling a more rapid changing parameter, flowrate or acceleration. It can be mathematically proven that the working frequency of the controller is increased and the time constant of the object is reduced by using cascaded PID controller.. Physical implementation of PID control In the early history of automatic process control the PID controller was implemented as a mechanical device. These mechanical controllers used a lever, spring and a mass and were often energized by compressed air. These pneumatic controllers were once the industry standard. Electronic analog controllers can be made from a solid-state or tube amplifier, a capacitor and a resistance. Electronic analog PID control loops were often found within more complex electronic systems, for example, the head positioning of a disk drive, the power conditioning of a power supply, or even the movement-detection circuit of a modern seismometer. Nowadays, electronic controllers have largely been replaced by digital controllers implemented with microcontrollers or FPGAs. Most modern PID controllers in industry are implemented in programmable logic controllers (PLCs) or as a panel-mounted digital controller. Software implementations have the advantages that they are relatively cheap and are flexible with respect to the implementation of the PID algorithm. Variable voltages may be applied by the time proportioning form of Pulse-width modulation (PWM) – a cycle time is fixed, and variation is achieved by varying the proportion of the time during this cycle that the controller outputs +1 (or −1) instead of 0. On a digital system the possible proportions are discrete – e.g., increments of .1 second within a 2 second cycle time yields 20 possible steps: percentage increments of 5% – so there is a discretization error, but for high enough time resolution this yields satisfactory performance. Alternative nomenclature and PID forms Ideal versus standard PID form The form of the PID controller most often encountered in industry, and the one most relevant to tuning algorithms is the standard form. In this form the gain is applied to the , and terms, yielding: where is the integral time is the derivative time In this standard form, the parameters have a clear physical meaning. In particular, the inner summation produces a new single error value which is compensated for future and past errors. The addition of the proportional and derivative components effectively predicts the error value at seconds (or samples) in the future, assuming that the loop control remains unchanged. The integral component adjusts the error value to compensate for the sum of all past errors, with the intention of completely eliminating them in seconds (or samples). The resulting compensated single error value is scaled by the single gain . In the ideal parallel form, shown in the controller theory section PID controller the gain parameters are related to the parameters of the standard form through 309 and . This parallel form, where the parameters are treated as simple gains, is the most general and flexible form. However, it is also the form where the parameters have the least physical interpretation and is generally reserved for theoretical treatment of the PID controller. The standard form, despite being slightly more complex mathematically, is more common in industry. Basing derivative action on PV In most commercial control systems, derivative action is based on PV rather than error. This is because the digitised version of the algorithm produces a large unwanted spike when the SP is changed. If the SP is constant then changes in PV will be the same as changes in error. Therefore this modification makes no difference to the way the controller responds to process disturbances. Basing proportional action on PV Most commercial control systems offer the option of also basing the proportional action on PV. This means that only the integral action responds to changes in SP. While at first this might seem to adversely affect the time that the process will take to respond to the change, the controller may be retuned to give almost the same response - largely by increasing . The modification to the algorithm does not affect the way the controller responds to process disturbances, but the change in tuning has a beneficial effect. Often the magnitude and duration of the disturbance will be more than halved. Since most controllers have to deal frequently with process disturbances and relatively rarely with SP changes, properly tuned the modified algorithm can dramatically improve process performance. Tuning methods such as Ziegler-Nichols and Cohen-Coon will not be reliable when used with this algorithm. King[12] describes an effective chart-based method. Laplace form of the PID controller Sometimes it is useful to write the PID regulator in Laplace transform form: Having the PID controller written in Laplace form and having the transfer function of the controlled system makes it easy to determine the closed-loop transfer function of the system. PID controller 310 PID Pole Zero Cancellation The PID equation can be written in this form: When this form is used it is easy to determine the closed loop transfer function. If Then This can be very useful to remove unstable poles Series/interacting form Another representation of the PID controller is the series, or interacting form where the parameters are related to the parameters of the standard form through , , and with . This form essentially consists of a PD and PI controller in series, and it made early (analog) controllers easier to build. When the controllers later became digital, many kept using the interacting form. Discrete implementation The analysis for designing a digital implementation of a PID controller in a Microcontroller (MCU) or FPGA device requires the standard form of the PID controller to be discretised.[13] Approximations for first-order derivatives are made by backward finite differences. The integral term is discretised, with a sampling time ,as follows, The derivative term is approximated as, Thus, a velocity algorithm for implementation of the discretised PID controller in a MCU is obtained by differentiating , using the numerical definitions of the first and second derivative and solving for and finally obtaining: PID controller 311 Pseudocode Here is a simple software loop that implements the PID algorithm in its 'ideal, parallel' form: previous_error = setpoint - process_feedback integral = 0 start: wait(dt) error = setpoint - process_feedback integral = integral + (error*dt) derivative = (error - previous_error)/dt output = (Kp*error) + (Ki*integral) + (Kd*derivative) previous_error = error goto start PI controller A PI Controller (proportional-integral controller) is a special case of the PID controller in which the derivative (D) of the error is not used. The controller output is given by Basic block of a PI controller. where is the error or deviation of actual measured value (PV) from the set-point (SP). . A PI controller can be modelled easily in software such as Simulink using a "flow chart" box involving Laplace operators: where = proportional gain = integral gain Setting a value for is often a trade off between decreasing overshoot and increasing settling time. The lack of derivative action may make the system more steady in the steady state in the case of noisy data. This is because derivative action is more sensitive to higher-frequency terms in the inputs. Without derivative action, a PI-controlled system is less responsive to real (non-noise) and relatively fast alterations in state and so the system will be slower to reach setpoint and slower to respond to perturbations than a well-tuned PID system may be. PID controller References [1] Araki, M.. "PID Control" (http:/ / www. eolss. net/ ebooks/ Sample Chapters/ C18/ E6-43-03-03. pdf). . [2] Bennett, Stuart (1993). A history of control engineering, 1930-1955. IET. p. p. 48 (http:/ / books. google. com/ books?id=VD_b81J3yFoC& pg=PA48). ISBN 9-780863412998 [3] Ang, K.H., Chong, G.C.Y., and Li, Y. (2005). PID control system analysis, design, and technology, IEEE Trans Control Systems Tech, 13(4), pp.559-576. http:/ / eprints. gla. ac. uk/ 3817/ [4] Jinghua Zhong (Spring 2006). PID Controller Tuning: A Short Tutorial (http:/ / saba. kntu. ac. ir/ eecd/ pcl/ download/ PIDtutorial. pdf). . Retrieved 2011-04-04. [5] Bennett, Stuart (November 1984). "Nicholas Minorsky and the automatic steering of ships" (http:/ / ieeexplore. ieee. org/ iel5/ 37/ 24267/ 01104827. pdf?arnumber=1104827). IEEE Control Systems Magazine 4 (4): 10–15. doi:10.1109/MCS.1984.1104827. ISSN 0272-1708. [6] "A Brief Building Automation History" (http:/ / www. building-automation-consultants. com/ building-automation-history. html). . Retrieved 2011-04-04. [7] (Bennett 1993, p. 67 (http:/ / books. google. com/ books?id=VD_b81J3yFoC& pg=PA67)) [8] Bennett, Stuart (June 1986). A history of control engineering, 1800-1930. IET. pp. 142–148 (http:/ / books. google. com/ books?id=1gfKkqB_fTcC& pg=PA142). ISBN 978-0-86341047-5. [9] Li, Y. and Ang, K.H. and Chong, G.C.Y. (2006) PID control system analysis and design - Problems, remedies, and future directions. IEEE Control Systems Magazine, 26 (1). pp. 32-41. ISSN 0272-1708 (http:/ / eprints. gla. ac. uk/ 3815/ ) [10] Yang, T. (June 2005). "Architectures of Computational Verb Controllers: Towards a New Paradigm of Intelligent Control". International Journal of Computational Cognition (Yang's Scientific Press) 3 (2): 74–101. [11] Liang, Y.-L.() et al. (2009). "Controlling fuel annealer using computational verb PID controllers". Proceedings of the 3rd international conference on Anti-Counterfeiting, security, and identification in communication (IEEE): 417–420. [12] King, Myke. Process Control: A Practical Approach. Wiley, 2010, p. 52-78 [13] "Discrete PI and PID Controller Design and Analysis for Digital Implementation" (http:/ / www. scribd. com/ doc/ 19070283/ Discrete-PI-and-PID-Controller-Design-and-Analysis-for-Digital-Implementation). Scribd.com. . Retrieved 2011-04-04. • Minorsky, Nicolas (1922). "Directional stability of automatically steered bodies". J. Amer. Soc. Naval Eng. 34 (2): 280–309 • Liptak, Bela (1995). Instrument Engineers' Handbook: Process Control. Radnor, Pennsylvania: Chilton Book Company. pp. 20–29. ISBN 0-8019-8242-1. • Tan, Kok Kiong; Wang Qing-Guo, Hang Chang Chieh (1999). Advances in PID Control. London, UK: Springer-Verlag. ISBN 1-85233-138-0. • King, Myke (2010). Process Control: A Practical Approach (http://eu.wiley.com/WileyCDA/WileyTitle/ productCd-0470975873.html). Chichester, UK: John Wiley & Sons Ltd.. ISBN 978-0-470-97587-9. • Van, Doren, Vance J. (July 1, 2003). "Loop Tuning Fundamentals" (http://old.controleng.com/article/ 268148-Loop_Tuning_Fundamentals.php.html). Control Engineering (Reed Business Information). • Sellers, David. "An Overview of Proportional plus Integral plus Derivative Control and Suggestions for Its Successful Application and Implementation" (http://web.archive.org/web/20070307161741/http://www. peci.org/library/PECI_ControlOverview1_1002.pdf) (PDF). Archived from the original (http://www.peci. org/library/PECI_ControlOverview1_1002.pdf) on March 7, 2007. Retrieved 2007-05-05. • Graham, Ron (10/03/2005). "FAQ on PID controller tuning" (http://web.archive.org/web/20050206113949/ www.tcnj.edu/~rgraham/PID-tuning.html). Retrieved 2009-01-05. 312 PID controller External links PID tutorials • PID Tutorial (http://www.engin.umich.edu/group/ctm/PID/PID.html) • P.I.D. Without a PhD (http://igor.chudov.com/manuals/Servo-Tuning/PID-without-a-PhD.pdf): a beginner's guide to PID loop theory with sample programming code • What's All This P-I-D Stuff, Anyhow? (http://electronicdesign.com/article/analog-and-mixed-signal/ what-s-all-this-p-i-d-stuff-anyhow-6131.aspx) Article in Electronic Design • Shows how to build a PID controller with basic electronic components (http://asl.epfl.ch/research/projects/ VtolIndoorFlying/rapports/rapportSemStauffer.pdf) (pg. 22) • Virtual PID Controller Laboratory (http://www.pidlab.com) • PID Design & Tuning (http://www.pidlab.com/en/pid-design-and-tuning) • Online PID Tuning applet from University of Texas Control Group (http://www.che.utexas.edu/course/ che360/documents/tuner/Process_Tuner.html) • example PID implementation in 16F887 microchip (http://simf1.blogspot.com/) • PID Control with MATLAB and Simulink (http://www.mathworks.com/discovery/pid-control.html) • Improving the Beginner’s PID – Introduction (http://brettbeauregard.com/blog/2011/04/ improving-the-beginners-pid-introduction) Special topics and PID control applications • Proven Methods and Best Practices for PID Control (http://www.controlguru.com/pages/table.html) • PID Control Primer (http://www.embedded.com/story/OEG20020726S0044) Article in Embedded Systems Programming 313 Article Sources and Contributors Article Sources and Contributors Systems analysis Source: http://en.wikipedia.org/w/index.php?oldid=467516213 Contributors: 1ForTheMoney, Aitias, Al.Ragsdale, Albamuth, AllTheCoolNamesAreAlreadyTaken, Architectchao, Ask123, Beowulph, BigHaz, Bigs slb, Bobbyanalog, Bobo192, Brian48, Btyner, Burzmali, CMG, Cask05, Charleenmerced, Cyrus Grisham, DVdm, Deipnosophista, EagleFan, El C, Erkan Yilmaz, Fooziex, Freakmighty, Goethean, Ian Tabi, Impaciente, Irmtraining, Jac16888, Jeffmartincg, Jesgab, Jheald, Johnbrownsbody, JonHarder, Jpbowen, Kenyon, Kingturtle, Kuru, Larsobrien, Liridon, Logicalgregory, Luna Santin, Malcolmxl5, Maria C Mosak, Mark Renier, Materialscientist, Maurreen, Mcsharry, Mdd, Mnent, Mr Stephen, MrOllie, Mydogategodshat, Mårten Berglund, NawlinWiki, Nbarth, Nick Green, Nono64, Octahedron80, Philip Trueman, Philosopher06, Piano non troppo, Pinethicket, Qst, R'n'B, RJBurkhart, Rajah, RattleMan, Remuel, Riana, Rich Farmbrough, Ronz, S0aasdf2sf, Shreshth91, SimonClinch, Skittleys, Soresumakashi, Srikant.sharma, Stevertigo, Straker, Strangnet, Stypex, Sverdrup, Tawker, Tbackstr, Ted Wilkes, The Missing Hour, The Rambling Man, Tide rolls, Toxickore, Velella, WereSpielChequers, Wiki Roxor, Willking1979, Wpociengel, X!, Yamamoto Ichiro, Zoara, 217 anonymous edits Hardware architecture Source: http://en.wikipedia.org/w/index.php?oldid=456923882 Contributors: Alan Liefting, Basar, Bwilkins, CoderGnome, Dpv, EagleFan, FlyHigh, Frap, Giraffedata, Hetar, J04n, JonHarder, Kubanczyk, Mboverload, MichaelBillington, Neparis, Normxxx, Overjive, Rettetast, Rror, Squids and Chips, TDS, Tribaal, Unused0030, YoavShapira, 29 anonymous edits Stability theory Source: http://en.wikipedia.org/w/index.php?oldid=444273131 Contributors: Arcfrk, Bird of paradox, BradLipovsky, Dmitri83, El C, Grafen, Gurch, Hans Adler, Infovarius, Jbergquist, Linas, MathMartin, Mathmanta, Mathuvw, Mdd, Megus, Michael Hardy, Pleasantville, R.e.b., Sanfranman59, Sniedo, Spinningspark, TedPavlic, Tim1357, 14 anonymous edits Control engineering Source: http://en.wikipedia.org/w/index.php?oldid=461797703 Contributors: AJStadlin, Ahmedsaieed, Allan McInnes, Alll, Americanhero, AndrewHowse, Andy Dingley, Atique1619, Attilios, AutomationBooks, Billymac00, Caporaletti, Cburnett, Chmod007, Chzz, Cjlim, Control.optimization, ControlsGuru, Conversion script, Curtwulf, Cvgittings, Cyrius, DRivera44, Darigan, DeNihiloNihil, Delldot, Dicklyon, Djgreenfield, Dthomsen8, Dúnadan, Edcolins, El C, Energon, Engelec, Firebladed, GB fan, Ghostal, Glenn, Idril, Iswantoumy, Iterator12n, IvanLanin, JanHRichter, Jimmayjr, Jiuguang Wang, Jluedem, Jugander, JustAGal, KKoolstra, KarenW, Kdcarver, Krellis, Light current, MWOAP(AWB), Mani excel, Mdd, Mduench, Michael Hardy, Mohammad1985, MrOllie, Nbarth, Ninly, Nmnogueira, Nposs, Oleg Alexandrov, Oli Filth, Orderud, Pan Dan, Pascalv, PaulGarner, Petedarnell, PeterWoolf, Phuzion, PoccilScript, Politepunk, Pro crast in a tor, Rangoon11, Ranilb5, Rich Farmbrough, Rjpbi, Rrburke, Rs2, Saatwik Katiha, Sanguinity, Saty85, Savh, SchreyP, SeanMon411, Seb, Seh444, Sigmundg, SimonMayer, Sitar Physics, Spalding, Spradlig, Style, Swrkoab, Sympatycznyfacet, Tdadamemd, Tobias Hoevekamp, Toffile, Triddle, User A1, Viral sachde, Wayne Slam, WhatisFeelings?, WikHead, 148 anonymous edits Signal processing Source: http://en.wikipedia.org/w/index.php?oldid=465772966 Contributors: 10metreh, Adoniscik, Alexkin, Alinja, Altenmann, Andrus Kallastu, Antandrus, Bact, Bethnim, Biasoli, Borgx, Burhem, Cdworetzky, Cedars, Charles Matthews, Chester Markel, Christopherlin, Conversion script, Cookie90, Dicklyon, Dwachter, EagleFan, Emperorbma, Firefly322, Funandtrvl, Geoeg, Giftlite, Glenn, Glrx, Grebaldar, Groovamos, GyroMagician, Helwr, Heron, Hezarfenn, Isheden, Ixfd64, Jamilsamara, Janki, Jennifer Levinson, Jimmaths, Johnuniq, Jovial4u, Jusdafax, Kjkolb, Kvng, Light current, Llorenzi, Luckyz, MER-C, Mako098765, Mange01, Mingramh, Monig1, Muriel Gottrop, Nabla, Ninly, Northumbrian, Oicumayberight, Oleg Alexandrov, OrgasGirl, Redgecko, Rememberway, Robert.Harker, Robsavoie, Rockyrackoon, Saddhiyama, SamShearman, Sandeeppalakkal, Sergio.ballestrero, Sfeldm2, Shyamal, Sl, Smack, Spinningspark, The Photon, Tonto Kowalski, TransporterMan, Triddle, Vishalbhardwaj, Voidxor, Wiki contributor 21, Wikihitech, Yk Yk Yk, Yswismer, Zvika, Ömer Cengiz Çelebi, 75 anonymous edits Feedback Source: http://en.wikipedia.org/w/index.php?oldid=465317525 Contributors: 0, 130.159.254.xxx, 16@r, 28bytes, 4twenty42o, A8UDI, ABoerma, Aapo Laitinen, Abmcdonald, AbsolutDan, Achler, AdjustShift, Ahmedsaieed, Alan McBeth, Ale2006, Aliekens, Altenmann, Andre Engels, AndrewHowse, Anupmalavia, Anwormy, ArglebargleIV, Arman Cagle, Arthena, Asinclair, Athkalani, Banjodog, Bart133, Barticus88, Belovedfreak, Bhuston, Blueyoshi321, Brews ohare, Brian787, Bushytails, Butseriouslyfolks, Bwhack, CH3374H, Cancun771, CapitalLetterBeginning, Charles Matthews, Chjoaygame, Chuunen Baka, Circuit dreamer, Cliff smith, Constructive editor, Conti, Cutler, Darrik2, Davidrust, Dcoetzee, Dicklyon, Dino, Dlewis3, DocWatson42, Dragana666, Drphilharmonic, E-Kartoffel, Eaefremov, Edudobay, Eekerz, El C, Elwikipedista, Erckvlp, Erianna, Euryalus, Examtester, Fjellstad, Fredbauder, Furrykef, Fyrael, GeorgeMoney, Giftlite, Glane23, Glenn, Gloo, Gogo Dodo, Goto, Heron, Holgerson0, Hoof Hearted, Houtlijm, Huangdi, Id447, Intgr, Isourou, IstvanWolf, Jaeger5432, Jbergste, JenVan, Jh51681, Jim.henderson, Jimmaciejewski, Jiuguang Wang, Jkeene, John, Johnuniq, Jpbowen, Jwdietrich2, KD5TVI, Kansas Sam, Keegan, Kenneth M Burke, Kingturtle, Klemen Kocjancic, Ktpeil, Kwharris, Kzzl, L Kensington, Lachatdelarue, Lakinekaki, Lecochonbleu, Levineps, Lexor, Lightmouse, Linuxlad, Luna Santin, Mac, Mahanga, Masgatotkaca, Mausy5043, Mdd, Michael Hardy, Mike Rosoft, Mintleaf, Mirwin, Mrs.arna11, Mschlindwein, Musicbox1987, Mwtoews, Nbarth, Neelix, Nick Green, Nigelj, Nikoladie, Nitsky416, Nixdorf, Nonstandard, Nuno Tavares, Nvillega, Nzd, Nzzl, Oatmeal batman, Odie5533, Oobyduby, Ott, Owlgorithm, Patrick, Pengo, Pgreenfinch, Phanerozoic, Piano non troppo, Poccil, Pokipsy76, Pol098, Postglock, Poweroid, Profero, Qwert, RFerreira, RainbowOfLight, ReCover, Reviewsnap, Rjwilmsi, Rod57, Rogerbrent, Ronz, Rurigok, Ryguasu, SEWilco, Sandshoe, Santryl, Shirulashem, SimonP, SoftwareLibre, Sommers, Sophie means wisdom, Steve10345, Strife911, SusanLesch, Taxman, Thseamon, Torc2, Trevithj, Troy 07, Ttwaring, Underpants, Vonkje, Wayne Slam, WhatamIdoing, Wikiborg, Williamoxley, Wji, Woohookitty, XJamRastafire, Xanzzibar, Zlerman, Zzuuzz, Мирослав Ћика, ﮐﺎﺷﻒ ﻋﻘﯿﻞ, 297 anonymous edits Negative feedback Source: http://en.wikipedia.org/w/index.php?oldid=469176142 Contributors: ABF, Abmcdonald, Achler, Aeluwas, Alex Liebsch, Alfred Centauri, AlsatianRain, Anthere, Arcadian, BenFrantzDale, Bensaccount, Binksternet, Bn, Brews ohare, Brian0918, Calrosfing, Catgut, Chjoaygame, Chris fluppy, Coppertwig, CyrilB, DRosenbach, Darthsco, Davidmcnatt, Dcoetzee, Dicklyon, Discospinster, Dominick, Epbr123, Evercat, Flyingidiot, GcSwRhIc, Giftlite, Giridhar, Gizziiusa, Glenn, Heybutch, Hooperbloob, IMSoP, J.delanoy, Jncraton, Johnuniq, Jpbowen, Jwdietrich2, KD5TVI, Ktims, Ktpeil, Kuyabribri, Kyk, Leafyplant, Lexor, Lexw, Light current, Linuxlad, Lir, Makawity, Marshall Williams2, Matthew Yeager, Mdd, Michael Hardy, Mitch994, Moondoll, Nathan Johnson, Nbarth, Nigelj, Nikoladie, Novangelis, OlEnglish, Oli Filth, Omegatron, Patrick, Pearle, Pebkac, Philip Trueman, Prashanthns, PrestonH, RJaguar3, RUL3R, Rogerbrent, Rohitbd, Rspanton, Rustyfence, RyanGerbil10, Ryulong, Shell Kinney, Sheridp, SineWave, Snippingo, So God created Manchester, Soliloquial, Stepp-Wulf, Teapeat, Tech77jp, The Utahraptor, Tide rolls, Tswsl1989, Two Bananas, Uthbrian, William Ortiz, Wlodzimierz, Wolfkeeper, XRDoDRX, Xcentaur, Yettie0711, Youre dreaming eh?, Yves-Laurent, ZimZalaBim, Zundark, 226 anonymous edits Positive feedback Source: http://en.wikipedia.org/w/index.php?oldid=469820378 Contributors: Ahoerstemeier, Allthingstoallpeople, Alta-Snowbird, Altenmann, Andareed, AndrewHowse, Andy M. Wang, Angela, Anthere, Arcadian, Athkalani, BD2412, Beland, Bgwhite, Bhuston, Brendan19, C-randles, Cafzal, Calrosfing, Caltas, Cantus, Chjoaygame, Circuit dreamer, Cisba, CyborgTosser, DavidLevinson, Dcoetzee, DeadEyeArrow, Denelson83, Denniskakai, Derek Ross, Djr32, Drrdudley, Dudecon, Eleassar777, Emiehling, Epbr123, Erkan Yilmaz, Every name is taken12345, Ewlyahoocom, Fredrick day, Geologyguy, Giftlite, Ginsengbomb, Gizziiusa, Glenn, GliderMaven, Hammersoft, Harryboyles, Herenthere, Hulten, Id447, ImperfectlyInformed, Indium, Jenks1987, Johnuniq, Jonathanstray, Jopienator, Joriki, Jpbowen, KD5TVI, Kakuchirana, Ktpeil, Lexor, LibLord, Light current, LilHelpa, Luigifan, Magwo, Mandarax, Mannafredo, Maurice Carbonaro, Mdd, Member, Mild Bill Hiccup, Mluo2010, ModManDan, NESFreak92, NHRHS2010, NSR, Nascar1996, Nashhinton, Nathan Johnson, Nbarth, NeonMerlin, Nigelj, Nihiltres, Nikoladie, Nimur, Nivix, Novangelis, Oli Filth, Onionmon, Oxymoron83, Patrick, Peripitus, Phanerozoic, Philmcrae, Pmarshal, Pstudier, R'n'B, RandyBurge, ReCover, Recognizance, Richard001, RichardVeryard, Riking8, Rinconsoleao, Robertmacl, Robthepiper, Roger Mexico, Rogerbrent, Ryan Roos, RyanGerbil10, Scwlong, Semiwiki, Slakr, Sokari, Spalding, Spinningspark, Sporkot, Squidonius, Superm401, Swarm, Teapeat, Thinboy00P, Thorsten1, USMCSoftLove, VandalCruncher, Vivio Testarossa, WMCEREBELLUM, Walk&check, William M. Connolley, Wimt, Winderful1, Wolfkeeper, Wotnow, Wtmitchell, Wtshymanski, Xcentaur, Yill577, Yuckhil, Yworo, 212 anonymous edits Systems design Source: http://en.wikipedia.org/w/index.php?oldid=452494858 Contributors: Ahoerstemeier, Allan McInnes, Anaxial, Andreas Kaufmann, Architectchao, Arthena, Avalon, Beber, Contemplator, Danhiel, Dominic, Dudboi, Eastlaw, Esoltas, FlyHigh, Freakofnurture, FritzSolms, G1langos, Grafen, H.ehsaan, Hadal, Hari, Imjustmatthew, Imran, Jcw69, Jerryobject, Joy, Jpbowen, K.Nevelsteen, Keilana, Lawynholds, LightAnkh, LouH68, Mannamalai, Maqingpei, Mdd, Miker9asim, Msh210, NSR77, Nav102, Nscheffey, Rjwilmsi, Ruud Koot, SE SME, Shaun F, Shenme, Shirik, SimonMayer, Stefanson, Tee Owe, Tonyshan, Toxickore, West.andrew.g, Wikipelli, Xyzzyplugh, 82 anonymous edits Requirement Source: http://en.wikipedia.org/w/index.php?oldid=466427773 Contributors: 16@r, 9Nak, Aaronbrick, Abhinashak, AbsolutDan, Al Lemos, Allan McInnes, Andreas Kaufmann, Architectchao, Beetstra, Bigglesjames, BirgitteSB, Bloodshedder, Cassbeth, Cheesy UK, Chiswick Chap, Chocmah, Chris Howard, Danikolt, Davewild, Dekisugi, Dhulis, Discospinster, Ellengott, Epbr123, Ewlyahoocom, Fernandoacorreia, Furrykef, H-b-g, HappySneezy, Herbee, Iansommerville, Ijansch, Ilia Kr., Indon, Iridescent, ItsZippy, Jadadad, Jamelan, Janiejones56, JathanBiddlecomb, Jc3, Jpbowen, Katalaveno, KelleyCook, Khalid hassani, Kuru, Lakeworks, Lcg.wda, Lgalumbeck, MPerel, Magioladitis, Mark Renier, Martarius, Mdd, Michael Hardy, Michig, Mjchonoles, Mwfnwa, Nasrani, PedroLamarao, Ph4g3, Pm master, Pnoble805, Project2501a, Qweedsa, RL THe Joker, Rbarroso, Rich Janis, Robert Weemeyer, Ruud Koot, Salsb, Samiroy, Scmbwis, Shahrome, Shambhaviroy, Shanemcd, Starrymessenger, Strellean, Struway2, Tasma3197, Thoughtpod, ThreePD, TimBentley, Triddle, Useight, Vincehk, Wknight94, Wmahan, 146 anonymous edits Systems architecture Source: http://en.wikipedia.org/w/index.php?oldid=464190956 Contributors: Allan McInnes, Andreas Kaufmann, Architectchao, Basar, Brighterorange, Cask05, CoderGnome, Coreurus corporation, Craigb81, Creacon, Daduzi, Deemery, DonFiresmith, Dpv, Dvavasour, EagleFan, Eric-Wester, Erkan Yilmaz, FlyHigh, Graham Berrisford, Hans Adler, Hetar, Horatio Huxham, Ian Pitchford, Iskander s, J04n, JaGa, Jhbendz, Jpbowen, Latiligence, Leutha, Marcika, Maria C Mosak, Mattisse, Mausy5043, Mdd, Michael Hardy, MichaelBillington, Nav102, Normxxx, Onlygains, PaulBoxley, Pegej, Poor Yorick, Psychohistorian, Ronz, Ruud Koot, SEI Publications, Scope creep, Squids and Chips, Tedickey, Tribaal, Walter Görlitz, Willardsimmons, Workingboy, YoavShapira, 35 anonymous edits Function model Source: http://en.wikipedia.org/w/index.php?oldid=459132849 Contributors: Architectchao, Banana Concoction, Bender235, Chatkan Bunnag, Cpt2slow, EagleFan, Grafen, J04n, Jncraton, John of Reading, LittleHow, M-le-mot-dit, Mark Renier, Mdd, Mr pand, Pallaverm, R'n'B, Rayhennessey, TeunSpaans, Topfix, WikHead, Woohookitty, 雁太郎, 10 anonymous edits 314 Article Sources and Contributors Function block diagram Source: http://en.wikipedia.org/w/index.php?oldid=468331594 Contributors: BrianOfRugby, EagleFan, Feyyaz, Jncraton, KSchutte, Karol Langner, KathrynLybarger, Lmatt, Mdd, Nameless23, R'n'B, SeventyThree, Slearl, Spalding, Swen, Vonkje, Zabadab, Zundark, 9 anonymous edits Transfer function Source: http://en.wikipedia.org/w/index.php?oldid=469624290 Contributors: A multidimensional liar, Ale2006, Ap, Attilios, AxelBoldt, BenFrantzDale, Brad7777, Bryan Derksen, Colin Marquardt, Conversion script, Crazycasta, Cyrius, D1ma5ad, Dbroadwell, Dysprosia, Elwikipedista, Fresheneesz, Giftlite, Grubber, H2g2bob, HappySophie, HeroTsai, Hypergeek14, Ihatetoregister, Imjustmatthew, Jiuguang Wang, Jorge Stolfi, Josh1238, Jwdietrich2, Kku, L Kensington, Light current, Linas, Maitchy, Mastermeister, Mckee, Mebden, Melcombe, Metacomet, Michael Hardy, Mschlindwein, Mwilde, Mwtoews, NathanHagen, Nejko, Oarih, Oleg Alexandrov, Oli Filth, Omegatron, Rbj, Rhetth, Ro8269, Roadrunner, Rwestafer, Sagaciousuk, SamuelRiv, Smack, Spinningspark, Splash, Sterrys, TedPavlic, The Anome, Thermochap, Tijfo098, Tlotoxl, Tobias Hoevekamp, Tuos, Vonkje, Wallers, Wowo53, Xlation, Zfeinst, 107 anonymous edits Convolution Source: http://en.wikipedia.org/w/index.php?oldid=467161674 Contributors: 16@r, 1exec1, AhmedFasih, Aitias, Ajb, Akella, Almwi, Anonymous Dissident, Antandrus, Anthony Appleyard, Ap, Arcenciel, Armin Straub, Auclairde, AxelBoldt, Bdiscoe, Bdmy, Belizefan, Ben pcc, BenFrantzDale, Berland, BigHaz, BjarteSorensen, Bob K, Boing! said Zebedee, Bombshell, Bracchesimo, Btyner, CRGreathouse, Carolus m, CaseInPoint, Cdnc, Charles Matthews, Chato, Chowbok, Coco, Coffee2theorems, Comech, Conversion script, Crisófilax, Cronholm144, CryptoDerk, Cwkmail, Cyp, D1ma5ad, Daniele.tampieri, DaveWF, Dbfirs, December21st2012Freak, Dicklyon, DirkOliverTheis, Dmmaus, Dmn, Dpbsmith, Drew335, Dysprosia, Dzordzm, Edward Z. Yang, Eleassar, Emc2, Emote, Esoth, Feraudyh, Flavio Guitian, Frozenport, Furrykef, GabrielEbner, Gareth Owen, Gene Ward Smith, Giftlite, Giuscarl, Greenec3, Greg newhouse, H2g2bob, Hankwang, Headbomb, Helder.wiki, Hellbus, HenningThielemann, Hippasus, Hurricane111, Hyacinth, IMSoP, Inductiveload, Intelliproject, JSKau, Jeff G., JethroElfman, Jevinsweval, Jschuler16, Jshadias, Justin545, Kallikanzarid, Karada, Kazkaskazkasako, Kelly Martin, Kenyon, Kielhorn, Kku, Konradek, Kurykh, Lamoidfl, Larvy, Lautarocarmona, Liotier, Lklundin, Loisel, Looxix, Loren Rosen, Lovibond, Lwalt, MFH, Madmath789, MarcLevoy, Matteo.Bertini, Mckee, Mct mht, Melchoir, Michael Hardy, Mild Bill Hiccup, Minna Sora no Shita, Mircha, Mitrut, Miym, Mmendis, Modster, Mojodaddy, Mormegil, Mpatel, MrOllie, Mschlindwein, Mwilde, N.MacInnes, Naddy, Nbarth, Neffk, Ngchen, Nuwewsco, Odedee, Oleg Alexandrov, Oli Filth, Omegatron, Orderud, Ortho, Oski, P4lm0r3, PMajer, Patrick, Paul Rako, Perey, Phys, Pinethicket, Pion, PowerWill500, Rade Kutil, RapidR, Rlcook.unm, Robert K S, RobertStar20, Rogerbrent, Rubybrian, Schneelocke, Seabhcan, SebastianHelm, Shanes, Silly rabbit, Slawekb, Smack the donkey, Smithwn, Stevenj, SuburbaniteFury, Sullivan.t.j, Sławomir Biały, Tarquin, Tblackma222, TheObtuseAngleOfDoom, Thenub314, Tinos, Tomash, User A1, UtherSRG, Vuo, Waqas946, Wayne Slam, Wikomidia, WojciechSwiderski, Wwoods, Wxm29, Xe7al, Yaris678, Zoltan808, Zvika, 294 anonymous edits Dynamical system Source: http://en.wikipedia.org/w/index.php?oldid=468735819 Contributors: 0, 195.186.254.xxx, 7&6=thirteen, Aaronp808, Adam majewski, Aleksandar Guzijan, Aliotra, Altenmann, Americanhero, AntOnTrack, Ap, Athkalani, AxelBoldt, Bluemoose, Brazzouk, CBM, CX, Caesium, Charles Matthews, Chetvorno, Chopchopwhitey, Complex01, Complexica, Cumi, Cutler, Daniele.tampieri, Dhollm, Dinhxuanduyet, Dino, Dmharvey, Dratman, Duoduoduo, Dysprosia, EPM, El C, Epbr123, Epolk, Everyking, Evilphoenix, Filemon, Filur, Finn-Zoltan, Frederick.d.abraham, Fredrik, Gandalf61, Giftlite, Graham87, Headbomb, Hesam7, Highlightened, Hve, Hydroli, Iamthedeus, Jabernal, Jay Gatsby, Jeff3000, Jeffrey Smith, JerrySteal, Jitse Niesen, JocK, Jowa fan, Jugander, K-UNIT, KYPark, Karol Langner, Kayvan45622, Kenneth M Burke, Kotepho, Kzzl, LaHonda MBA, Lakinekaki, LeilaniLad, Levineps, Lightmouse, Linas, LokiClock, ManiacK, Marj Tiefert, Math.geek3.1415926, MathMartin, Mathmanta, Mct mht, Mdd, Meersan, Michael Hardy, Milly.mortimer, MrOllie, Msh210, Myasuda, Neelix, Neruo, Nick Number, Nn123645, Noeckel, Oleg Alexandrov, Orange Death, OrgasGirl, Paine Ellsworth, Patrickdepinguin, PetaRZ, Pgan002, Phys, PlatypeanArchcow, Profyorke, RDBury, RedWolf, Reddi, Reinderien, Revolver, Rhetth, Rhythmiccycle, Rich Farmbrough, Rintrah, Rjwilmsi, Roesser, Rschwieb, SEIBasaurus, Sadi Carnot, Salgueiro, Salix alba, Sam Korn, Samuelbf85, Schmloof, SilverSurfer314, Snoyes, Solace098, Sverdrup, Tedder, Template namespace initialisation script, The Anome, The wub, Tiled, Tobias Hoevekamp, Tomisti, Tommyjs, Torcini, Tosha, Ttrantow, Volfy, Voretus, Waitati, WaysToEscape, Wclxlus, WhiteC, WillowW, XJamRastafire, XaosBits, Yellow octopus, Zsniew, 147 anonymous edits Graph dynamical system Source: http://en.wikipedia.org/w/index.php?oldid=418085477 Contributors: Charvest, Docu, Eslip17, Harryboyles, Henning.Mortveit, Jim.belk, Jyoshimi, Mhym, Michael Hardy, 3 anonymous edits First-order logic Source: http://en.wikipedia.org/w/index.php?oldid=469501602 Contributors: .digamma, 34jjkky, Ahmed saeed, Ahouseholder, Alastair Haines, Almit39, Apolkhanov, Arthur Rubin, AshtonBenson, AugPi, Avakar, AxelBoldt, Axl, Banazir, Be hajian, BenBaker, Bender235, Blaisorblade, Bookandcoffee, Borgx, Brick Thrower, CBM, CBM2, CRGreathouse, Cacadril, Caesura, Carbo1200, Cdills, Chalst, Charles Matthews, Childoftv, Chinju, Classicalecon, Cloudyed, ConcernedScientist, Conversion script, Creidieki, Cronholm144, Crowne, Dan Gluck, Danman3459, David Eppstein, David.Monniaux, Dbtfz, Dcoetzee, Dhruvbaldawa, Dhruvee, Diannaa, Djk3, Dkf11, Dpol, Dwheeler, Dysprosia, ELDRAS, Ejars, Ekspiulo, Eleuther, Elwikipedista, EmilJ, English Subtitle, Eubulide, Exostor, Expensivehat, Filemon, Fl, Foxjwill, Francvs, Frecklefoot, Fredrik, Gf uip, Gherson2, Giftlite, Greenrd, Gregbard, Grim23, Gubbubu, Hakeem.gadi, Hans Adler, Harburg, Henning Makholm, Heyitspeter, Hilverd, History2007, Histre, Hobsonlane, Hq3473, Ht686rg90, Iannigb, Igodard, Inquam, Iridescent, Ixfd64, JRSpriggs, Jan1nad, Jay Gatsby, Jayme, Jleedev, Jod, Jon Awbrey, Jorend, Joriki, Jsnx, Juansempere, Julian Mendez, JulianMendez, Karl-Henner, Katieh5584, Kelly Martin, Kendrick Hang, Kgoarany, Kim Bruning, Kku, Klausness, Kubigula, Kumioko, Kusunose, Kwertii, LBehounek, Lambiam, Lauri.pirttiaho, Lethe, Linas, Ljf255, Loadmaster, Looxix, Lord British, Lucidish, MarSch, Mark Renier, Marner, Meloman, Mets501, Mhss, Michael Hardy, Michael Slone, Mike Fikes, Mike Segal, Mild Bill Hiccup, Minesweeper, Mistercupcake, Mormegil, Mpeisenbr, Msh210, Nahaj, Nanmus, Nanobear, NavarroJ, NicDumZ, Nick Number, NoBu11, Norman Ramsey, Nortexoid, Nzrs0006, Obradovic Goran, Oleg Alexandrov, Omphaloscope, Onceler, Otto ter Haar, Palnot, Patrick, Paul August, Pcap, Pdibner, Penumbra2000, Per Olofsson, Philogo, Phuzion, Physis, Phyte, Pkrecker, Pomte, R.e.b., RG2, RJaguar3, Randall Holmes, Randomblue, Reach Out to the Truth, Rich Farmbrough, Richard L. Peterson, Rjwilmsi, Rlcolasanti, RobHar, Sampletalk, Saric, Siroxo, SixWingedSeraph, Slaniel, SouthLake, Spayrard, SpigotMap, Spug, Starlord, Stevertigo, Subversive.sound, TechnoFaye, Tesseran, The Anome, The Tetrast, Tigranes Damaskinos, Tijfo098, Timwi, TitusCarus, Tizio, Tkuvho, ToastieIL, Tobias Bergemann, Trovatore, Turms, Utcursch, Vanden, Velho, VictorAnyakin, Virago250, Voorlandt, Whitepaw, WikHead, Willhig, Wireless99, Xplat, Youandme, Zeno Gantner, Zero sharp, 265 ,. דניאל בanonymous edits List of first-order theories Source: http://en.wikipedia.org/w/index.php?oldid=469791986 Contributors: Ben Standeven, Bernard the Varanid, Bwhack, CBM, Charles Matthews, DA3N, Dmharvey, EmmetCaulfield, FlyHigh, GabrielEbner, Giftlite, Gregbard, Hans Adler, JackSchmidt, Knotwork, Lethe, Madmardigan53, Magmalex, ManoaChild, Mcpk, Mets501, Michael Hardy, Ott2, R.e.b., Rjwilmsi, RobHar, Shellgirl, SomeStranger, SunCreator, Tesseran, Trovatore, Uncia, Wmahan, Ynhockey, ZeroOne, 12 anonymous edits Pole splitting Source: http://en.wikipedia.org/w/index.php?oldid=356754552 Contributors: BillC, Brews ohare, Circeus, Giftlite, Michael Hardy, Nbarth, Spinningspark, Toffile, WakingLili, 6 anonymous edits Damping ratio Source: http://en.wikipedia.org/w/index.php?oldid=470153952 Contributors: ASSami, AndrewDressel, Apoorvajsh, Barkman, BillC, Btyner, Cheeto81, Craigerv, Daniele.tampieri, Dannya222, Dicklyon, EdJogg, Gene Nygaard, Giftlite, HappyCamper, Jgreeter, MarSch, Michael Hardy, Nbarth, Nmnogueira, Oleg Alexandrov, PigFlu Oink, Remuel, Robobeg, Salsb, Sanpaz, Smit, SyntheticDNA, Teapeat, TedPavlic, Tlee123, Warut, Yeokaiwei, 41 anonymous edits Second-order logic Source: http://en.wikipedia.org/w/index.php?oldid=468830873 Contributors: Aminorex, AshtonBenson, AugPi, BrideOfKripkenstein, CBM, Chalst, Charles Matthews, Chinju, Chira, Chris the speller, Creidieki, David Eppstein, Dcoetzee, Delirium, DividedByNegativeZero, Dysprosia, E-boy, EdJohnston, EmilJ, Eric Yurken, Ericbodden, Giftlite, Glaucus, Gregbard, Gruepig, Hairy Dude, Iannigb, Inego, JRSpriggs, Jamelan, Jason Quinn, Jkcmsal, Joriki, Julian Mendez, Kan8eDie, Knight Of The Woeful Countenance, Kumioko, LOL, Lipedia, Lpgeffen, Mets501, Michael Hardy, Michaeljsouth, Mistercupcake, Modify, Nortexoid, Ntmatter, Oleg Alexandrov, Omnipaedista, Ott2, Postcrypto, Rich Farmbrough, Rjwilmsi, Root4(one), Russell C. Sibley, Salgueiro, Sapphic, Sergey Dmitriev, Shellgirl, Tesseran, That Guy, From That Show!, The Tetrast, Wclark, Zero sharp, 44 anonymous edits Impulse response Source: http://en.wikipedia.org/w/index.php?oldid=442978665 Contributors: Adoniscik, Ahoerstemeier, Aluvus, Anjosacruca, Beeline bc, BenFrantzDale, Benjamin.friedrich, Binksternet, Bob K, Bobblewik, CodyARay, Colonies Chris, Dicklyon, Firefly322, Giftlite, Iain, Jag123, Jerryfishenden, Karatsobanis, Kvng, LRG, Lightmouse, Lindosland, Magioladitis, Melcombe, Metacomet, Michael Hardy, Msa11usec, Mwilde, Nbarth, Nolaiz, Oleg Alexandrov, Oli Filth, PeterEmanuelRoos, Phase Theory, Rinconsoleao, Ro8269, Rs2, SUL, Spectrogram, Stephenellwood99, Voyaging, Wolfkeeper, 62 anonymous edits LTI system theory Source: http://en.wikipedia.org/w/index.php?oldid=465030887 Contributors: Batsmacks, BenFrantzDale, Bob K, Bodrell, CBM, Cbs228, Cihan, Cxw, DerHexer, Dicklyon, Gbruin, Giftlite, Grebaldar, GreyCat, Infzy, Interferometrist, J04n, JABoye47x, Jamelan, Jpkotta, Juansempere, Kb9wte, Mako098765, Marlynboke, Mdd, Melcombe, Metacomet, Michael Hardy, Mmeijeri, Mwilde, Natalya, Nejko, Nish-NJITWILL, Niteowlneils, No-body, Nuwewsco, Oli Filth, Onkelringelhuth, Rbj, Rsrikanth05, Schizodelight, Sonick, Splash, Sterrys, TedPavlic, ThePianoMan, Toffile, Zvika, 62 anonymous edits Routh–Hurwitz stability criterion Source: http://en.wikipedia.org/w/index.php?oldid=469091140 Contributors: Becritical, Billymac00, Bletchley, Brews ohare, Catslash, Charles Matthews, Dicklyon, Djenicek, Duncharris, Ekotkie, FourBlades, Fweeky, Giftlite, Insertesla, Jitse Niesen, Jiuguang Wang, Julien Tuerlinckx, Kevinsane, Kmellem, Linuxlad, Mdd, Mdupont, Michael Hardy, Nanasid, Novangelis, Olaf, Oleg Alexandrov, Pelotas, Spradlig, Tbsmith, Titoxd, User A1, Vivek7de, Zaxxonal, 48 anonymous edits Control theory Source: http://en.wikipedia.org/w/index.php?oldid=469715053 Contributors: A Sengupta, Absurdburger, Adam Keller, Aleksandar Guzijan, Americanhero, Anakata, Ancheta Wis, Ap, Arman Cagle, Aschwarm, Attilios, Aushulz, Bact, Barak, BenFrantzDale, Benbest, Billymac00, Bn, Bob12794, Boden Rodenberger, Brad7777, Brews ohare, Bullhaddha, CRGreathouse, Canglesea, Capricorn umair, Cburnett, Ceyockey, Cha'yu, Chandu15, ChangChienFu, Chimin 07, Chongkian, ChristianSchluchter, Complexica, Constant314, Control.optimization, Conversion script, Copeland.James.H, Coronafire, Cvgittings, D1ma5ad, D6, DARTH SIDIOUS 2, DCwom, Dan Polansky, Daniel Kellis, Daniel Sperry, David Pardy, Dedmonds, Didier wkpd, Dinhxuanduyet, Dja25, DrJunge, Dudered, EUvin, EmreDuran, Engelec, Erik the Red 2, Erkan Yilmaz, Euyyn, EverGreg, Fenice, First Harmonic, Frederick12, Frenchwhale, GRAHAMUK, Gaius Cornelius, Gandalf61, Gauge, GeneralAlgernon, GeorgeLouis, Giftlite, Glenn, Gnusbiz, Graham87, GregorB, Gwernol, Hammertime, Hazard, Heron, Holnet, IPSOS, Ignacioerrico, Ignignot, Ikelos, Jamesontai, JanHRichter, Jbergquist, Jeff3000, Jimmaths, Jiuguang Wang, Jiy, Jmdaly, John43ny, JohnBlackburne, Jpbowen, Jpkotta, Jugander, Julien Tuerlinckx, Jwdietrich2, Kdcarver, Khukri, Kku, Kurochka, Kurykh, Lexor, Lightmouse, Lommer, LuckyWizard, Magmi, Mani excel, Marek69, Markeditwiki, Marklar2007, Martin451, Mastlab, 315 Article Sources and Contributors Mathmanta, Matt Gies, Mbbradford, Mdd, Michael Hardy, Mickraus, Mindmatrix, Mmeijeri, Mononoke, Mrdarrett, Nbarth, Neonil, Netlad, Oleg Alexandrov, Oli Filth, Onco p53, Pak21, PeR, Phmoreno, PierreAbbat, Pmg, Poor Yorick, Poree, Ppntori, Pulkitgrover, Ralesk, Rawling, Regancy42, Revolver, Riceplaytexas, Rich Farmbrough, Rjwilmsi, Ro8269, Rod57, RoyKok, Royalguard11, Rs2, S Roper, Sahodaran, Sattyam, Scott Dial, Seb, Seh444, Serrano24, Shadowjams, Sigmundg, Sigmundur, Simetrical, Simoneau, Siroxo, Someguy1221, Spalding, Spike Wilbury, Sponge, Spradlig, StephenReed, Style, Suruena, Swrkoab, Tailboom22, Tamtamar, Taral, TedPavlic, The Anome, The Transhumanist, Thelctrclengnr, Theraft, Tide rolls, Tilin, Titoxd, Tommy2010, Torzsmokus, Tupeliano, Tuxa, User A1, Vonkje, Vsh3r, Wavelength, Wknight94, Wmahan, Wojciech.ziniewicz, XieChengnuo, Yamamoto Ichiro, Ykhwong, Yuriybrisk, Zander the Mander, ﺳﻌﯽ, 312 anonymous edits Control system Source: http://en.wikipedia.org/w/index.php?oldid=469294718 Contributors: 28421u2232nfenfcenc, Aasdasdas, Abarry, Ahmedsaieed, Alansohn, Andres, Angela, Anna Lincoln, AugPi, Beaufanamus, Benbest, Biancasimone, Bruyninc, Buhrerf, Can't sleep, clown will eat me, Chrisdotb, Christophenstein, Chzz, Circuit dreamer, Crunchy Numbers, Cybercobra, D1ma5ad, Darigan, Data8504, Daz 90, Dicklyon, Dmsar, Dougofborg, Edcolins, Engelec, Engineer Bob, Epbr123, Erkan Yilmaz, Excirial, Firien, Frappucino, Gary Cziko, Gepwiki, Giftlite, Glenn, Groogle, Gurch, Haipa Doragon, Iautomation, Intelliw, JHunterJ, Jbrasla, Jiang, JoanneB, Joel Saks, Joeschmoe35, Jpbowen, Kanonkas, KarenW, Keegscee, Kku, Klilidiplomus, Kmellem, KnightRider, Krellis, Krish matlab, Kyle01 is, L Kensington, L337p4wn, Lambiam, Light current, LittleOldMe, M@sk, MER-C, Masgatotkaca, Mbeychok, Mdd, Mentifisto, Mindmatrix, Mishlai, MrOllie, My Username is..., Nbarth, Nick, Nigelj, Nirvana888, OmarNJITWILL, Palica, Passw0rd, PatrickJCollins, Pauldouglas72, Paulpw, PeaceNT, Petedarnell, Pnm, Pyrophilliac, Radagast83, Rafmag, RazorICE, Rdhunkins, Reaper Eternal, Recurring dreams, Redvers, Rjwilmsi, Rob 52240, Roman12345, Rror, S Roper, SCH56, SchreyP, Shell Kinney, Softy, Spalding, Spinningspark, Spradlig, Srinivasasha, StaticGull, StephenReed, Swrkoab, Tabletop, TedPavlic, The Thing That Should Not Be, Ttwaring, User A1, Vonkje, Vrenator, Wdl1961, Wintonian, Xionbox, Yk Yk Yk, Zander the Mander, ﺳﻌﯽ, 250 anonymous edits Controller (control theory) Source: http://en.wikipedia.org/w/index.php?oldid=469506307 Contributors: Archelon, Attilios, Aushulz, Benbest, CALR, Caporaletti, Coppertwig, DJ LoPaTa, Drtelnet, Duoduoduo, EagleFan, Eightkinarrows, Enboifre, Gaius Cornelius, Icairns, Joechao, Jpbowen, JustAGal, Jwdietrich2, K3rb, Marlysse, Mellery, Mikeblas, Palica, RJFJR, Robofish, Robosorne, Rod57, TheTito, Think outside the box, Tom, ViM, Vonkje, ﺳﻌﯽ, 45 anonymous edits Process control Source: http://en.wikipedia.org/w/index.php?oldid=468297917 Contributors: Attilios, Aushulz, AutomationBooks, Ayrton Prost, Ben Ben, Billymac00, BjKa, Bob, COGDEN, Caesura, Cflm001, Charles Matthews, Chemical Processing, Chrisportelli, Citect, Control.optimization, Cvgittings, Dysprosia, E Wing, Engelec, Ewlyahoocom, Finejon, Fubar Obfusco, GB fan, Giftlite, Grandviewmedia, HaPi, Heron, J04n, Jerry, Jiuguang Wang, Jmuenzing, Jschwa1, Kiefer.Wolfowitz, Kwill, Materialscientist, Mdd, MegaSloth, Melcombe, Michael Hardy, Mojtaba Cazi, Mschel, Mydogategodshat, N4AEE, Netlad, Nsaa, PeterWoolf, Phmoreno, Pimlottc, Pm master, Pugges, Pumpmeup, Ramanraj, Reconsider the static, Requestion, RichardVeryard, Rifleman 82, Ripogenus77, Rlsheehan, RockMFR, Ronz, RoyKok, Sciurinæ, Scjessey, Scott white85, Search255, Smalljim, Spalding, Spallen, SteffiKl, Stevenknudsen, SuperMarioMan, Supersteve04038, Swrkoab, Tagishsimon, Timwi, Tnxman307, Unregistered.coward, User A1, Uthbrian, VQuakr, Vdegroot, Weetoddid, WhiteOakTree, Wtshymanski, Zhenqinli, 96 anonymous edits Proportionality (mathematics) Source: http://en.wikipedia.org/w/index.php?oldid=469841540 Contributors: 28421u2232nfenfcenc, Aidepikiw0000, Alansohn, Alberto da Calvairate, Andres, Aniten21, Asifkemalpasha, Audiosmurf, AugPi, Aughost, Auron Cloud, AxelBoldt, Baseball Watcher, Belajane123, Brandon, Btyner, CSTAR, Capitalist, Capricorn42, CarbonUnit, Cbowman713, Charles Matthews, Chris the speller, Commander Keane, Constructive editor, Cybercobra, DARTH SIDIOUS 2, Dadude3320, David R. Ingham, Dcirovic, Dendodge, DerHexer, Dhollm, Dicklyon, Dmyersturnbull, Ecemaml, Eighty, Enviroboy, Evan.oltmanns, Ewen, Excirial, Fdskjs, Finell, Funandtrvl, Fuzzform, Giftlite, Gigabyte, Hazard-SJ, Heron, Hq3473, In actu, Inflector, Irbisgreif, Isheden, Iwilllearn2010, Ja 62, JaGa, Jacobusmaximus34, JerroldPease-Atlanta, Jerzy, Jncraton, Jusdafax, Jóna Þórunn, Kaihsu, Kjarda, Kozuch, Krellis, Kuddle143, La goutte de pluie, Levineps, Luk, Marc Venot, Marek69, Mayfare, Miccoh, Michael Hardy, Mindlurker, Mpatel, Mutt20, Nbarth, Netalarm, Nicoli nicolivich, Noodle snacks, Omegatron, Oxymoron83, Paste, Patrick, Paul August, Pfctdayelise, Pjvpjv, Pred, Pstanton, Randywombat, Rgdboer, Rjd0060, Sakkura, Scottish lad, Skater, Sohmc, Sonett72, SpaceFlight89, Spinningspark, Spoon!, TBster2001, TejasDiscipulus2, The Pikachu Who Dared, The Thing That Should Not Be, Theoprakt, Tide rolls, Tobias Bergemann, Tom harrison, Tommy2010, Tompw, Traxs7, Tricore, Viridae, WeijiBaikeBianji, WikHead, Willking1979, Yeungchunk, Zonuleofzinn, 百家姓之四, 217 anonymous edits Integral Source: http://en.wikipedia.org/w/index.php?oldid=469216818 Contributors: 129.174.59.xxx, 4C, 6birc, A bit iffy, Ace Frahm, Acegikmo1, Admiral Norton, Adrruiz, Ahknor, Ais523, Alansohn, Aleksander.adamowski, Alexius08, Alsandro, Aly89, Amahoney, AnOddName, Andrei Stroe, Andrew Moylan, Andrewcmcardle, Anonymous Dissident, Antillarum, Apokrif, Arcfrk, ArnoldReinhold, Arunirde, AxelBoldt, Azuredu, Baccala@freesoft.org, BarretB, Bdesham, Bdmy, Beefman, Bemoeial, BenFrantzDale, Benjaminwill, Benzi455, Berria, Bethnim, Bnitin, Bo Jacoby, Bobo192, Bomac, Borb, Boreas231, Bovineone, Brad7777, Brews ohare, Brufydsy, Bsmntbombdood, Butko, CSTAR, CalebNoble, Calltech, Caltas, Calvin 1998, Capefeather, Caramdir, Cardamon, Cassini83, Cat2020, Catgut, Cemenarist, Centrx, Cflm001, Chadernook, Chait1027, Charles Matthews, Chetvorno, Chinju, ChrisHodgesUK, Ciko, Closedmouth, Conversion script, Crakkpot, Cronholm144, D.Lazard, DHN, Da nuke, DabMachine, Daryl Williams, Davewild, DavidCBryant, Dchristle, Den fjättrade ankan, Der Spion, Diberri, Dima373, Discospinster, Diza, Djradon, Dmcq, Dmitrey, Doctormatt, Dojarca, Drdonzi, Dugwiki, Dysprosia, E2eamon, Eagleal, Edward, Edward Knave, Einsteins37, ElTchanggo, Electron9, Emaalt, Emet truth, Enochlau, Eomund, Epbr123, Espressobongo, Euty, Evil saltine, Favonian, Ferengi, FerrousTigrus, Filemon, FilipeS, Fintor, FractalFusion, FrankTobia, Franklin.vp, Fraqtive42, Fredrik, Frokor, Fulvius, Fyyer, Gadykozma, Gandalf61, Garo, Gary King, Geometry guy, Gesslein, Giftlite, Glaurung, Glenn, Gmcastil, Gnixon, Goethean, Greg Stevens, HairyFotr, Hajhouse, Hakeem.gadi, Hakkasberra, Hal0920, Herald747, Heron, Hotstreets, Hrafeiro, Icairns, Ider88, Igny, Inter, Introareforcommonpublic, Iridescent, Isheden, Iulianu, Ivan Štambuk, JB Gnome, JForget, JKBlock, JRSpriggs, Jackfork, Jagged 85, JakeVortex, Jakob.scholbach, Jalesh, Jdlambert, Jfgrcar, Jim.belk, Jitse Niesen, Johnlemartirao, JonMcLoone, JonezyKiDx, Jose77, Josh dos, Jugander, Jynus, KSmrq, Kapoor Amit, Karada, Karimjb, Karol Langner, Katzmik, Kawautar, Keegan, Kevin Baas, KhoousesWiki, KieferSkunk, Kiensvay, King Bee, Kingpin13, Kirbytime, Knakts, Kntrabssi, Kumioko, Kurykh, Kusunose, Kwantus, Kyle1278, LOL, Lambiam, Leland McInnes, Lethe, Levi.vaieua, LiDaobing, Light current, Lightdarkness, Lindberg G Williams Jr, Lir, Loisel, Loom91, Luís Felipe Braga, MC10, MJBurrage, MONGO, Madmath789, Magioladitis, Marek69, MarkSweep, Materialscientist, Matqkks, Matsoftware, Maxvcore, Mcld, Mcorazao, Mecanismo, Melchoir, Mets501, MiNombreDeGuerra, Michael Hardy, MightyBig, Mikael Häggström, Mike2vil, Mindmatrix, Minestrone Soup, Miquonranger03, Momo san, Mormegil, MrOllie, Ms2ger, Mtz1010, MuZemike, Mìthrandir, Nbarth, Nickrds09, Nikai, Nitya Dharma, Nnedass, NonvocalScream, Obeattie, Oleg Alexandrov, OrgasGirl, Ourhomeplanet, Ozob, PMG, Paolo.dL, Patrick, Paul August, Paul Matthews, Paxsimius, Pcb21, PericlesofAthens, PhilipMW, Phillip J, PhySusie, Physicistjedi, PiMaster3, Pie4all88, Planetary Chaos Redux, Plasticup, Point-set topologist, Pooryorick, Programmar, Python eggs, Quuxplusone, RDBury, RJFJR, Raamin, Radomir, Rama's Arrow, Randomblue, Raven4x4x, Razorflame, RedWordSmith, Rich Farmbrough, Ringbang, Rjwilmsi, Rklawton, Robbyjo, Roboquant, Rracecarr, Rubybrian, SJP, Saforrest, Salix alba, Sankalpdravid, Satori42, Schneelocke, Schoolscienceteacher, Scott MacLean, Sdornan, Seaphoto, Seresin, Shadowjams, Shipmaster, Showgun45, Siddhant, Silly rabbit, Slakr, Sligocki, Smithpith, Snigbrook, Sodin, Somedaydoctor, Someguy1221, Spellcast, Splashkid2, Ssd, StaticGull, Stca74, Stevertigo, Stlrams22, StradivariusTV, Stw, Super-real dance, Sushant gupta, Susurrus, Sławomir Biały, TJRC, TStein, Tabletop, Tbsmith, Template namespace initialisation script, The Anome, Thegeneralguy, Thenub314, Theshadow27, Thiseye, Tjdw, Tkuvho, Tobby72, Tobias Bergemann, Tomyumgoong, Topology Expert, TroyBurm, Twp, Ukexpat, Urdutext, VKokielov, Van helsing, Vanished User 0001, Velvetron, Viames, Vladislav Pogorelov, Vrenator, Waabu, Waltpohl, Wik, Wile E. Heresiarch, Wilsonater23456, Wimt, WinterSpw, Witchinghour, Wknight94, Wood Thrush, Wordsoup, Wtshymanski, Xantharius, Yacht, Ybbor, Yosha, Youandme, Your Lord and Master, Yuyudevil, Zfr, Zoicon5, ZooFari, Zundark, 510 anonymous edits Derivative Source: http://en.wikipedia.org/w/index.php?oldid=468104358 Contributors: 10p12a0195, 123Mike456Winston789, 127, 24.44.206.xxx, A. Parrot, A876, ABCD, AbcXyz, Acannas, Ace Frahm, Ahmad87, Akshaysrinivasan, Alex Bakharev, Algebraist, AllTheThings, AllyUnion, Aly89, Andre Engels, Andrejj, Andres, Anonymous Dissident, Anwar saadat, ArglebargleIV, Art LaPella, Arthur543, Ash4Math, AstroHurricane001, Audiovideo, AugPi, AvicAWB, Aw.rootbeer1, AxelBoldt, AySz88, Az1568, AzaToth, Azuredu, BJjake, Baccyak4H, Baprameya, Barak Sh, Bcrowell, Bdesham, Bdmy, Ben b, BenFrantzDale, Bethnim, Bethpage89, Bhxho, BiT, Binary TSO, Blaze2010, Bo Jacoby, Bobianite, Bobo192, Bomac, Boothy443, Brian Tvedt, Bryan Derksen, Bsadowski1, Bsmntbombdood, Buckyboy314, Burk, Buttseckks, CBM, CSTAR, Caesura, CanadianLinuxUser, Canthusus, Cataclysm12, Catgut, Celestianpower, Cenarium, Ceyockey, ChP94, Charles Matthews, Charvest, Cheeser1, Chenyu, ChrKrabbe, Chrishmt0423, Chrisportelli, Chuck SMITH, Cj67, Cje, Cobaltcigs, Complex (de), Conversion script, Cooolsduudes, Courcelles, Crisófilax, Cronholm144, Cyp, DGaw, DHN, Daniel.Cardenas, Darrel francis, DarrylNester, Den fjättrade ankan, Deor, DerHexer, Dialecticas, Dino, Dmcq, Dmharvey, Doctormatt, Dojarca, Dolphin51, Dotancohen, DrBob, Drmies, DroEsperanto, Dtm142, DuKot, Duoduoduo, Dysprosia, ENIAC, EconoPhysicist, Elektron, Eman2129, Enigmaman, Enviroboy, Epbr123, Eric119, Erik Sandberg, Error792, Essjay, Estudiarme, Evan2718281828, Everyking, Evil saltine, Evolauxia, Extransit, Ezh, Fazan1983, Fcelikler, Fintor, Foobarnix, Foxandpotatoes, Fredrik, Fresheneesz, Frickylick, Fsiler, GTBacchus, Gabbe, Gabn1, Gagueci, Garrooney, Gary King, Geometry guy, Georgia guy, Gesslein, Giftlite, Gilliam, Ginkgo100, Glenn, Gmaxwell, Graham87, Greg von greg, Guardian of Light, Gurch, Haham hanuka, Hashar, Headbomb, Heimstern, HenningThielemann, Hephaestos, Heron, Hoolycheeese, Hteen, Iamthedeus, Icairns, Ideyal, If I Am Blocked I Will Cry, InkKznr, Innerproduct, IstvanWolf, Iulianu, Izno, JAn Dudík, JB82, JRSpriggs, Jacj, Jackzhp, Jacobolus, JamesBWatson, Janus Shadowsong, JeLuF, Jeff G., Jim.belk, Jimothytrotter, Jitse Niesen, Jixzad123, Jketola, Joey-das-WBF, John254, JohnOwens, Johnuniq, Jon Harald Søby, Jonathan48, Josh dos, Jrtayloriv, Jshadias, Jsjsjs1111, KSmrq, Kaimbridge, Karol Langner, Katzmik, Kbolino, Keilana, Keithcc, Kerdek, Keryk, Kevin Saff, King Bee, KleinKlio, KnowledgeOfSelf, Knutux, Kocher2006, Konradek, Korovyev, Kourtney88, Kozuch, Kpennington82, Kukini, Kumioko, Kurykh, Kwantus, Lethe, Lewallen, Lightmouse, Livius3, Loisel, Loodog, Lorrybizzle314159, Lotje, Ltmboy, Lynxmb, MC10, MStankie, Mac, Mac Davis, Madhero88, Maelin, Magister Mathematicae, MagnaMopus, Mandavi, Mani1, Manulinho72, MarSch, Marc Venot, Marek69, Markjoseph125, Marquez, Masterofpsi, Materialscientist, MatrixFrog, Maurice Carbonaro, Mboverload, Mcorazao, Melchoir, Melongrower, Metacomet, Michael Hardy, Mike2vil, Mindmatrix, MisterSheik, Mo0, Mormegil, Mountain, Mouse20080706, Mpatel, Mphilip1, Mr Ape, MrOllie, Ms2ger, Nahum Reduta, Najoj, Nandesuka, Nbarth, NeonMerlin, Netrapt, Newone, NickBush24, Nickrds09, No Guru, Nobaddude, Noel Bush, Nonagonal Spider, ObserverFromAbove, OlEnglish, Oleg Alexandrov, Omegatron, Onebravemonkey, Orderud, Oscabat, Ozob, Palfrey, Paolo.dL, Pascal.Tesson, Patrick, Paul August, PaulSmith641, Pbroks13, Pcb21, PeteX, Peterwhy, Phgao, Phillip J, Pinkadelica, Pizza Puzzle, Pmanderson, Polx, Populus, Potatoswatter, Poulter7777777, Powowatushtoosha, Programmar, Pseudolus42, Qef, Quadrescence, Quondum, Qz, R3m0t, RDBury, RMFan1, Ralian, Ramcdpai, Rdsmith4, Realmestup, RedWolf, Revolver, Rhyshuw1, Ricardo sandoval, Rick Norwood, Ritchy, Rjwilmsi, Robbjedi, Rosasco, Rossami, Rrenner, Ruud Koot, Rx5674, SJP, SQGibbon, Salix alba, Sanu.soysa, Saotomi5102, Saretakis, Scientific29, Scifitechie360, Sergiobgaspar, Shellgirl, Shellreef, Sigmalmtd, Silly rabbit, Simetrical, Simfish, Skizzik, Sky Attacker, SkyMachine, Slakr, Sligocki, SlipperyHippo, Slysplace, Smithpith, Snaxe920, Sonett72, Spartan S58, Srleffler, Starr sam, Stevertigo, Stomme, StradivariusTV, Stuart07, Stw, Sukolsak, SunCreator, Sushant gupta, Svick, Sławomir Biała, Sławomir Biały, THEN WHO WAS PHONE?, TStein, TakuyaMurata, Tammojan, Tcncv, Tcnuk, Template namespace initialisation script, Terminaterjohn, Tewy, Thalesfernandes, Thariath1, The Anome, The Font, The Wordsmith, TheNightFly, TheRanger, TheTito, Theda, Theunixgeek, ThirdParty, Tholme, Thomasmeeks, Tiddly Tom, Timothy Clemans, Timwi, Titoxd, Tkuvho, Tobias Bergemann, Tomaxer, Tosha, Trombe29, Turenar, Turgidson, Ufnoise, Ukexpat, Ukikonno, Unint, Urdutext, User A1, Usuwiki, VKokielov, Vangos, Vanished User 0001, VectorPosse, Vevek, WatermelonPotion, Wavelength, Wik, Wikid77, Wikifixi, 316 Article Sources and Contributors Wikithesource, Willking1979, Wisdom89, Wolfmankurd, WriterHound, X!, XP1, Xantharius, Yacht, Yazaq, Yk Yk Yk, Youssefsan, Zack wadghiri, Zchenyu, Zoicon5, Zundark, Περίεργος, 758 anonymous edits OLE for process control Source: http://en.wikipedia.org/w/index.php?oldid=450498911 Contributors: A876, AdAstraRG, Aligilman, Alutov, Anrie Nord, Arotto, ArséniureDeGallium, Automationgirl, BBird, Beetstra, Boostaldo, CALR, CL, Cp2020, Dpc1962, Harksaw, Honzinus, Hu12, Iamdaman2, Inductiveautomation, JonHarder, KenJackson.US, Khrasmussen, Kku, Krackersk, KrakatoaKatie, Kuru, Langerwolfgang, Lastorset, Lcfenwick, Leirith, Lem1968, Limited Atonement, Merbabu, Mihju, Mikedi, Modster, Mr.opc, Ninokurtalj, Pawel Duda, Pdcook, Pg msk, Plrk, Pnm, Poccil, Poniard, Rail, Ripogenus77, Roland Abt, SCH56, Sdddlt, SoManySpammers, Steveh7, Supersteve04038, Tabletop, Techtonik, Texture, The Utahraptor, Themfromspace, Thingg, Thumperward, USMstudent09, Veinor, Vikramrc, Vivers, VladNick, Warren, Whoelie, Z10x, Zemke98, 118 anonymous edits Nonlinear control Source: http://en.wikipedia.org/w/index.php?oldid=450610110 Contributors: Encyclops, Favonian, Jiuguang Wang, Jmdaly, Kwiki, Lambiam, Lovewarcoffee, Mild Bill Hiccup, Mmeijeri, Neelix, Nick Number, Ohconfucius, Pakaraki, Pivs, Pmg, R'n'B, Ronz, Serra66, TedPavlic, Woohookitty, 18 anonymous edits Proportional control Source: http://en.wikipedia.org/w/index.php?oldid=452386886 Contributors: Antandrus, Cheet, Dollars, Dougsim, Edward, Heron, Hytar, Javaboy.ms, Jeffthejiff, Malcolma, Nbarth, Nillerdk, SiliconDioxide, Sonett72, Spalding, Zwarepeet, 12 anonymous edits Optimal control Source: http://en.wikipedia.org/w/index.php?oldid=469477092 Contributors: Alansohn, Alschwartz123, Amillar, Arishth, Arsen.palestini2, Arthur Rubin, Attilios, Awhan, BenFrantzDale, Billymac00, Bruyninc, Charles Matthews, Daniel5127, Delaszk, Dralexccf, Drusillo, Duoduoduo, EagleFan, Eisv, Encyclops, Fph, Gaius Cornelius, GavinTing, Giftlite, Hanshuo, Hess88, Hqb, Ikaros321, Jamelan, Jcoffland, Jitse Niesen, Jiuguang Wang, Jmdaly, Johan.akesson modelon, John of Reading, Kimchi.sg, Mangogirl2, Mani excel, Mat cross, Mdd, Medvall, Michael Hardy, Michaelnikolaou, Mindmatrix, Nbarth, Nima3, Nmilich, Ocuser, Ogo, Oleg Alexandrov, Oli Filth, Optctl, Oscarjquintana, PS User, PSContributor, PeR, Robbbb, SimonP, Smartcat, Subodha0, Sudhanshu1, TedPavlic, Themfromspace, Trey Urqhart, Ummit, V madhu, Vinzklorthos, Vmbecerra, Wolfkeeper, YUL89YYZ, Zsniew, 69 anonymous edits Robust control Source: http://en.wikipedia.org/w/index.php?oldid=461282929 Contributors: Aminrahimian, Anantsrao1988, CRGreathouse, Cretog8, Dja25, Encyclops, Hfst, Jmoini, Jugander, LachlanA, Melcombe, Michael Hardy, Oleg Alexandrov, PhilDWraight, Rjwilmsi, Robbbb, Sniedo, Tanja-Else, TedPavlic, TheLightbringer, Tilin, Tribaal, Yakudza, 17 anonymous edits Adaptive control Source: http://en.wikipedia.org/w/index.php?oldid=452054329 Contributors: Aushulz, Billymac00, Celithemis, Control.optimization, Encyclops, J04n, JForget, James086, Jason Quinn, Jdpipe, Jiuguang Wang, Letranova, LilHelpa, Mdd, Misrom, Pekaje, Pjetter, Pol098, Rjpbi, Sonymontuno, Sympatycznyfacet, Taggard, Tomas e, Trebor, User A1, Vonkje, Wizard191, Іванко1, ﺇﻣﺎﺭﺍﺗﻲ1971, 16 anonymous edits Stochastic control Source: http://en.wikipedia.org/w/index.php?oldid=435585994 Contributors: Aryaniae, Chris the speller, Duoduoduo, Encyclops, Jiuguang Wang, Jonkerz, Melcombe, Pulkitgrover, Raymondwinn, 1 anonymous edits Neural network Source: http://en.wikipedia.org/w/index.php?oldid=469341914 Contributors: A tzoumerkioti, Aavindraa, Adams7, Adodge, Ahoerstemeier, Ahyeek, Alan Au, Alanbly, Alansohn, Aleksey Vayner, AnAj, Andrew C. Wilson, Andrew4010, Anetode, Antandrus, Anthony Appleyard, AnthonyQBachler, Anthonyhcole, Antonio Lopez, Aomarks, Apmonitor, Appljax123, Arcadian, ArielGold, Arru, Arthena, Arthur Rubin, Arun143123, Aupif, Avenue, Az1568, B9 hummingbird hovering, BSTRhino, Batch1, Benbread, Bethnim, BeverlyCrusher, Bhadani, BlaiseFEgan, Bonnarj, Bpavel88, Brian0918, BrokenSegue, Brunton, Butros, CX, Caana, Caltas, Cameltrader, Canley, Cantus, Cap'n Refsmmat, Cburnett, Ceran, Chaosdruid, CharlesGillingham, Chase me ladies, I'm the Cavalry, Chire, ChrisCooper1991, ChrisKennedy, Clarissaelsinger, Clementi, Coladie, CommodiCast, Cs007, D.Strand, DVdm, Daev, Dan Guan, Dan100, Dank, Darkride, Daryakav, David R. Ingham, Daytona2, Ddtuttle, Delirium, Denisarona, Denn, Denoir, Deodar, Dicklyon, Diegotorquemada, Digfarenough, Dlohcierekim's sock, Doczilla, Doyley, Driper, Dstorkey, Duncharris, Dysprosia, Dzkd, E. Ripley, Ecalot, Efbfweborg, Elsendero, Epsilon60198, EyrianAtWork, Fangz, Fbarl, Fenice, Ferdinandopo, Figma, Fippy Darkpaw, Foobar, Francob, G.A.S, Gauss, Ghepeu, Giftlite, Gilhamto, Gioto, GoingBatty, Goyston, Gray1, Guntram, Gurch, Hebbian Learning, Herschel, HighKing, Hmains, Holme215, Hongooi, Hu, Hu12, Icktoofay, Imersion, Imroy, Ines it, Iohannes Animosus, Itistoday, J.Davis314, Jacj, Jafeluv, James-davis, Jhedengren, Jiteshchougule, Jitse Niesen, John Broughton, John Quiggin, JohnJBarton, JohnWilliams, Jon Awbrey, JonEAhlquist, JonHarder, Josephorourke, Jpbowen, Justaguy0479, Justin W Smith, Jwdietrich2, Jwmillerusa, Kaimiddleton, Karbinski, Khoikhoi, Kl4m, Klaus, Kuchhadiyaa, Kuru, La goutte de pluie, LaggedOnUser, Lampsalot, Levin, LilHelpa, LindsayH, Looie496, Lordvolton, Luna Santin, MER-C, MSchmahl, MaBe, Maian, Majoreditor, Male1979, Mamling, Mangst, Manop, Mareino, Margaret52, Mathaddins, Mathemajor, Matthew, Mattisse, Maxversace, Mbell, Mcasl, Mcsee, Mdd, Mehdiabbasi, Meier99, Mekem, Melcombe, Michael C Price, Michael Hardy, Mietchen, MikeWren, Minimac, Mosquitopsu, Mr.winston, Murx, Mwanner, Mysid, Nguyengiap84, Nicholasnice, NickW557, Nikai, Nikoladie, Ninly, Nomad, NotAnonymous0, Nunh-huh, Oleg Alexandrov, Olethros, Oli Filth, Omegatron, Onasraou, Opelio, Originalname37, Ostracon, Pakcw, Pankaj Sharma Computers, Passino, Pavan206, Pavel Vozenilek, Pepperpot80, Pescadero, Peter.kabai, Pietro16, Pjetter, Prolog, PseudoSudo, Psyphen, Qwfp, Qwyrxian, R'n'B, RJASE1, RWillwerth, Random User 937494, Raven1977, Rayc, Reddi, Reedbeta, Reedy, Requestion, RexNL, Riana, Rich Farmbrough, Richard001, Ritchy, Riturajrr, Rjwilmsi, Rmyeid, RobertMel, Romanm, Ronz, Royalguard11, Rtcpenguin, Rwp, SDas, Salehjoon, SallyForth123, SamuelRiv, Satishsnandihalli, Schwnj, Seans Potato Business, Shadmanzafar, Shawndkc, Shinosin, Slakr, Sonyjose, SparkOfCreation, Spazzm, Spudtater, Steven Zhang, Stevenj, Stoni, Strangerer, Strife911, Sunny256, Sunsetsky, Supten, Svea Kollavainen, Tarotcards, Tbutzon, Tectonicura, That Guy, From That Show!, The sunder king, TheJosh, Themfromspace, Thomasmeeks, Thompsma, Titoxd, Tribaal, Trilobitealive, Tryptofish, Twri, User A1, Vendettax, Violetriga, Vrenator, Waggers, Wavelength, Wbrameld, Wduch, Wernerseyfried, Wikiklrsc, Will316, Wingchi, Wknight8111, Wmahan, Wraithdart, X7q, Xanzzibar, Yoshua.Bengio, Zeiden, Zybler, Ömer Cengiz Çelebi, 537 anonymous edits Fuzzy logic Source: http://en.wikipedia.org/w/index.php?oldid=469291037 Contributors: AK Auto, Abtin, Academic Challenger, Ace Frahm, Acer, Adrian, Ahoerstemeier, Aiyasamy, Ajensen, Alarichus, Alca Isilon, Allmightyduck, Amitauti, Amram99, Andreas Mueller, Andres, AndrewHowse, Anidaane, Anonymous Dissident, Ap, Aperezbios, Arabani, ArchonMagnus, Arjun01, Aronbeekman, Arthur Rubin, Atabəy, AugPi, Avoided, Aydos, Aylabug, Babytoys, Bairam, BenRayfield, BertSeghers, Betterusername, Bjankuloski06en, Blackmetalbaz, Blainster, BlaiseFEgan, Boffob, Borgx, Brat32, Brentdax, C2math, CLW, CRGreathouse, Catchpole, Causa sui, Cedars, Cesarpermanente, Chalst, Charvest, Christian List, Chronulator, Ck lostsword, Clemwang, Closedmouth, Cnoguera, Crasshopper, Crunchy Numbers, Cryptographic hash, Cybercobra, Damian Yerrick, Denoir, Dethomas, Dhollm, Diegomonselice, Dragonfiend, Drift chambers, Drwu82, Duniyadnd, EdH, Elockid, Elwikipedista, Em3ryguy, Eric119, Eulenreich, EverettColdwell, Evert Mouw, Expensivehat, EyeSerene, False vacuum, Felipe Gonçalves Assis, Flaminchimp, Flewis, Fratrep, Fullofstars, Furrykef, Fyyer, GManNickG, Gauss, Gbellocchi, George100, Gerhardvalentin, Gerla, Gerla314, GideonFubar, Giftlite, Gregbard, Grim23, Gryllida, Guard, Gurch, Gurchzilla, Gyrofrog, Gökhan, H11, Hargle, Harry Wood, Heron, History2007, Hkhandan, Honglyshin, Hypertall, ISEGeek, Icairns, Ignacioerrico, Igoldste, Ihope127, Intgr, Ioverka, Iridescent, Ixfd64, J.delanoy, J04n, JHTaler, Jack and Mannequin, Jadorno, Jaganath, Jbbyiringiro, Jchernia, Jcrada, Jeff Silvers, JesseHogan, Jim1138, JimBrule, Joriki, Junes, JustAnotherJoe, K.Nevelsteen, KSmrq, Kadambarid, Kariteh, Katzmik, Kilmer-san, Kingmu, Klausness, Koavf, Kuru, Kzollman, L353a1, LBehounek, Labnoor, Lambiam, LanguidMandala, Lars Washington, Lawrennd, Lbhales, Lese, Letranova, Leujohn, Loopy48, Lord Hawk, Loren Rosen, Lynxoid84, M.r.ebraahimi, MC MasterChef, MER-C, Maddie!, Malcolmxl5, Mani1, Manop, Marcus Beyer, Mastercampbell, Materialscientist, Mathaddins, Maurice Carbonaro, Mdd, Mdotley, Megatronium, Melcombe, Mhss, Michael Hardy, Mkcmkc, Mladifilozof, Mneser, Moilleadóir, Mr. Billion, Nbarth, Ndavies2, Nekura, Nortexoid, Ohka-, Oicumayberight, Oleg Alexandrov, Olethros, Oli Filth, Omegatron, Omicronpersei8, Oroso, Palfrey, Panadero45, Paper Luigi, Passino, Paul August, PeterFisk, Peterdjones, Peyna, Pickles8, Pit, Pkrecker, Pownuk, Predictor, Ptroen, Quackor, Qwfp, R. S. Shaw, RTC, Rabend, RedHouse18, Repep, Requestion, Rgheck, Riyad parvez, Rjstott, Rjwilmsi, Rohan Ghatak, Ronaldloui, Ronz, Ruakh, Rursus, Ruud Koot, S. Neuman, SAE1962, Sahelefarda, Saibo, Samohyl Jan, Saros136, Scimitar, Scriber, Sebastjanmm, Sebesta, Sector001, Serpentdove, Sesanker, Shervink, Slashme, Smmurphy, Snespeca, SparkOfCreation, Srikeit, Srinivasasha, StephenReed, Stevertigo, SuzanneKn, Swagato Barman Roy, T2gurut2, T3hZ10n, TankMiche, Tarquin, Teutonic Tamer, ThornsCru, Thumperward, Tide rolls, Topher385, Traroth, TreyHarris, Trovatore, Trusilver, Turnstep, Typochimp, Ultimatewisdom, Ululuca, Vansskater692, Velho, Vendettax, Virtk0s, Vizier, Voyagerfan5761, Wavelength, Williamborg, Wireless friend, Woohookitty, Xaosflux, Xca777, Xezbeth, Yamamoto Ichiro, Zaphod Beeblebrox, Zfr, Zoicon5, 499 anonymous edits Expert system Source: http://en.wikipedia.org/w/index.php?oldid=468790751 Contributors: .mdk., 2help, 4twenty42o, 767Mobile, A5b, Abaddon314159, Abdull, AbsoluteFlatness, Acroterion, Adrian.walker, Agutie, Ahoerstemeier, Alansohn, Aldarsior, Alksub, Alphachimp, Aluion, Amypro, Andrew Parke Yang Yu, AndyTheGrump, Angela, Anonymous Dissident, Anselmobattisti, Arnehans, Arpabr, Arthur Rubin, Arved, Asheeshgoja, Barneca, Bdonlan, Beetstra, Beland, Bentogoa, BertSeghers, Bevan.coleman, BlackHatMentor, BlueNovember, Boing! said Zebedee, Bookandcoffee, Boothy443, Brandonm0720, Brat32, CaliforniaDreaming, Calltech, Canley, Carlosliu, CatherineMunro, Caulde, CharlesGillingham, Chocolateboy, Chris 73, Chris the speller, Christian75, Clemwang, Coll7, Control.optimization, Conversion script, Courcelles, Criffin, Crysb, Cse.buddhika, DARTH SIDIOUS 2, DXBari, Dan Gan, Davehi1, Dbfirs, DieSkurk, Discospinster, Doc Tropics, Dottydotdot, Dreadstar, Drwu82, Dspradau, Dullfig, Dupz, EAi, EagleFan, Econrad, Ee79, Eh7, Ehheh, Elphington, Epbr123, Erkan Yilmaz, Eternal dragon, Excirial, Favonian, Floris V, Frecklefoot, Freek Verkerk, Furrykef, Fvasconcellos, Fyyer, Fæ, Gadfium, Gh23, Gilliam, Gmstanley, GoingBatty, Graham king 3, Graymornings, Greenchilefrog, Greenrd, Gtg204y, Gurch, Gwernol, Gökhan, H-ko, Harvey the rabbit, Hetar, Hingram, Hirzel, History2007, Hobartimus, Hoziron, Hubyrod, Hut 8.5, Ike9898, Iluvcapra, Imran, Informavores, Ingolfson, IvanLanin, Ixfd64, JMSwtlk, JTN, Jahiegel, JamesBWatson, Jeff G., Jerome Charles Potts, Jez9999, Jim McKeeth, Jim15936, Jmaio2, Jmchauvet, Joachim.baumeister, Joanna Bryson, JonHarder, Joydurgin, Joyous!, Jpbowen, Jtp15, Kaimiddleton, Karthikanandraj, Kelovy, Kenyon, Khalid hassani, Kingpin13, Komandobb, Kpjas, Kristensson, Kuru, Kvng, Kyle1278, Lastorset, Lbhales, Lear's Fool, Lights, LittleOldMe old, Looxix, Louharris, Lradrama, Luciandrei, Luk, Luna Santin, Lupo, Lycurgus, MC MasterChef, MCrawford, MER-C, Madelinefelkins, Manop, Mariannehgv, MarkJFAllen, Marudubshinki, Massic80, Materialscientist, MattBan, Maurreen, Mav, Mdd, Merbabu, Mi ai guy, Michael Hardy, MichaelBillington, Mikael Häggström, Mlaffs, Mmernex, Mononomic, MonsterSRM, Mormegil, Mr. Lefty, Mrholybrain, Mrmatiko, Mrseibert, Muchness, Murdomck, Mwi33, Mydogategodshat, Mzucherman, Müslimix, Nabeth, Neilc, Nick Number, Noodlez84, Oddbodz, OingoBoingo2, OldManInACoffeeCan, Oliver202, Ooz dot ie, OrgasGirl, Paddles, Pakaran, Pat grenier, Pavel Vozenilek, Pd75, Pearle, Perfectblue97, Pgr94, Pharaoh of the Wizards, Philip Trueman, Piano non troppo, Pinethicket, Prakash Nadkarni, Predictor, Programmar, Prokopenya Viktor, Proofreader77, PseudoSudo, Quintote, Qwertyus, Qxz, R Calvete, R'n'B, RDBury, RapidR, Reach Out to the Truth, Reconsider the static, Rich Farmbrough, Richardhenricks, RickK, Rmosler2100, Robert Kowalski, Ronz, Rspeer, Rursus, Ryulong, SNowwis, Sam Korn, Sam42, Sander Säde, SarekOfVulcan, Sarnholm, Saswat012, SatuSuro, Savant13, Scott Dial, Semperf, Serdar XoXo, Sesu Prime, Shadowjams, Shaggyjacobs, Shepard, Shizhao, SimonArlott, SimonLyall, Skinneal, Skyezx, Skysmith, SlackerMom, Snowolf, Solitude, Sortior, SpeedyGonsales, Spellmaster, Sprocc, Squidfish, Srleffler, Stephen Bain, Stevag, Storkk, Supten, Swollib, Sylvain Mielot, Talk2xpert, TerrorTowers, TexMurphy, The Epopt, Thistheman, Tide rolls, Tim1357, Titov Anton, Tmg1165, Tobias Bergemann, Tom Pippens, Topiarydan, Treyt021, Ulric1313, Useight, Van helsing, Vangelis12, VictorAnyakin, Vrenator, WaltBusterkeys, Wayiran, West Brom 4ever, WikiLaurent, Wikilibrarian, WillWare, Woohookitty, 317 Article Sources and Contributors Wrathchild, X96lee15, Yonkie, Youblend2, Youssefsan, ZENDELBACH, Zoicon5, Zzuuzz, Δ, 662 anonymous edits Root locus Source: http://en.wikipedia.org/w/index.php?oldid=456907370 Contributors: Adoniscik, AhmedHan, Alphachimp, Angela, AvicAWB, Biezl, Brews ohare, Burns512, Celieber, Charles Matthews, Cuaxdon, Daniel fady, David Ayton, Dja25, Fredrik, Glrx, Gtnash, Hvn0413, J.delanoy, Kgartner, Kir360, Lpkeys, Marcosagliano, Markrkrebs, Masgatotkaca, Mdd, Melhawar, Michael Hardy, Musically ut, OlavN, Oleg Alexandrov, Petrb, Quinwound, Raiker, Rasputin243, RickK, Rs2, Salamurai, Silverfish, Spradlig, Sss41, Topbanana, Trevor MacInnis, Tuvas, Updatehelper, WhiteOak2006, Wigan25, Wj32, Xeno, Yamamoto Ichiro, Zfeinst, Zzxterry, 102 anonymous edits Compensation (engineering) Source: http://en.wikipedia.org/w/index.php?oldid=416361288 Contributors: Bearcat, EagleFan, Fratrep, HGB, Longhair, Metrax, Nickjwebb, Pichpich, R'n'B, Radiojon, Tabletop, 5 anonymous edits Frequency compensation Source: http://en.wikipedia.org/w/index.php?oldid=434185903 Contributors: Anoneditor, Brews ohare, Cburnett, CyborgTosser, Gaius Cornelius, Glenn, Hooperbloob, Light current, MichaelBillington, Nbarth, Omegatron, PhilKnight, TedPavlic, Toffile, Wdwd, 13 anonymous edits Error detection and correction Source: http://en.wikipedia.org/w/index.php?oldid=470214539 Contributors: 0nlyth3truth, 4th-otaku, Abdull, Aboeing, Agentbla, Anon lynx, Antaeus Feldspar, Apantomimehorse, Arthena, Arvindn, Auntof6, Avocade, Awmorp, Babybahamut, Beamishboy, Bekant, Bigs slb, Binary TSO, Bird of paradox, BitJam, Bob.v.R, Bobblewik, Bryan Derksen, Bumm13, Caesura, Cantalamessa, Cerireid, Chekholko, Cihan, Conversion script, Copeland.James.H, Cryptonaut, Cunchem, D6, DGerman, DanielPharos, DavidCBryant, DavidCary, Davidhorman, Deflective, Dogposter, Drhex, Dthede, Dysprosia, Eastlaw, Em3ryguy, Eric119, EugeneZelenko, EverGreg, Eyreland, Firien, Flying hazard, Francs2000, Gaius Cornelius, GearCutter, Giftlite, Gmazeroff, Gnorthup, GringoCroco, Gwern, Helder.wiki, Icecats11, Inventio, Iridescent, Izab1, Jaidan, Jao, Jarble, Jcea, Jdseymour, Jerry, Jkl, Joey7643, Jorge Stolfi, Kenyon, Linas, Lir, Lotje, Lowellian, Mandarax, Mange01, Margin1522, Martarius, Masterpiece2000, Materialscientist, Mathuranathan, Michael Hardy, Mitchan, Miterdale, Mmeijeri, Morten, Music Sorter, MuthuKutty, Mysid, Nageh, Netpilot43556, Niceguyedc, Nihiltres, Nubiatech, Ojl, Oleg Alexandrov, Oli Filth, Phantomsteve, Phr, Pinethicket, Piotrus, Pnm, Poeloq, Pol098, Pouya, PrologFan, Pudhed, R'n'B, Radagast83, Ranjithsutari, Rbrewer42, Reedy, Reetep, Requestion, RexNL, Rgray001, Ricky81682, Ring0, Rjstott, Ruud Koot, Ryan Roos, SDS, SNIyer12, Sepia tone, Seven of Nine, Sikilai, Sixequalszero, Snowolf, Snoyes, Splash, Splintercellguy, Spoon!, Suffusion of Yellow, Supercoop, Swerdnaneb, T-rex, Taral, Technopilgrim, Teejayo, Tejas81, The Anome, The Rationalist, The Thing That Should Not Be, Tobias Bergemann, Traal, Trachten, Ubunaki, Unco, Vipinhari, WatchAndObserve, Wavelength, WestwoodMatt, Woohookitty, Ykhwong, ZeroOne, 231 anonymous edits Linear encoder Source: http://en.wikipedia.org/w/index.php?oldid=463085769 Contributors: C mobile, CWA-FRB, Colonies Chris, Fiducial, Fresheneesz, Lightmouse, Magioladitis, Malcolma, Matesy, Mechgyver, Mild Bill Hiccup, Nagle, Pt130135, R'n'B, Radiojon, 28 anonymous edits Gray code Source: http://en.wikipedia.org/w/index.php?oldid=469363225 Contributors: A876, AJP, Acerperi, Ahoerstemeier, Ahseaton, AileTheAlien, Andy Dingley, Animagi1981, BAxelrod, BD2412, Bender235, Bggoldie, Bobblehead, Bogdangiusca, BrotherE, Bubba73, Bunyk, Cburnett, Charles Matthews, Charvest, Cholling, Chris the speller, ChrisEich, CosineKitty, Cyp, David Eppstein, David.Monniaux, DavidCary, Daxx wp, Dcoetzee, Deacs33, Demi, Denelson83, Dicklyon, Dina, Dispenser, Djfeldman, Dr0b3rts, Droll, Edratzer, El C, Elias, Elphion, Emvee, Fcdesign, Fresheneesz, Gerbrant, Giftlite, Gnorthup, GoingBatty, Haham hanuka, Hariva, Heron, Hv, Iamfscked, Inductiveload, Iridescent, Isaac Dupree, Isiah Schwartz, JLaTondre, Jaiguru42, Jan olieslagers, Jason Recliner, Esq., Jeff 113, Jim1138, Jjbeard, Johnmorgan, Johnuniq, Jos.koot, Jturkia, Kanesue, Kateshortforbob, Kjkjava, Law, Leonard G., LilHelpa, Linas, Lipedia, MER-C, MSGJ, Machine Elf 1735, MarkSweep, MattGiuca, Matusz, Max711, Maxal, Maxalot, Mellum, Michael Hardy, Mike1024, Mild Bill Hiccup, MooMan1, MrOllie, MyrddinE, Nick Pisarro, Jr., Nikevich, Nneonneo, Noe, Oashi, Ocatecir, Ohiosoil, Pacifier, Pakaran, Pgimeno, PierreAbbat, Piet Delport, Plamka, Pleasantville, Plugwash, Prashantgonarkar, PuerExMachina, Qwyrxian, RTC, RayKiddy, Requestion, Rich Farmbrough, Ricky81682, Rjwilmsi, RobH (2004 account), Ronz, Sakurambo, SciCompTeacher, Sciyoshi, Seantellis, Seraphimblade, Shellreef, Sjock, Smalljim, Snowolf, SuneJ, Suruena, Svick, Tedickey, TheNightFly, Tomo, TutterMouse, Vanish2, Vanka5, Wapcaplet, Wikkrockiana, Winzurf, Wwoods, XJamRastafire, Yahya Abdal-Aziz, Yoshigev, Yworo, ZeroOne, Zeycus, 197 anonymous edits Rotary encoder Source: http://en.wikipedia.org/w/index.php?oldid=465481711 Contributors: Ace Frahm, AlexHe34, Alhaiksandher, AliasFan4Lyfe, Aliasgherman, Americanhero, Artie p, Attilios, Bazkaz, Beta16, Birdav, Bongwarrior, Calltech, Cat5nap, Cburnett, Coasterlover1994, CommonsDelinker, Cristan, Damian Yerrick, Dancter, Danim, David-bel, Dhollm, Dina, Dr0b3rts, Dtgriscom, EUROFARAD, Erik9, Evanh, Gamekock, Halfgaar, Heron, Hooperbloob, IP83, Jaho, Jamesooders, Jcdonelson, JohnCD, Johncatsoulis, Julesd, Katana, Kithira, Kku, Mgros, Mike1024, Nagle, North8000, Ocaasi, Oceano2012, Oli Filth, Omegatron, Onionmon, Overand, Phlake, RoySmith, Rqtfi, Rspanton, SBunce, Sagsaw, Sdec25, Seaphoto, Semiwiki, Shai-kun, Shubinator, Sinaerem, Sintaku, Snibble, Srleffler, Sscott328, Steelerdon, Suruena, THEN WHO WAS PHONE?, TheDog, Therealpaulie, Tpikonen, Victor Korniyenko, WikHead, Wikkrockiana, Wolfch, Wtshymanski, חובבשירה, ﻣﺎﻧﻲ, 136 anonymous edits Bode plot Source: http://en.wikipedia.org/w/index.php?oldid=466646364 Contributors: A2Kafir, Andy.ackland, Anna Lincoln, Arcturus4669, Avicennasis, Avoided, BenFrantzDale, Betacommand, Bobblehead, Brews ohare, Buffetrand, Chaosgate, Clam0p, Clemens vi, Dbtfz, Derdeib, Dicklyon, Dr.K., Drew335, Ec5618, ExportRadical, Falcorian, Fresheneesz, Gaius Cornelius, Ganzor, Giftlite, Gmoose1, Graibeard, Gravix, Heron, Hooperbloob, Huntthetroll, Jamelan, Jeff P Harris, Jiuguang Wang, Jive Dadson, Joy, Jshen6, KPH2293, Kenyon, Kmellem, Lambertch, Mdd, Mehran, Mellery, Michael Devore, Michael Hardy, Miguelgoldstein, Mivey6, Mr. PIM, Nabla, Nbarth, Niketmjoshi, Nitin.mehta, Nodlit, Omegatron, Patrick, Peter.scottie, Phosphoricx, Poulpy, Private Pilot, Redheylin, Rehnn83, Requestion, Shadowjams, SimonP, Sommacal alfonso, Spradlig, Stw, Tim Starling, Toffile, Tomer shalev, User A1, Vukg, WakingLili, Wtshymanski, Zoomzoom1, 114 anonymous edits Nyquist stability criterion Source: http://en.wikipedia.org/w/index.php?oldid=467069219 Contributors: Aminrahimian, Attilios, Beta16, Brews ohare, Buffetrand, Charles Matthews, Cuaxdon, Dicklyon, Duncharris, Ewlyahoocom, Giftlite, Haditya, Hvn0413, Ikiwaner, Jenblower, Jiuguang Wang, Jm1234567890, Krishnavedala, LachlanA, Linuxlad, Lupo, Lv131, Mailer diablo, Matutano, Merovingian, Minhlong87, Musically ut, Nillerdk, Novangelis, Pharaoh of the Wizards, Phobulos, PoqVaUSA, Preben Holm, Smalljim, SunCreator, Toffile, Trusilver, Wknight8111, XMxWx, Пика Пика, 58 anonymous edits Instability Source: http://en.wikipedia.org/w/index.php?oldid=464134809 Contributors: Anticipation of a New Lover's Arrival, The, Art Carlson, Bluemoose, Chris the speller, Chuckiesdad, Chyeburashka, Complexica, Crowsnest, Cyrus Grisham, David Kendall, David R Merritt, Djbaniel, Donarreiskoffer, Eassin, Evolauxia, Gasheadsteve, Glenn, Grm wnr, Headbomb, Hornandsoccer, Iantresman, JIP, Jenesis, Jiuguang Wang, Johnwsimpson, Jujutacular, JzG, Kvng, Mais oui!, Mgmirkin, Mleconte, Mstrickl, Nherm, Nikai, Oleg Alexandrov, Open2universe, Originalwana, Ottojack, Pearle, Petri Krohn, Rs2, Salih, Silverfish, The RedBurn, Tokamac, UlmPhysiker, Valentinian, Vsmith, WilyD, XJamRastafire, Ytrottier, 20 anonymous edits Frequency response Source: http://en.wikipedia.org/w/index.php?oldid=468175898 Contributors: Andrei Stroe, Antaya, Askbojesen, BenFrantzDale, Binksternet, C6H3N3O3, Cburnett, CosineKitty, Furrykef, GRAHAMUK, Giftlite, Glenn, H.sh8241, Headbomb, Hooperbloob, Jag123, Janyllp, Java13690, Jj137, Jorge Stolfi, Krishnavedala, Laurascudder, Light current, Materialscientist, Mbrennwa, Metacomet, Mulad, Mwilde, Neelix, Nillerdk, Nnh, Oarih, Omegatron, Onco p53, PoqVaUSA, Redheylin, Someguy1221, Spinningspark, Steve Quinn, Tomasz Zuk, Tomer shalev, Trumpeter96, Tsiaojian lee, Wolps, Yaris678, Zoicon5, 35 anonymous edits Nyquist plot Source: http://en.wikipedia.org/w/index.php?oldid=468278645 Contributors: A. B., Attilios, Bazz, Brews ohare, Cuaxdon, CyrilB, Engelec, Flehmen, Giftlite, Hemanshu, Heron, Hooperbloob, Jeff P Harris, Jiuguang Wang, John of Reading, JorisvS, Linuxlad, MarkusHagenlocher, Mdd, Mets501, Michael Hardy, Mpassman, Nbarth, Omegatron, Patrick, Pjvpjv, Pladask, PoqVaUSA, Reconsider the static, Ro8269, Rod57, Salasks, So God created Manchester, Spiff, The Anome, Titan, Toffile, Trojancowboy, Vonkje, XMxWx, Yorrak, 36 anonymous edits Nichols plot Source: http://en.wikipedia.org/w/index.php?oldid=464927680 Contributors: B4hand, Bazz, Blair Bonnett, Chthonicdaemon, Encyclops, Engelec, Giftlite, Hooperbloob, Mdd, Michael Hardy, Peter17, The Anome, Tuvosi, 4 anonymous edits Bandwidth (signal processing) Source: http://en.wikipedia.org/w/index.php?oldid=460610886 Contributors: 16@r, 198.211.186.xxx, 203.109.250.xxx, A3r0, Academic Challenger, Agi896, Ahoerstemeier, Alfio, Ali@gwc.org.uk, Alinja, Andonic, Andre Engels, Andrewpmk, Andy120290, Archer3, Archimerged, Atlant, AxelBoldt, Bandwith, BenFrantzDale, Binksternet, Blobglob, Bobblewik, Bobo192, Brews ohare, CONFIQ, Carbonite, Cart73, Cburnett, Christian Bienia, Cihan, Cmathas, Conversion script, Coredesat, CosineKitty, DVD R W, DarkArcher, Dbroadwell, Delinquer, DerHexer, Dicklyon, Don4of4, Dysprosia, Epbr123, Escapepea, Falling Cloud, Fatsamsgrandslam, FreplySpang, FrozenUmbrella, Gbleem, Gene Nygaard, Giftlite, Gilliam, Graham87, Green Giant, Green Heart 1985, GunnerJr, Gwernol, Henrikb4, Heron, Httpwebwitch, Ianharvey, Ianneub, Inductiveload, Jacobolus, Jamelan, Jasonrichmond, Jestar, Jhbdel, Jim.henderson, JohnTechnologist, JonHarder, Jongleur100, Jordav, JordoCo, Kumioko, KyraVixen, Light current, Lightmouse, Lloydsmart, Lord Snoeckx, Madewokherd, Maglev2, Mange01, Marek69, Mav, Mbell, Mejor Los Indios, Melaniemm, Metacomet, Michael Hardy, Mike Logghe, Mnmngb, Motmajor, Mrdvt92, Mrendo, Munford, MuthuKutty, N6gn, Nbarth, Nekkensj, Noisy, Noosentaal, Nposs, Ohconfucius, Ojw, Oleg Alexandrov, Omegatron, Onceonthisisland, OverlordQ, Oysterguitarist, Patrick, PatrikR, Patstuart, PeterFlannery, Phe, Philip Trueman, Prenz, R'n'B, Redheylin, Reinthal, RexNL, Ribilla, Richard001, Rintrah, Rmaarts, RobertG, RoyBoy, Rs2, Rwestafer, S415046, Saihtam, Scorpion, Sharewarepro, Shark96z, Shawn 357, Shawnc, Skicavs, Smack, Smyth, Srleffler, Steve8675309, Stevonova, The Anome, Tiddly Tom, Tim Starling, Tins128, Titoxd, TonyTiger22, Vriullop, Vspengen, Waveguy, Weihao.chiu, West London Dweller, Whiskers9, Wknight94, Wolfkeeper, Wtshymanski, X736e65616b, Yonatan, Yosri, Yudiweb, ZZninepluralZalpha, Zalgo, Zereshk, Zowie, 311 anonymous edits Lead–lag compensator Source: http://en.wikipedia.org/w/index.php?oldid=451172418 Contributors: Ademkader, Chris Melton, Cmichael, Dicklyon, LachlanA, Michael Hardy, Ojw, Oli Filth, Oxnard28, Razorflame, Rogerbrent, Shoez, Tariqabjotu, TedPavlic, Vonkje, XieChengnuo, 33 anonymous edits Static VAR compensator Source: http://en.wikipedia.org/w/index.php?oldid=470109427 Contributors: Anders751004, ArséniureDeGallium, BillC, Clampower, Cmdrjameson, Desert Tripper, Dhollm, Eassin, Fireglowe, Grafen, Iridescent, Karada, Linas, Mariraja2007, Paul Comfort, Rjwilmsi, Scampiandchips, Sladen, Solipsist, SpK, Tommy2010, Tony Fox, Uncle G, Volpix0, Wtshymanski, 28 anonymous edits 318 Article Sources and Contributors PID controller Source: http://en.wikipedia.org/w/index.php?oldid=469730997 Contributors: 2m man, A. Carty, A3RO, AaronMK, Abarry, Acdx, Ademkader, Aeolian, Alexius08, Alll, Altzinn, Aluvus, AnitaS, Anna Lincoln, Attilios, Aturevsk, AutomationBooks, Bdijkstra, BeastRHIT, Ben pcc, BenFrantzDale, Benjamander, Bentogoa, Billymac00, Bobblewik, Bora Eryilmaz, Bpavel88, BradBeattie, Briceb, Bsodmike, Buesser, CanisRufus, CapitalR, Carmichael95, Caslon, Charles Matthews, Chiklit, ChristianG2, Ciphergoth, Cometstyles, Computationalverb, Control.optimization, Copeland.James.H, Coredesat, Crinoid, Crohnie, CyberneticSheepCZ, DARTH SIDIOUS 2, Dabozz88, Dangrayorg, DanielRigal, Dbaechtel, Dedmonds, Dethme0w, Deville, Dhatfield, Dicklyon, Dinhxuanduyet, Dja25, Djgreenfield, Dmcq, Dominick, Dzkd, Ejwong, Elcobbola, Encyclops, Engelec, Epolk, FastIkarus, Femto, Fenn fukai, Frank Lofaro Jr., Frappucino, Fæ, GB fan, Gabelstaplerfahrer, Gaius Cornelius, GarrettSocling, Gene Nygaard, Giftlite, Glane23, GunnerJr, HK-90, Hankwang, Harriv, HenkvD, Heron, HexaChord, Hughjack, Icairns, J.delanoy, JLLCuba, JamieStapleton, Jbcmf, Jdigangi, Je bonilla, Jeff G., Joel Saks, Jon Mansfield, JonDePlume, JonathonReinhart, Jrdioko, Karthikkumar411, Kbosak, Kerotan, Kku, Kreuzer5, Kvng, LHOON, Laurentsavaete, Learnfpga, LiDaobing, LieAfterLie, Liu.Yunxia, Liyang, LyonL, Marsian, Masgatotkaca, Mausy5043, Mav, Mbbradford, Mbeychok, Mbutts, Md2perpe, Meatmanek, Meestaplu, Michael Hardy, Mikiemike, Miles underwood, Mmeijeri, Mononoke, Motafa, Movedgood, MrOllie, Mustafa Al-mosawi, N2e, Nave.notnilc, Nbarth, Nigelj, Nitin.mehta, NunoSoares, Oh Snap, Oli Filth, Omegatron, Orzetto, Othompson, Papa November, Patrick, Piano non troppo, Pko, Printer222, ProcessControlGuy, Profr ettore, Pteromys, Q Science, Ray Van De Walker, Rdautel, Redbeanpaste, Relilles, Requestion, Rich Farmbrough, Robert - Northern VA, Rod57, Ronz, RoyKok, Rspadim, Russvdw, S.borchers, SD5, SMC89, Saxbryn, Seanp, Seaphoto, Sectryan, Sesc, Shadowjams, Shanes, Shimei, Shriramrs31, Signalhead, SilverStar, Sinasgay, Sk wiki, Skorkmaz, Smzoha, Sonett72, Spalding, Spiel496, Spradlig, Stephenb, Steveaa, Strait, Swrkoab, Teoryn, The Anome, Tremilux, Trusilver, Ummit, Unbitwise, Undead warrior, Unregistered.coward, User A1, Van helsing, Vary, Vivek.cd, Vizcarra, Wahapure, WakingLili, Warniats, WikiDao, Wmahan, Wtshymanski, Xenodevil, XieChengnuo, Yasirniazkhan, Zephalis, Zerodamage, Zoomzoom1, 680 anonymous edits 319 Image Sources, Licenses and Contributors Image Sources, Licenses and Contributors Image:LOCKHEED F-117A NIGHT HAWK.png Source: http://en.wikipedia.org/w/index.php?title=File:LOCKHEED_F-117A_NIGHT_HAWK.png License: Public Domain Contributors: Cobatfor, D-Kuru, Joshbaumgartner, Matthewedwards, Orlovic, Slomox, Stahlkocher, SvonHalenbach, 1 anonymous edits Image:F-117A GBU-28.JPEG Source: http://en.wikipedia.org/w/index.php?title=File:F-117A_GBU-28.JPEG License: Public Domain Contributors: Original uploader was Koalorka at en.wikipedia (Original text : MSGT EDWARD SNYDER) Image:Schnittbild drehnatrieb 01.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Schnittbild_drehnatrieb_01.jpg License: Creative Commons Attribution-Sharealike 2.0 Contributors: HerbstrittMse Image:insulin pump with infusion set.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Insulin_pump_with_infusion_set.jpg License: Public Domain Contributors: Original uploader was Mbbradford at en.wikipedia Image:SRH025-p40.jpg Source: http://en.wikipedia.org/w/index.php?title=File:SRH025-p40.jpg License: Public Domain Contributors: Original uploader was user:Megapixie at en.wikipedia Image:Space Shuttle Columbia launching.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Space_Shuttle_Columbia_launching.jpg License: Public Domain Contributors: NASA Image:Ideal feedback model.svg Source: http://en.wikipedia.org/w/index.php?title=File:Ideal_feedback_model.svg License: Public Domain Contributors: Me (Intgr) Image:ACTH Negative Feedback.svg Source: http://en.wikipedia.org/w/index.php?title=File:ACTH_Negative_Feedback.svg License: Creative Commons Attribution 3.0 Contributors: DRosenbach File:Herdwick Stampede.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Herdwick_Stampede.jpg License: Creative Commons Attribution 2.0 Contributors: Andy Docker from England File:Birmingham Northern Rock bank run 2007.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Birmingham_Northern_Rock_bank_run_2007.jpg License: Creative Commons Attribution-Sharealike 2.0 Contributors: 159753, Infrogmation, Man vyi, OSX, Yarl Image:Positive feedback bistable switch.svg Source: http://en.wikipedia.org/w/index.php?title=File:Positive_feedback_bistable_switch.svg License: Public Domain Contributors: Naj-GMU, Squidonius, THEN WHO WAS PHONE?, 1 anonymous edits Image:Phanerozoic Biodiversity.svg Source: http://en.wikipedia.org/w/index.php?title=File:Phanerozoic_Biodiversity.svg License: GNU Free Documentation License Contributors: SVG version by Albert Mestre File:Regenerartive Receiver-S7300056.JPG Source: http://en.wikipedia.org/w/index.php?title=File:Regenerartive_Receiver-S7300056.JPG License: Public Domain Contributors: Ozguy89 Image:Smitt hysteresis graph.svg Source: http://en.wikipedia.org/w/index.php?title=File:Smitt_hysteresis_graph.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: FDominec Image:R-S mk2.gif Source: http://en.wikipedia.org/w/index.php?title=File:R-S_mk2.gif License: Creative Commons Attribution 2.0 Contributors: Napalm Llama File:Technics SL-1210MK2.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Technics_SL-1210MK2.jpg License: GNU Free Documentation License Contributors: Cschirp. Original uploader was Cschirp at de.wikipedia File:Adam Savage HOPE.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Adam_Savage_HOPE.jpg License: Creative Commons Attribution-Sharealike 3.0 Contributors: Original uploader was Porkrind at en.wikipedia Image:IDEF Diagram Example.jpg Source: http://en.wikipedia.org/w/index.php?title=File:IDEF_Diagram_Example.jpg License: Public Domain Contributors: Defense Acquisition University Image:Fo2ufg823rhf832hfdorfg.JPG Source: http://en.wikipedia.org/w/index.php?title=File:Fo2ufg823rhf832hfdorfg.JPG License: Public Domain Contributors: DFD guru Image:Functional Flow Block Diagram Format.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Functional_Flow_Block_Diagram_Format.jpg License: Public Domain Contributors: Original uploader was Mdd at en.wikipedia Image:IPO-S Model.gif Source: http://en.wikipedia.org/w/index.php?title=File:IPO-S_Model.gif License: Public Domain Contributors: DXBari Image:N2 chart definition.JPG Source: http://en.wikipedia.org/w/index.php?title=File:N2_chart_definition.JPG License: Public Domain Contributors: NASA Image:SADT.svg Source: http://en.wikipedia.org/w/index.php?title=File:SADT.svg License: Public Domain Contributors: UlrichAAB Image:BPMN-AProcesswithNormalFlow.jpg Source: http://en.wikipedia.org/w/index.php?title=File:BPMN-AProcesswithNormalFlow.jpg License: Public Domain Contributors: Tttt1234 Image:FEA BRM Hierachy.JPG Source: http://en.wikipedia.org/w/index.php?title=File:FEA_BRM_Hierachy.JPG License: Public Domain Contributors: doi.gov Image:PD-icon.svg Source: http://en.wikipedia.org/w/index.php?title=File:PD-icon.svg License: Public Domain Contributors: Alex.muller, Anomie, Anonymous Dissident, CBM, MBisanz, Quadell, Rocket000, Strangerer, Timotheus Canens, 1 anonymous edits Image:Functional block diagram of the attitude control and maneuvering electronics system .jpg Source: http://en.wikipedia.org/w/index.php?title=File:Functional_block_diagram_of_the_attitude_control_and_maneuvering_electronics_system_.jpg License: Public Domain Contributors: McDonnell, "Project Gemini Familiarization Charts," File:Convolution of box signal with itself2.gif Source: http://en.wikipedia.org/w/index.php?title=File:Convolution_of_box_signal_with_itself2.gif License: Creative Commons Attribution-Sharealike 3.0 Contributors: Convolution_of_box_signal_with_itself.gif: Brian Amberg derivative work: Tinos (talk) File:Convolution of spiky function with box2.gif Source: http://en.wikipedia.org/w/index.php?title=File:Convolution_of_spiky_function_with_box2.gif License: Creative Commons Attribution-Sharealike 3.0 Contributors: Convolution_of_spiky_function_with_box.gif: Brian Amberg derivative work: Tinos (talk) File:Convolution3.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Convolution3.PNG License: Public Domain Contributors: Original uploader was w:en:User:EmoteEmote at en.wikipedia Later version(s) were uploaded by w:en:User:McLoafMcLoaf at en.wikipedia. File:Lorenz attractor yb.svg Source: http://en.wikipedia.org/w/index.php?title=File:Lorenz_attractor_yb.svg License: Creative Commons Attribution-Sharealike 2.5 Contributors: User:Dschwen, User:Wikimol File:LinearFields.png Source: http://en.wikipedia.org/w/index.php?title=File:LinearFields.png License: Creative Commons Attribution 2.5 Contributors: XaosBits Image:circ-4-nor.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Circ-4-nor.jpg License: Creative Commons Attribution 3.0 Contributors: Henning.Mortveit Image:circ-4-nor-1234.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Circ-4-nor-1234.jpg License: Creative Commons Attribution 3.0 Contributors: Henning.Mortveit File:Prop-tableau-4.svg Source: http://en.wikipedia.org/w/index.php?title=File:Prop-tableau-4.svg License: GNU Free Documentation License Contributors: Original uploader was Tizio at en.wikipedia. Later version(s) were uploaded by RobHar at en.wikipedia. Image:Pole Splitting Example.png Source: http://en.wikipedia.org/w/index.php?title=File:Pole_Splitting_Example.png License: GNU Free Documentation License Contributors: Brews ohare Image:Pole Splitting Example with Miller Transform.png Source: http://en.wikipedia.org/w/index.php?title=File:Pole_Splitting_Example_with_Miller_Transform.png License: Creative Commons Attribution-Share Alike Contributors: Brews ohare Image:Two-pole Bode magnitude plot.png Source: http://en.wikipedia.org/w/index.php?title=File:Two-pole_Bode_magnitude_plot.png License: Creative Commons Attribution-Share Alike Contributors: Brews ohare Image:Compensation capacitance.png Source: http://en.wikipedia.org/w/index.php?title=File:Compensation_capacitance.png License: GNU Free Documentation License Contributors: Brews ohare Image:Damped spring.gif Source: http://en.wikipedia.org/w/index.php?title=File:Damped_spring.gif License: Public Domain Contributors: Oleg Alexandrov Image:2nd Order Damping Ratios.svg Source: http://en.wikipedia.org/w/index.php?title=File:2nd_Order_Damping_Ratios.svg License: Public Domain Contributors: Inductiveload Image:Impulse.png Source: http://en.wikipedia.org/w/index.php?title=File:Impulse.png License: Public domain Contributors: w:User:IainIain 09:41, 7 June 2006 (UTC) Image:LTI.png Source: http://en.wikipedia.org/w/index.php?title=File:LTI.png License: Public Domain Contributors: Flappiefh, Joelholdsworth, Ma-Lik, Maksim File:Feedback loop with descriptions.svg Source: http://en.wikipedia.org/w/index.php?title=File:Feedback_loop_with_descriptions.svg License: Creative Commons Attribution-Share Alike Contributors: Myself File:Boulton and Watt centrifugal governor-MJ.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Boulton_and_Watt_centrifugal_governor-MJ.jpg License: Creative Commons Attribution 3.0 Contributors: Dr. Mirko Junge File:simple feedback control loop2.png Source: http://en.wikipedia.org/w/index.php?title=File:Simple_feedback_control_loop2.png License: unknown Contributors: Corona 320 Image Sources, Licenses and Contributors Image:Variables proporcionals.png Source: http://en.wikipedia.org/w/index.php?title=File:Variables_proporcionals.png License: GNU Free Documentation License Contributors: M. Romero Schmidkte Image:Academ_homothetic_rectangles.svg Source: http://en.wikipedia.org/w/index.php?title=File:Academ_homothetic_rectangles.svg License: Creative Commons Attribution-Sharealike 3.0 Contributors: Yves Baelde File:Integral example.svg Source: http://en.wikipedia.org/w/index.php?title=File:Integral_example.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: KSmrq File:ArabicIntegralSign.svg Source: http://en.wikipedia.org/w/index.php?title=File:ArabicIntegralSign.svg License: Public Domain Contributors: File:Integral approximations.svg Source: http://en.wikipedia.org/w/index.php?title=File:Integral_approximations.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: KSmrq File:Integral Riemann sum.png Source: http://en.wikipedia.org/w/index.php?title=File:Integral_Riemann_sum.png License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: KSmrq File:Riemann sum convergence.png Source: http://en.wikipedia.org/w/index.php?title=File:Riemann_sum_convergence.png License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: KSmrq Image:RandLintegrals.png Source: http://en.wikipedia.org/w/index.php?title=File:RandLintegrals.png License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: Aaron Rotenberg, Darapti, Dubhe File:Improper integral.svg Source: http://en.wikipedia.org/w/index.php?title=File:Improper_integral.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: KSmrq File:Improper integral unbounded internally.svg Source: http://en.wikipedia.org/w/index.php?title=File:Improper_integral_unbounded_internally.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: KSmrq File:Volume under surface.png Source: http://en.wikipedia.org/w/index.php?title=File:Volume_under_surface.png License: Public Domain Contributors: Oleg Alexandrov File:Line-Integral.gif Source: http://en.wikipedia.org/w/index.php?title=File:Line-Integral.gif License: GNU Free Documentation License Contributors: Cronholm144, Darapti, McZusatz, Nandhp, SkiDragon File:Surface integral illustration.png Source: http://en.wikipedia.org/w/index.php?title=File:Surface_integral_illustration.png License: Public Domain Contributors: Darapti, Jahobr, Oleg Alexandrov, WikipediaMaster File:Numerical quadrature 4up.png Source: http://en.wikipedia.org/w/index.php?title=File:Numerical_quadrature_4up.png License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: KSmrq File:Linear PK Example.png Source: http://en.wikipedia.org/w/index.php?title=File:Linear_PK_Example.png License: Creative Commons Attribution 3.0 Contributors: Alfie↑↓© File:Tangent to a curve.svg Source: http://en.wikipedia.org/w/index.php?title=File:Tangent_to_a_curve.svg License: Public Domain Contributors: Original uploader was Jacj at en.wikipedia Later versions were uploaded by Oleg Alexandrov at en.wikipedia. File:Graph of sliding derivative line.gif Source: http://en.wikipedia.org/w/index.php?title=File:Graph_of_sliding_derivative_line.gif License: Public Domain Contributors: en:User:Dino Image:Tangent-calculus.svg Source: http://en.wikipedia.org/w/index.php?title=File:Tangent-calculus.svg License: GNU Free Documentation License Contributors: derivative work: Pbroks13 (talk) Tangent-calculus.png: Rhythm Image:Secant-calculus.svg Source: http://en.wikipedia.org/w/index.php?title=File:Secant-calculus.svg License: GNU Free Documentation License Contributors: derivative work: Pbroks13 (talk) Secant-calculus.png: Shizhao Image:Lim-secant.svg Source: http://en.wikipedia.org/w/index.php?title=File:Lim-secant.svg License: GNU Free Documentation License Contributors: derivative work: Pbroks13 (talk) Lim-secant.png: Original uploader was CSTAR at en.wikipedia File:Right-continuous.svg Source: http://en.wikipedia.org/w/index.php?title=File:Right-continuous.svg License: Public Domain Contributors: w:User:JacjJacj File:Absolute value.svg Source: http://en.wikipedia.org/w/index.php?title=File:Absolute_value.svg License: GNU Free Documentation License Contributors: This hand-written SVG version by Qef Original bitmap version Image:Absolute_value.png by Ævar Arnfjörð Bjarmason Image:Lur'e_Problem_Block.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Lur'e_Problem_Block.jpg License: Creative Commons Attribution 3.0 Contributors: Pivs File:Centrifugal governor.png Source: http://en.wikipedia.org/w/index.php?title=File:Centrifugal_governor.png License: Public Domain Contributors: Dicklyon, Glenn, Guam, Liftarn, Mdd, MdeVicente, Pieter Kuiper, Tano4595, WikipediaMaster, 2 anonymous edits Image:MRAC.svg Source: http://en.wikipedia.org/w/index.php?title=File:MRAC.svg License: Public Domain Contributors: Original uploader was Pekaje at en.wikipedia Image:MIAC.svg Source: http://en.wikipedia.org/w/index.php?title=File:MIAC.svg License: Public Domain Contributors: Original uploader was Pekaje at en.wikipedia Image:Neural network example.svg Source: http://en.wikipedia.org/w/index.php?title=File:Neural_network_example.svg License: Public Domain Contributors: User:Wiso File:Forest of synthetic pyramidal dendrites grown using Cajal's laws of neuronal branching.png Source: http://en.wikipedia.org/w/index.php?title=File:Forest_of_synthetic_pyramidal_dendrites_grown_using_Cajal's_laws_of_neuronal_branching.png License: Creative Commons Attribution 2.5 Contributors: Mietchen, NeverDoING Image:Fuzzy logic temperature en.svg Source: http://en.wikipedia.org/w/index.php?title=File:Fuzzy_logic_temperature_en.svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: fullofstars Image:RL&ZARL-(1 2)-(1 3 5 1).png Source: http://en.wikipedia.org/w/index.php?title=File:RL&ZARL-(1_2)-(1_3_5_1).png License: Public Domain Contributors: Mintz l File:Root-locus-example-diagram.png Source: http://en.wikipedia.org/w/index.php?title=File:Root-locus-example-diagram.png License: Public Domain Contributors: Sss41 Image:Step response.png Source: http://en.wikipedia.org/w/index.php?title=File:Step_response.png License: GNU Free Documentation License Contributors: Brews ohare File:Optical_Encoders.png Source: http://en.wikipedia.org/w/index.php?title=File:Optical_Encoders.png License: Creative Commons Attribution-Share Alike Contributors: Fiducial File:Visualisierung der magnetischen Struktur eines Linearencoders (Aufnahme mit MagView).jpg Source: http://en.wikipedia.org/w/index.php?title=File:Visualisierung_der_magnetischen_Struktur_eines_Linearencoders_(Aufnahme_mit_MagView).jpg License: Creative Commons Attribution-Sharealike 3.0 Contributors: User:Matesy File:Circular Lissajous.gif Source: http://en.wikipedia.org/w/index.php?title=File:Circular_Lissajous.gif License: Creative Commons Attribution-Sharealike 3.0 Contributors: Fiducial File:Quadrature Diagram.svg Source: http://en.wikipedia.org/w/index.php?title=File:Quadrature_Diagram.svg License: Public Domain Contributors: Sagsaw Image:Reflected binary Gray 2632058.png Source: http://en.wikipedia.org/w/index.php?title=File:Reflected_binary_Gray_2632058.png License: Public Domain Contributors: Bulwersator, Dicklyon, Liftarn, Tassedethe Image:US02632058 Gray.png Source: http://en.wikipedia.org/w/index.php?title=File:US02632058_Gray.png License: Public Domain Contributors: Original uploader was Dicklyon at en.wikipedia Image:Encoder Disc (3-Bit).svg Source: http://en.wikipedia.org/w/index.php?title=File:Encoder_Disc_(3-Bit).svg License: Public Domain Contributors: jjbeard File:Gray code rotary encoder 13-track opened.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Gray_code_rotary_encoder_13-track_opened.jpg License: Public Domain Contributors: Mike1024 File:Binary-reflected Gray code construction.svg Source: http://en.wikipedia.org/w/index.php?title=File:Binary-reflected_Gray_code_construction.svg License: Public Domain Contributors: Inductiveload File:Gray code permutation matrix 16.svg Source: http://en.wikipedia.org/w/index.php?title=File:Gray_code_permutation_matrix_16.svg License: Public Domain Contributors: Lipedia Image:Enkelspoors-Graycode.svg Source: http://en.wikipedia.org/w/index.php?title=File:Enkelspoors-Graycode.svg License: Creative Commons Attribution-Sharealike 2.5 Contributors: Gerbrant Image:Rotatory EnDat Encoder.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Rotatory_EnDat_Encoder.jpg License: GNU Free Documentation License Contributors: IP83 Image:Encoder disc (3-Bit binary).svg Source: http://en.wikipedia.org/w/index.php?title=File:Encoder_disc_(3-Bit_binary).svg License: Creative Commons Attribution-ShareAlike 3.0 Unported Contributors: en:User:Cburnett File:ROD420 HEIDENHAIN.jpg Source: http://en.wikipedia.org/w/index.php?title=File:ROD420_HEIDENHAIN.jpg License: Creative Commons Attribution 3.0 Contributors: Victor Korniyenko Image:Quadrature Diagram.svg Source: http://en.wikipedia.org/w/index.php?title=File:Quadrature_Diagram.svg License: Public Domain Contributors: Sagsaw 321 Image Sources, Licenses and Contributors Image:Odometergearcropped.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Odometergearcropped.jpg License: GNU Free Documentation License Contributors: Original uploader was Nagle at en.wikipedia Image:Bode High-Pass.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_High-Pass.PNG License: GNU Free Documentation License Contributors: Brews ohare Image:Bode Low-Pass.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Low-Pass.PNG License: Creative Commons Attribution-Share Alike Contributors: Brews ohare Image:Bode Low Pass Magnitude Plot.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Low_Pass_Magnitude_Plot.PNG License: GNU Free Documentation License Contributors: Brews ohare Image:Bode Low Pass Phase Plot.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Low_Pass_Phase_Plot.PNG License: GNU Free Documentation License Contributors: Brews ohare Image:Bode Pole-Zero Magnitude Plot.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Pole-Zero_Magnitude_Plot.PNG License: GNU Free Documentation License Contributors: Brews ohare Image:Bode Pole-Zero Phase Plot.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Bode_Pole-Zero_Phase_Plot.PNG License: GNU Free Documentation License Contributors: Brews ohare Image:Magnitude of feedback amplifier.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Magnitude_of_feedback_amplifier.PNG License: Creative Commons Attribution-Share Alike Contributors: Brews ohare Image:Phase of feedback amplifier.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Phase_of_feedback_amplifier.PNG License: GNU Free Documentation License Contributors: Brews ohare Image:Gain Margin.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Gain_Margin.PNG License: GNU Free Documentation License Contributors: Brews ohare Image:Phase Margin.PNG Source: http://en.wikipedia.org/w/index.php?title=File:Phase_Margin.PNG License: GNU Free Documentation License Contributors: Brews ohare Image:Bodeplot.png Source: http://en.wikipedia.org/w/index.php?title=File:Bodeplot.png License: Public Domain Contributors: derivative work: Stw (talk) Bodeplot.JPG: Vukg Image:Nyquist.svg Source: http://en.wikipedia.org/w/index.php?title=File:Nyquist.svg License: Public Domain Contributors: Engelec Image:Nichols.svg Source: http://en.wikipedia.org/w/index.php?title=File:Nichols.svg License: Public domain Contributors: Engelec (talk) Image:Nyquist example.svg Source: http://en.wikipedia.org/w/index.php?title=File:Nyquist_example.svg License: Public Domain Contributors: Utkarsh Upadhyay,--Krishnavedala (talk) 20:21, 3 June 2011 (UTC) (python version) Image:Unstable3.svg Source: http://en.wikipedia.org/w/index.php?title=File:Unstable3.svg License: GNU Free Documentation License Contributors: Ma-Lik, Pokipsy76 Image:HD-Rayleigh-Taylor.gif Source: http://en.wikipedia.org/w/index.php?title=File:HD-Rayleigh-Taylor.gif License: Public Domain Contributors: Shengtai Li, Hui Li Image:Fluid Instability.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Fluid_Instability.jpg License: Creative Commons Attribution-Sharealike 3.0 Contributors: Sungjune Jung Image:Butterworth response.svg Source: http://en.wikipedia.org/w/index.php?title=File:Butterworth_response.svg License: Creative Commons Attribution-Sharealike 3.0,2.5,2.0,1.0 Contributors: derivative work: Krishnavedala (talk) Butterworth_response.png: Omegatron Image:Baseband.svg Source: http://en.wikipedia.org/w/index.php?title=File:Baseband.svg License: Public Domain Contributors: Jhbdel (talk) Image:Bandwidth 2.svg Source: http://en.wikipedia.org/w/index.php?title=File:Bandwidth_2.svg License: Public Domain Contributors: Inductiveload File:Static VAR Compensator 2a.png Source: http://en.wikipedia.org/w/index.php?title=File:Static_VAR_Compensator_2a.png License: Creative Commons Attribution-Sharealike 3.0 Contributors: User:Clampower Image:PID en.svg Source: http://en.wikipedia.org/w/index.php?title=File:PID_en.svg License: Creative Commons Attribution-Sharealike 3.0 Contributors: Zerodamage Image:Change with Kp.png Source: http://en.wikipedia.org/w/index.php?title=File:Change_with_Kp.png License: Public Domain Contributors: Skorkmaz Image:Change with Ki.png Source: http://en.wikipedia.org/w/index.php?title=File:Change_with_Ki.png License: Public Domain Contributors: Skorkmaz Image:Change with Kd.png Source: http://en.wikipedia.org/w/index.php?title=File:Change_with_Kd.png License: unknown Contributors: Skorkmaz File:Scross helmsman.jpg Source: http://en.wikipedia.org/w/index.php?title=File:Scross_helmsman.jpg License: Creative Commons Attribution-Sharealike 2.5 Contributors: Original uploader was Fishdecoy at en.wikipedia Image:PI controller.png Source: http://en.wikipedia.org/w/index.php?title=File:PI_controller.png License: Public Domain Contributors: Email4mobile (talk)Email4mobile 322 License License Creative Commons Attribution-Share Alike 3.0 Unported //creativecommons.org/licenses/by-sa/3.0/ 323