Lecture 1 Lecture 1
Introduction to knowledge-base Introduction to knowledge-base
intelligent systems intelligent systems
■■ Intelligent Intelligent machinesmachines, or , or whatwhat machinesmachines cancan dodo
■■ The The historyhistory of of artificialartificial intelligenceintelligence or or fromfrom the the
“Dark Ages” to
“Dark Ages” to knowledgeknowledge--based systemsbased systems
■■ SummarySummary
Intelligent
Intelligent machines machines , or , or what what machines
machines can can do do
■■ Philosophers have been trying for over 2000 yearsPhilosophers have been trying for over 2000 years to understand and resolve two
to understand and resolve two Big QuestionsBig Questions of the of the Universe:
Universe: How does a human mind work, andHow does a human mind work, and Can non-humans have minds?
Can non-humans have minds? These questions These questions are still unanswered.
are still unanswered.
■■ IntelligenceIntelligence is their ability to understand and learn is their ability to understand and learn things. 2
things. 2 IntelligenceIntelligence is the ability to think and is the ability to think and understand instead of doing things by instinct or understand instead of doing things by instinct or
automatically.
automatically.
((Essential English DictionaryEssential English Dictionary, Collins, London, 1990), Collins, London, 1990)
■■ In order to think, someIn order to think, someoneone or some or somethingthing has to have has to have a brain,
a brain, or an organ thator an organ that enables some enables someoneone or or
somesomethingthing to learn and understand things, to solve to learn and understand things, to solve problems and to make decisions. So we can define problems and to make decisions. So we can define
intelligence as
intelligence as thethe ability to learn and understand, ability to learn and understand, to solve problems and to make
to solve problems and to make decisionsdecisions..
■■ The goal of The goal of artificial intelligenceartificial intelligence (AI) as a science (AI) as a science is to make machines do things that would require is to make machines do things that would require
intelligence if done by humans. Therefore, the intelligence if done by humans. Therefore, the
answer to the question
answer to the question Can Machines Think? wasCan Machines Think? was vitally important to the discipline.
vitally important to the discipline.
■■ The answer is not a simple “The answer is not a simple “YesYes” or “” or “No”.No”.
■■ Some people are smarter in some ways than others.Some people are smarter in some ways than others.
Sometimes we make very
Sometimes we make very intelligent decisions but intelligent decisions but sometimes we also make very silly mistakes. Some sometimes we also make very silly mistakes. Some
of us deal with complex mathematical and of us deal with complex mathematical and
engineering problems but are moronic in engineering problems but are moronic in
philosophy and history. Some people are good at philosophy and history. Some people are good at
making money, while others are better at spending making money, while others are better at spending it. As humans, we all have the ability to learn and it. As humans, we all have the ability to learn and
understand, to solve problems and to make understand, to solve problems and to make
decisions; however, our abilities are not equal and decisions; however, our abilities are not equal and lie in different areas. Therefore, we should expect lie in different areas. Therefore, we should expect that if machines can think, some of them might be that if machines can think, some of them might be smarter than others in some ways.
■■ One of the One of the most significantmost significant papers on machine papers on machine intelligence,
intelligence, “Computing Machinery and“Computing Machinery and Intelligence
Intelligence””,, was written by the British was written by the British mathematician
mathematician Alan TuringAlan Turing over fifty years ago . over fifty years ago . However, it still stands up well under the test of However, it still stands up well under the test of
time, and the Turing’s approach remains universal.
time, and the Turing’s approach remains universal.
■■ He asked: He asked: Is there thought without experience? IsIs there thought without experience? Is there mind without communication? Is there
there mind without communication? Is there language without living? Is there intelligence language without living? Is there intelligence
without life?
without life? All these questions, as you can see, All these questions, as you can see, are just variations on the fundamental question of are just variations on the fundamental question of
artificial intelligence,
artificial intelligence, Can machines think?Can machines think?
■■ Turing did not provide definitions of machines andTuring did not provide definitions of machines and thinking, he just avoided semantic arguments by thinking, he just avoided semantic arguments by
inventing a game, the
inventing a game, the Turing Imitation GameTuring Imitation Game..
■■ The imitation game originally included two phases.The imitation game originally included two phases.
In the first phase, the interrogator, a man and a In the first phase, the interrogator, a man and a
woman are each placed in separate rooms. The woman are each placed in separate rooms. The
interrogator’s objective is to work out who is the interrogator’s objective is to work out who is the man and who is the woman by questioning them.
man and who is the woman by questioning them.
The man should attempt to deceive the interrogator The man should attempt to deceive the interrogator
that
that hehe is the woman, while the woman has to is the woman, while the woman has to convince the interrogator that
convince the interrogator that sheshe is the woman. is the woman.
Turing Imitation Game: Phase 1
Turing Imitation Game: Phase 1
Turing Imitation Game: Phase 2 Turing Imitation Game: Phase 2
■■ In the second phase of the game, the man isIn the second phase of the game, the man is
replaced by a computer programmed to deceive the replaced by a computer programmed to deceive the
interrogator as the man did. It would even be interrogator as the man did. It would even be
programmed to make mistakes and provide fuzzy programmed to make mistakes and provide fuzzy
answers in the way a human would. If the answers in the way a human would. If the
computer can fool the interrogator as often as the computer can fool the interrogator as often as the
man did, we may say this computer has passed the man did, we may say this computer has passed the
intelligent behaviour test.
intelligent behaviour test.
Turing Imitation Game: Phase 2
Turing Imitation Game: Phase 2
The Turing test has two remarkable qualities The Turing test has two remarkable qualities
that make it really universal.
that make it really universal.
■■ ByBy maintaining communication between the human maintaining communication between the human and the machine via terminals, the test gives us an and the machine via terminals, the test gives us an
objective standard view on intelligence.
objective standard view on intelligence.
■■ TheThe test itself is quite independent from the details test itself is quite independent from the details of the experiment. It can be conducted
of the experiment. It can be conducted as a twoas a two-- phase
phase game, or even as a singlegame, or even as a single-phase game when-phase game when the interrogator needs to choose between the
the interrogator needs to choose between the
human and the machine from the beginning of the human and the machine from the beginning of the
test.
test.
■■ Turing believed that by the end of the 20th centuryTuring believed that by the end of the 20th century it would be possible to program a digital computer it would be possible to program a digital computer
to play the imitation game. Although modern to play the imitation game. Although modern
computers still cannot pass the Turing test, it computers still cannot pass the Turing test, it
provides a basis for the verification and validation provides a basis for the verification and validation
of knowledge-based systems.
of knowledge-based systems.
■■ AA program thought intelligent in some narrow program thought intelligent in some narrow area of expertise is evaluated by comparing its area of expertise is evaluated by comparing its
performance with the performance of a human performance with the performance of a human
expert.
expert.
■■ To build an intelligent computer system, we have toTo build an intelligent computer system, we have to capture, organise and use human expert knowledge capture, organise and use human expert knowledge
The history of artificial intelligence The history of artificial intelligence
■■ The first work recognised in the field of AI wasThe first work recognised in the field of AI was presented by
presented by WarrenWarren McCulloch McCulloch and Walter and Walter Pitts
Pitts in 1943 in 1943. They. They proposed a model of anproposed a model of an artificial
artificial neural neural networknetwork andand demonstrated demonstrated thatthat simple network structures could learn.
simple network structures could learn.
■■ McCulloch, the second “founding father” of AIMcCulloch, the second “founding father” of AI after Alan Turing, had created the corner stone of after Alan Turing, had created the corner stone of
neural computing and artificial neural networks neural computing and artificial neural networks
(ANN).
(ANN).
The birth of artificial intelligence (1943 – 1956)
The birth of artificial intelligence (1943 – 1956)
■■ The third founder of AI was The third founder of AI was JohnJohn von Neumann von Neumann,, the brilliant Hungarian-born mathematician. In the brilliant Hungarian-born mathematician. In
1930, he joined the Princeton University, lecturing 1930, he joined the Princeton University, lecturing
in mathematical physics.
in mathematical physics. He was an adviser for theHe was an adviser for the Electronic Numerical Integrator and Calculator
Electronic Numerical Integrator and Calculator project at the University of Pennsylvania and project at the University of Pennsylvania and
helped to design the
helped to design the ElectronicElectronic Discrete Variable Discrete Variable Calculator
Calculator. He was influenced by McCulloch. He was influenced by McCulloch and and Pitts’s
Pitts’s neural network model. When neural network model. When MarvinMarvin Minsky
Minsky and and Dean EdmondsDean Edmonds, two , two graduategraduate
students in the Princeton mathematics department, students in the Princeton mathematics department,
built the first neural network computer in 1951,
built the first neural network computer in 1951, von von Neumann
Neumann encouraged and supported them. encouraged and supported them.
■■ Another of the first generation researchers wasAnother of the first generation researchers was Claude Shannon
Claude Shannon. He graduated from . He graduated from MIT andMIT and joined Bell Telephone Laboratories
joined Bell Telephone Laboratories in 1941. in 1941.
Shannon shared Alan Turing’s ideas on the Shannon shared Alan Turing’s ideas on the
possibility of machine intelligence. In 1950, he possibility of machine intelligence. In 1950, he
published a paper on chess-playing machines, published a paper on chess-playing machines,
which pointed out that a typical chess game which pointed out that a typical chess game
involved about 10
involved about 10120120 possible moves (Shannon, possible moves (Shannon, 1950). Even if the new
1950). Even if the new von Neumann von Neumann-type-type computer could examine one move per
computer could examine one move per microsecond, it would take
microsecond, it would take 3 3 ×× 1010106106 years to make years to make its first move. Thus Shannon demonstrated the
its first move. Thus Shannon demonstrated the
need to use heuristics in the search for the solution.
need to use heuristics in the search for the solution.
■■ In 1956, In 1956, John McCarthyJohn McCarthy, Martin Minsky, Martin Minsky and and Claude Shannon
Claude Shannon organised a summer workshop at organised a summer workshop at Dartmouth College. They brought together
Dartmouth College. They brought together researchers interested in the study of machine researchers interested in the study of machine
intelligence, artificial neural nets and automata intelligence, artificial neural nets and automata
theory. Although there were just ten researchers, theory. Although there were just ten researchers, this workshop gave birth to a new science called this workshop gave birth to a new science called
artificial intelligence artificial intelligence..
The The rise rise of of artificial artificial intelligence intelligence , or the , or the era era of of great
great expectations expectations (1956 – late 1960s) (1956 – late 1960s)
■■ The early works on neural computing and artificialThe early works on neural computing and artificial neural networks started by
neural networks started by McCulloch McCulloch and and Pitts Pitts was continued. Learning methods were improved was continued. Learning methods were improved and and FrankFrank Rosenblatt Rosenblatt proved the proved the perceptronperceptron
convergence theorem
convergence theorem, demonstrating that his, demonstrating that his learning algorithm could adjust the connection learning algorithm could adjust the connection
strengths of a
strengths of a perceptron.perceptron.
■■ One of the most ambitious projects of the era ofOne of the most ambitious projects of the era of great expectations was the
great expectations was the General ProblemGeneral Problem Solver (GPS)
Solver (GPS). Allen Newell. Allen Newell and and Herbert SimonHerbert Simon from the Carnegie Mellon University developed a from the Carnegie Mellon University developed a
general-purpose program to simulate human- general-purpose program to simulate human-
solving methods.
solving methods.
■■ Newell and Simon postulated that a problem to beNewell and Simon postulated that a problem to be solved could be defined in terms of
solved could be defined in terms of statesstates. They. They used the mean-end analysis to determine a
used the mean-end analysis to determine a
difference between the current and desirable or difference between the current and desirable or
goal state
goal state of the problem, and to choose and apply of the problem, and to choose and apply operators
operators to reach the goal state. The set of to reach the goal state. The set of
■■ However, GPS failed to solve complex problems.However, GPS failed to solve complex problems.
The program was based on formal logic and could The program was based on formal logic and could generate an infinite number of possible operators.
generate an infinite number of possible operators.
The amount of computer time and memory that The amount of computer time and memory that
GPS required to solve real-world problems led to GPS required to solve real-world problems led to
the project being abandoned.
the project being abandoned.
■■ In the sixties, AI researchers attempted to simulateIn the sixties, AI researchers attempted to simulate the thinking process by inventing
the thinking process by inventing general methodsgeneral methods for solving
for solving broad classes of problemsbroad classes of problems. They used. They used the general-purpose search mechanism to find a the general-purpose search mechanism to find a solution to the problem. Such approaches, now solution to the problem. Such approaches, now
referred to as
referred to as weak methods, applied weakweak methods, applied weak information about the problem domain.
information about the problem domain.
■■ By 1970, the euphoria about AI was gone, and mostBy 1970, the euphoria about AI was gone, and most government funding for AI projects was cancelled.
government funding for AI projects was cancelled.
AI was still a relatively new field, academic in AI was still a relatively new field, academic in
nature, with few practical applications apart from nature, with few practical applications apart from
playing games. So, to the outsider, the achieved playing games. So, to the outsider, the achieved results would be seen as toys, as no AI system at results would be seen as toys, as no AI system at
that time could manage real-world problems.
that time could manage real-world problems.
Unfulfilled
Unfulfilled promises promises , or the , or the impact impact of of reality reality (late 1960s – early 1970s)
(late 1960s – early 1970s)
TheThe main difficulties for AI in the late 1960s were: main difficulties for AI in the late 1960s were:
■■ Because AI researchers were developing generalBecause AI researchers were developing general methods for broad classes of problems, early
methods for broad classes of problems, early
programs contained little or even no knowledge programs contained little or even no knowledge
about a problem domain. To solve problems, about a problem domain. To solve problems,
programs applied a search strategy by trying out programs applied a search strategy by trying out
different combinations of small steps, until the right different combinations of small steps, until the right
one was found. This approach was quite feasible for one was found. This approach was quite feasible for
simple
simple toy problemstoy problems, so it seemed reasonable that,, so it seemed reasonable that, if the programs could be “scaled up” to solve large if the programs could be “scaled up” to solve large
■■ Many of the problems that AI attempted to solveMany of the problems that AI attempted to solve were
were too broad and too difficulttoo broad and too difficult. A typical task for. A typical task for early AI was machine translation. For example, the early AI was machine translation. For example, the
National Research Council, USA, funded the National Research Council, USA, funded the
translation of Russian scientific papers after the translation of Russian scientific papers after the launch of the first artificial satellite (Sputnik) in launch of the first artificial satellite (Sputnik) in
1957. Initially, the project team tried simply 1957. Initially, the project team tried simply
replacing Russian words with English, using an replacing Russian words with English, using an
electronic dictionary. However, it was soon found electronic dictionary. However, it was soon found that translation requires a general understanding of that translation requires a general understanding of
the subject to choose the correct words. This task the subject to choose the correct words. This task was too difficult. In 1966, all translation projects was too difficult. In 1966, all translation projects
funded by the US government were cancelled.
funded by the US government were cancelled.
■■ In 1971, the British government also suspendedIn 1971, the British government also suspended support for AI research. Sir
support for AI research. Sir JamesJames Lighthill Lighthill had had
been commissioned by the Science Research Council been commissioned by the Science Research Council of Great Britain to review the current state of AI. He of Great Britain to review the current state of AI. He
did not find any major or even significant results did not find any major or even significant results
from AI research, and therefore saw no need to have from AI research, and therefore saw no need to have
a separate science called “artificial intelligence”.
a separate science called “artificial intelligence”.
The The technology technology of of expert expert systems systems , , or or the the key key to to success
success (early 1970s – mid-1980s) (early 1970s – mid-1980s)
■■ Probably the most important development in theProbably the most important development in the seventies was the realisation that the domain for seventies was the realisation that the domain for
intelligent machines had to be sufficiently intelligent machines had to be sufficiently
restricted. Previously, AI researchers had believed restricted. Previously, AI researchers had believed
that clever search algorithms and reasoning that clever search algorithms and reasoning
techniques could be invented to emulate general, techniques could be invented to emulate general,
human-like, problem-solving methods. A general- human-like, problem-solving methods. A general-
purpose search mechanism could rely on purpose search mechanism could rely on
elementary reasoning steps to find complete elementary reasoning steps to find complete
solutions and could use weak knowledge about solutions and could use weak knowledge about domain.
■■
When weak methods failed, researchers finally When weak methods failed, researchers finally realised that the only way to deliver practical realised that the only way to deliver practical
results was to solve typical cases in narrow results was to solve typical cases in narrow
areas of expertise, making large reasoning areas of expertise, making large reasoning
steps.
steps.
DENDRAL DENDRAL
■■ DENDRAL was developed at Stanford University DENDRAL was developed at Stanford University toto determine the molecular structure of Martian soil,
determine the molecular structure of Martian soil, based on the mass spectral data provided by a mass based on the mass spectral data provided by a mass
spectrometer.
spectrometer. The project was supported by The project was supported by NASA.NASA.
Edward Feigenbaum, Bruce Buchanan (a computer Edward Feigenbaum, Bruce Buchanan (a computer
scientist) and Joshua Lederberg (a Nobel prize winner scientist) and Joshua Lederberg (a Nobel prize winner
in genetics) formed
in genetics) formed a a team.team.
■■ There was no scientific algorithmThere was no scientific algorithm for mapping the for mapping the mass spectrum into its molecular structure.
mass spectrum into its molecular structure.
Feigenbaum’s job was to incorporate the expertise of Feigenbaum’s job was to incorporate the expertise of
Lederberg into a computer program to make it Lederberg into a computer program to make it
perform at a human expert level. Such programs were perform at a human expert level. Such programs were
■■ DENDRAL marked a major “paradigm shift” in AI: aDENDRAL marked a major “paradigm shift” in AI: a shift from
shift from general-purpose, knowledge-sparse weak general-purpose, knowledge-sparse weak methods to domain-specific, knowledge-intensive methods to domain-specific, knowledge-intensive
techniques.
techniques.
■■ The aim of the project was to develop a computerThe aim of the project was to develop a computer program to attain the level of performance of an program to attain the level of performance of an
experienced human chemist. Using heuristics in the experienced human chemist. Using heuristics in the
form of high-quality specific
form of high-quality specific rules,rules, rules-of-thumb rules-of-thumb , the, the DENDRAL team proved that
DENDRAL team proved that computers could equal an computers could equal an expert in narrow,
expert in narrow, well definedwell defined, problem areas, problem areas..
■■ The DENDRAL project originated the fundamental ideaThe DENDRAL project originated the fundamental idea of expert systems –
of expert systems – knowledge engineeringknowledge engineering, which, which encompassed techniques of capturing, analysing and encompassed techniques of capturing, analysing and
■■ MYCIN was a rule-based expert system for theMYCIN was a rule-based expert system for the
diagnosis of infectious blood diseases. It also provided diagnosis of infectious blood diseases. It also provided
a doctor with therapeutic advice in a convenient, user- a doctor with therapeutic advice in a convenient, user-
friendly manner.
friendly manner.
■■ MYCIN’s knowledge consisted of about 450 rulesMYCIN’s knowledge consisted of about 450 rules derived from human knowledge in a
derived from human knowledge in a narrow domain narrow domain through extensive interviewing of experts.
through extensive interviewing of experts.
■■ The The knowledgeknowledge incorporated in the form of rules was incorporated in the form of rules was clearly separated from the reasoning mechanism. The clearly separated from the reasoning mechanism. The
system developer could easily manipulate knowledge system developer could easily manipulate knowledge in the system by inserting or deleting some rules. For in the system by inserting or deleting some rules. For
example, a domain-independent version of MYCIN example, a domain-independent version of MYCIN
MYCIN
MYCIN
■■ PROSPECTOR was an expert system for mineralPROSPECTOR was an expert system for mineral exploration developed by the Stanford Research exploration developed by the Stanford Research
Institute. Nine experts contributed their knowledge and Institute. Nine experts contributed their knowledge and
expertise. PROSPECTOR used a combined structure expertise. PROSPECTOR used a combined structure
that incorporated rules and a semantic network.
that incorporated rules and a semantic network.
PROSPECTOR had over 1000
PROSPECTOR had over 1000 rules.rules.
■■ The user, an exploration geologistThe user, an exploration geologist, , waswas asked to input asked to input the characteristics of a suspected deposit: the geological the characteristics of a suspected deposit: the geological
setting, structures, kinds of rocks and minerals.
setting, structures, kinds of rocks and minerals.
PROSPECTOR compared
PROSPECTOR compared these characteristics with these characteristics with models of ore
models of ore deposits and made an assessment of thedeposits and made an assessment of the suspected mineral deposit. It could also explain the suspected mineral deposit. It could also explain the
PROSPECTOR
PROSPECTOR
■■ A 1986 survey reported a remarkable number ofA 1986 survey reported a remarkable number of successful expert system applications in different successful expert system applications in different
areas: chemistry, electronics, engineering, geology, areas: chemistry, electronics, engineering, geology,
management, medicine, process control and management, medicine, process control and
military science (
military science (WatermanWaterman, 1986). Although, 1986). Although Waterman
Waterman found nearly 200 expert systems, most found nearly 200 expert systems, most of the applications were in the field of medical
of the applications were in the field of medical diagnosis. Seven years later a similar survey diagnosis. Seven years later a similar survey reported over 2500 developed expert systems reported over 2500 developed expert systems ((DurkinDurkin, 1994). The new growing area was, 1994). The new growing area was
business and manufacturing, which accounted for business and manufacturing, which accounted for
about 60% of the applications. Expert system about 60% of the applications. Expert system
technology had clearly matured.
technology had clearly matured.
However However : :
■■ Expert systems are restricted to a very narrowExpert systems are restricted to a very narrow
domain of expertise. For example, MYCIN, which domain of expertise. For example, MYCIN, which was developed for the diagnosis of infectious blood was developed for the diagnosis of infectious blood
diseases, lacks any real knowledge of human diseases, lacks any real knowledge of human
physiology. If a patient has more than one disease, physiology. If a patient has more than one disease,
we cannot rely on MYCIN. In fact, therapy we cannot rely on MYCIN. In fact, therapy
prescribed for the blood disease might even be prescribed for the blood disease might even be
harmful because of the other disease.
harmful because of the other disease.
■■ Expert systems Expert systems can show the sequence of the rulescan show the sequence of the rules they applied
they applied to reach a solution, but cannot relate to reach a solution, but cannot relate accumulated, heuristic knowledge to any deeper accumulated, heuristic knowledge to any deeper
■■ Expert systems have difficulty in recognising domainExpert systems have difficulty in recognising domain boundaries. When given a task different from the
boundaries. When given a task different from the typical problems
typical problems, an expert system might attempt to, an expert system might attempt to solve it and fail in rather unpredictable ways.
solve it and fail in rather unpredictable ways.
■■ Heuristic rules represent knowledge in abstract formHeuristic rules represent knowledge in abstract form and lack even basic understanding of the domain
and lack even basic understanding of the domain area. It makes the task of identifying incorrect, area. It makes the task of identifying incorrect,
incomplete
incomplete or inconsistent knowledge difficult. or inconsistent knowledge difficult.
■■ Expert systems, especially the first generation, haveExpert systems, especially the first generation, have little or no ability to learn from their experience.
little or no ability to learn from their experience.
Expert systems are built individually and cannot be Expert systems are built individually and cannot be
developed fast.
developed fast. Complex systems can take over 30Complex systems can take over 30 person-years to build.
person-years to build.
How to
How to make make a a machine machine learn learn , or the , or the rebirth rebirth of of neural
neural networks networks (mid-1980s – onwards) (mid-1980s – onwards)
■■ In the mid-eighties, researchers, engineers andIn the mid-eighties, researchers, engineers and experts found that building an expert system experts found that building an expert system
required much more than just buying a reasoning required much more than just buying a reasoning system or expert system shell and putting enough system or expert system shell and putting enough rules in it. Disillusions about the applicability of rules in it. Disillusions about the applicability of
expert system technology even led to people expert system technology even led to people
predicting an
predicting an AI “winter”AI “winter” with severely squeezed with severely squeezed funding for AI projects. AI researchers decided to funding for AI projects. AI researchers decided to
have a new look at neural networks.
have a new look at neural networks.
■■ By the late sixties, most of the basic ideas andBy the late sixties, most of the basic ideas and concepts necessary for neural computing had concepts necessary for neural computing had
already been formulated. However, only in the already been formulated. However, only in the
mid-eighties did the solution emerge. The major mid-eighties did the solution emerge. The major
reason for the delay was technological: there were reason for the delay was technological: there were
no PCs or powerful workstations to model and no PCs or powerful workstations to model and
experiment with artificial neural networks.
experiment with artificial neural networks.
■■ In the eighties, because of the need for brain-likeIn the eighties, because of the need for brain-like information processing, as well as the advances in information processing, as well as the advances in
computer technology and progress in neuroscience, computer technology and progress in neuroscience, the field of neural networks experienced a dramatic the field of neural networks experienced a dramatic resurgence. Major contributions to both theory and resurgence. Major contributions to both theory and
■■ GrossbergGrossberg established a new principle of self- established a new principle of self- organisation (
organisation (adaptive resonance theoryadaptive resonance theory), which), which provided the basis for a new class of neural
provided the basis for a new class of neural networks (
networks (GrossbergGrossberg, 1980)., 1980).
■■ HopfieldHopfield introduced neural networks with feedback introduced neural networks with feedback –– HopfieldHopfield networks, which attracted much attention networks, which attracted much attention
in the eighties (
in the eighties (HopfieldHopfield, 1982)., 1982).
■■ KohonenKohonen published a paper on published a paper on self-organising mapsself-organising maps ((KohonenKohonen, 1982)., 1982).
■■ BartoBarto, Sutton and Anderson published their work on, Sutton and Anderson published their work on reinforcement learning
reinforcement learning and its application in and its application in control (
control (BartoBarto et al., 1983 et al., 1983).).
■■ But the real breakthrough came in 1986 when theBut the real breakthrough came in 1986 when the back-propagation learning algorithm
back-propagation learning algorithm, first, first
introduced by Bryson and Ho in 1969 (Bryson &
introduced by Bryson and Ho in 1969 (Bryson &
Ho, 1969), was reinvented by Rumelhart and Ho, 1969), was reinvented by Rumelhart and
McClelland in
McClelland in Parallel Distributed ProcessingParallel Distributed Processing (1986).
(1986).
■■ Artificial neural networks have come a long wayArtificial neural networks have come a long way from the early models of McCulloch and Pitts to an from the early models of McCulloch and Pitts to an interdisciplinary subject with roots in neuroscience, interdisciplinary subject with roots in neuroscience, psychology, mathematics and engineering, and will psychology, mathematics and engineering, and will
continue to develop in both theory and practical continue to develop in both theory and practical
applications.
applications.
The The new new era era of of knowledge knowledge engineering engineering , or , or computing
computing with with words words (late 1980s – onwards) (late 1980s – onwards)
■■ Neural network technology offers more naturalNeural network technology offers more natural interaction with the real world than do systems interaction with the real world than do systems
based on symbolic reasoning. Neural networks can based on symbolic reasoning. Neural networks can learn, adapt to changes in a problem’s environment, learn, adapt to changes in a problem’s environment,
establish patterns in situations where rules are not establish patterns in situations where rules are not
known, and deal with fuzzy or incomplete known, and deal with fuzzy or incomplete
information. However, they lack explanation information. However, they lack explanation facilities and usually act as a black box. The facilities and usually act as a black box. The
process of training neural networks with current process of training neural networks with current technologies is slow, and frequent retraining can technologies is slow, and frequent retraining can
■■ Classic expert systems are especially good forClassic expert systems are especially good for
closed-system applications with precise inputs and closed-system applications with precise inputs and logical outputs. They use expert knowledge in the logical outputs. They use expert knowledge in the form of rules and, if required, can interact with the form of rules and, if required, can interact with the
user to establish a particular fact. A major user to establish a particular fact. A major
drawback is that human experts cannot always drawback is that human experts cannot always
express their knowledge in terms of rules or explain express their knowledge in terms of rules or explain
the line of their reasoning. This can prevent the the line of their reasoning. This can prevent the expert system from accumulating the necessary expert system from accumulating the necessary knowledge, and consequently lead to its failure.
knowledge, and consequently lead to its failure.
■■ Very important technology dealing with vague,Very important technology dealing with vague, imprecise and uncertain knowledge and data is
imprecise and uncertain knowledge and data is fuzzyfuzzy logic
logic..
■■ Human experts do not usually think in probabilityHuman experts do not usually think in probability values, but in such terms as
values, but in such terms as oftenoften, , generallygenerally,, sometimes
sometimes, , occasionallyoccasionally and and rarelyrarely. Fuzzy logic is. Fuzzy logic is concerned with
concerned with capturing the meaning of words,capturing the meaning of words, human reasoning and decision making.
human reasoning and decision making. Fuzzy logicFuzzy logic provides the way to break through the computational provides the way to break through the computational
bottlenecks of traditional expert systems.
bottlenecks of traditional expert systems.
■■ At the heart of fuzzy logic lies the concept of aAt the heart of fuzzy logic lies the concept of a linguistic variable
linguistic variable. The values of the linguistic. The values of the linguistic variable are words rather than numbers.
variable are words rather than numbers.
■■ Fuzzy logic or Fuzzy logic or fuzzy set theoryfuzzy set theory was introduced by was introduced by Professor
Professor Lotfi ZadehLotfi Zadeh, Berkeley’s electrical, Berkeley’s electrical engineering department chairman, in
engineering department chairman, in 1965. It1965. It provided
provided a means of computing with words. a means of computing with words.
However, acceptance of fuzzy set theory by the However, acceptance of fuzzy set theory by the
technical community was slow and difficult. Part technical community was slow and difficult. Part
of the problem was the provocative name – “fuzzy”
of the problem was the provocative name – “fuzzy”
– it seemed too light-hearted to be taken seriously.
– it seemed too light-hearted to be taken seriously.
Eventually, fuzzy theory, ignored in the West, was Eventually, fuzzy theory, ignored in the West, was
taken seriously in the East – by the Japanese. It has taken seriously in the East – by the Japanese. It has
been used successfully since 1987 in Japanese- been used successfully since 1987 in Japanese-
designed dishwashers, washing machines, air designed dishwashers, washing machines, air
conditioners, television sets, copiers, and even cars conditioners, television sets, copiers, and even cars..
Benefits derived from the application of fuzzy Benefits derived from the application of fuzzy
logic models in knowledge-based and logic models in knowledge-based and
decision-support systems can be summarised decision-support systems can be summarised
as follows:
as follows:
■■ Improved computational power:Improved computational power: Fuzzy rule- Fuzzy rule- based systems perform faster than conventional based systems perform faster than conventional
expert systems and require fewer rules. A fuzzy expert systems and require fewer rules. A fuzzy
expert system merges the rules, making them more expert system merges the rules, making them more
powerful.
powerful. Lotfi Zadeh Lotfi Zadeh believes that in a few years believes that in a few years most expert systems will use fuzzy logic to solve most expert systems will use fuzzy logic to solve
highly
highly nonlinear nonlinear and computationally difficult and computationally difficult problems.
problems.
■■ Improved cognitive modelling:Improved cognitive modelling: Fuzzy systems allow Fuzzy systems allow the encoding of knowledge in a form that reflects the the encoding of knowledge in a form that reflects the
way experts think about a complex problem. They way experts think about a complex problem. They
usually think in such imprecise terms as
usually think in such imprecise terms as high and high and low,low, fast and fast and slow, slow, heavyheavy and and light. In order to buildlight. In order to build
conventional rules, we need to define the crisp conventional rules, we need to define the crisp
boundaries for these
boundaries for these terms by breakingterms by breaking down the down the expertise into fragments.
expertise into fragments. This fragmentationThis fragmentation leads to leads to the poor performance of conventional expert systems the poor performance of conventional expert systems
when they deal
when they deal with complex problems. In contrast,with complex problems. In contrast, fuzzy expert systems model imprecise information, fuzzy expert systems model imprecise information,
capturing expertise
capturing expertise similarsimilar to to the way it is representedthe way it is represented in the expert mind, and thus improve cognitive
in the expert mind, and thus improve cognitive modelling of the problem.
modelling of the problem.
■■ The ability to represent multiple experts:The ability to represent multiple experts:
Conventional expert systems are built for
Conventional expert systems are built for a narrowa narrow domain. It makes the system’s performance
domain. It makes the system’s performance fully fully dependent on the right choice of experts.
dependent on the right choice of experts. WhenWhen a a more complex expert system is being built or when more complex expert system is being built or when
expertise is not well defined,
expertise is not well defined, multiple expertsmultiple experts might be might be needed.
needed. However, multiple experts seldom reach closeHowever, multiple experts seldom reach close agreements; there are often differences in opinions and agreements; there are often differences in opinions and even conflicts. This is especially true in areas, such as even conflicts. This is especially true in areas, such as
business and management
business and management, where no simple solution, where no simple solution exists and conflicting views should be taken into
exists and conflicting views should be taken into
account. Fuzzy expert systems can help to represent account. Fuzzy expert systems can help to represent
the expertise of multiple experts when they have the expertise of multiple experts when they have
■■ Although fuzzy systems allow expression of expertAlthough fuzzy systems allow expression of expert knowledge in a more natural way, they still depend knowledge in a more natural way, they still depend
on the rules extracted from the experts, and thus on the rules extracted from the experts, and thus
might be smart or dumb. Some experts can provide might be smart or dumb. Some experts can provide
very clever fuzzy rules – but some just guess and very clever fuzzy rules – but some just guess and
may even get them wrong. Therefore, all rules may even get them wrong. Therefore, all rules
must be tested and tuned, which can be a prolonged must be tested and tuned, which can be a prolonged
and tedious process. For example, it took
and tedious process. For example, it took HitachiHitachi engineers several years to test and tune only 54 engineers several years to test and tune only 54
fuzzy rules to guide the Sendal Subway System.
fuzzy rules to guide the Sendal Subway System.
■■ In recent years, several methods based on neuralIn recent years, several methods based on neural network technology have been used to search
network technology have been used to search
numerical data for fuzzy rules. Adaptive or neural numerical data for fuzzy rules. Adaptive or neural fuzzy systems can find new fuzzy rules, or change fuzzy systems can find new fuzzy rules, or change and tune existing ones based on the data provided.
and tune existing ones based on the data provided.
In other words, data in – rules out, or experience in In other words, data in – rules out, or experience in
– common sense out.
– common sense out.
Summary Summary
■■ ExpertExpert, neural and fuzzy systems have now, neural and fuzzy systems have now matured and been applied to a broad range of matured and been applied to a broad range of
different problems, mainly in engineering, different problems, mainly in engineering,
medicine, finance, business and management.
medicine, finance, business and management.
■■ EachEach technology handles the uncertainty and technology handles the uncertainty and
ambiguity of human knowledge differently, and ambiguity of human knowledge differently, and
each technology has found its place in knowledge each technology has found its place in knowledge engineering. They no longer compete; rather they engineering. They no longer compete; rather they
complement each
complement each other.other.
■■ A synergy of expert systems with fuzzy logic andA synergy of expert systems with fuzzy logic and neural computing improves adaptability,
neural computing improves adaptability, robustness, fault-tolerance and speed of robustness, fault-tolerance and speed of
knowledge-based systems. Besides, computing knowledge-based systems. Besides, computing
with words makes them more “human”. It is now with words makes them more “human”. It is now
common practice to build intelligent systems using common practice to build intelligent systems using
existing theories rather than to propose new ones, existing theories rather than to propose new ones, and to apply these systems to real-world problems and to apply these systems to real-world problems
rather than to “toy
rather than to “toy” problems.” problems.
Period Key Events
The birth of Artificial Intelligence
(1943–1956)
McCulloch and Pitts, A Logical Calculus of the Ideas Immanent in Nervous Activity, 1943
Turing, Computing Machinery and Intelligence, 1950 The Electronic Numerical Integrator and Calculator
project (von Neumann)
Shannon, Programming a Computer for Playing Chess, 1950
The Dartmouth College summer workshop on machine intelligence, artificial neural nets and automata theory, 1956
Main events in the history of AI
Main events in the history of AI
Period Key Events
The rise of artificial intelligence
(1956–late 1960s)
LISP (McCarthy)
The General Problem Solver (GPR) project (Newell and Simon)
Newell and Simon, Human Problem Solving, 1972
Minsky, A Framework for Representing Knowledge, 1975
The disillusionment in artificial
intelligence (late 1960s–early 1970s)
Cook, The Complexity of Theorem Proving Procedures, 1971
Karp, Reducibility Among Combinatorial Problems, 1972 The Lighthill Report, 1971
Period Key Events
The discovery of expert systems (early 1970s–mid-1980s)
DENDRAL (Feigenbaum, Buchanan and Lederberg, Stanford University)
MYCIN (Feigenbaum and Shortliffe, Stanford University) PROSPECTOR (Stanford Research Institute)
PROLOG - a logic programming language (Colmerauer, Roussel and Kowalski, France)
EMYCIN (Stanford University)
Waterman, A Guide to Expert Systems, 1986
Period Key Events
The rebirth of artificial neural networks
(1965–onwards)
Hopfield, Neural Networks and Physical Systems with Emergent Collective Computational Abilities, 1982 Kohonen, Self-Organized Formation of Topologically
Correct Feature Maps, 1982
Rumelhart and McClelland, Parallel Distributed Processing, 1986
The First IEEE International Conference on Neural Networks, 1987
Haykin, Neural Networks, 1994
Neural Network, MATLAB Application Toolbox (The MathWork, Inc.)
Period Key Events
Evolutionary
computation (early 1970s–onwards)
Rechenberg, Evolutionsstrategien - Optimierung
Technischer Systeme Nach Prinzipien der Biologischen Information, 1973
Holland, Adaptation in Natural and Artificial Systems, 1975.
Koza, Genetic Programming: On the Programming of the Computers by Means of Natural Selection, 1992.
Schwefel, Evolution and Optimum Seeking, 1995
Fogel, Evolutionary Computation –Towards a New Philosophy of Machine Intelligence, 1995.
Period Key Events
Computing with Words
(late 1980s–onwards)
Zadeh, Fuzzy Sets, 1965
Zadeh, Fuzzy Algorithms, 1969
Mamdani, Application of Fuzzy Logic to Approximate Reasoning Using Linguistic Synthesis, 1977
Sugeno, Fuzzy Theory, 1983
Japanese “fuzzy” consumer products (dishwashers, washing machines, air conditioners, television sets, copiers)
Sendai Subway System (Hitachi, Japan), 1986 The First IEEE International Conference on Fuzzy
Systems, 1992
Kosko, Neural Networks and Fuzzy Systems, 1992 Kosko, Fuzzy Thinking, 1993
Cox, The Fuzzy Systems Handbook, 1994
Zadeh, Computing with Words - A Paradigm Shift, 1996 Fuzzy Logic, MATLAB Application Toolbox (The