A Constraint-Based Approach to the Description and Detection Of

Linkoping Electronic Articles in

Computer and Information Science

Vol nr

A ConstraintBased Approach

to the Description and Detection

of FitnessforPurp ose

Simon White and Derek Sleeman

Linkoping University Electronic Press

Linkoping Sweden

httpwwwepliuseeacis

PublishedonDecember by

Linkoping University Electronic Press

Linkoping Sweden

Linkoping Electronic Articles in

Computer and Information Science

ISSN

Series editor Erik Sandewal l

Simon White and Derek Sleeman

Typeset by the author using L T X

Formattedusingetendu style

Recommended citation

Author Title Linkoping Electronic Articles in

Computer and Information Science Vol nr

httpwwwepliuseeacisDecember

This URL wil l also contain a link to the authors home page

The publishers wil l keep this article online on the Internet

or its possible replacement network in the future

for a periodofyears from the date of publication

barring exceptional circumstances as describedseparately

The online availability of the article implies

apermanent permission for anyone to read the article online

to print out single copies of it and to use it unchanged

for any noncommercial research and educational purpose

including making copies for classroom use

This permission can not berevoked by subsequent

transfers of copyright Al l other uses of the article are

conditional on the consent of the copyright owner

The publication of the article on the date statedabove

included also the production of a limited number of copies

on paper which werearchived in Swedish university libraries

like al l other written works publishedinSweden

The publisher has taken technical and administrative measures

to assure that the online version of the article wil l be

permanently accessible using the URL statedabove

unchanged and permanently equal to the archivedprintedcopies

at least until the expiration of the publication period

For additional information about the Linkoping University

Electronic Pressanditsprocedures for publication and for

assur anceofdocument integrity please refer to

its WWW home page httpwwwepliuse

or by conventional mail to the address statedabove

Abstract

This pap er intro duces the notion of tnessforpurpose presents

a tractable approximate approach to the recognition of tness

forpurp ose and describ es a working implementation using con

straint programming

The prop erty of tnessforpurpose states whether running a

software comp onent with a supplied set of inputs can satisfy a

given goal Our interest is to assess whether a chosen problem

solver together with one or more knowledge bases can satisfy

a given problemsolving goal In general this is an intractable

problem We therefore intro duce an eective practical approxi

to tnessforpurp ose based on the plausibility of the goal mation

Webelieve that constraint logic programming provides a natu

ral approach to the implementation of such approximations We

to ok the Common LISP constraints library SCREAMER and

extended its symb olic capabilities to suit our purp oses

Authors address

Department of Computing Science

Kings College University of Ab erdeen

Ab erdeen AB UE

Scotland UK

Email hswhite dsleemanicsdabdnacuk

Intro duction

The prop ertyof tnessforpurpose states whether running a software com

ponent with a supplied set of inputs can satisfy a given goal Fitnessfor

purp ose is related to the notion of comp etence Wielinga Akkermans

Schreib er but with some imp ortant dierences Firstly whilst com

p etence describ es the general inputoutput relationship of a problemsolver

the verication of tnessforpurp ose involves a test of a problem solver and

a sp ecic set of input instances against a given goal Secondly tnessfor

purp ose is a descriptive notion b ecause it is applied to an existing system

whereas comp etence is a prescriptive term applied to a system that is to

b e built

Descriptions of tnessforpurp ose can enable human or machine agents

to assess the suitability of application of the describ ed comp onent for some

particular task Some established approaches to assessing whether a soft

ware comp onent is suitable for solving the task at hand are by design test

ing verication validation proving properties and syntacticstructural

matching with requirements Suitability by design aims to develop new com

ponents to satisfy requirements Suitability by testing aims to detect faults

in existing comp onents by careful preparation of test case inputs Suitability

by verication validation aims to check whether knowledgebased sys

tems meet their users requirements by for example identifying redundant

or conicting rules in a knowledge base Suitability by proving properties

is p otentially more informative However when done by hand it is a dif

cult and involved pro cess and only feasible for small programs When

automated it can also b e problematic for example MacKenzie notes that

wholly automatic theorem provers have so far b een considered inadequate

to these tasks MacKenzie

The semiautomatic assessment of tnessforpurp ose is b ecoming in

creasingly imp ortant for two main reasons Firstly as the range and sophis

tication of software increases it b ecomes dicult for a human user to make

an informed choice of the b est software solution for any particular task The

same p oint applies equally to domain indep endent reasoning comp onents

such as the problem solving metho ds PSMs Benjamins Fensel

We b elieve that a novice user of such comp onents in particular could b ene

t greatly from advice generated as a result of a tnessforpurp ose analysis

Secondlywe observe a demand for software brokers which given some soft

ware requirements return either the software itself or a reference to such

software In the knowledge acquisition community the Ibrow pro ject

Benjamins et al intends to build such a broker for the distribution

of problem solving comp onents

Recent work on the description of comp etence includes that of Ben

jamins et al in the context of the Ibrow pro ject Benjamins et al

Benjamins et al Fensel et al as part of an ongoing investigation

into the role of assumptions Fensel Schonegge Fensel Schonegge

and Wielinga et al as a metho dological approach to the op erational

isation of knowledgelevel sp ecications Wielinga Akkermans Schreib er

We consider Fensels approach to b e the nearest to our work b ecause

it investigates the context dependency of an existing problem solverPSM

through the discovery of assumptions Our work also investigates the con

text dep endency of problem solvers but through the question of task suit

ability with the available knowledge Thus whilst Fensel investigates a

problem solving metho d in isolation of its inputs in order to derive suitabil

ity conditions we take a problem solver together with its inputs and test

Figure Comparison of the Fensel approach i and our approach

ii to the discovery of problem solving prop erties

their combined suitability for solving a sp ecic task Both lines of inquiry

are intractable and therefore demand some compromises in any imple

mentation Fensels compromise concerns the level of automation of his

pro of mechanism whichisaninteractive verier rather than a fully auto

mated pro of mechanism Since wewould like to generate advice at runtime

for the p otential user of a problem solver we compromise instead with the

deductivepower of our pro of mechanism We b elieve however that what

our constraint satisfaction mechanism may lack in expressivepower it gains

in its abilities to combine the results of multiple problem solvers through

propagation mechanisms and to run as a batch pro cess Figure compares

Fensel et als pro cess of assumption hunting with our approach to matching

a problem solver to a to olkit users goal

Our approach to tnessforpurp ose analysis was develop ed to assist

in the generation of advice for novice users of knowledge acquisition to ols

KA to ols and problemsolvers in the Muskrat system Graner Slee

man Sleeman White White Sleeman White

Muskrat is a MUltiStrategy Knowledge Renement and Acquisition

To olb ox that makes it easier for novice users to apply and combine the

incorp orated software to ols to solve some sp ecic task When generating

advice on the application of a chosen problemsolver Muskrat should b e

able to dierentiate between the following three cases for the knowledge

bases available to the problemsolver

Case The problemsolver can be applied with the knowledge bases

already available ie no acquisition or mo dication of knowledge is neces

sary

Case The problemsolver needs knowledge bases not currently

available therefore these knowledge bases must b e rst acquired

Case The problemsolver needs knowledge bases not currently

available but the requirements can be met by mo difying existing knowl edge bases

The phrase tnessforpurp ose and the computational approximation

to it as describ ed in this pap er represents our approach to case We

exp ect subsequent researchtoinvestigate issues relating to cases and

In the next section we dene more exactly what wemeanby tness

forpurp ose and explain the computational diculties which arise in its

analysis In section we describ e how we approach these diculties by

considering an approximation of tnessforpurp ose rather than the actual

tnessforpurp ose In section we explain howwe are implementing these

ideas using constraint logic programming in Common Lisp Finally in

section we summarise the ideas relate them to other work in the eld

and indicate some p ossible directions for future work

FitnessforPurp ose

When the advisory subsystem of Muskrat addresses the problem of tness

forpurp ose it is in eect p osing the question Is it p ossible to solve the

given task instance with the available problemsolver knowledge bases

and KA to ols Clearlythis is a very dicult question to answer with

out actually running the problemsolver regardless of whether the answer

is generated by a human or a machine Indeed the theory of computa

tion has shown that it not p ossible in general to determine whether a

program in this case a problemsolver terminates This is the Halting

Problem Turing Therefore the only way to arm the suitability

of a problemsolver for solving a particular task is to run it and test the

outcome against the goal For the purp oses of generating advice in a mul

tistrategy to olb ox however we cannot aord this luxury particularly since

running the problemsolver could often be computationally intensive We

prefer instead to pose the weaker question Is it plausible that the given

task instance could b e solved with the available problemsolver knowledge

bases and KA to ols Wehaveshown that it is p ossible to demonstrate

computationally that some congurations of a task problemsolver exist

ing knowledge bases and KA to ols cannot even in principle generate an

acceptable solution to the task Such situations form a set of recognisably

implausible congurations with resp ect to the problem at hand Further

more the computational cost asso ciated with recognising this implausibility

is in many cases far less than that asso ciated with running the problem

solver

For example consider a question from simple arithmetic is it true that

Rather than evaluate the left hand side

of the equation straightaway let us rst insp ect the problem to see if it is

reasonable We knowthat when an even number is multiplied by an o dd

numb er the result is always even and that an o dd number multiplied by

an o dd number is always o dd Therefore the left hand side of the equation

could b e rewritten as heveni hoddi hoddi Likewise an even number

addedtoanoddnumberisalways o dd and the sum of twooddnumbers is

always even Then evaluating left to right wehave hoddi hoddiwhich

is heveni Since is not even it cannot b e the result of the evaluation

Wehavethus answered the question without having to do the hard work

of evaluating the actual value of the left hand side

As another example consider the truth of the statement If Pigs can

y then Im the Queen of Sheba which we write as P Q Given that

For simplicity we currently assume the application of a single chosen

problemsolver

the premise P is false we can use the truth table for logical implication to

derive that the whole statement is true since any implication with a false

premise is true Notice that we derived our result without having to know

the truth of the consequent Q In a similar way it is p ossible to investigate

the outputs of programs in particular problemsolvers without needing

complete information ab out their inputs This issue b ecomes imp ortantif

running a problem solver has a high cost asso ciated with it such as the

time it takes to p erform an intensivesearch In such cases a preliminary

investigation of the plausibilityofthe task at hand could savemuch time

if the intended problem solver conguration can b e shown to b e unsuitable

for the task We consider suchaninvestigation to b e a kind of plausibility

test that should b e carried out b efore running the actual problem solver

The idea was suggested as part of a general problem solving framework

a In his b o ok How to SolveIt he prop osed some byPolyaPoly

general heuristics for tackling problems of a mathematical nature including

an initial insp ection of the problem to understand the condition that must b e

satised by its solution For example is the condition sucient to determine

the problems unknown And is there redundancy or contradiction in the

condition Polya summarised the issue by asking the following

Is our problem reasonable This question is useful at an

early stage of our work if we can answer it easily If the answer

is dicult to obtain the trouble wehave in obtaining it may

outweigh the gain in interest Polya

It is interesting to note that the arithmetic example given ab ove rst

abstracts the problem instance to a dierent space ie from integers to

that of o dd and even numb ers to which a simpler algebra can b e applied

We note that much of the problem instance detail has b een ignored to keep

the plausibility test simple On the other hand enough detail has been

retained for the test to reect the current problem instance and for it to b e

useful In this sense the plausibility test has approximated the task In the

next section we dene a more precise notion of plausibilityapproximation

and explain how it can b e applied to problemsolvers

The Plausibility Approximation to Fitness

forPurp ose

We classify a prop osed output value of a problem solver to b e one of im

plausible plausible possible or actual Avalue that is implausible will never

be the output of the problem solver Informallya plausible value cannot

b e ruled out as a solution through reasoning such as that in the arithmetic

or logical examples ab ove A possible valuecan b e a solution to the problem

in some cases An actual value is the solution in a given case It is worth

noting that all possible values are also plausible and any actual value is

always possible and hence also plausible see Figure

Many AI programs p erform searches of problem spaces that grow exp onen

tially with the size of the problem

One way to determine whether the plausibilitytest is useful is to compare

the computational complexities of the problem solver and the plausibility test If

the complexity of the plausibilitytestislower than that of the problem solver we

might assume it is reasonable to apply the plausibility test rst Unfortunately

this mo del takes no account of the utility of the information gained from the

plausibilitytest

Figure Venn Diagram showing the relationships b etween plausible

p ossible and actual values

As an example of this classication consider the formula for the area

of a circle r Supp ose the radius r and therefore r

The range of p ossible values is but for convenience we might

approximate to for estimating the lower b ound of the range and for

estimating the upp er b ound of the range This gives the plausible range of

r as Note that the range of p ossible values lies completely within

the range of plausible values When evaluating the expression an actual

value might b e

The set of plausible values is an approximation to the set of possible

values but we dene it such that the set of possible values is always con

tained within the set of plausible values This guarantees that an implausible

value is not p ossible so we can approximate the test for a p ossible value

with a test for plausibility Note that a plausible value may not necessarily

b e p ossible plausible but imp ossible values are the small sh that slip

through the net Note also that for any gain in computational eort testing

an element for memb ership of the set of plausible values must b e in some

sense less exp ensive than testing it for memb ership of the set of possible

values

Inferences Using the Plausibility Approximation

Our approach recognises that a problem solving task can only b e solved if

there is at least one memb er of the p ossible set of problemsolver outputs

that is also a solution to the task Let us denote the set of acceptable output

values ie those that solve the task by goal If the problem solver PS uses

fR R R g then we denote the inputs I fI I I g in roles R

n n

the set of p ossible problem solver outputs by p ossible

P SI R

We then note the following three lemmas

Lemma The goal is only satisable if

pp p ossible p goal

PSIR

or equivalently

p ossible g oal

P SI R

Lemma If possible g oal then the problem solver cannot

PSIR satisfy the goal

Figure Insp ecting the relationship between the problem solving

goal and the plausibility approximation

Lemma If every p ossible output value satises the goal ie

p ossible goal then the goal must b e satised

PSIR

The plausibilityapproximation to p ossible is the set of values

PSIR

which are plausibly output by the problem solver PSgiven the inputs I

fI I I g in roles R fR R R g We denote this approximation

n n

by plausible where plausible p ossible Let us

PSIR P SI R PSIR

now insp ect the intersection plausible goal without additional

P SI R

knowledge of p ossible

P SI R

There are four cases to consider see gure

Case Goal is implausible if plausible goal This follows

P SI R

b ecause the plausible set of values contains the p ossible set of values by

denition so

plausible goal p ossible goal

P SI R P SI R

It follows from lemma that the goal is unsatisable

Case The goal is plausible if goal plausible In this case

PSIR

it do es not necessarily followthat goal p ossible There are two

PSIR

cases to consider either p ossible goal whichwould imply

PSIR

that the goal is unsatisable from lemma or p ossible g oal

PSIR

which leaves the goal plausible from lemma Without knowing the

contents of p ossible we do not know whichofthetwo cases applies

P SI R

so we must draw the weaker of the two conclusions ie that the goal is

plausible

Case Goal is plausible if plausible goal with the same

PSIR

reasoning as for case

Case Goal is satised if p ossible goal If the plausibility

PSIR

approximation is completely contained within the goal then all the p ossible

problem solver outputs must also satisfy the goal That is

plausible goal p ossible goal

P SI R P SI R

ust b e satised Therefore from lemma the goal m

Application to ProblemSolving

We are applying these ideas to problem solving scenarios in whichanumber

of knowledge bases KB KB KB are assigned input roles

R R R to a problemsolver and a desired goal is stated We call

this a problemsolver conguration left hand side of Figure Note that

at this stage wehave made no assumptions ab out the representation of the

knowledge when we say know ledge base we do not imply a set of rules

Neither do wemakeany assumptions ab out the specicity of the knowledge

bases with resp ect to the given problem instance That is a knowledge

base might remain constantover all problem instances it mightchange o c

casionallyor it might change with every instance In essence any input

to the problemsolver has b een lab elled as a knowledge base We use the

knowledge obtained from these inputs and a plausibility description of the

behaviour of the problemsolver to generate a set of plausible output val

ues Note that under this scheme a problemsolver description is itself

a program that given appropriate inputs generates the problem solvers

plausible output The description therefore represents a kind of mo del of

the problem solving task it is a metaproblemsolver right hand side of

Figure The purp ose of building the metaproblemsolver is to determine

whether the goal is consistent with the set of plausible values that it gen

erates b ecause if the goal is not consistent with this set then it will also

not b e satised by the output of the problemsolver itself In such cases we

need not run the problemsolver to test its outcome

But how can this functionalitybe implemented In a naive generate

andtest approach we mightanswer the plausibility question by testing ev

ery p oint in the plausible output set against the goal G Unfortunatelythis

is b oth computationally inecient OHara Shadb olt and enables

only the mo delling of sets of nite size For a more exible and computa

tionally more ecientversion we prefer a constraintbased approach The

idea is that a set of plausible values is represented using constraints The

plausibilityset could be of innite or indenite size such as the set of

even counting numb ers or the set of all p ersons whose surname b egins with

W To test the plausibilityofa given goal the goal is rst represented as

a set of constraints These constraints are then applied to the plausibility

set generated by a metaproblemsolver If the resulting set is the empty

set then the goal was not plausible if the resulting space is nonemptyor

cannot easily b e demonstrated to b e empty then the goal remains plausi

ble Furthermore unlike the metalevel reasoning required for the halting

problem the question of plausibility if dened appropriately can b e guar

anteed to terminate within a nite numb er of program steps b ecause the

metalevel need only b e an approximation to the problem solving level

A role represents a way in which an input is used for problem solving Our

notion of a role corresp onds very closely to the notion of a knowledge role in

CommonKADS Schreib er et al p

Consider the whimsical analogy of a man wishing to buy a pair of sho es he

do es not walk into a b o okshop pickupevery b o ok and insp ect it for its sho elike

qualities the generateandtest approach Instead he recognizes from the outset

that b o okshops sell b o oks ie the plausibility space is a set of b o oks and that

this is not consistent with the goal of buying sho es

Consider the case of the halting problem in whichwe should like to determine

whether a program runs forever or halts At the metalevel wemaycho ose to run

the program for some given length of time to see what happ ens If the program

halts within the allotted time then weknow that it halts Otherwise it remains

plausible that it runs forever

Figure Problemsolver and Metaproblemsolver Congurations

Note that the plausibility refers to the combination of knowledge bases

their problemsolving roles and the sp ecic goal As an exploration of all the

KBrole pairings might take a considerable amount of time wemakethis

pro cess more ecient by assigning preconditions to each problem solving

role and rst checking that these are fullled b efore conducting a plausi

bilitytest However note that in some cases wemay still have to explore a

numb er of KBrole combinations b efore deciding that no plausible solution

exists

More general scenarios are also p ossible since a plausibility set gener

ated by one metaproblemsolver can also b e used as an input to a further

metaproblemsolver When cascading descriptions in this way the approx

imate input to the second metaproblemsolver is used to generate a further

approximate output Note that the approximation generated by the second

metaproblemsolver is less precise than when particular inputs are given

Nevertheless the generated plausible output set may still contain enough

knowledge to answer some interesting questions ab out the combination of

problemsolvers

Implementation Approach

We mentioned earlier that we prefer a constraintbased approach to answer

ing the question Is the goal G contained in the set of plausible output v al

ues P In fact it is crucial to the implementation that constraints derived

from the goal can b e used to reduce the size of the plausibilityset without

enumerating it This technique makes the plausibility test less exp ensive

than running the problemsolver b ecause the whole search space do es not

have to b e explored to determine whether a goal is implausible It also leads

one naturally to consider an implementation using constraint logic pro

gramming As the to ols we wished to mo del and the rest of the Muskrat

framework were already implemented in Common Lispwe implemented the

plausibilityapproximation to tnessforpurp ose using Screamer a con

straint logic programming library for Common Lisp Siskind McAllester

a Siskind McAllester b

A Brief Intro duction to Screamer and Screamer

Screamer intro duced two paradigms into Common Lisp rstlyitintro

duced a nondeterminism similar to the backtracking capabilities of PRO

LOG Secondly and more imp ortantly for our purp oses it used these back

tracking capabilities as the foundation for a declarative constraints package

The Screamer constraints package provides Lisp functions that enable a

programmer to create constraint variables often referred to simply as vari

ables assert constraints on those variables and search for assignments of

values to variables that are consistent with the asserted constraints

Although Screamer forms a very go o d basis for solving numeric con

straint problems in Common Lispwe found it less go o d for tackling prob

lems based on constraints of a more symb olic nature We therefore extended

the Screamer library in three ma jor directions rstly to improve the ex

pressiveness of the library with resp ect to constraints on lists secondlyto

Strictly sp eaking this should read no more precise

Version available at the time of writing from

httpwwwcisupenneduscreame rtoo lsh ome html

intro duce higher order constraint functions such as constraintoriented ver

sions of the Lisp functions every some and map car and thirdly to enable

constraints to b e imp osed on CLOS ob jects and their slots We called the

extended library Screamer A more detailed accountofScreameris

given in our technical rep ort White Sleeman

In the next sections we outline an approach to the construction of meta

problemsolvers using Screamer and provide an example of its usage to

mo del the tnessforpurp ose of the setpruning problemsolving metho d

Building a MetaProblemSolver

To capture a description of a problem solver in terms of its inputs and

outputs we prop ose that the Muskrat develop er p erforms the following

activities

Elicit the necessary prop erties of each of the problem solvers

input roles and express those prop erties as constraints

Elicit the prop erties necessary for interrole consistencyand

express those prop erties as constraints across the input roles

Elicit guaranteed prop erties of the problem solvers outputs

and express those prop erties as constraints Often prop erties of the

outputs are expressed in terms of the input roles

These three sets of constraints then form the basis of the metaproblem

solver The aim of the pro cess is to collect constraints that capture detailed

glassb ox knowledge of the problem solver Once captured this knowl

edge can subsequently b e made available as blackboxknowledge to a user

of the Muskrat to olb ox The acquisition of these constraints is a onceonly

activity b ecause the problem solver description is also goalindep endent In

other words the same metaproblemsolver can be reused for the tness

forpurp ose verication of dierentknowledge bases for dierent tasks

Note that initially there maybemany plausible KBrole assignments

so the verication of tnessforpurp ose represents a search sub ject to the

ab ovegiven constraints through the space of the available KBs and the

known input roles of the problem solvers

Example Set Pruning

As an example of the approach consider the set pruning metho d Benjamins

et al Gro ot ten Teije v an Harmelen This metho d iterates

over the memb ers of a set removing those memb ers that do not exhibit a

given prop erty It can therefore b e regarded as a simple lter A problem

solver with this functionality can b e written in Lisp as follows

defun setprune inputset test

removeifnot test inputset

The setpruning functionality can b e demonstrated as follows

A higher order function is a function that accepts another function as argu

ment

CLOS is the Common Lisp Object System a sp ecication of functions to

enable ob jectoriented programming in Lisp which was adopted by the ANSI

committee XJ in as part of the Common Lisp standard

Create a set for testing represented by a list

setq myset to two too

TO TWO too

Prune away all the nonintegers

setprune myset integerp

The imp ortant issue here is not writing the problem solver itself but

the abilityto describ e the problem solver with a metaprogram We now

develop a metaproblemsolver according to the guidelines set out in section

We also demonstrate how the metaproblemsolver can b e used to prove

the implausibility of a problem solving goal

Necessary Prop erties of the Input Roles

The set pruner accepts two inputs a set to b e pruned and a test which

eachmemb er of the output must satisfy In our implementation the set is

represented by a list and the test is represented by a function In this case

the prop erty for a plausible role can be succinctly represented as a Lisp

typ e the role of a set can b e dened as a synonym for the Lisp typ e list

deftype plausibleset

list

We dene a plausible test as any function or CLOS metho d of one

argument The corresp onding typ e for a plausible test can b e dened in

Allegro Common Lisp as follows

Predicate to check whether given function accepts

a single argument

defun oneargfunctionp f

let args arglist f

cond

length args t

member car args rest optional t

t nil

Defines a specialisation of the function type for

functions of one argument

deftype plausibletest

and function satisfies oneargfunctionp

These denitions allow us to constrain an input to b e either a

plausibleset or a plausibletest

Prop erties Necessary for Interrole Consistency

For interrole consistencywecheckthemutual compatibilityofthe test and

the set These two inputs would b e inconsistent for example if the test is

for the primeness of a number the set contains strings as well as numbers

Note that technically CLOS metho ds are also classied as functions

We make the distinction b etween the prop ertyof primeness which can only

meaningfully b e applied to an integer and the prop erty of b eing a prime number

which can b e applied to anytyp e but nonintegers would always return the value

false As the given test is for the primeness of an integer inconsistencies can

arise and a test for interrole consistency is appropriate

defmethod primep n integer

setq n abs n

when n returnfrom primep t

unless evenp n

do divisor divisor check odd numbers from

divisor sqrt n t up to root of n

when zerop mod n divisor

returnfrom primep nil

Assign test to be our implemented test for primeness

setq test primep

STANDARDGENERICFUNCTION PRIMEP

Assign set to be a plausible set which includes a string

setq set list m

Assign test to be our definition of primep

setq test primep

STANDARDGENERICFUNCTION PRIME

Demonstrate that errors can occur if the test cannot be

applied to every member of the set

setprune set test

Error No methods applicable for generic function

STANDARDGENERICFUNCTION PRIMEP with args m of

classes STRING

However a metaproblemsolver can recognise the condition of interrole

inconsistency that causes such an error For the example of primeness the

idea can b e realised byintro ducing a new function called testable which

returns true T if the given test can b e applied to a supplied argument and

false NIL otherwise

defun testable inputset test

Conclude that test is plausible if it is

not implemented asa CLOS method

when not equal typeof test standardgenericfunction

returnfrom testable tnbsp

Otherwise inspect the types of input that the method

accepts to determine plausibility

dolist element inputset

when null computeapplicablemethods test list element

returnfrom testable nil

t returns true by default

The function works by considering two cases Firstly if the test is a

true function ie not a CLOS metho d then Lisp cannot detect the

exp ected typ es of its arguments and it is not checked further for its com

patibility with the given arguments If however the test is implemented

as a generic function with one or more CLOS metho ds then the function

determines whether the generic function is applicable to eachofthevalues

in the given set That is the function checks whether a metho d exists

that accepts arguments of the same typ e as the elements of the supplied set

Note that in Lisp the generic function approach using defmethod allows

for stronger typing of input arguments than the weakly typ ed defun form

The generic function approach therefore provides go o d supp ort for checking

interrole consistency

Notice that testable has b een implemented in a very general wayand

do es not enco de any prior knowledge of the test Furthermore it will also b e

acheap test to execute certainly faster than executing the test itself We

can b e sure of this b ecause we know that the Lisp system must compute the

applicable metho ds of a generic function with anygiven arguments b efore

a metho d can b e applied Also as this is such an imp ortant part of Lisps

runtime evaluation system we can b e condent that it has b een optimised

Given the denition of testablewe can nowcheck whether test can

safely b e applied to each of the memb ers of set

testable set test

NIL

We can also assert that test b e safely applied to all memb ers of setas

follows Note that with the value of set given ab ove the assertion fails

as exp ected

assert funcallv testable set test

FAILS

Guaranteed Prop erties of the Problem Solvers Out

puts

When pruning a set it is guaranteed that

The output is of typ e plausibleset

The outputset is a subset of the inputset ie outputset input

set and

Every memb er of the outputset satises the given test

These three statements are captured by the following three assertions

assert typepv outputset plausibleset

assert subsetpv outputset inputset

assert everyv constraintfn test outputset

I also tried an alternative implementation which only computed the appli

cabilityof the test for each distinct element type in the given set This was a

promising approach b ecause an input set maycontain many elements of the same

typ e and the applicable metho ds would b e computed once for eachtyp e instead

of once p er memb er of the set However myinvestigations led me to the conclu

sion that this approachwas no faster than the given implementation probably

b ecause determining the typ e of a Lisp value is just as computationally exp ensive

as computing a generic functions applicable metho ds

Recall from chapter that constraintfn accepts a conventional Lisp func

tion as an argument and returns a constraintbased function as its result

By combining these constraints with the constraints given earlier for ensur

ing satisfaction of role properties and interrole consistency the following

denition of the metapruner is derived

defun metaprune inputset test

let outputset makevariable

Assert properties of input roles

assert typepv inputset plausibleset

assert typepv test plausibletest

Assert interrole consistency

assert funcallv testable inputset test

Assert known properties of the output

assert typepv outputset plausibleset

assert subsetpv outputset inputset

assert everyv constraintfn test outputset

outputset outputset is the return value

In this denition the inputs inputset and test corresp ond to the in

gure The metaproblemsolver rst generates puts KB and KB of

a constraintvariable to be used as its output then asserts the necessary

prop erties of its input roles Next it guarantees the interrole consistency

of its inputs by asserting that the supplied test must b e applicable to each

memb er of the inputset Finally the prop erties of the output are asserted

namelythateachmemb er of the outputset is also a memb er of the input

set and that the supplied test is satised byeach memb er of the outputset

Note also that a metaproblemsolver must return the constraintvariable

which represents the plausible output value of the problem solver

Verifying FitnessforPurp ose

The metaproblemsolver is interesting b ecause it enables reasoning ab out

the output of the problem solver without executing it In particular many

cases in which the problem solver would not satisfy the goal are detected

early for example even b efore the inputs to the problem solver are consid

ered Wenow describ e some examples in which failures are detected by the

metapruner

In the following example the test integerp can meaningfully b e ap

plied to all the memb ers of the input set but the goal data structure

is rejected b ecause it is not a subset of the given input recall that set has

the value m

setq plausibleoutput metaprune set integerp

plausibleset

assert equalv plausibleoutput

FAILS

Here the symbolic argument notafunction representing the test

is not a function and is therefore incompatible with the exp ected typ e

plausibletest

metaprune m notafunction

FAILS

In this example the supplied test for whether an integer is prime cannot

meaningfully b e applied to all the memb ers of the inputset

metaprune m primep

FAILS

Searching for Plausible Congurations

Supp ose that several inputsets were available as well as a single goal By

using Screamer together with the metapruner it is p ossible to discover

which of the inputsets plausibly satisfy the goal For example supp ose that

the inputsets were fg f g fA B Cg fAg and that the test prunes

away items that are not integers Supp ose further that the goal is to obtain

the set fg

Then the problem is rst set up as follows

setq kbs amemberofv a b c a

nonnumber enumerateddomainNIL A B C A

setq goal

assert equalv metaprune kbs integerp goal

NIL

Notice that no failures have b een generated so far indicating that the

task instance is still plausible The following Screamer search idiom re

turns all inputsets that plausibly satisfy the goal

allvalues solution kbs staticordering linearforce

In eect the Muskrat metho d is recommending to try the inputsets

f g and fAg to satisfy the goal It has ltered out the emptysetfg

and the set fab cg b ecause they do not plausibly satisfy the goal Notice

that fg do es not actually satisfy the goal but fAg do es

returns nil because

setprune integerp returns

equal setprune integerp goal

NIL

returns t because

setprune A integerp returns which

is the same as the goal

equal setprune A integerp goal

Example Meal Planning

Supp ose we are applying a problemsolver which constructs solutions con

sistent with some given domain and temp oral constraints a design task

Wehavechosen to apply the problemsolver to the domain of meal prepara

tion b ecause the domain is richandchallenging but also easily understo o d

Furthermore the problemsolving task is analogous to exible design and

manufacturing also called justintime manufacturing In the domain of

The task is also soluble by two separate problem solvers the domain con

straints are solved by the rst and the temp oral constraints satised by the

second A detailed discussion of the cascading of problem solvers is outside the scop e of this pap er

Figure Problem Solving in the Domain of Meal Preparation

meal preparation the designer is used to comp ose or construct a dish

which is consistent with culinary temp oral and resource constraints also

taking a set of dish preferences into consideration see Figure

Now supp ose that we would liketo use plausibility mo delling on this

problemsolver by constructing its corresp onding metaproblemsolver A

metadesigner is easy to construct if we neglect the constraints dish pref

erences and the available resources Note that neglecting knowledge of

inputs will havethe eect of pro ducing more general outputs b ehaviour

that is consistent with the approximation describ ed in section The

metadesigner pro duces a set of plausible dishes together with their plau

sible preparation times These times are represented as upp er and lower

b ounds on the preparation of whole dishes and are derived from knowledge

of the preparation times of dish comp onents The upp er b ound of the dish

preparation time is set as the sum of the durations of the comp osite tasks

the lower b ound is the duration of the lengthiest task Constraints from

the goal can then b e applied to the set of plausible scheduled dishes to see

if inconsistencies can be detected see Figure Note that although we

advo cate the use of the constraint satisfaction paradigm for the implemen

tation of the metalevel this choice is indep endentofthe implementation

metho d used by the problem solver itself The indep endence arises b ecause

the metalayer treats the problem solver as a blackbox describing only its

inputoutput relationship and not howitworks internally

We have implemented the example using the knowledge provided in

Table a list of dish comp onents together with their comp onenttyp e a

measure of their dietary value their cost and preparation time In addition

the designer must be told what it should construct from this knowledge

This information is provided b y a dish ontology which states that a dish

consists of a main comp onent together with a carb ohydrate comp onent

and two other vegetables Likewise the cost of a dish is dened to b e the

sum of the costs of its comp onent parts and the number of diet points

asso ciated with a dish is the sum of the diet p oints of its comp onents The

Figure Plausibility Reasoning in the Domain of Meal Preparation

metadesigner generates upp er and lower b ounds for the preparation time

of a dish as describ ed in the previous section

Ingredient name Ingredienttyp e Diet Cost Prepn

Points Time

Lamb Chop Main meat

Gammon Steak Main meat

Fillet of Plaice Main sh

Sausages Main meat

Chicken Kiev Main meat

Quorn Taco Main vegetarian

Jacket Potato Carb ohydrate

FrenchFries Carb ohydrate

Rice Carb ohydrate

Pasta Carb ohydrate

Runner Beans Vegetable

Carrots Vegetable

Cauliower Vegetable

Peas Vegetable

Sweetcorn Vegetable

Table Meal Preparation Knowledge Used in Plausibility Set Generation

a Points schemes like these are commonly used by slimmers as a simplication of

caloric value When using suchschemes slimmers allow themselves to consume

no more than say p oints worth of fo o d in a single day the actual limit usually

dep ends on the slimmers sex height and weight

The Screamer implementation makes use of the CLOS facilities oered

by dening the set of plausible dishes as a single object in which each slot

such as the main comp onent of the dish the preparation time dietary

points value and cost is constrained to be plausible In practice this

means that constraints are expressed across the slots within a single ob ject

If the domain of one of the slots constraint variables changes then this

causes propagation to the other related slots

For example let us insp ect some plausible dish preparation times and

costs

Create an instance of the object

An after method asserts the appropriate constraints across

the objects slots at object creation time

USER setq mydish makeinstance plausibledish

PLAUSIBLEDISH xca

Retrieve the value of the duration slot

USER slotvalue mydish duration

integer

Retrieve the value of the cost slot

USER slotvalue mydish cost

integer enumerateddomain

This tells us that a plausible dish takes b etween ten minutes and two

hours to prepare and costs or Now let us restrict our searchto

a sh dish and again insp ect the plausible values

USER assert typepv slotvaluev mydish main fish

NIL

USER slotvalue mydish duration

integer

USER slotvalue mydish cost

The plausible range size for the preparation time of the dish has de

creased and the cost of the dish has b ecome bound without having to

makeany choices of vegetables If the goal were to have such a sh dish

ready in less than quarter of an hour it would fail without having to know

the details of the dish preferences or kitchen resources used bythe actual

problemsolver and without needing to run the problemsolver

USER assert v slotvaluev mydish duration

FAILS

In a similar wayifwe dene a quick dish to b e one that takes less than

half an hour to prepare and a healthy dish to b e one that has a dietary

points count of or less then it is easy to discover that there is no such

thing as a quick dish that includes a jacket p otato or a healthy dish that

includes gammon and french fries

Discussion

In this section we discuss the contributions which Muskrat is making and

indicate some p ossible directions for further work

Contributions

Muskrats contributions include the following

A notation for representing the capabilities of problemsolvers to

gether with their relationships to the contents of knowledge sources and a

desired goal

The further developmentofthetechnology of constraint satisfaction

through our extension to the Screamer package

Formulation of the idea of economical problem solving and the in

vestigation of its relationship to the description of problem solving metho ds

Progress towards the provision of a framework that unies problem

solving knowledge acquisition and knowledge transformationrenement

The development of a metho d to determine whether it is plausi

blelikely that knowledge bases can b e reused by a particular problem solver

to solve a particular task

Our notation for representing the comp etencies of problem solvers and

the technology we have chosen for proving their prop erties diers consid

erably from the related works of Wielinga et al Wielinga Akkermans

Schreib er Fensel et al Fensel Schonegge Fensel

Schonegge and PierretGolbreich PierretGolbreich The

dierences haveevolved b ecause wehave b een working towards a dierent

but asso ciated ob jective Unlike the metho dology of Wielinga et al for

example we assume our problem solvers exist and seek to describ e them as

they are rather than attempting to construct them as wewould likethemto

be Furthermore since we are building an advisory system for novice users

of problem solvers wehave strong requirements for operational descriptions

and a fully automatic pro of technique PierretGolbreich argues that formal

metho ds should b e used to describ e problemsolving metho ds and that this

oers a means to decide on a comp onents reusability Since suchmethods

are not op erational however this approach cannot b e applied to our prob

lem Fensel Schonegge have op erationalised their pro of technique but its

po werful pro of mechanisms are necessarily interactiveFensel Schonegge

and not suitable to b e driven bynovices Our notation on the other

hand is declarative op erational and suciently expressiveto enable the

automatic derivation of interesting prop erties of a problem solver and the

knowledge it employs

Our extension to the Screamer package driven by our requirementfor

a declarative and op erational description of problem solvers and knowledge

sources has reached a level of expressiveness that contributes to the eld

of constrainttechnology Screamer can b e used to provide elegantsolu

tions to many of the nontrivial problems cited as soluble by commercially

available systems such as CHIP Simonis and ECLIPSE Wallacee

Novello Schimpf The Lispbased nature of Screamer however

has also help ed it to gain advantages over its PROLOGbased counterparts

such as the ability to assert constraints on expressions that take functions

as arguments This is imp ortant in the currentcontext when one considers

that a function might b e the realisation of a PSM

y the re Our ideas of economical problem solving have b een motivated b

quirement to generate advice on the suitability of problem solvers on the y

at runtime Conceptually some of the techniques applied can b e seen vari

ously as applications of abstraction Giunchiglia Walsh or sp ecial

cases of generalisation transformations within the plausibility framework

of Collins and Michalski Collins Michalski Oroumchian

In contrast to these approaches which classify their metho ds according to

the morphology of their mappings we are investigating the knowledgelevel

dep endencies of eachtechnique in terms of domain task and metho d

The fourth contribution of Muskrat is to create a unied framework

for problem solving directed knowledge acquisition and knowledge trans

formation This aim is shared by the generalised directive mo dels GDM

work of OHara Shadb olt and van Heijst OHara Shadb olt van Heijst

but there are signicant dierences as their work takes place in the

context of analysing source materials and attempting to coevolve domain

ontologies as well as the appropriate mo del for the task in some cases a KA

task is activated In the case of Muskrat the task to b e solved and the

problem solver to b e used have already b een identied Muskrat seeks to

discover whether existing knowledge bases can b e used without change with

the identied problem solver whether it can b e transformed b efore use or

whether it is necessary to acquire a completely new knowledge base using

aKA to ol Other work that has soughtto provide a common framework

for problem solving and KA includes MOBAL Morik et al VITAL

Motta OHara Shadb olt and NOOS Arcos Plaza Arcos

Plaza

Our contribution to the topic of knowledge reuse is most closely related

to Pupp es work Pupp e and the Protege pro ject Gennari et al

Pupp e has realised as do OHara et al OHara Shadb olt van

Heijst that as exp erts learn more ab out the domain their p ersp ec

tives on the task maychange and hence it is highly desirable that acquired

knowledge can b e used with a dierent algorithm from the one for whichit

was initially acquired Pupp e connes his attention to classication tasks

and notes a number of dierent classication algorithms that pro cess the

same information in dierent ways He discusses a number of algorithms

including CATEGORICAL which pro duces decision treestables heuris

tic classiers which use heuristic rules and casebase reasoners Pupp es

imp ortant insightwas to reimplementseveral classiers so that they have

common data les that can b e readily reused In Muskrat however we

have set up a more general framework in which a knowledge base initially

used with a PSM for a classication task might b e reused with a dierent

PSM for synthesis or planning

Proteges longterm goal is to build a to olset and metho dology for the

construction of domainsp ecic KA to ols and knowledgebased systems from

reusable comp onents The pro ject plans to develop a variety of metho ds and

knowledge bases all packaged as CORBA Common Ob ject Request Bro

k er Architecture comp onents Knowledge bases are made available on a

server which uses the OKBC Op en Knowledge Base Connectivity pro

to col enabling develop ers to query the framebased knowledge with func

tions such as getclassallsubs to retrieve classes getframeslotsto retrieve

slots and getclassallinstances to retrieve the instances of a class A sin

gle metho d proposeandrevise is available within the framework to date

but there are several knowledge bases which have b een used for the VT

elevatorconguration task UHaul truck selection as well as the Ri

b osome and tRNA conguration tasks In order for a particular metho d

to reuse knowledge from a particular knowledge base a mediator must b e

manually enco ded for each metho dKB pair Howev er Gennari et al Gen

nari et al argue that given the similarities in the representations of

the knowledge bases there will b e many commonalities b etween the vari

ous mediators needed This already reduces the workload required to pro

duce the mediators and the Stanford group envisages partially automating

mediator pro duction in the future Muskrats third case see section

corresp onds to the situation for whichProtege currently creates mediators

Whilst Protege assumes that a mediator is necessary Muskrat has evolved

a test to determine whether an available knowledge base mightbe directly

reusable for the given task If this is the case no mediation or adaptation

is required

Further Work

The arithmetic example of tnessforpurp ose presented in section showed

how prop erties of parityininteger arithmetic can b e used to assess the plau

sibility of an arithmetic equation Arithmetic expressions were abstracted

to their equivalent parity expressions b efore the precondition was applied

Moreover there was a very close relationship b etween the precondition and

the original arithmetic expression In such cases it is p ossible for the pre

condition to b e generated automatically Rob ertson This represents

a class of tasks to b e investigated

As an example supp ose again that wewould like to compute the plau

sibility of the arithmetic expression

That is the lefthand side forms the input to the metaproblemsolverand

the goalis that the expression plausibly evaluates to the righthand side We

can rst rewrite the expression into Lisps prex notation giving

Then since the metalevel in this case applies a dierent algebra to the

same numbers the corresp onding metalevel expression can be generated

lifted by replacing the groundlevel op erators and by their

metalevel counterparts meta meta and meta giving

meta meta meta meta meta

With appropriate constraintbased denitions of the metalevel func

tions given in White the expression fails signifying that the goal is

inconsistent with the value of the arithmetic expression

assert meta meta meta meta