Software Agents and the Law Part. 1. Lovely Rita What Are They? Their

Software Agents and the Law Part. 1. Lovely Rita What are they?

U t r e c h t , De c e m b e r 1 0 , 2 0 0 3 ò Software agents (called also digital or electronic agents) ò computational entities Giovanni Sartor ò capable of autonomous action in complex contexts. CIRSFID – Re s e a r c h Ce n t r e f o r Co m p u t e r s a n d L a w U n i v e r s i t y o f B o l o g n a - It a l y A L FE B IIT E P r o j e c t

Their competence (I) Their competence (II)

ò Agent are characterised (to different degrees – the capability to interact with others coherently and in different combinations) by the following: (agents use ontologies and message-based communication), – reactiveness to the changing conditions of the environment (agents adjust their own behaviours – intelligence (agents are able to acquire and process according to the changes they sense), information and to learn from experience), – flexibility (agents are able to face unpredictable – pro-activeness (agents are able to decide according to their goals, i.e. they are goal driven), scenarios), – mobility (agents move through networks, in order to – persistency (the life of agents is supposed to be long lasting), use resources and to locate partners).

3 © G. Sartor 4 © G. Sartor

Rational agents: the BDI model Agent technologies: Reactive agents

ò BDI (Beliefs-Desires-Intentions) ò The agent has the propensity (disposition) to ò The agent has react to stimuli coming from its environment – Beliefs (about the environment and about itself) activating certain behavioural models (if hot, get – Desires (he would like that some states obtain) ò On the basis of its beliefs and desires, the agent away!), (if good, eat it!) – Builds cognitive models of its environment (theoretical ò Those behavioural model can overlap with each reasoning) other, yielding a combined result – Bilds plans on how to achieve its goals (practical reasoning) ò Finally, the agent ò When the agent receives a stimulus, it behaves – Forms the intention to implement those plans according to this combined results – Acts accordingly

5 © G. Sartor 6 © G. Sartor

1 Agent technogies: adaptive agents A further feature of agents

ò The agent ò Each of them has its own purposes and its own – Replicates old behaviour knowledge (no master plan to govern all their activities): – Sometimes, tries new ones (randomly, or according – Advantage: Each agent behaves according to its purposes and to some strategy) knowledge, reacts in a flexible and creative way to its specific circumstances. – If the new behaviour is effective under certain – Disadvantage: no necessary co-ordination with other agent. conditions, the attitude to replicate it under similar Sometime when each agent is separately following its agenda conditions is increased the result may be negative for everybody (prisoner dilemma, – If the new behaviour is ineffective under certain tragedy of the commons). conditions, the attitude to replicate it under the same – Problem: how to reconcile the autonomy of individuals and the conditions is diminished preservation (or achievement) of certain systemic properties?

7 © G. Sartor 8 © G. Sartor

Solutions Agents in the internet

ò Central planning (but then we lack the advantage of ò Agents help in coping with the excess of flexibility and autonomy ) – accessible information ò Setting constraints on the behaviour of agents (build agent that cannot lie, or cannot destroy each other, etc.) – possible actions ò Setting norms for agent societies (agents that can ò They contribute to accelerate the dynamics of include rules in their decisional processes, and possibly electronic commerce: that can adopt shared rules when the circumstances are appropriate) – Searching for products ò Building altruistic agents, which besides being self- – Searching for partners interested (trying to persist and to achieve the benefit of – Electronic negotiation their user) also take into account the common good (and try to get to a shared view of the common good) 9 © G. Sartor 10 © G. Sartor

Agents and the economy of the net Types of agents

ò Agents may provide everybody with knowledge about ò Information agents market opportunities: – on the basis of information needs of their users, they – Improve the position of customers? select relevant sources, and present the information – Reduce differences on prices – Put pressure of smaller sellers in an appropriate way. – Promote different types of competition – They support advanced pull technologies, so allowing one to select and download (via one’s own agent) exactly the ò Agent reduce transaction costs. This can have major impacts: information one needs – In particular, they may look in the internet for the commercial – All contractual clauses can be bargained offers that are more convenient for their users – Defeasible legal rules may become irrelevant – Disposable legal rights can become irrelevant to economic choices (Coase) 11 © G. Sartor 12 © G. Sartor

2 Interface agents Negotiation agents

ò They facilitate the use of software programs and ò The contribute to the making of contracts the interaction with virtual environments – They find possible partners, ò Providing help facilities – They make offers, – They evuate offers ò Acting as virtual guides – They make agreement ò Acting as virtual shop assistants – They execute contract ò Etc. – They control the contractual execution of the counterparty – Etc.

13 © G. Sartor 14 © G. Sartor

Legal issues and agents Legal issues and agents (II) – Can electronic agents be legal persons? What does it mean to be a person? – Can electronic agents be agents (representatives) in the – A computer systems made of agents is (or legal sense (have the power of making contact and should be) a kind of legal system? other legal acts for their users) – How shall we replicate legal institution (e.g. – Can agents have legally relevant cognitive states legislation, courts, police) in the agents’ world (intentions, beliefs, good faith, ecc.)? – What connections between between such a – Can agents have normative attitudes (can they adopt of violate norms)? normative system and the legal system?

15 © G. Sartor 16 © G. Sartor

Other legal issues

ò In the domains of – Contract law – Tort law – Consumers law It is 17:55. Enrico leaves his office, hurries up – Agency law toward is car. It is too late though, Rita a meter – Intellectual property law maid has just placed a fine on the windscreen of ò Let us see an example: the lovely Rita case his car.

17 © G. Sartor 18 © G. Sartor

3 No room for negotiating, but maybe the occurrence is not so adverse as it seemed… RRI, Radio Rock International, is broadcasting music Enrico asks Rita to have a coffee together Enrico has an idea, he remembers of an old Beatles’ song, “Lovely Rita Meter Maid” and wishes to dedicate it to the girl. He calls Jonathan, the DJ, and asks him to play the song.

Who are you, Hermes? (I) Meanwhile, inside RRI studio, Alexandra, the technical assistant fills a ò Hermes is a software agent: form for their software agent, Hermes – implementing a model patented as “Music Middleman” by AECS (Applied E-Commerce Science), a US corporation; – Build through a software copyrighted by ZNS (ZweiNullSieben) a German corporation; HERMES – task description – Licensed to RRI (Radio Rock International), an Italian radio; ò The terms of the AECS-ZNS contract: Author(s) Lennon, McCartney – ZNS is allowed to build systems implementing “Music Middleman”, it is forbidden to make any other use of this Title Lovely Rita Meter Maid technology, it is obliged to pay a fee to AECS, etc....

Performer(s) ò The terms of the ZNS-RRI contract: Beatles – RRI is allowed to use Hermes, empowered to assign him tasks, forbidden to access or modify his code, obliged to pay a monthly fee … – Zns is obliged to maintain Hermes, to provide him with market © G. Sartor 22 © Gi.n Sfaortorrmation …

Who are you, Hermes? (II) Who are you, Hermes? (III)

ò Who owns Hermes? ò Who controls Hermes? – The owner of the technology (the patent holder)? – The patent holder (who decides upon the use of the – The owner of the software (the copyright holder)? technology)? – The owner of a right of use (the licensee and the “sub- licensees”)? – The copyright holder (who can access and modify software)? ò Hermes gets information from: – – ZNS database manager; The licensee and the “sub-licensee” (who issue – Alexandra and RRI staff; assignments and communicate information)? – his own autonomous inquires; – The agent himself (who knows the circumstances – other agents and decides what to do)?

23 © G. Sartor 24 © G. Sartor

4 Who are you, Hermes? (IV) Who are you, Hermes? (V)

ò Who controls Hermes? ò Who is responsible for Hermes (if he harms somebody)? – The patent holder (who decides how to use the technology)? – The patent holder (who designed the technology)? – The copyright holder (who implemented the software and – The copyright holder (who can access and modify provided general knowledge)? software)? – The licensee and the “sub-licensee” (who assigned tasks and – The licensee and the “sub-licensee” (who issue endowed him with specific knowledge)? assignments and transmit information)? – Hermes himself (who evaluated circumstances and took his own decisions)? – The agent himself (who knows the circumstances and decides what to do)? – A devious third party (who manipulated his environment or his internal state)?

25 © G. Sartor 26 © G. Sartor

Hermes’s story (V) Hermes’s story (VI)

ò The statement of Hermes’s task ò To execute his tasks, Hermes: – Specification of the author: John Lennon, Paul McCartney – Consults the register of musical recordings – Specification of the title of the work: "Lovely Rita Meter Maid" – Accesses Zns (his builder) database of on-line music providers – Preferred performing artist: The Beatles – Clones himself and contact simultaneously different providers – Alternative performing artist: Any – Negotiates with the offerors – Availability: 6.17 pm – Evaluates alternative offers against his task descriptions – Price: the lowest possible, below 5 Euros – Decides when an offer is good enough to make a deal – License type: One time only, before 8pm current date, public – Makes a contract performance by broadcast – Executes it

27 © G. Sartor 28 © G. Sartor

H e r m e s d e c i d e s t o p u r c h a s e t h e l i c e n s e f o r H e r m e s p a y s t h e f e e t o A g e n t B a n d t h e l a t t e r b r o a d c a s t i n g L o v e l y Ri t a M e t e r M a i d b y A g e n t s e n d s i t a c o m p r e s s e d e n c r y p t e d v e r s i o n o f t h e B o f T i m e W a r n i n g c o r p o r a t i o n . s o n g

> Payment is made through an electronic micropayment system (e.g., Mondex)

> Handshake is made through the interchange of encrypted time-stamped messages > The song is compressed and encrypted; RRI is given the key for launching it

Agent B public key encrypted msg: date: Feb. 12, 2002 Hermes pays time: 18:15:44 using song: Lovely Rita… price: 2,50 Euro Mondex protocol …. Buying OK

certified certified certified certified Agent A public key key encrypted msg: key key key date: Feb. 12, 2002 Hermes’s public key time: 18:15:50 encrypted file: Hermes song: Lovely Rita… Music selling agent B Hermes “LovelyRita.cmp” Music selling agent B price: 2,50 Euro …. Hermes’ public key Selling OK encrypted key 29 © G. Sartor 30 © G. Sartor for launching the song

5 In t h e m e a n t i m e , H e r m e s c e a s e s t r a n s a c t i o n s w i t h o t h e r s e l l i n g a g e n t . M e a n w h i l e a t RRI, A l e x a n d r a l a u n c h e s t h e f i l e . It i s 1 8 : 1 7 : 4 0 . O u t s i d e t h e b a r n i g h t i s f a l l i n g . T h e s o n g What if Hermes does not inform the other i s i n t h e a i r . “ L o v e l y Ri t a m e t e r m a i d , n o t h i n g c a n agents that he ceases transaction, so that they c o m e b e t w e e n u s … L o v e l y Ri t a m e t e r m a i d , m a y I wait and lose other business opportunities i n q u i r e d i s c r e t e l y , w h e n a r e y o u f r e e , t o t a k e s o m e t e a What if Hermes does not pay the song? w i t h m e ?

Agent A public key encrypted msg: date: Feb. 12, 2002 time: 18:15:47 song: Lovely Rita… price: 6 Euro certified …. certified Buying NO key key Hermes Music selling agent A

31 © G. Sartor © G. Sartor

Hermes’s legal qualification Who “wants” to make the contract?

ò What is Hermes? ‹ The principal (Alexandra). – A messenger, who only transmits an offer from his ‹ She assigned Hermes his tasks and established the principal (RRI and Alexandra) to the offerees constraints, ‹ However, she is not aware of the context and the contents of – A representative, who autonomously determines the the negotiation content of a contract which he will conclude in the ‹ However, she does not know the precise content of the name of his principal (who will acquire rights and contract which is concluded duties ensuing from the contract) ‹ Hermes. – An independent contractor (an agent without representation), who acquires rights and duties on ‹ He autonomously decided when and how to make the deal his own, and later transfers them to his principal. ‹ However he followed her instructions (and his built-in rules) ‹ However he was not “conscious” 33 © G. Sartor 34 © G. Sartor

What is the extent of Hermes’s autonomy? Defects of consent

‹ What if Hermes goes beyond his powers ò Invalid consent doctrines: A contract is invalid when one ‹ As stated in the certificate communicate to the counterparty party has certain states of mind, resulting of certain ‹ As stated by RRI to Hermes itself circumstances – Mistake: a false belief which lead to the stipulation of the ‹ What if Hermes goes against his assignment? contract and was recognisable to counterparty ‹ And the counterparty is/should be aware of that – Violence and menace: an unjust threat which determined the ‹ And the counterparty is not/should not be aware of that stipulation of the contract and was recognisable to counterparty – Misrepresentation (deceit): a false belief induced by the ‹ What if Hermes “extends” his assignment? counterparty which lead to the stipulation of the contract ‹ And he behaves reasonably ò Whose are the relevant states of minds? ‹ And he behaves with evident absurdity/inconsistency – Hermes’s? – Alexandra’s?

6 Can agents have a legal Has Hermes legal personality? personality? (I) ò We need to distinguish various meaning of ò Can he have his own legal rights and duties (which are legal personality: distinguished from the rights and duties of his principal)? 1. An entity is a legal person if is in capable of – Are his interest relevant for the law (directly or indirectly)? acquiring rights and duties – His he capable of looking after himself (so that he can exercise his own rights)? 2. An entity is a legal person if it has an independent – Has he got his own property (which is separated from the property, which unsatisfied creditors can attack property of his owner)? 3. An entity is a legal person if it has the basic set of ò What about the slaves? In Roman law rights and duties (human rights) that each human – They could (if authorised) make contracts in the name of their owners; has or should have – They could have a certain amount of money (peculium), which the third parties could rely upon.

Can agents have a legal Further problems with the legal personality? (II) personality of agents

ò Agents may be legal persons under description ò Giving rights and duties to agents may lead to 1 (when the rights and duties are appropriate to problems, since their nature) – Agents may disappear ò Agents may also be persons under description – They may clone themselves 2, if appropriate institutional arrangements are – They have no physical location (where will their provided creditor find them and sue them?) ò What is their residence ò Agents are NOT legal persons under description ò What is their domicile 3 – Etc.

What solutions (I) What solutions (II)

ò Assign a legal location to agents and to their patrimony ò Form a company, and have agents act for the company. (e.g. in the bank were the money is located): – This would solve the issue of legal personality (no personality – Agents have legal personality (can have their own rights and for agents will be needed: their actions will count as actions of duties, which are guaranteed by their assets) the company) – Agents will identify themselves by using a key which is – The shareholder of the company will be liable only for their associated with their assets shares – The user with be liable within the limits of the money he has – Third parties could ask the company to make transaction in their transferred to the agent’s assets (which will be made known to interest (though their agents) third parties) – Third parties could hire agents which will then act in their name

7 The responsibility for the wrongful behaviour of Hermes Now, back to e-business

ò Is the owner liable since the agent’s behaviour in reality the behaviour of his owner? ò A spy-agent, Mata Hari was delivered along with ò Is the “owner” liable as being the custodian of a thing? the song. Mata has to: ò Is the “owner” liable as being the guardian of the agent? – Allow playing the song (acting as a start-up program) (as if the agent was a child) – check RRI compliance with the terms of the license; ò Is the “owner” liable as being the employer of the agent (vicarious liability)? (but then Hermes must realise his – Delete the recording after playing it own torts) – Bring back to Time Warning a Report. ò Is the “owner” liable as being the principal (mandator) of the agent? ò Is the owner not liable (since only Hermes is only liable on his own for his behaviour) Mata Hari ò …© G . Sartor © G. Sartor 43 44

Agents and privacy What legal constraints?

ò Is Hermes’s privacy legally protected? ‹ Hari – When he contains data concerning his owners? ‹ must inform Alexandra that somebody is hiding in her computer – Also when the data only concerns himself (his plans, preferences, or constraints) ‹ has to limit the scope of her inquiry to the terms of contracts at hand ò What if Hari violates RRI’s privacy? ‹ must not gather information, even related to previous – Does an agent knowledge of other people’s data contracts or other issues (e.g., looking for “cracks”, realise a forbidden “treatment” of personal data? illegal reproductions, relations with other companies, – Who is responsible for the agent’s wrongful etc.) behaviour?

Future issues

ò Develop the various directions of research so far sketched ò Analyse the legal implication of agents’ behavious ò Suggest appropriate legal arrangements (for different business models) ò Find ways of implementing legal values in social relationships mediated by agents ò Transfer legal institutions into the cyberspace.