Intro duction
Remote Pro cedure Calls RPC are a p opu-
lar mo del for building client/server applica-
tions
Overview of Remote Pro cedure
{ ONC RPC and OSF DCE are widely available RPC
to olkits
Calls RPC
RPC forms the basis for many client/server
applications
Douglas C. Schmidt
{ e.g., NFS
Washington University, St. Louis
Distributed object computing DOC frame-
works may be viewed as an extension of
RPC RPC on sterio ds
http://www.cs.wustl.ed u/s chmi dt/
{ e.g., OMG CORBA
RPC falls somewhere b etween the transp ort
layer and application layer in the OSI mo del
{ i.e., it contains elements of session and presenta-
tion layers
1 2
Motivation
IPC Overview
RPC tries to simplify distributed application
rogramming by making distribution trans-
p CLIENT HOST SERVER HOST
parent
RPC to olkits automatically handle CLIENT SERVER
PROCESS PROCESS
{ Reliability
e.g., communication errors and transactions
. REQUEST RESPONSE
{ Platform heterogeneity
e.g., p erforms parameter \marshaling" of com-
. NETWORK
plex data structures and handles byte-ordering
di erences
Many applications require communication among
{ Service lo cation and selection
multiple pro cesses
{ Service activation and handler dispatching
{ Pro cesses may b e remote or lo cal
{ Security
3 4
Message Passing Mo del
Message passing is a general technique for
exchanging information b etween twoormore
pro cesses
Message Passing Mo del cont'd
Basically an extension to the send/recv I/O
In general, message passing do es not make
API
an e ort to hide distribution
{ e.g., UDP, VMTP
{ e.g., network byte order, p ointer linearization, ad-
dressing, and security must b e dealt with explicitly
Supp orts a numb er of di erent communica-
tion styles
This makes the mo del ecient and exible,
{ e.g., request/resp onse, asynchronous oneway, mul-
but also complicate and time consuming
ticast, broadcast, etc.
May serve as the basis for higher-level com-
munication mechanisms such as RPC
6 5
Monolithic Application Structure
Message Passing Design Consideratio ns
FILE
Blo cking vs. nonblo cking SYSTEM FILESYSTEM
API
{ A ects reliablility, resp onsiveness, and program struc-
ture RECALC RECALC SPREAD API
SHEET
Bu ered vs. unbu ered
DATABASE
A ects p erformance and reliability { API DATABASE
GUI
API
Reliable vs. unreliable
GUI
{ A ects p erformance and correctness
7 8
Basic Principles of RPC
RPC Application Structure
1. Use traditional programming style for dis-
application development FILE tributed SYSTEM FILESYSTEM RECALC
API
Enable selective replacement of lo cal pro ce- RPC 2.
GENERATED RPC
calls with remote pro cecure calls
RECALC dure SPREAD API CLIENT GENERATED SHEET STUBS SERVER
STUBS
Lo cal Pro cedure Call LPC DATABASE
API DATABASE
{ A well-known metho d for transferring control
GUI
rt of a pro cess to another API from one pa
GUI
. Implies a subsequent return of control to the
caller
Remote Pro cedure Call RPC
Note, RPC generators automate most of
{ Similar LPC, except a lo cal pro cess invokes a
the work involved in separating client and
pro cedure on a remote system
server functionality
. i.e., control is transferred across pro cesses/hosts
9 10
A Temp oral View of RPC
A Layered View of RPC
CLIENTCLIENT SERVER SERVER KERNEL
USER CLIENT HOST SERVER HOST REQUEST
CLIENT SERVER PROCESS REMOTE PROCESS REMOTE USER SERVICE PROCEDURE PROCEDURE KERNEL NETWORK CLIENT EXECUTES CALL APPLICATION APPLICATIONSERVER BLOCKED CODE CODEPROCESS
STUB CODE STUB CODE RESPONSE RPC RPC RUNTIME RUNTIME LIBRARY LIBRARY
REQUEST RESPONSE
An RPC proto col contains two sides, the
sender and the receiver i.e., client and server
NETWORK
{ However, a server might also b e a client of another
server and so on:::
11 12
Typical Server Startup Behavior
RPC Automation
CLIENT HOST SERVER HOST
To help make distribution transparent, RPC
hides all the network co de in the client stubs
server skeletons and SERVER PROCESS
(1) REGISTER INTERFACE
{ These are usually generated automatically:: : (2) CREATE BINDING INFO (3) ADVERTISE SERVER LOCATION (4) REGISTER ENDPOINTS
(5) LISTEN FOR CALLS
This shields application programs from net-
RPC RUNTIME
working details
LIBRARY
{ e.g.,sockets, parameter marshalling, network byte
order, timeouts, ow control, acknowledgements,
retransmissions, etc. ENDPOINT
MAP
It also takes advantage of recurring com-
NAME SERVICE
muncation patterns in network servers to
CDS HOST
generate most of the stub/skeleton co de
automatically
13 14
Typical Client Startup Behavior
Typical Client/Server Interaction
CLIENT HOST SERVER HOST
CLIENT HOST SERVER HOST CLIENT PROCESS (6) MAKE REMOTE CLIENT SERVER PROCEDURE PROCESS PROCESS (14) EXECUTE APPLICATION CALL REMOTE CODE SERVER APPLICATION PROCEDURE (7) FIND CODE S ERVER PROCESS (10) PREPARE SYSTEM (15) PREPARE (13) CONVERT INPUT STUB (18) CONVERT OUTPUT INPUT STUB OUTPUT (8) FIND
S ERVER (11) TRANSMIT (12) RECEIVE PROCESS INPUT RPC AND RPC RUNTIME (16) TRANSMIT DISPATCH (9) BIND TO RUNTIME (17) RECEIVE TO STUB LIBRARY S ERVER LIBRARY OUTPUT OUTPUT ENDPOINT ENDPOINT MAP MAP
NAME SERVICE NAME CDS SERVICE HOST CDS
HOST
15 16
RPC Mo dels
RPC Mo dels
SYNCHRONOUS RPC
There are several variations on the stan-
CLIENT SERVER
rd RPC \synchronous request/resp onse"
da PROCESS mo del
NOWAIT RPC
CLIENT SERVER
(NOWAIT)
Each mo del provides greater exibility, at
VOID REPLY
the cost of less transparency
CALLBACK RPC
CLIENT (NOWAIT) SERVER
Certain RPC to olkits supp ort all the di er- VOID REPLY INTERNE
WINDOW SYNCHRONOUS CALLBACK RPC mo dels ent LOOP OR T
WAITFOR_RPC
{ e.g., ONC RPC BATCH RPC
CLIENT SUB1
VOID REPLY SERVER NETWORK SUB2
SUB3
Other DOC frameworks do not due to p orta- SUB1 VOID REPLY
ACCESS SUB2
y concerns bilit SUB3
SUB3 RETURN
{ e.g., OMG CORBA and OSF DCE
18 17
RPC Mo dels cont'd
BROADCAST RPC
SERVER
ransparency Issues BROADCAST T CLIENT APPLICATION REQUEST DOESPROCESS NOT REPLY
BROADCAST SERVER RPC has a numb er of limit ati ons that must REQUEST BROADCAST BROADCAST HOST-TO-
COLLECTION REPLY
be understo o d to use the mo del e ectively ROUTINE HOST BROADCAST BROADCAST REPLY REQUEST
SERVER
{ Most of the limitations center around transparency INTERNET NETWORK
THREADED RPC
SERVER
Transforming a simple lo cal pro cedure call APPLICATION
CLIENT PROCESS
into system calls, data conversions, and net-
work communications increases the chance
SERVER
something going wrong (SYNCHRONOUS RPC) HOST-TO- of
HOST
i.e., it reduces the transparency of distribution SERVER { INTERNET
NETWORK
20 19
Tranparency Issues cont'd
Parameter Passing
Key Asp ects of RPC Transparency
Functions in an application that runs in a
1. Parameter passing single pro cess may collab orate via parame-
ters and/or global variables
2. Data representation
3. Binding
Functions in an application that runs in mul-
tiple pro cesses on the same host may col-
4. Transp ort proto col
lab orate via message passing and/or non-
distributed shared memory
5. Exception handling
6. Call semantics
However, passing parameters is typically the
only way that RPC-based clients and servers
7. Security
share information
{ Hence, we have already given up one typ e of transparency:: :
8. Performance
21 22
Parameter Passing cont'd
Passing parameters across pro cess/host b ound-
Parameter Passing cont'd
aries is surprisingly tricky:::
Typical solutions include:
Parameters that are passed by value are fairly
simple to handle
{ Have the RPC proto col only allow the client to
pass arguments by value
{ The client stub copies the value from the client
and packages into a network message
. However, this reduces transparency even further!
{ Presentation issues are still imp ortant, however
{ Use a presentation data format where the user
sp eci cally de nes what the input arguments are
and what the return values are
Parameters passed by reference are much
. e.g., Sun's XDR routines
harder
{ e.g., in C when the address of a variable is passed
{ RPC facilities typically provide an \interface de -
nition language" to handle this
. e.g., passing arrays
. e.g., CORBA or DCE IDL
{ Or more generally, handling p ointer-based data
structures
. e.g., p ointers, lists, trees, stacks, graphs, etc.
24 23
Data Representatio n
RPC systems intended for heterogeneous
Data Representation cont'd
environments must b e sensitive to byte-ordering
di erences
Examples cont'd
{ They typically provide to ols for automatically p er-
forming data conversion e.g., rpcgen or idl
{ DCE RPC NDR
. Supp orts multiple presentation layer formats
Examples:
. Supp orts \receiver makes it right" semantics:: :
{ Sun RPC XDR
Allows the sender to use its own internal for-
mat, if it is supp orted
. Imp oses \canonical" big-endian byte-ordering
. The receiver then converts this to the appropri-
. Minimum size of any eld is 32 bits
ate format, if di erent from the sender's format
{ Xerox Courier
This is more ecient than \canonical" big-
endian format for little-endian machines
. Uses big-endian
. Minimum size of any eld is 16 bits
25 26
Binding
Binding cont'd
Binding is the pro cess of mapping a request
There are two comp onents to binding:
for a service onto a physical server some-
where in the network
1. Finding a remote host for a desired service
{ Typically, the client contacts an appropriate name
2. Finding the correct service on the host
server or \lo cation broker" that informs it which
remote server contains the service
{ i.e., lo cating the \pro cess" on a given host that
is listening to a well-known p ort
. Similar to calling 411:::
There are several techniques that clients use
If service migration is supp orted, it may be
to lo cate a host that provides a given typ e
necessary to p erform this op eration multiple
of service
times
{ These techniques di er in terms of their p erfor-
{ Also may b e necessary to leave a \forwarding" ad-
mance, transparency, accuracy, and robustness
dress
28 27
Binding cont'd
Binding cont'd
\Hard-co de" magic numb ers into programs
ugh:: : ;-
Sup erserver: inetd and listen
{ Motivation
Another technique is to hard-co de this in-
formation into a text le on the lo cal host
. Originally, system daemon pro cesses ran as sep-
{ e.g., /etc/services
arate pro cesses that started when the system
was b o oted
{ Obviously, this is not particularly scalable:: :
. However, this increases the numb er of pro cesses
on the machine, most of which are idle much of
Another technique requires the client to name
the time
the host they want to contact
{ Solution ! sup erserver
{ This host then provides a \sup erserver" that knows
the p ort numb er of any services that are available
. Instead of having multiple daemon pro cesses asleep
on that host
waiting for communication, inetd or listen lis-
tens on b ehalf of all of them and dynamically
{ Some example sup er servers are:
starts the appropriate one \on demand"
. inetd and listen -- ID byport numb er
i.e., up on receipt of a service request
. tcpmux -- ID by name e.g., "ftp"
29 30
Binding cont'd
Binding cont'd
Sup erservers cont'd
Lo cation brokers and traders
{ This reduces total numb er of system pro cesses
{ These more general techniques maintain a dis-
tributed database of \service ! server" mappings
{ It also simpli es writing of servers, since many
start-up details are handled by inetd
{ Servers on any host in the network register their
willingness to accept RPCs by sending a sp ecial
. e.g., socket, bind, listen, accept
registration message to a mapping authority, e.g.,
portmapper -- ID by PROGRAM/VERSION numb er
{ See /etc/inetd.conf for details:: :
orbixd -- ID by \interface"
{ Clients contact the mapping authority to lo cate a
{ Note that these sup er servers combine several ac-
particular service
tivities
. Note, one extra level of indirection:: :
. e.g., binding and execution
31 32
Transp ort Proto col
Binding cont'd
Some RPC implementations use only a sin-
gle transp ort layer proto col
{ Others allow proto col section either implicitly or
Lo cation brokers and traders
explicitly
{ A lo cation broker manages a hierarchy consisting
of pairs of names and object references
Some examples:
. The desired object reference can b e found if its
name is known
{ Sun RPC
. Earlier versions supp ort only UDP, TCP
{ A trader service can lo cate a suitable object given
a set of attributes for the object
. Recent versions are \transp ort indep endent"
. e.g., supp orted interface s , average load and
resp onse times, or p ermissions and privileges
{ DCE RPC
. Runs over many, many proto col stacks
{ The lo cation of a broker or trader may b e set via
a system administratoror determined via a name
. And other mechanisms that aren't stacks
server discovery proto col
e.g., shared memory
. e.g., may use broadcast or multicast to lo cate
name server:::
{ Xerox Courier
. SPP
33 34
Exception Handling
Transp ort Proto col cont'd
With a lo cal pro cedure call there are a lim-
When a connectionless proto col is used, the
ited numb er of things that can go wrong,
client and server stubs must explicitl y han-
b oth with the call/return sequence and with
dle the following:
the op erations
1. Lost packet detection e.g., via timeouts
{ e.g., invalid memory reference, divide by zero, etc.
2. Retransmissions
With RPC, the p ossibility of something go-
3. Duplicate detection
ing wrong increases, e.g.,
1. The actual remote server pro cedure itself generate
an error
This makes it dicult to ensure certain RPC
2. The client stub or server stub can encounter net-
reliabili ty semantic guarantees
work problems or machine crashes
A connection-oriented proto col handles some
Two typ es of error co des are necessary to
of these issues for the RPC library, but the
handle two typ es of problems
overhead may b e higher when a connection-
oriented proto col is used
1. Communication infrastructure failures
{ e.g., due to the connection establishment and ter-
mination overhead
2. Service failures
35 36
Exception Handling cont'd
Exception Handling cont'd
Both clients and servers may fail indep en-
dently
DCE and CORBA de ne a set of standard
{ If the client pro cess terminates after invoking a
remote pro cedure but b efore obtaining its result,
\communication infrastructure errors"
the server reply is termed an orphan
For C++ mappings, these errors are often
Imp ortant question: \how do es the server
translated into C++ exceptions
indicate the problems back to the client?"
In addition, DCE provides a set of C macros
for use with programs that don't supp ort
Another exception condition is a request by
exception handling
the client to stop the server during a com-
putation
37 38
Call Semantics cont'd
When an RPC can b e executed any numb er
of times, with no harm done, it is said to
Call Semantics
be idemp otent.
{ i.e., there are no harmful side-e ects:::
When a lo cal pro cedure is called, there is
{ Some examples of idemp otent RPCs are:
never any question as to how many times
the pro cedure executed
. Returning time of day
. Calculating square ro ot
With a remote pro cedure, however, if you
do not get a resp onse after a certain inter-
val, clients may not know how many times
. Reading the rst 512 bytes of a disk le
the remote pro cedure was executed
. Returning the current balance of a bank account
{ i.e., this dep ends on the \call semantics"
{ Some non-idemp otent RPCs include:
{ Of course, whether this is a problem or not is
\application-de ned"
. Apro cedure to app end 512 bytes to the end of
a le
. Apro cedure to subtract an amount from a bank
account
39 40
Call Semantics cont'd
There are three di erent forms of RPC call
semantics:
1. Exactly once same as lo cal IPC
Call Semantics cont'd
{ Hard/imp ossible to achieve, b ecause of server
crashes or network failures :: :
2. At most once
Handling non-idemp otent services typically
{ If normal return to caller o ccurs, the remote pro-
requires the server to maintain state
cedure was executed one time
{ If an error return is made, it is uncertain if re-
However, this leads to several additional com-
mote pro cedure was executed one time or not
plexities:
at all
1. When is it acceptable to relinquish the state?
3. At least once
{ Typical for idemp otent pro cedures, client stub
2. What happ ens if crashes o ccur?
keeps retransmitting its request until a valid re-
sp onse arrives
{ If client must send its request more than once,
there is a p ossibility that the remote pro cedure
was executed more than once
. Unless resp onse is cached:::
41 42
Security
Call Semantics cont'd
Typically, applications making lo cal pro ce-
dure calls do not have to worry ab out main-
taining the integrityor security of the caller/callee
Note that if a connectionless transp ort pro-
to col is used then achieving \at most once"
{ i.e., calls are typically made in the same address
semantics b ecomes more complicated
space
{ The RPC framework must use sequence numb ers
. Note that shared libraries may complicate this:: :
and cache resp onses to ensure that duplicate re-
quests aren't executed multiple times
Lo cal security is usually handled via access
Note that accurate distributed timestamps
control or sp ecial pro cess privileges
are useful for reducing the amount of state
that a server must cache in order to detect
Remote security is handled via distributed
duplicates
authentication proto cols
{ e.g., Kerb eros:::
43 44
Performance cont'd
Performance
RPC also tends to b e much slower than us-
ing lower-level remote IPC facilities such as
Usually the p erformance loss from using RPC
so ckets directly due to overhead from
is an order of magnitude ormore, compared
with making a lo cal pro cedure call due to
1. Presentation conversion
1. Proto col pro cessing
2. Data copying
2. Context switching
3. Flow control
3. Data copying
{ e.g., stop-and-wait, synchronous client call be-
havior
4. Network latency
4. Timer management
5. Congestion
{ Non-adaptive consequence of LAN upbringing
Note, these sources of overhead are ubiqui-
Note, these sources of overhead are typical
tous to networking:::
of RPC:::
45 46
Performance cont'd
Performance cont'd
Another imp ortant asp ect of p erformance is
how the server handles multiple simultane-
ous requests from clients
In many situation, a concurrent RPC server
should be used:
{ An iterative RPC server p erforms the following
functionality:
lo op f
wait for RPC request;
lo op f
receive RPC request;
wait for RPC request;
deco de arguments;
receive RPC request;
spawn a pro cess or thread f
deco de arguments;
execute desired function;
execute desired function;
reply result to client;
reply result to client;
g
g
{ Thus the RPC server cannot accept new RPC re-
g quests while executing the function for the previ-
ous request
. This is undesirable if the execution of the func-
Threading is often preferred since it requires
tion takes a long time
less resources to execute eciently
e.g., clients will time out and retransmit, in-
creasing network and host load
48 47
Performance cont'd
Servers are often the b ottleneck in distributed
Performance cont'd
communication
However, the primary justi cation for RPC
is not just replacing lo cal pro cedure calls
Therefore, another p erformance consider-
ation is the technique used to invoke the
server every time a client request arrives,
{ i.e., it is a metho d for simplifying the development
e.g.,
of distributed applications
{ Iterative -- server handles in the same pro cess
In addition, using distribution may provide
. May reduce throughput and increase latency
higher-level improvements in:
{ Concurrent -- server forks a new pro cess or thread
1. Performance
to handle each request
2. Functionality
. May require subtle synchronization, programming,
and debugging techniques to work successfully
3. Reliability
Thread solutions may b e non-p ortable
. Note also that multi-threading removes the need
for synchronous client b ehavior:: :
49 50
Summary
RPC is one of several mo dels for implement-
ing distributed communication
{ It is particular useful for transparently supp orting
request/resp onse-style applications
{ However, it is not appropriate for all applications
due to its p erformance overhead and lack of exi-
bility
Before deciding on a particular communica-
tion mo del it is crucial to carefully analyze
the distributed requirements of the applica-
tions involved
{ Particularly the tradeo of securityfor p erformance:: : 51