Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!cornell!rcbc From: rcbc@honir.cs.cornell.edu (Robert Cooper) Newsgroups: comp.sys.isis Subject: "Object oriented" remote interfaces in ISIS Summary: Viewing an ISIS group as a module implementing an interface. Message-ID: <28783@cornell.UUCP> Date: 10 Jun 89 15:38:41 GMT References: <3358@cps3xx.UUCP> <28754@cornell.UUCP> Sender: nobody@cornell.UUCP Organization: Cornell Univ. CS Dept. Ithaca NY Lines: 98 In-reply-to: ken@gvax.cs.cornell.edu's message of 10 Jun 89 01:09:55 GMT In article 22 or comp.sys.isis Ken Birman has started a discussion of "object oriented" remote interfaces in ISIS. I'll elaborate on some of our current plans to support the notion of a group as a "module" which implements a well-defined "interface". This could provide a basis for a more "object oriented" ISIS programming style, but I'll leave that subject alone for now. The problem: ------------ ISIS provides broadcast and other primitives with well-defined behaviour. But when using these primitives the programmer still has many choices to make. For instance the initiator of a broadcast must know whether to wait for replies from ALL group members, a MAJORITY of members, or wait for just 1 reply. The initiator must know whether to use ABCAST, which provides a totally ordered, atomic broadcast, or CBCAST which has weaker semantics but is more efficient. The receivers of the broadcast must know whether to send a nullreply or an abortreply to message that are not of interest to it. One way to look at this issue is to say that an ISIS program implements some application level protocol on top of the ISIS primitives. These issues become problematical when doing programming-in-the-large. The different "modules" in your application must have well-defined interfaces so that diverse clients know exactly what the interface provides and how to correctly use the interface. Clearly aspects such as ALL versus MAJORITY, and the flavour of broadcast algorithm used are crucial to the overall correctness of the resulting system, and ought to be expressed in the interface. Another issue is managing software upgrades. In a reliable distributed application the lifetime of an "execution" of the application (i.e. the time between total simultaneous failure of all system components) can be longer the the time between modifications to the source code for that system. If we are to produce truly reliable distributed applications we must be able to change the implementation of its components while the system continues to operate. And we must be able to detect when components with incompatible versions of an interface try to communicate. Group and entry attributes: --------------------------- We plan to address this problem by providing viewing a process group as a "module" which implements some "interface" and which uses some "protocol" (defined in terms of ISIS broadcast primitives) for remote communication. We plan to associate the following "attributes" with a group. The first two attributes already exist but the third is new: o size The number of member processes. o incarnation A number incremented each time the group recovers after a total failure. o entries The set of messages, or entry points, that every member of the group will accept. Associated with each entry are the following attributes: o entry number This identifies the function to be performed by the service, and is the equivalent of the procedure number in many RPC interfaces. Entry numbers range from 1 to 100, and must be unique within a particular group. o ordering A set chosen from TOTAL, ATOMIC, CAUSAL, or FIFO, corresponding to GBCAST, ABCAST, CBCAST, and FBCAST respectively, specifying the permissible broadcast methods that a client of this group should use. o replies Specifies the number of successful replies to be required from a broadcast. One of ALL, MAJORITY, ONE, or NONE. o format This specifies the number and types of data objects in the request message and any reply message. o version A version number associated with this entry, which is incremented whenever some attribute of this entry changes. When an interface changes we invisage that group members will support both the old and new versions of the interface while a management and configuration tool incrementally crashes and restarts existing clients until no obsolete clients exist. Currently much of this kind of information about groups and entries is encoded in the implementation of group members or in the client application. By making these attributes explicit we can more easily ensure that clients and group members are behaving compatibly with respect to a group interface. The benefits for program correctness are clear. A group member would specify these attributes when it joined a group. If its set of attributes did not match those of the existing group members an error would occur. On the client side, information about group attributes would be used to automatically select the appropriate broadcast protocol. Thus we free the application programmer from continually reimplementing common communication patterns. And this extra information may provide opportunites for the ISIS system to optimize some kinds of operations. We can imagine other candidates for group and entry attributes. For instance, whether a group uses logging to survive total failure (i.e. simultaneous failure of all its members), or whether an entry is idempotent, making retries upon failure easy. Since multiple groups may accept the same entry list, perhaps we ought to define a notion of a "group type", to capture this. -- Robert Cooper (rcbc@cs.cornell.edu)