A Distributed Event Messaging System for Mobile Agent Communication

Communication

John McCormick, Daria Chacón, Susan McGrath, and Craig Stoneking

Lockheed Martin Advanced Technology Laboratories

1 Federal Street A&E 3WCamden, NJ 08102

{jmccormi, dchacon, smcgrath, cstoneki}@atl.lmco.com

Abstract: Agent mobility presents distinct challenges not handled bytraditional, distributed, communication frameworks. By nature, mobile agentschange location over time, making targeted messaging difficult to achieve.Also, a communication infrastructure must handle the case that the agent maybe migrating when the message is sent. We have implemented a DistributedEvent Messaging System (DEMS) that satisfies the communicationrequirements unique to mobile agents. This infrastructure has enabled efficientcontrol and monitoring of our mobile agents and has facilitated mobile agentcollaboration and coordination.

1 Introduction

Agent mobility presents distinct challenges to communication frameworks. A mobileagent poses new requirements, such as message delivery to mobile recipients, thatdistributed object systems cannot currently address. Traditional, distributed object,communication systems, such as those provided by Common Object Request BrokerArchitecture (CORBA), Distributed Component Object Model (DCOM), and RemoteMethod Invocation (RMI), provide a satisfactory infrastructure for stationary agentsystems, by using name directories and static binding to locate and communicate withagents. In mobile agent systems, these location mappings must change over time, andthe difficulty of providing a reference to a mobile object is aggravated when both thesender and receiver are mobile. Race conditions threaten communication integritywhen the recipient is migrating. Mobility invalidates the static binding method usedby the previously mentioned architectures. A general purpose framework shouldsatisfy these requirements for asynchronous and synchronous communication.Lockheed Martin Advanced Technology Laboratories (LM ATL) has developed aDistributed Event Messaging System (DEMS) that provides a lightweight,communication infrastructure for our Java-based Extendable Mobile AgentArchitecture (EMAA) [1]. DEMS meets all the communication challenges presentedby mobile agents in general and sentinel agent behavior in particular.

Mobile systems require such a robust communication delivery framework tocontrol and monitor dispatched agents. While a mobile agent exhibits a high level of

autonomy, it should be able to receive and respond to commands from a controllingauthority. A typical controlling authority would be the system, user, or agent thatinstantiated it. The agent should also be able to send messages regarding its state backto an interested listener, especially for logging and debugging purposes. Much of ourrecent work with EMAA has concentrated on sentinel agent behavior, which wedefine as agents persistently monitoring an environment for user-defined criterion andalerting all relevant parties when that criterion has been met. Achieving sentinelbehavior frequently requires multiple collaborating agents, with subordinate childagents reporting status information back to a managing agent or system.

Most of the related work that we review in Section 2 uses remote methodinvocation techniques for communication. The DEMS approach leverages Java’sevent model to provide the flexibility required by mobile agent systems. DEMSassumes full responsibility for message distribution, relieving the message sourcethread from this activity, and delivers the messages in a very efficient manner. Wedetail the technical approach to building DEMS in Section 3 and demonstrate how itenables mobile agent control and monitoring in Section 4. We also provideperformance metrics in Section 5 and lay out our road map for future enhancements toDEMS in Section 6.

2 Related Work

Investigating related work in agent communication techniques yields a wealth ofinterest in representation of the communicated content. However, the deliverymechanisms for the communication often seem to be taken for granted or presumethat the agents are static. Solutions to the message distribution problem for mobileagents are being pursued on several fronts:

• Open agent and distributed system architecture specifications.• Commercial and academic agent system frameworks.• Government-funded agent infrastructure research.

Two of the most significant open specifications addressing mobile agentcommunication come from the Foundation for Intelligent Physical Agents (FIPA) andthe Object Management Group (OMG). Of the major agent systems supporting agentmobility, the two most prevalent approaches to communication distribution are use ofa proprietary message routing architecture and extending a distributed object system,such as CORBA. We will compare these approaches as implemented by JATLite(proprietary routing) and Jumping Beans (Mobile CORBA). Finally we will reviewthe Defense Advanced Research Project Agency’s (DARPA) major agent framework,Control of Agent Based Systems (CoABS) GRID, with respect to agentcommunication infrastructure.

The FIPA draft specification for Agent Management [2] defines an AgentCommunication Channel (ACC) that is responsible for routing messages amongagents within the platform and to agents resident on other platforms. Twocommunication options are specified for agent-to-agent communication:

•Agents can request their local ACC to route messages to target agents and

ACC.

•Agents can contact the ACC of target platforms directly responsible forrouting messages to target agents.

ACC support for agent mobility is only optional at this stage of the specification, andthe specification does not address the means of delivery.

OMG has published a Mobile Agent Facility (MAF) specification [3] as a CORBAfacility. The objective of the specification is to promote a standard interface to diversemobile agent architectures. The specific issues addressed are agent management,tracking, and transport. Interestingly, agent communication is declared outside thescope of the MAF specification due to CORBA's extensive coverage of objectcommunication. This implies that CORBA's communication framework is thought tobe sufficient to support mobile agent communication, but the specification laterclearly acknowledges that current distributed objects systems do not meet thecommunication requirements of mobile agents. Both the MAF and FIPAspecifications focus exclusively on unicast or one-to-one communication. The MAFspecification is constrained by the CORBA communication framework, while theFIPA specification appears more flexible; Multicast messaging could be an optionalfeature of the ACC. One important objective of both specifications not currentlysupported by DEMS is agent platform interoperability, and we address this issue inSection 6.

Jumping Beans™ is a Java, mobile, application library developed by Ad AstraEngineering [4]. Their architecture is heavily client/server-oriented and uses acommunication infrastructure named Mobile CORBA. All messaging is routedthrough a central server that maintains dynamic bindings to the mobile agents. Thiscontrasts with DEMS peer-to-peer approach of enabling each host to deliver messagesto any other host in the system. What is unclear from Ad Astra’s white paper is therelationship, if any, between their Mobile CORBA implementation and OMG's MAF.Again the emphasis is placed on agent-to-agent communication, while DEMSsupports a Multicast distribution mechanism in which the event source is unaware ofthe total number of listeners or their locations. JATLite, an agent frameworkdeveloped at Stanford University, uses a specially developed router application formessage delivery [5]. The router maintains a file system depository of messages thatagents or other components can access as connectivity permits. While the typicalimplementation is a centralized message depository, the framework supports multiplerouters, each with its own depository.

Another major initiative in agent systems is the CoABS GRID being developed aspart of DARPA's CoABS program. This infrastructure for heterogeneous, agentsystem collaboration is built on Jini. Its design standardizes on the FIPA-ACLspecification for its Agent Communication Language, using a registry service toimplement the ACC functionality. The registry is currently only capable offorwarding messages to static registered agents. Techniques for supporting messagedelivery to mobile agents are being proposed as part of a new mobility service for theGRID.

3 Technical Approach

DEMS provides an infrastructure for the creation and distribution of event messagesamong distributed objects that may be mobile. Farley's text, Java DistributedComputing [6], presents a number of designs for message passing systems fordelivering information to agents in a distributed system. The message passing withJava events best matched the behavior needed by DEMS. DEMS expands on Farley’sdesign by incorporating multiple distribution modes, event caching for mobilelisteners, and pipelined event delivery.

DEMS is modeled after the Java event model, but the event listeners are handleddifferently. The Java event model is built on three main classes: event objects, eventsources, and event listeners. Event sources create event objects that are then handledby the event listeners registered with it. Each event listener registers with an eventsource to handle specific types of events. In the traditional Java event model, theevent source maintains internally a list of registered listener objects. Upon eventfiring, each listener is notified directly by the event source. This model breaks downin two critical ways in a distributed system with mobile objects:

•The event source may be on a separate system from the listener.

•The event source and event listener may be mobile, making registrationrendezvous difficult to coordinate.

Farley provides an Event Transceiver in his text that handles the first problem. DEMSexpands on Farley's design with an EventTransceiverServer that mitigates bothproblems. One significant advantage of extending from the base Java event modelwas the ease of integrating the distributed system with Java Swing or AWT interfacesthat were based on the Java event model.

Farley also presents a message passing system built on RMI. This system has theslight advantage of easy extensibility, but it was inappropriate for our needs for anumber of reasons:

•RMI presumes non-mobile objects in the distributed system.

•EMAA is not RMI-based, so there is no guarantee of a running RMIregistry.

•The RMI solution requires that the source of the message knows all of thelisteners.

•Multi-threading of delivery would have to be implemented at the eventsource.

The current implementation of RMI assigns a new, unique identifier to an object if ittransfers to a new system. This is the source of the CoABS GRID limitation to staticagents. The third and fourth issues are related to the communication model used byRMI and CORBA. If the event source object is responsible for invoking the listenermethods, then the source object must know all of the listeners and its thread will bededicated to the communication activity until complete—unless special multi-threaded communication logic is added. By using the EventTransceiverServer, DEMSallows delegation of the communication activity and built-in, multi-threaded delivery,event pipelining, which improves average delivery time.

The EventTranceiverServer, the primary class in the DEMS architecture, has tworesponsibilities (Figure 1):

•Maintain registration tables of listeners located on the local host and theevents to be delivered to them.

•Deliver incoming events to the appropriate local listeners.

Listeners may register to receive events from a specific DistributedEventSource or byDistributedEvent class type. Registering by source is accomplished by referencing theUniqueID of the DistributedEventSource. The EventTranscieverServer maintains thelistener registration in three internal Hashtables: source registration, type registration,and unicast registration. If a DistributedEventListener needs to migrate to anothersystem, then it must first unregister with the local EventTransceiverServer. The actualevent transmission and receipt functionality is inherited from the TransceiverServer,which uses transmission agents to perform the remote delivery operation. The use of atransmission agent enables multi-threaded event delivery, which we refer to as eventpipelining.

Dock11EventTranceiverServer1UniquelyIdentifiable<>TransceiverServer<>{ordered}**1..*DistributedEventListener<>DistributedEventSource<>11EventObject*DistributedEvent1UniqueIDFig. 1. DEMS Conceptual Class Diagram Builds on Java’s Event Mode

The DistributedEventListener interface specifies a single method required byimplementing classes, handleDistributedEvent. This is the method called by theEventTransceiverServer when delivering a DistributedEvent to theDistributedEventListener (Figure 2). Adapter classes for a listener interface arefrequently provided to facilitate the demultiplexing of event to specific listenermethod. Examples of this will be shown in Section 4.

The DistributedEventSource interface must be implemented by all classes desiringto transmit DistributedEvents via the EventTransceiverServer. The only methodrequired is getUniqueID, which should return the UniqueID of the source object. ADistributedEventSourceAdapter is provided as a convenience to enable existing classto become sources and to enable an object to have multiple identities with respect to

TransmissionAgentArrivalServer ActivatedRegistration ChangeTransmissionAgent ThreadNew EventReceive EventDelivery ThreadListener ThreadAdd/RemoveListener*for each remote hostTransmitAccess LocalListeners*For each listenerTrigger EventHandlerFig. 2. EventTransceiverServer Activity Diagram Shows Built-in, Multi-threaded Delivery

the events it generates. An example of this is an agent that uses one adapter to issuecommands to child agents and another to report its state events.

The DistributedEvent extends Java’s standard EventObject to meet the needs ofDEMS. It maintains the UniqueID and Class of the DistributedEventSource thatcreated it. Additionally, each DistributedEvent has its own UniqueID and a sequencenumber to help ensure proper delivery order and avoid delivery duplication. Onecritical requirement of classes extending DistributedEvent is that all the classmembers must be serializable or defined as transient. The EventTranceiverServersupports two transmission modes: Multicast and Unicast. Multicast is the defaulttransmission mode, and its rules follow:

•Local delivery to all listeners registered to event source.•Local delivery to all listeners registered to event class.•Event transmitted to all docks in the host list.

•Delivery at remote docks follows local delivery rules, but the event is notpropagated.

Multicast events do not guarantee delivery to any particular listener. Currently theEventTransceiverServer broadcasts the event to all docks in its host list. Its parentclass, TransceiverServer, supports limiting the broadcast to a subset of docks, and thiscapability could be added to the EventTransceiverServer at a future date with littleeffort. This would allow better control of network bandwidth use.

The Unicast transmission mode allows targeting a message to a specific listener.The delivery of a UnicastEvent is guaranteed. The transmission rules forUnicastEvents follow:

•Check for target listener locally.

•Event delivered if listener registered to receive UnicastEvent from source.

•If not local, event transmitted to all docks.•Delivery at remote docks follows same rules.

Although required to register with the local EventTransceiverServer upon arrivaland unregister before departure, events transmitted during the migration process willbe undelivered. This behavior is unacceptable for control events, which must haveguaranteed delivery. A DistributedEvent caching mechanism was developed toguarantee delivery of UnicastEvents (Figure 3).

Machine ASending ETSAgentTransmission AgentReceiving ETSMachine BcheckInterruptswait oncachingAcknowledgedend wait oncachingAcknowledgedrequestRemoteCachingsendCachingRequestreceiveObjectgetAcknowledgereceiveAcknowledgeunregisterListenersmigrateInitialize (on Machine B)THREADBLOCKregister listenersremove forwardingmergeput in cachexFig. 3. Event Caching Sequence Diagram Shows Guaranteed UnicastEvent Delivery

Before the agent unregisters and migrates, it invokes the caching mechanism on thelocal EventTransceiverServer, passing the InetAddress of its migration destination.The local EventTransceiverServer sends a message via a TransmissionAgent to thedestination EventTransceiverServer to begin caching UnicastEvents for the agent.Only after acknowledgement that caching has been established is the agent permittedto continue the migration process. Upon its arrival to the new host, the agent registerswith the EventTransceiverServer. Any UnicastEvents for the agent that had beencached are delivered at that time and the caching is terminated. TheEventTransceiverServer uses synchronized methods when listeners are registered,forwarding is removed, and cached events are being ordered and merged with eventscurrently ready to be delivered. This prevents the possibility of duplicate events beingdelivered due to the event receipt while the cache is in an inconsistent state.

4 DEMS Application: Agent Monitoring and Control

DEMS may be used to monitor agents’ progress and/or to control their execution. Abasic set of events, event listeners, and adapters are included in a child package tosupport this functionality. The AgentEvent class defines the root agent monitoringevent. It conveys that an agent has started, stopped, finished, failed, departed, or

arrived. It can also be used to acknowledge a control event or show that lower-levelcheckpoints have been met. This class is extensible, so an agent application maydefine additional notification reasons. The AgentEventListener interface defines atemplate listener that the EventTranscieverServer calls when delivering the event. Aclass wishing to receive an AgentEvent may implement the AgentEvent Listener; itcould alternately extend or contain an instance of the AgentEventAdapter class, whichprovides a default listener implementation. The AgentEvent is not a Unicast event; itis always transmitted to all EventTransceiverServers, so any listener registered forevents from the source will receive all of its AgentEvents. Additionally, anAgentEventListener can register to receive all AgentEvents from all sources. This isparticularly useful for enabling overall system monitoring. An example of this isshown in Figure 4.

RunningPausedStoppedFinishedUnprocessedFig. 4. GUI Listeners can Provide Visual Agent Itinerary Monitoring

The CommandEvent class defines the root agent control event (Figure 5). It specifiesthat all command events are unicast; therefore, the only agent that receives acommand event is the agent to which it is directed. Three types of control events havebeen predefined, and they all inherit from the CommandEvent class:AgentControlEvent, VocalControlEvent, and AgentRetaskEvent.

UnicastEventCommandEventAgentControlEventVocalControlEventAgentRetaskEventFig. 5. CommandEvent Class Diagram

The AgentControlEvent may be used to stop, pause, suspend, or resume an agent’sexecution. The VocalControlEvent may be used to turn vocality on and off for aVocalAgent or any agent descended from it. The AgentRetaskEvent can be used tochange the parameters of an agent’s task. Default listeners and adapters have beenpredefined to handle all of these events as well. An agent application may either usethe predefined events and listeners or adapters or it may inherit from the existing onesto define application-specific control events.

5 System Performance

We selected the average message distribution time as an appropriate benchmark forevaluating performance of a communication framework. The distribution time isdefined as the elapsed time from when the message is generated to when the listenerreceives the message. With respect to DEMS, the time of generation is when the eventis submitted to the EventTransceiverServer by the event source. For our preliminaryanalysis of DEMS, we established an objective of determining the efficiency of theEventTransceiverServer delivery without concurrency. We set up the experiment withthe following parameters:•Number of listeners.

•Event generation frequency.•Event handler processing time.

The number of listeners varied from 1 to 100 and, at each run, all of the listenersused the same processing. This experiment used a single event source whose eventgeneration rate was slow enough to allow all the listeners to have processed a givenevent before the next was generated. Because we are sure there will not be concurrentevent delivery, we can compute the theoretical best average delivery time. Figure 6shows the typical results for the experiment case.

60005000400030002000100001510# Listeners5010010.990.980.970.960.950.940.930.92510# Listeners50100% EfficiencyFig. 7. High Efficiency Distribution by EventTransceiverServer, Event Handling Time =

100ms

An important property of the DEMS architecture is the parallel delivery to otherhosts. If a source has 10 total listeners divided equally across two machines, theaverage distribution time will be much closer to 200ms than 450ms. This contrastswith the typical, remote, method invocation technique, where the listeners areprocessed sequentially even if delivery is delegated to a new thread.

6 Conclusions and Future Enhancements

DEMS was very successful in providing the agent control and monitoring behaviorrequired for sentinel agent behavior. As part of our research effort, we developedseveral demonstration applications that allowed a user to launch an agent and monitorits state and migration progress as it performed its itinerary. Users were also able tosend command messages to the agents and receive confirmation of their execution.One of the important additional benefits was a alternate method of informationdelivery to another component in the agent system. Prior to DEMS, an agent needingto deliver information back to a user or system had two options:•Migrate back to the system and deliver the information.•Spawn a child agent to deliver the information.

DEMS effectively provides a standardized way of implementing the second option.Using agents as the delivery mechanism enabled the event pipeline behavior of theEventTransceiverServer. Most importantly we were able to achieve an effective,mobile agent, communication framework that was much more lightweight than RMIor CORBA solutions.

DEMS is still a work-in-progress and we are pursuing several enhancements.While it currently suits the needs of EMAA, the EventTransceiverServer does notfacilitate interoperability with other agent platforms. We are investigating methods forextending the EventTransceiverServer to be aware of the CoABS GRID.

Additionally, we intend to continue our performance analysis of the DEMS. We arequite pleased with the initial results and are ready to focus on the concurrent eventpipelining performance. The first goal is to determine how to calculate a theoreticalbest case for the experiments. We anticipated using the results of the existing andfuture experiments to identify potential optimizations for the infrastructure.

7 Acknowledgements

We wish to thank Julius Etzl, Martin Hofmann, and Ken Whitebread for their aid indefining requirements for monitoring and control within agent systems. In addition,we are also grateful to Russ Lentini for thoroughly testing and validating ourDistributed Event Messaging System, and to Virginia Garrison, who developedreusable graphical user interfaces utilizing DEMS for controlling and monitoringEMAA agents. Finally, we are grateful for the developers at ATL who acted as betatesters by using DEMS within their projects and supplied many useful suggestions.

References

1.Lentini,, R., Rao, G., Thies, J., and Kay, J.: EMAA: An Extendable Mobile Agent

Architecture. (1997)

2.FIPA. FIPA ’98 Draft Specification: Part 1 Agent Management. (1998).3.OMG. Mobile Agent Facility Specification. (2000)

4.Ad Astra Engineering. Jumping Beans White Paper. http://www.JumpingBeans.com/,

October 1998. Sunnyvale, California.

5.Stanford University. The JATLite Router. http://java.stanford.edu/JATLiteRouter.html6.Farley, J.: Java Distributed Computing. (1998)7.Lentini, R., Rao, G., Thies, J.: Agent Itineraries: An Alternative Data Structure For Agent

Systems. Dr. Dobbs Journal of Software Tools, 24(5). (1999) 60-70.